Sponsored Content
Top Forums Shell Programming and Scripting Merge 70 files into one data matrix Post 302258204 by labrazil on Friday 14th of November 2008 03:13:18 AM
Old 11-14-2008
this is what i made:
Code:
#!/usr/bin/perl -w

#creates a hash and reads in all files in a given directory.  
# all files are then read once, inserted into hash and file is closed
#this script will merge all data from multiple files into one big data matrix.

#create new hash
my %hash;

#open directory and read in file names into an array
opendir (DIR, "/original_files/") or die "Cannot open $!";
my @files = readdir DIR;
close DIR;

#open each file
foreach my $files (@files) {
open (FILE , "/original_files/$files") or die "Cannot open $files!";

#read in each line of the file and dump it into a hash, if it exists, append at end.
	while (<FILE>)
	{
	   chomp;
	   my ($key, $val) = split /\t/;
	   $hash{$key} .= exists $hash{$key} ? "\t$val" : $val;
	}
	close FILE;
}

#printout the hash and have fun with new datamatrix
open (OUT, ">merged.txt") or die "cannot open $!";
foreach $key (sort keys %hash)
    {
	print OUT "$key\t$hash{$key}\n";
    }

what do you think Anni?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extract data from a data matrix with filter criteria

Here is what old matrix look like, IDs X1 X2 Y1 Y2 10914061 -0.364613333 -0.362922333 0.001691 -0.450094667 10855062 0.845956333 0.860396667 0.014440333 1.483899333... (7 Replies)
Discussion started by: ssshen
7 Replies

2. Shell Programming and Scripting

two-column data to matrix in AWK

Howdy, I need to convert an association data matrix, currently in a two-column format, into a matrix with numbers indicating the number of associations. I've been looking around for AWK code in the list, but could not find anything. Here's an example of what I want to perform: original... (10 Replies)
Discussion started by: sramirez
10 Replies

3. Ubuntu

How to convert full data matrix to linearised left data matrix?

Hi all, Is there a way to convert full data matrix to linearised left data matrix? e.g full data matrix Bh1 Bh2 Bh3 Bh4 Bh5 Bh6 Bh7 Bh1 0 0.241058 0.236129 0.244397 0.237479 0.240767 0.245245 Bh2 0.241058 0 0.240594 0.241931 0.241975 ... (8 Replies)
Discussion started by: evoll
8 Replies

4. Shell Programming and Scripting

Reformatting data in matrix form

Hi, Some assistance with respect to the following problem will be very helpful. I want to reformat my dataset in the following manner for subsequent analysis. I have first column values (which repeat for each value of 2nd column) which are names, the second column specifies position ad the... (1 Reply)
Discussion started by: newbie83
1 Replies

5. Shell Programming and Scripting

Merge multiple tables into big matrix

Hi all, I have a complex (beyond my biological expertise) problem at hand. I need to merge multiple files into 1 big matrix. Please help me with some code. Inp1 Ang_0 chr1 98 T A Ang_0 chr1 352 G A Ang_0 chr1 425 C T Ang_0 chr2 ... (1 Reply)
Discussion started by: newbie83
1 Replies

6. Shell Programming and Scripting

AWK to match and merge data from 2 files into 1.

Hello, hopefully this is an easy on for the AWK guru's out there. I'm having some trouble figuring out how to match+merge data in 2 files into 1 single report. I've got my 2 files filtered and delimited, just need to MATCH $3 in file1 to $1 in file2, then put $0 from File1 and $2+$3 from File2... (6 Replies)
Discussion started by: right_coaster
6 Replies

7. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

8. Shell Programming and Scripting

How to merge the multiple data files as a single file?

Hi Experts, I have created multiple scripts and send the output to new file, getting this output to my mailbox on daily basis. I would like to send the all outputs to a single file, need to merge all file outputs on a single file. For example, Created script for df -h > df.doc grep... (7 Replies)
Discussion started by: seenuvasan1985
7 Replies

9. Shell Programming and Scripting

Merge files and copy some data using sed or awk

Copy data from other file to paste cat file1: Name: server1.data.com data1 server1 running Name: server3.data.com data3 server3 running cat file2: server1 good server2 bad network not ok server3 good Output: (10 Replies)
Discussion started by: kenshinhimura
10 Replies

10. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies
Hash::Util(3pm) 					 Perl Programmers Reference Guide					   Hash::Util(3pm)

NAME
Hash::Util - A selection of general-utility hash subroutines SYNOPSIS
use Hash::Util qw(lock_keys unlock_keys lock_value unlock_value lock_hash unlock_hash); %hash = (foo => 42, bar => 23); lock_keys(%hash); lock_keys(%hash, @keyset); unlock_keys(%hash); lock_value (%hash, 'foo'); unlock_value(%hash, 'foo'); lock_hash (%hash); unlock_hash(%hash); DESCRIPTION
"Hash::Util" contains special functions for manipulating hashes that don't really warrant a keyword. By default "Hash::Util" does not export anything. Restricted hashes 5.8.0 introduces the ability to restrict a hash to a certain set of keys. No keys outside of this set can be added. It also introduces the ability to lock an individual key so it cannot be deleted and the value cannot be changed. This is intended to largely replace the deprecated pseudo-hashes. lock_keys unlock_keys lock_keys(%hash); lock_keys(%hash, @keys); Restricts the given %hash's set of keys to @keys. If @keys is not given it restricts it to its current keyset. No more keys can be added. delete() and exists() will still work, but will not alter the set of allowed keys. Note: the current implementation prevents the hash from being bless()ed while it is in a locked state. Any attempt to do so will raise an exception. Of course you can still bless() the hash before you call lock_keys() so this shouldn't be a problem. unlock_keys(%hash); Removes the restriction on the %hash's keyset. lock_value unlock_value lock_value (%hash, $key); unlock_value(%hash, $key); Locks and unlocks an individual key of a hash. The value of a locked key cannot be changed. %hash must have already been locked for this to have useful effect. lock_hash unlock_hash lock_hash(%hash); lock_hash() locks an entire hash, making all keys and values readonly. No value can be changed, no keys can be added or deleted. unlock_hash(%hash); unlock_hash() does the opposite of lock_hash(). All keys and values are made read/write. All values can be changed and keys can be added and deleted. AUTHOR
Michael G Schwern <schwern@pobox.com> on top of code by Nick Ing-Simmons and Jeffrey Friedl. SEE ALSO
Scalar::Util, List::Util, Hash::Util perl v5.8.0 2002-06-01 Hash::Util(3pm)
All times are GMT -4. The time now is 04:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy