Writing a loop to merge multiple files by common column Post: 302562597

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join multiple files based on 1 common column

I have n files (for ex:64 files) with one similar column. Is it possible to combine them all based on that column ? file1 ax100 20 30 40 ax200 22 33 44 file2 ax100 10 20 40 ax200 12 13 44 file2 ax100 0 0 4 ax200 2 3 4

2. UNIX for Dummies Questions & Answers

loop? print max column in each line for 800 files and merge

Hello, I have 800 or so files with 3 columns each and >10000 lines each. For each file and each line I would like to print the maximum column number for each line. Then I would like to 'paste' each of these files together (column-wise) so that the file with expression in label '_1' is the...

3. UNIX for Dummies Questions & Answers

Writing a for loop that processes multiple input files

I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store the...

4. UNIX for Advanced & Expert Users

merge two column multiple files into one

Hi I have multiple files each with two columns and I need to combine all those file into a tab delimited file. (multiple entry with same name separated by a comma) The content of the files are as follows: --- file1.txt: name var1 aaa xx aaa gg bbb yy ddd zz --- file2.txt ...

5. UNIX for Dummies Questions & Answers

Merge rows with common column

Dear all I have big file with two columns A_AA960715 GO:0006952 A_AA960715 GO:0008152 A_AA960715 GO:0016491 A_AA960715 GO:0007165 A_AA960715 GO:0005618 A_AA960716 GO:0006952 A_AA960716 GO:0005618 A_AA960716...

6. UNIX for Dummies Questions & Answers

Writing a for loop to manipulate multiple files

Hi, I have 1000 text files in a folder that are labeled data1.txt all the way to data1000.txt. I want to write a small script that manipulates the text files in this way: (1) cut the 2nd and 9th columns of the text files (2) sort by the numerical value in the 9th column (3) then save the rows...

7. Shell Programming and Scripting

Count and merge using common column

I have the following records from multiple files. 415 A G 415 A G 415 A T 415 A . 415 A . 421 G A 421 G A,C 421 G A 421 G A 421 G A,C 421 G . 427 A C 427 A ...

8. Shell Programming and Scripting

Merge with common column

hi i have two files and i wanted to join them using common column. try to do this using "join" command but that did not help. File 1: 123 9a.vcf hy92.vcf hy90.vcf Index Ref Alt Ref Alt Ref Alt 315 14 0 7 4 ...

9. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions.

10. Shell Programming and Scripting

Paste columns based on common column: multiple files

Hi all, I've multiple files. In this case 5. Space separated columns. Each file has 12 columns. Each file has 300-400K lines. I want to get the output such that if a value in column 2 is present in all the files then get all the columns of that value and print it side by side. Desired output...

LEARN ABOUT DEBIAN

flow-merge

flow-merge(1)						      General Commands Manual						     flow-merge(1)

NAME

       flow-merge -- Merge flow files.

SYNOPSIS

       flow-merge [-aghm]  [-b big|little]  [-C comment]  [-d debug_level]  [-o filename]  [-z z_level]  [file|directory ...]

DESCRIPTION

       The  flow-merge	utility processes files and/or directories of files in the flow-tools format.  The resulting merged data set is written to
       the standard output or file specified by -o.  If file is a single dash (`-') or absent, flow-merge  will  read  from  the  standard  input.
       Unlike flow-cat, flow-merge interleaves flow records preserving the relative chronological order.

OPTIONS

       -a	 Do not ignore filenames that begin with tmp.

       -b big|little
		 Byte order of output.

       -C Comment
		 Add a comment.

       -d debug_level
		 Enable debugging.

       -g	 Sort file list by capture start time before processing.

       -h	 Display help.

       -m	 Disable the use of mmap().

       -p	 Preload headers.  Use to preserve meta information such as lost flows.

       -o file	 Write to file instead of the standard out.

       -z z_level
		 Configure compression level to  z_level.  0 is disabled (no compression), 9 is highest compression.

       file|directory...
		 Process the files and/or directory.

EXAMPLES

       Merge all flow files begining with ft-v05.2001-05.01, use flow-print to display the results.

	   flow-merge ft-v05.2001-05-01.* | flow-print

BUGS

       None known.

AUTHOR

       Larry Lidz ellidz@eridu.uchicago.edu

SEE ALSO

       flow-tools(1)

																     flow-merge(1)