Sponsored Content
Top Forums Shell Programming and Scripting Merge columns from multiple files Post 302903280 by Don Cragun on Monday 26th of May 2014 04:23:14 PM
Old 05-26-2014
What operating system are you using?

Do all of your input files have all of the same values in columns 2 and 3 and do they all have those values in the same order? (If not, how should the output denote no data for a given $2,$3 pair for a given input file?)

Do all 144 files have a common naming scheme? (Are the names really File1 through File144? If so, is it important for the output columns to have the files listed in numeric sort order rather than alphanumeric sort order?)

Your input files seem to have tab separated fields (except the header line), should the output be tab separated or use sequences of spaces for separators? Is it important for the output columns to be aligned and adjust to the maximum input field width for each column (including filenames in the headers)? Why do some of the header fields have $6 filename (with a space) and some have $6filename (without a space)? How is your script supposed to know which output field headers are supposed to have a space?

Are there other files in the directory where you are going to run this script other than the 144 input files you're trying merge? (I assume there are and that is why you're seeing extra columns.) Show us the output you're getting from SriniShoo's script and RavinderSingh13's script for your three sample files. What "extra columns" are you getting?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge the two files which has contain columns

Hi may i ask how to accomplish this task: I have 2 files which has multiple columns first file 1 a 2 b 3 c 4 d second file 14 a 9 .... 13 b 10.... 12 c 11... 11 d 12... I want to merge the second file to first file that will looks like this ... (2 Replies)
Discussion started by: jao_madn
2 Replies

2. UNIX for Dummies Questions & Answers

Merge two files with two columns being similar

Hi everyone. How can I merge two files, where each file has 2 columns and the first columns in both files are similar? I want all in a file of 4 columns; join command removes the duplicate columns. 1 Dave 2 Mark 3 Paul 1 Apple 2 Orange 3 Grapes to get it like this in the 3rd file:... (9 Replies)
Discussion started by: Atrisa
9 Replies

3. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

4. UNIX for Dummies Questions & Answers

How do I merge multiple columns into one column?

Hi all, I'm looking for a way to merge multiple columns (from one file) into a single column in an output file. The file I have looks somewhat like this: @HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII @HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies

5. UNIX for Dummies Questions & Answers

Merge columns from multiple files

Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files. I know paste will append columns like so: paste file1 file2 file3 file4 file5 ... But this becomes inconvenient when you want to append a large number of files into a single file. ... (2 Replies)
Discussion started by: torchij
2 Replies

6. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

7. Shell Programming and Scripting

Merge 2 files with one reference columns

Hi All Source1 servername1,patchid1 servername1,patchid2 servername1,patchid3 servername2,patchid1 servername2,patchid2 servername3,patchid4 servername3,patchid5 Source2 servername1,appname1 servername1,appname2 servername1,appname3 servername2,appname1 servername2,appname2... (13 Replies)
Discussion started by: mv_mv
13 Replies

8. Shell Programming and Scripting

Merge records based on multiple columns

Hi, I have a file with 16 columns and out of these 16 columns 14 are key columns, 15 th is order column and 16th column is having information. I need to concate the 16th column based on value of 1-14th column as key in order of 15th column. Here are the example file Input File (multiple... (3 Replies)
Discussion started by: Ravi Agrawal
3 Replies

9. UNIX for Beginners Questions & Answers

Merge multiple columns into one using cat

I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file. file1.txt: 1234 1234 1234 file2.txt: aaaa bbbb cccc dddd cat file1.txt file2.txt > file3.txt file3.txt: 1234 1234 1234aaaa bbbb cccc... (13 Replies)
Discussion started by: geomarine
13 Replies

10. Shell Programming and Scripting

Join and merge multiple files with duplicate key and fill void columns

Join and merge multiple files with duplicate key and fill void columns Hi guys, I have many files that I want to merge: file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: (5 Replies)
Discussion started by: yjacknewton
5 Replies
getcol(1)						      General Commands Manual							 getcol(1)

Name
       getcol - Extract specified columns from an ASCII table file

Synopsis
       getcol [-amv][-n num][-r lines][-s num] filename [column number range]

Description
       Extract specified columns from an ASCII table file

Options
       filename
	      Name  of a ASCII table file.  At least one of these must be present for any values to be printed.  If it is stdin or STDIN, an ASCII
	      table is expected as standard input.  If there is no input file, standard input is assumed.

       @filename
	      Name of a file containing a list of ASCII table files.  If this is present, any other  file  names  on  the  command  line  will	be
	      ignored.

       field range
	      Print  value  of	these  columns for the number of lines of the table specified by the -n argument after the skippiing the number of
	      lines specified by the -s argument.  A value of 0 causes the entire input line to be printed.

       -a     Sum all numeric columns selected, printing the sum on the line following the result.  Columns with  no  sum  are	filled	with  ___.
	      (Added in version 2.6.9)

       -b     Input is bar-separate table file

       -c     Add count of number of lines in each column at end

       -d <number>
	      Number of decimal places in f.p. output

       -e     Compute medians of selected columns

       -f     Print range of values in selected columns

       -h     Print Starbase tab table header

       -i     Input is tab-separate table file

       -k     Print number of columns on first line

       -l <number>
	      Number of lines to add to each line

       -m     Compute the means of all numeric columns selected, printing the mean on the line following the result (or the line following the sum
	      if -a is used).  Columns with no mean are filled with ___.  (Added in version 2.6.9)

       -n num Print selected columns for this many lines.  If not specified, all lines will be read after the number of lines specified by -s have
	      been skipped.

       -o     OR conditions insted of ANDing them

       -p     Print only sum, mmean, sigma, median, or range, not entries

       -r @listfile
	      -r  line	range  Print  columns from the lines specified as either the first nonzero number on each line of the file listfile or the
	      comma- and hyphen- delimitied range; i.e. 1-5,10-12 will print values from lines 1, 2, 3, 4, 5, 10, 11, and 12.  (added  in  version
	      2.6.12)

       -s num Skip this many line before starting to print values.  If not specified, no lines will be skipped.

       -t     Starbase (tab-separated) table output

       -v     Print more information about process.

       Web Page
	      http://tdc-www.harvard.edu/software/wcstools/getcol.html

Author
       Doug Mink, SAO (dmink@cfa.harvard.edu)

8 November 2001 						     WCSTools								 getcol(1)
All times are GMT -4. The time now is 06:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy