Combine columns from 100 files with same structure
Hi,
I have a bunch of files with the following format.
I would like to have col2-4 from all files output side by side with just col1 being the unique identifier across all files.
result.txt
Please help
The solution provided my mirni seems to work OK except that it doesn't print the key at the start of each output line. (It also prints the 0 value as 0.000, but I don't see why that really matters.) It does however invoke awk once for each input file and once for each key, and it invokes grep on each input file once for each key.
I believe that the following provides all of the requested features only invoking awk once and sort once (with no need for grep). However, this script requires that all of the data in the input files be accumulated in awk's address space at once while mirni's script just requires that the entries for a single key be kept in memory at once.
To run this, save the following script in a file (e.g.,combine):
Make it executable (chmod +x combine), and run it with something like the following:
Although the script above specifies ksh, it will also work with at least bash and sh.
This User Gave Thanks to Don Cragun For This Post:
Both the scripts worked very well Thank you.
Is it possible to have the filenames as 1st rows in the results file.
Thanks
!GH
Since the script I gave doesn't know what files will be passed in as arguments, thels PUR.*won't work in the general case. But, if you change:
in the script I provided to:
(Note that the first argument to ls (-1) uses the digit one; not the letter ell. It prints only the filenames in a single column, which is what you requested. With-l(the letter ell), you could get a long format listing of the files to be processed.
Using the-1(digit one) only matters when output from the script is not redirected to a file. This is the default when output is not directed to a terminal device file.)
or you could use:
to print the filename of each file as it is processed. And, if the output is not redirected to a file, it provides a progress report as processing starts for each file. Obviously, you can change the printf format string to format the display of the filenames in whatever manner you wish.
This User Gave Thanks to Don Cragun For This Post:
i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Need some help with this ... please
60644,NJ090237_0263_GRP,NJ090237_0263_VIEW,NJ090237_0263_PSGRP,NJ090237_0263_GOLD_CSGRP,,06E:0_08E:0_09E:0_11E:0,0CE5,TDEV,34,VP_TIER... (3 Replies)
Hello everyone,
I searched the forum looking for answers to this but I could not pinpoint exactly what I need as I keep having trouble.
I have many files each having two columns and hundreds of rows.
first column is a string (can have many words) and the second column is a number.The files are... (5 Replies)
This may seem obvious but I am having problems doing this as columns get converted to rows when i try to write a script.
I have 2 files text1.txt and text2.txt each of which have 6 columns of numbers separated by a space.
I need to combine the 2 files so that the output file text3.txt maintains... (2 Replies)
Hi Guys,
I want to combine 2 files and and put together in 1 file and make two columns out it. See below desired output. Any help will be much appreciated.
inputfile1.txt
12345
67890
24580
inputfile2.txt
AAAAA
BBBBB
CCCCC (11 Replies)
Hello everybody,
I have a text file containing 10,000 rows and 5000 columns. The values are separated by a tab.
Ex.
file_ex.ped
1 mike 0 0 2 1 A A G G C T A G
1 jack 0 0 2 2 T A G T C A A C
1 Mary 0 0 1 2 A T G C A T G C
...
I would like a out put file
1 mike 0 0 2 1 AA GG CT AG
1... (7 Replies)
Hi there,
I was wondering if someone can help me with this.
I am trying the combine multiple columns from multiple files into one file.
Example file 1:
c0t0d0 c0t2d0 # hostname vgname
c0t0d1 c0t2d1 # hostname vgname
c0t0d2 c0t2d2 # hostname vgname
c0t1d0 c0t3d0 # hostname vgname1... (5 Replies)
Hi,
I am trying to write a script that will move all the files from source directory structure(multiple levels might exist) to destination directory structure. If a sub folder is source doesnot exist in destination then I have to skip and goto next level. I also need to delete the files in... (4 Replies)