Sponsored Content
Top Forums Shell Programming and Scripting Multiple file merge by column Post 302769085 by atzounis on Monday 11th of February 2013 06:49:21 AM
Old 02-11-2013
Hello and thanks a lot for your immediate reply.
It was my mistake to use "|" as separator because the actual separator is " " (single space) and files are .dat
Guess the code that does the work and has a file as an output is:

Code:
awk -F " " '{X[$1,$2]=X[$1,$2]?X[$1,$2] FS $3 "-" FILENAME : $0 "-" FILENAME}END{for(i in X){print X[i]}}' *.dat >output.dat

Right?

---------- Post updated at 01:49 PM ---------- Previous update was at 01:43 PM ----------

Just tried it. It really works, apart from one thing: This code puts one file at the end of the other.
Example:
file1-00.00
x y temp
1 1 10

and file2-00.10
x y temp
1 1 12

Become file output
x y temp
file1-00.00
1 1 10
file2-00.10
1 1 12

I would prefer the output to be like
x y temp-file1-00.00 temp-file2-00.10 temp-fileXX-23.50
1 1 10 12 zz
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge two two txt files into one file based on one column

Hi, I have file1.txt and file2.txt and would like to create file3.txt based on one column in UNIX Eg: file1.txt 17328756,0000786623.pdf,0000786623 20115537,0000793892.pdf,0000793892 file2.txt 12521_74_4.zip,0000786623.pdf 12521_15_5.zip,0000793892.pdf Desired Output ... (5 Replies)
Discussion started by: techmoris
5 Replies

2. Shell Programming and Scripting

Help with merge two file based on similar column content

Input file 1: A1BG A1BG A1BG A1CF A1CF BCAS BCAS A2LD1 A2M A2M HAT . . Input file 2: A1BG All A1CF TEMP (5 Replies)
Discussion started by: perl_beginner
5 Replies

3. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

4. UNIX for Advanced & Expert Users

merge two column multiple files into one

Hi I have multiple files each with two columns and I need to combine all those file into a tab delimited file. (multiple entry with same name separated by a comma) The content of the files are as follows: --- file1.txt: name var1 aaa xx aaa gg bbb yy ddd zz --- file2.txt ... (8 Replies)
Discussion started by: mary271
8 Replies

5. Shell Programming and Scripting

How to merge multiple rows into single row if first column matches ?

Hi, Can anyone suggest quick way to get desired output? Sample input file content: A 12 9 A -0.3 2.3 B 1.0 -4 C 34 1000 C -111 900 C 99 0.09 Output required: A 12 9 -0.3 2.3 B 1.0 -4 C 34 1000 -111 900 99 0.09 Thanks (3 Replies)
Discussion started by: cbm_000
3 Replies

6. UNIX for Dummies Questions & Answers

How do I merge multiple columns into one column?

Hi all, I'm looking for a way to merge multiple columns (from one file) into a single column in an output file. The file I have looks somewhat like this: @HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII @HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies

7. Shell Programming and Scripting

Merge the multiple text files into one file

Hi All, I am trying to merge all the text files into one file using below snippet cat /home/Temp/Test/Log/*.txt >> all.txt But it seems it is not working. I have multiple files like Output_ServerName1.txt, Output_ServreName2.txt I want to merge each file into one single file and... (6 Replies)
Discussion started by: sharsour
6 Replies

8. Shell Programming and Scripting

Seperated by columns, merge in a file, sort them on common column

Hi All, I have 4 files in below format. I took them as an example. File 1: Cut from position 1-4 then 6-7 then 8-14 then rest left and make them as columns in one new file. Inserting character H to the initial of all line like HCTOT. CTOT 456787897 Low fever CTOR 556712345 High fever... (2 Replies)
Discussion started by: Mannu2525
2 Replies

9. Shell Programming and Scripting

Merge column file

Hi All, I have on file1 and file2 some, $cat file1 aaa bbb ccc ddd eee fff ggg hhh iii jjj with line blank, and (12 Replies)
Discussion started by: aav1307
12 Replies

10. UNIX for Beginners Questions & Answers

Cut specific column from 2 file and merge

Hi ALL, I have two file. I need to combine these two file based on a layout. I used the below code and able to extract the record. But now able to insert that to a 3'rd file in between the extract FILE 1 CAID NUMBER 1-20 TID NUMBER 21-22 LABEL CHAR 23-44 BASE 45-60... (5 Replies)
Discussion started by: arunkumar_mca
5 Replies
GMT_SHELL_FUNCTIONS.SH(1gmt)				       Generic Mapping Tools				      GMT_SHELL_FUNCTIONS.SH(1gmt)

NAME
gmt_shell_functions.sh - Practical functions to be used in GMT bourne shell scripts SYNOPSIS
gmt_init_tmpdir gmt_remove_tmpdir gmt_clean_up [prefix] gmt_message message gmt_abort message gmt_nrecords file(s) gmt_nfields string gmt_get_field string gmt_get_region file(s) [options] gmt_get_gridregion file [options] gmt_get_map_width -R -J gmt_get_map_height -R -J gmt_set_psfile file gmt_set_framename prefix framenumber gmt_set_framenext framenumber DESCRIPTION
gmt_shell_functions.sh provides a set of functions to Bourne (again) shell scripts in support of GMT. The calling shell script should include the following line, before the functions can be used: . gmt_shell_functions.sh Once included in a shell script, gmt_shell_functions.sh allows GMT users to do some scripting more easily than otherwise. The functions made available are: gmt_init_tmpdir Creates a temporary directory in /tmp or (when defined) in the directory specified by the environment variable TMPDIR. The name of the temporary directory is returned as environment variable GMT_TMPDIR. This function also causes GMT to run in `isolation mode', i.e. all temporary files will be created in GMT_TMPDIR and the .gmtdefaults file will not be adjusted. gmt_remove_tmpdir Removes the temporary directory and unsets the GMT_TMPDIR environment variable. gmt_cleanup Remove all files and directories in which the current process number is part of the file name. If the optional prefix is given then we also delete all files and directories that begins with the given prefix. gmt_message Send a message to standard error. gmt_abort Send a message to standard error and exit the shell. gmt_nrecords Returns the total number of lines in file(s) gmt_nfields Returns the number of fields or words in string gmt_get_field Returns the given field in a string. Must pass string between double quotes to preserve it as one item. gmt_get_region Returns the region in the form w/e/s/n based on the data in table file(s). Optionally add -Idx/dy to round off the answer. gmt_get_gridregion Returns the region in the form w/e/s/n based on the header of a grid file. Optionally add -Idx/dy to round off the answer. gmt_map_width Expects the user to give the desired -R -J settings and returns the map width in the current measurement unit. gmt_map_height Expects the user to give the desired -R -J settings and returns the map height in the current measurement unit. gmt_set_psfile Create the output PostScript file name based on the base name of a given file (usually the script name $0). gmt_set_framename Returns a lexically ordered filename stem (i.e., no extension) given the file prefix and the current frame number, using a width of 6 for the integer including leading zeros. Useful when creating animations and lexically sorted filenames are required. gmt_set_framenext Accepts the current frame integer counter and returns the next integer counter. NOTES
1. These functions only work in the bourne shell (sh) and their derivatives (like ash, bash, ksh and zsh). These functions do not work in the C shell (csh) or their derivatives (like tcsh), and cannot be used in DOS batch scripts either. 2. gmt_shell_functions.sh were first introduced in GMT version 4.2.2 and have since been regularly expanded with other practical scripting short-cuts. If you want to suggest other functions, please do so by mailing to the GMT mailing list: gmt-help@lists.hawaii.edu. SEE ALSO
GMT(1), sh(1), bash(1), minmax(1), grdinfo(1) GMT 4.5.7 15 Jul 2011 GMT_SHELL_FUNCTIONS.SH(1gmt)
All times are GMT -4. The time now is 03:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy