Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Merge columns from two files using awk
Top Forums Shell Programming and Scripting Merge columns from two files using awk Post 302978960 by Scrutinizer on Sunday 7th of August 2016 05:16:02 AM
Old 08-07-2016
RudiC has given you some good pointers to how you could solve your problem. This kind of problem is posted here frequently. What makes this problem stand out is the imperfections in the input files. b.csv contains spurious leading/trailing spaces in multiple fields, which will likely mess up the results.

So this can be fixed by getting rid of the space before storing and comparing, but also storing the imperfections, so they are not altered in the process:

Code:
awk '{r=$0; for(i=1; i<=NF; i++) gsub(/^ +| +$/,x,$i)} NR==FNR{A[$1,$2]=$3; next} {print r, A[$1,$2]}' FS=, OFS=, a.csv b.csv

Whereas with a proper input file, something like this would suffice:
Code:
awk 'NR==FNR{A[$1,$2]=$3; next} {print $0, A[$1,$2]}' FS=, OFS=,  a.csv b.csv

These approaches can be further augmented by first testing if a field pair is present in the other file..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

compare columns from 2 files and merge

Dear all, Being new to Unix i have a problem. I have 2 files: File 1: 118,1,0,2,3,0,5,0.3,0,0.3,0.6,1 118,2,1,2,2,0,5,0.4,0,0.4,0.4,1 118,4,2,0,3,0,5,0.7,0,0.3,0.6,1 118,6,4,1,0,0,5,0.8,0,0.2,0,1 File 2: 118,1,BFGL-NGS-109695,3610326,0,18,1,0.556,0.389,0.056,0.25,0.8183... (2 Replies)
Discussion started by: samwilkinson
2 Replies

2. Shell Programming and Scripting

merge the two files which has contain columns

Hi may i ask how to accomplish this task: I have 2 files which has multiple columns first file 1 a 2 b 3 c 4 d second file 14 a 9 .... 13 b 10.... 12 c 11... 11 d 12... I want to merge the second file to first file that will looks like this ... (2 Replies)
Discussion started by: jao_madn
2 Replies

3. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

4. Shell Programming and Scripting

Merge two files matching columns

Hi! I need to merge two files when col1 (x:x:x) matching and adds second column from file1.txt. # cat 1.txt aaa;a12 bbb;b13 ccc;c33 ddd;d55 eee;e11 # cat 2.txt bbb;b55;34444;d55 aaa;a15;35666;a44 I try with this awk and I get succesfully first column from 1.txt: # awk -F";"... (2 Replies)
Discussion started by: fhluque
2 Replies

5. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

6. Shell Programming and Scripting

Merge 2 files with one reference columns

Hi All Source1 servername1,patchid1 servername1,patchid2 servername1,patchid3 servername2,patchid1 servername2,patchid2 servername3,patchid4 servername3,patchid5 Source2 servername1,appname1 servername1,appname2 servername1,appname3 servername2,appname1 servername2,appname2... (13 Replies)
Discussion started by: mv_mv
13 Replies

7. Shell Programming and Scripting

Merge columns from multiple files

Hello and Good day I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files . I need to merge $2,$3,$6 from first file and $6 from another files. File1: $1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies

8. Shell Programming and Scripting

Merge files based on columns

011111123444 1234 1 20000 011111123444 1235 1 30000 011111123446 1234 3 40000 011111123447 1234 4 50000 011111123448 1234 3 50000 File2: 011111123444,Rsttponrfgtrgtrkrfrgtrgrer 011111123446,Rsttponrfgtrgtr 011111123447,Rsttponrfgtrguii 011111123448,Rsttponrfgtrgtjiiu I have 2 files... (4 Replies)
Discussion started by: vinus
4 Replies

9. Shell Programming and Scripting

awk merge matching columns

I know I'm not the first one asking this but my code still does not work: File 1: gi|1283| tRNAscan exon 87020 88058 . - . transcript_id "Parent=tRNA-Tyr5.r01"; gi|3283| tRNAscan exon 97020 97058 . + . transcript_id "Parent=tRNA-Tyr6.r01"; gi|4283| rRNAscan exon 197020 197058 . - . transcript_id... (5 Replies)
Discussion started by: 0sMoses
5 Replies

10. Shell Programming and Scripting

Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk

Hello, This post is already here but want to do this with another way Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: 1|123|jojo 1|NULL|bibi... (2 Replies)
Discussion started by: yjacknewton
2 Replies

LEARN ABOUT DEBIAN

rec2csv

REC2CSV(1)							   User Commands							REC2CSV(1)

NAME

       rec2csv - rec to csv converter

SYNOPSIS

       rec2csv [OPTIONS]... [REC_FILE]

DESCRIPTION

       Convert rec data into csv data.

       -t, --type=TYPE
	      record set to convert to csv; if this parameter is omitted then the default record set is used

       -S, --sort=FIELD
	      sort the output by the specified field.

       --help print a help message and exit.

       --version
	      show version and exit.

AUTHOR

       Written by Jose E. Marchesi.

REPORTING BUGS

       Report bugs to: bug-recutils@gnu.org
       GNU recutils home page: <http://www.gnu.org/software/recutils/>
       General help using GNU software: <http://www.gnu.org/gethelp/>

COPYRIGHT

       Copyright (C) 2010, 2011, 2012 Jose E. Marchesi.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       The  full  documentation  for  rec2csv  is maintained as a Texinfo manual.  If the info and rec2csv programs are properly installed at your
       site, the command

	      info rec2csv

       should give you access to the complete manual.

rec2csv 1.4.93							   January 2012 							REC2CSV(1)
All times are GMT -4. The time now is 07:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy