Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk
Top Forums Shell Programming and Scripting Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk Post 303040811 by yjacknewton on Thursday 7th of November 2019 07:24:16 AM
Old 11-07-2019
Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk
Hello,

This post is already here but want to do this with another way
Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files

file1.csv:

Code:
1|abc
1|def
2|ghi
2|jkl
3|mno
3|pqr

file2.csv:

Code:
1|123|jojo
1|NULL|bibi
3|789|zaza
3|012|NULL

file3.csv:

Code:
2|1a2b|1994|US
3|3c4e|1995|MG
4|5f6g|1996|FR

My desired output is like this, conserve the duplicate key and fill it with the data of anothers files, and the void columns is filling by NULL:

result.csv
Code:
1|abc|123|jojo|NULL|NULL|NULL
1|def|123|jojo|NULL|NULL|NULL
1|abc|NULL|bibi|NULL|NULL|NULL
1|def|NULL|bibi|NULL|NULL|NULL
2|ghi|NULL|NULL|1a2b|1994|US
2|jkl|NULL|NULL|1a2b|1994|US
3|mno|789|zaza|3c4e|1995|MG
3|pqr|789|zaza|3c4e|1995|MG
3|mno|012|NULL|3c4e|1995|MG
3|pqr|012|NULL|3c4e|1995|MG
4|NULL|NULL|NULL|5f6g|1996|FR

I already found code with JOIN but I want to do this with AWK
This is the "join" command :

Code:
join -t \| -a 1 -a 2 -e 'NULL' -o 0 1.2 1.3 1.4 2.2 2.3 2.4 \ 
\;(join -t \| -a 1 -a 2 -e 'NULL' -o 0 1.2 2.2 2.3 file1 file2) file3

Is there an idea with awk for more performance?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Left join on files using awk

nawk 'NR==FNR{a;next} {if($1 in a) print $1,"Found" else print}' OFS="," File_B File_A The above code is not working help is appreciated (6 Replies)
Discussion started by: pinnacle
6 Replies

2. Programming

sql,multiple join,outer join issue

example sql: select a.a1,b.b1,c.c1,d.d1,e.e1 from a left outer join b on a.x=b.x left outer join c on b.y=c.y left outer join d on d.z=a.z inner join a.t=e.t I know how single outer or inner join works in sql. But I don't really understand when there are multiple of them. can... (0 Replies)
Discussion started by: robbiezr
0 Replies

3. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7... (3 Replies)
Discussion started by: InfoSeeker
3 Replies

4. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

5. Shell Programming and Scripting

Remove ":" and join lines in outline file

I have a vim outliner file like this: Title title 2 :Testing now :testing 2 :testing 3 title 3 :testing :ttt :ttg Is there a way to use a script or command to remove... (7 Replies)
Discussion started by: jostber
7 Replies

6. Web Development

Perl join two files by "common" column

Hello; I am posting to get any help on my code that I have been struggling for some time. The project is to join two files each with 80k~180k rows. I want to merge them together by the shared common column. The problem of the shared column is partially matching, not exactly the same. File1:... (5 Replies)
Discussion started by: yifangt
5 Replies

7. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

8. Shell Programming and Scripting

left join using awk

Hi guys, I need to use awk to join 2 files file_1 A 001 B 002 C 003 file_2 A XX1 B XX2 output desired A 001 XX1 B 002 missing C 003 XX2 thank you! (2 Replies)
Discussion started by: g1org1o
2 Replies

9. Shell Programming and Scripting

left join using awk

Hi guys, I need AWK to merge the following 2 files: file1 1 a 1 1 2 b 2 2 3 c 3 3 4 d 4 4 file2 a a/a c/c a/c c/c a/a c/t c c/t c/c a/t g/g c/c c/t desired output: 1 a 1 1 a/a c/c a/c c/c a/a c/t 2 b 2 2 x x x x x x 3 c 3 3 c/t c/c a/t g/g c/c c/t 4 d 4 4 x x x x x x (2 Replies)
Discussion started by: g1org1o
2 Replies

10. Shell Programming and Scripting

Join and merge multiple files with duplicate key and fill void columns

Join and merge multiple files with duplicate key and fill void columns Hi guys, I have many files that I want to merge: file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: (5 Replies)
Discussion started by: yjacknewton
5 Replies

LEARN ABOUT DEBIAN

flow-merge

flow-merge(1)						      General Commands Manual						     flow-merge(1)

NAME

       flow-merge -- Merge flow files.

SYNOPSIS

       flow-merge [-aghm]  [-b big|little]  [-C comment]  [-d debug_level]  [-o filename]  [-z z_level]  [file|directory ...]

DESCRIPTION

       The  flow-merge	utility processes files and/or directories of files in the flow-tools format.  The resulting merged data set is written to
       the standard output or file specified by -o.  If file is a single dash (`-') or absent, flow-merge  will  read  from  the  standard  input.
       Unlike flow-cat, flow-merge interleaves flow records preserving the relative chronological order.

OPTIONS

       -a	 Do not ignore filenames that begin with tmp.

       -b big|little
		 Byte order of output.

       -C Comment
		 Add a comment.

       -d debug_level
		 Enable debugging.

       -g	 Sort file list by capture start time before processing.

       -h	 Display help.

       -m	 Disable the use of mmap().

       -p	 Preload headers.  Use to preserve meta information such as lost flows.

       -o file	 Write to file instead of the standard out.

       -z z_level
		 Configure compression level to  z_level.  0 is disabled (no compression), 9 is highest compression.

       file|directory...
		 Process the files and/or directory.

EXAMPLES

       Merge all flow files begining with ft-v05.2001-05.01, use flow-print to display the results.

	   flow-merge ft-v05.2001-05-01.* | flow-print

BUGS

       None known.

AUTHOR

       Larry Lidz ellidz@eridu.uchicago.edu

SEE ALSO

       flow-tools(1)

																     flow-merge(1)
All times are GMT -4. The time now is 02:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy