gawk: Merge N files into one


 
Thread Tools Search this Thread
Special Forums Windows & DOS: Issues & Discussions gawk: Merge N files into one
# 1  
Old 05-16-2012
gawk: Merge N files into one

hi all!

i'm an awk newbie and have been trying in vain to merge N files together.

ie.

file1.txt:
Code:
a b c
1 1 1
2 2 2
3 3 3
4 4 4

file2.txt:
Code:
a b c
5 5 5
6 6 6
7 7 7
8 8 8

file3.txt:
Code:
a b c
9 9 9
10 10 10


i want to end up with:
Code:
a b c
1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
6 6 6
7 7 7
8 8 8
9 9 9
10 10 10

And i also like to check if the 2nd or 3rd file has diferente number of colums, it stops.

Thanks in advance.

BR.

Last edited by Franklin52; 05-16-2012 at 08:09 AM.. Reason: Please use code tags for data and code samples
# 2  
Old 05-16-2012
Code:
awk 'NR==1{print;next}FNR==1{next}1' file1.txt file2.txt file3.txt > outfile

This User Gave Thanks to Franklin52 For This Post:
# 3  
Old 05-16-2012
Thanks

But can I validate that the number of columns of each file is the same?
# 4  
Old 05-16-2012
Yes, awk stores the number of the columns in the internal variable NF.

For example:
Code:
awk '{print FILENAME, NF}' file1.txt file2.txt file3.txt


Last edited by Franklin52; 05-16-2012 at 09:05 AM.. Reason: Adding sample
# 5  
Old 05-16-2012
can send me please the code integrated that before processing all files validate the headers - number of columns is equal in all.

Only creates or process the new merged file if the number of columns is equal in boths files.
# 6  
Old 05-16-2012
This prints the filename and the number of columns if it has more or less columns then the first row:
Code:
awk 'NR==1{n=NF}NF != n{print FILENAME " has " NF " columns"} 'file1.txt file2.txt file3.txt

# 7  
Old 05-16-2012
I believe that I did not explained well.

What I want is the following:

file1.txt:
Code:
a b c
1 1 1
2 2 2
3 3 3
4 4 4

file2.txt:
Code:
a b c
5 5 5
6 6 6
7 7 7
8 8 8

file3.txt:
Code:
a b c
9 9 9
10 10 10

i want to end up with:

Code:
a b c
1 1 1
2 2 2
3 3 3
4 4 4
5 5 5
6 6 6
7 7 7
8 8 8
9 9 9
10 10 10

but if the 3rd file was:

Code:
a b
9 9
10 10

the tool should not generate any files.

And can you explain what represent 1 in the code: {next}1'. I tried 2,3,etc numbers and it works.

Thanks for your time but I am a zero in gawk

Last edited by Scrutinizer; 05-16-2012 at 06:17 PM.. Reason: code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to merge two files?

Dear Frens, I have two files and need to merge into one file. Like File_1: Field1 Field2 1 4 File_2: Field1 Field2 3 5 I need one single output as File_1: Field1 Field2 1 4 3 5 This means taking header from either file. (8 Replies)
Discussion started by: manisha_singh
8 Replies

2. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies

3. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

4. Shell Programming and Scripting

Reading in all files from parent directory (GAWK)

Hi all, I'm very, very new to scripting (let alone SHELL) and was wondering if anyone could help me out as I seem to be in a spot of bother. I collect data (.dat files) which are automatically seperated into several sub directories, so the file paths I'm reading in at the moment would be... (11 Replies)
Discussion started by: gd9629
11 Replies

5. Shell Programming and Scripting

Merge 2 files

Hi , This might be the stupidest question I am asking, but I am struck with this problem. I am trying to merge 2 files, file1 has header and file2 has contents. while I merge them , it merges from the 1st line of file1. for ex: file1 col1|col2|col3| file2 123|234|456|... (2 Replies)
Discussion started by: rashmisb
2 Replies

6. Shell Programming and Scripting

Gawk / Awk Merge Lines based on Key

Hi Guys, After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines. I'm an SEO Consultant and provide... (9 Replies)
Discussion started by: Jamesfirst
9 Replies

7. Shell Programming and Scripting

How can i merge these two files into several...

Given are File A and File B File A has for example 5 lines: AAA BBB CCC DDD EEE File B has 3 lines: 111 222 333 How can i merge A and B into: 111 222 333 AAA (first line from A) then a new file: (4 Replies)
Discussion started by: Y-T
4 Replies

8. Shell Programming and Scripting

Using GAWK to combine files

Hello All, I have a folder containing few files. Each & every file contain only 1 column. I want to combine only column of all the files through GAWK, separate them by a delimiter and store it to a new file. So basically using GAWK, I want to combine '$1' of all files, separate them by a... (11 Replies)
Discussion started by: paragkalra
11 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. Shell Programming and Scripting

gawk - reading two files & re arrange the columns

Hi, I am trying to read 2 files and writing to the 3rd file if I find the same elements in 2 files. my first file is 1 0 kb12124819 766409 1.586e-01 1 0 kb17160939 773886 8.674e-01 1 0 kb4475691 836671 8.142e-01 1 0 ... (2 Replies)
Discussion started by: ezhil01
2 Replies
Login or Register to Ask a Question