Parse 2 or more files into one.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse 2 or more files into one.
# 1  
Old 07-31-2011
Parse 2 or more files into one.

Hi,
I have a really simple question...I think. I want to be able to parse two or more files into one by reading the first record from each file into new file then go back to the first file and start reading the second record in from each file into new file and so on. I am new to using awk and am not a programmer. I have played around and researched a bit and have only managed to read in the first rcord from each file and then it stops. (by using nextfile) I need to loop some how and keep track of where I am. I have hacked around with programming enough to be dangerous! Smilie I do understand for loops, while next etc.. Any suggestions are appreciated.


Thanks.
# 2  
Old 07-31-2011
I don't know if I understood your requirement correctly, but try this:
Code:
awk '{print > "record"FNR}' file1 file2 file3 ...

Records will be placed into "record.." files.
# 3  
Old 08-01-2011
that is almost what I want except I want all the records in one file but as broken out by the single files. So when I ran this code File one result had the first record from input files 1 2 and 3. then file 2 result had the second record from file 1 2 and 3 etc.. I want one result file containing all the results so the file looks like this. I hope that explains it a little better.

first file record 1
second file record 1
third file record 1
first file record 2
second file record 2
third file record 2
.......
......
.......
# 4  
Old 08-01-2011
You can accomplish it by:
Code:
cat record* > outfile

# 5  
Old 08-01-2011
I origianally had a space between record and FNR so my out put did not contain the prefix "record" file name was just 1 2 3 etc.. So I definately would need the literal record so I can use wildcard to concatenate as you show. I will have thousands of files. When I took the space out between record and FBNR this is the error I get. BTW thanks for all your input. If I am to much of a pain nt to worry at least you got me moving in the right direction.

Code:
C:\Users\Ray>gawk "{print > "record"FNR}" test.txt test3.txt text4.txt
gawk: (FILENAME=test.txt FNR=1) fatal: expression for `>' redirection has null string value


Last edited by Franklin52; 08-01-2011 at 11:08 AM.. Reason: Please use code tags for data and code samples, thank you
# 6  
Old 08-01-2011
You can't use double quotes like this... If you are using double quotes to enclose AWK code, then inside that code you have to escape them:
Code:
gawk "{print > \"record\"FNR}" test.txt test3.txt text4.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse input of two files to be the same in awk

I have two files that I am going to use diff to find the differences but need to parse them before I do that. I have include the format of each file1 and file2 with the desired output of each (the first 5 fields in each file). The first file has a "chr" before the # that needs to be removed. I... (1 Reply)
Discussion started by: cmccabe
1 Replies

2. Shell Programming and Scripting

Parse log files

Hi all, We are having a sample log like .... test.log:2015.03.17 06:16:24 >> ABC.generateMethod() MethodAException while processing Request! DataForm: Header --- dtd: template.dtd, titleName: berger, requestId: 1503170032131, documentName: invoice123, hostName: acme.net, userName: userABC... (10 Replies)
Discussion started by: tandrei
10 Replies

3. Shell Programming and Scripting

Parse csv files by their names

HI all I have multiple csv files with the names VAR1_VAR2_VAR3_VAR4.csv All the files have the same structure inside just values change. I am trying to retrieve data from those files by fixing at each time one or more VAR. I tried to write a script but I have 2 problems: 2-... (1 Reply)
Discussion started by: Jhon.c
1 Replies

4. Shell Programming and Scripting

awk to parse huge files

Hello All, I have a situation as below: (1) Read a source file (a single file of 1.2 million rows in it ) (2) Read Destination files one by one and replace the content ( few fields in it ) with the corresponding matching field from source file. I tried as below: ( please note I am not... (4 Replies)
Discussion started by: panyam
4 Replies

5. Shell Programming and Scripting

AWK failing to parse on certain files

Dear Unix Gurus, need your expertise to help troubleshoot a certain problem i'm having. I crated a shell script which will ftp get 1 crash log from multiple servers (listed in a text file). Each log will then be parsed by calling an awk script. The problem is, for certain log its parsing... (7 Replies)
Discussion started by: tarj
7 Replies

6. UNIX for Dummies Questions & Answers

How can i parse my Unix log files??

Hello, i would like to parse Unix log files and i would like to use a Unix syslog analyzer. I'm going to use Eucalyptus and i would like to parse its log files. Is there any open source/free syslog parser?? Thanks, in advance! (2 Replies)
Discussion started by: g_p
2 Replies

7. Shell Programming and Scripting

[bash] Parse files

Hi I've 2 folder A and B, they have files with the same name but different content. I mean A contain---------> aa.txt, bb.txt, cc.txt B contain---------> aa.txt, bb.txt, cc.txt but aa.txt in A has different content from aa.txt in B. I'd like to parse the homonyms files in... (7 Replies)
Discussion started by: Dedalus
7 Replies

8. Shell Programming and Scripting

parse files and delete

For the CSV file it should have one entry per line looking like this: pwd/filename,YYYYMMDDhhmmss,OK The date stamp should be from the file properties and OK will be a constant. find . name 'W*' gives me the proper list of files and date +%Y%m%d%H%M%S is the proper date format but I... (0 Replies)
Discussion started by: pimentelgg
0 Replies

9. Shell Programming and Scripting

How to parse 2 files simultaneously

Say I have 2 files of 2 rows of 3 columns each file1: cat catdata1 catdata2 dog dogdata1 dogdata2 file2: cat catdata3 catdata4 dog dogdata3 dogdata4 and I need to combine both files so that is appears like: cat catdata1 catdata2 catdata3 catdata4 dog dogdata1 dogdata2 dogdata3... (8 Replies)
Discussion started by: Awanka
8 Replies
Login or Register to Ask a Question