Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

[SOLVED] Handling multiple files using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [SOLVED] Handling multiple files using awk
# 1  
Old 08-27-2011
[SOLVED] Handling multiple files using awk
Hi,
I am trying to process 2 files simultaneously using awk satisfying following condition,

Both files contain 3 columns. It should take entry from column 1 from first file, look for that entry in file 2 and if found, add column 2 and column 3 from both files and output to third file. For e.g.

File1
Code:
1314354249		4		0
1314354250		0		3
1314354251		3		0
1314354252		1		6
1314354253		1		0
1314354254		3		7
1314354255		0		0
1314354256		3		2
1314354257		1		0
1314354258		2		0

File2
Code:
1314354249		2		0
1314354250		5		3
1314354252		7		6
1314354253		9		0
1314354256		3		2
1314354257		0		0
1314354258		0		0

File3
Code:
1314354249		6		0
1314354250		5		6
1314354251		3		0
1314354252		8		12
1314354253		10	0
1314354254		3		7
1314354255		0		0
1314354256		6		4
1314354257		1		0
1314354258		2		0

thanks alot for any help
Last edited by radoulov; 08-28-2011 at 03:52 AM.. Reason: Code tags.
# 2  
Old 08-27-2011
What have you tried so far? Please post the code.
# 3  
Old 08-27-2011
i tried following but its not working,

Code:
awk '{ arr[$1]=arr[$1]+$2} {  for (i in arr) print i,arr[i] } ' file1 file2

i was not sure how to deal with 3 columns, so i only tried working with 2 i.e. $1 and $2. I was trying to create an array with with $1 as index and $2 as value. But its not working Smilie

---------- Post updated at 02:39 AM ---------- Previous update was at 02:05 AM ----------

ok, i've got it to work with 2 columns. It matches column 1 from both files and add corresponding entry from column 2,

Code:
nawk ' BEGIN {
while ( getline <  "cluster2_data" > 0)
{
n[$1]=$2
}
}
{
n[$1]=n[$1]+$2
}
{
print $1,n[$1]
}' cluster1_data

but i am struggling with getting it to work with both columns 2 and 3. Please somebody help.

---------- Post updated at 02:50 AM ---------- Previous update was at 02:39 AM ----------

please somebody help...

---------- Post updated at 02:58 AM ---------- Previous update was at 02:50 AM ----------

done!!!!! Smilie

Code:
nawk ' BEGIN {
while ( getline <  "file1" > 0)
{
m[$1]=$2
n[$1]=$3
}
}
{
m[$1]=m[$1]+$2
n[$1]=n[$1]+$3
}
{
print $1,m[$1],n[$1]
}' file2

Last edited by radoulov; 08-28-2011 at 03:52 AM.. Reason: Code tags.
# 4  
Old 08-28-2011
Code:
awk ' BEGIN {OFS="\t\t";while(getline < "file2"){m[$1]=$2;n[$1]=$3}}{m[$1]+=$2;n[$1]+=$3}{print $1,m[$1],n[$1]}' file1

or using sort at the end
Code:
awk 'BEGIN{OFS="\t\t"}END{for(i in a) print i,b[i],c[i]}{a[$1];b[$1]+=$2;c[$1]+=$3}'  file1 file2 | sort

This User Gave Thanks to danmero For This Post:
muazfarooqaslam
# 5  
Old 08-28-2011
Sorry, despite the thread has benn marked as solved I'd like to rise a quick question. I'm a few days new using awk, 0 days with nawk. I noted there´s not an exact match between all the values in the first column of file 1 and the values of the first column at file 2. The first case of such condition is this:

File 1
Code:
1314354249		4		0 1314354250		0		3 1314354251		3		0

File 2
Code:
1314354249		2		0 1314354250		5		3 1314354252		7		6

As it's evident, File 2 lacks the row "1314354251", this is actually the question, seeing the solutions you posted:
Code:
awk ' BEGIN {OFS="\t\t";while(getline < "file2"){m[$1]=$2;n[$1]=$3}}{m[$1]+=$2;n[$1]+=$3}{print $1,m[$1],n[$1]}' file1



.. it semms to me that for every line in File 1 , you sum the values in $2 and $3 of File 2 into $2 and $3 in File 1, assuming sorted values in the first columns of the two files and that every value in column 1 of the File 1 exists in the column 1 of File 2, which semms not to be the case, ....

I'd appreciate a lot if you can briefly clarify that point to me,
Regards,

JRodrigoF
Last edited by Franklin52; 08-29-2011 at 03:48 AM.. Reason: Please use code tags for code and data samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Solved] How to remove multiple files?

Hi Gurus, I have below files in one directory. the file name has date and time portion which is exactly the file be created. I need keep only lasted created file which is abc_20140101_1550 and remove rest of the file. abc_20140101_1300 abc_20140101_1200 abc_20140101_1400 abc_20140101_1500... (2 Replies)
Discussion started by: ken6503
2 Replies

2. UNIX for Dummies Questions & Answers

[Solved] List Files With Multiple Name Patterns

Hi, We have created a script that would accept the an indicator as a parameter and archive files present in a directory. The indicator would drive what the name pattern of the files to be archived should be. If the indicator is 1, then the pattern to look out for is FACT*. If the indicator is... (2 Replies)
Discussion started by: jerome_rajan
2 Replies

3. Shell Programming and Scripting

[Solved] Multiple input files and output files

Hi, I have many test*.ft1 files to which I want to read as input for a script called pipe2txt.tcl and print the output in each separate file. For example, pipe2txt.tcl < test001.ft1 > test001.txt How can I read many files in this maner? thank you very much, Best, Pahuja (5 Replies)
Discussion started by: Pahuja
5 Replies

4. Shell Programming and Scripting

ISSUE in handling multiple same name files :-(

Dear all, My work is completely stuck cos of the following issue. Please find it here and kindly help me. Task is following: I have set of files with such pattern 1t-rw-rw-r-- 1 emily emily 119 Jun 11 10:45 vgtree_5_1_pfs.root 3t-rw-rw-r-- 1 emily emily 145 Jun 11 10:46 vgtree_5_3_pfs.root... (4 Replies)
Discussion started by: emily
4 Replies

5. Shell Programming and Scripting

[Solved] HP-UX awk sub multiple patterns

Hi, I am using sub to remove blank spaces and one pattern(=>) from the input string. It works fine when I am using two sub functions for the same. However it is giving error while I am trying to remove both spaces and pattern using one single sub function. Working: $ echo " OK => " |awk... (2 Replies)
Discussion started by: sai_2507
2 Replies

6. UNIX for Dummies Questions & Answers

[SOLVED] Rename multiple files

Hi. I have a large number of files with names like: t_ 0.20000E-02.dat There is actually a space after the underscore. These files are numbered numerically, i.e. t_ 0.20000E-02.dat, t_ 0.21000E-02.dat, t_ 0.22000E-02.dat and so on. What I would like to do is rename such that the file with... (8 Replies)
Discussion started by: lost.identity
8 Replies

7. Shell Programming and Scripting

[SOLVED] moving multiple files? mv

HI I have a list of files that are incorrectely names and I need to move them to new name .. I tried few things that has not worked so far can you help ? I need to rename all thes eifle ( tere are over 100 ) xldn0357bap.orig.new xldn0389bap.orig.new xldn0439bap.orig.new... (12 Replies)
Discussion started by: mnassiri
12 Replies

8. Shell Programming and Scripting

Handling 2 files simultaneously with awk

Hello, Is it possible to handle data from two different files at once in awk (latest version and platform is Fedora). I found on the net that we cannot nest awk. My requirement is that I have two similar files : File 1: Name: abc Val = 58 Name: cdf Val = 1; .................. File... (7 Replies)
Discussion started by: fifteate
7 Replies

9. Shell Programming and Scripting

handling multiple files using awk command and wants to get separate out file for each

hai all I am new to the world of shell scripting I wanted to extract two columns from multiple files say around 25 files and i wanted to get the separate outfile for each input file tired using the following command to extract two columns from 25 files awk... (2 Replies)
Discussion started by: hema dhevi
2 Replies

10. Shell Programming and Scripting

Handling multiple fields of a database file for toupper() function in awk

hello everyone.... script is: To convert the contents of a database file into uppercase my code is: printf "%s\n" , $2 | awk '{print toupper($2)}' emp.lst i m able to do only for one field.....didn't get any sources for handling multiple fields. please suggest me for multiple... (1 Reply)
Discussion started by: Priyanka Bhati
1 Replies
Login or Register to Ask a Question