Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare two files and set a third one using awk or perl

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two files and set a third one using awk or perl
# 1  
Old 10-22-2010
Compare two files and set a third one using awk or perl
Folks I need your help cuz I've a file with 100,000 records that need to be compared against a passwd file (300) and then create a third one with the data in the first one and the passwd from the second one set in it.

The format of the first file is:

Code:
host xxxxxx "" 0,0 Closed control00/ SOLARIS 0x0+0+0 "TCP"
host yyyyyy "" 0,0 Closed control01/ SOLARIS 0x0+0+0 "TCP"
...
host 222222 "" 0,0 Closed control30/ Linux 0x0+0+0 "TCP"

the second one has a key and the passwd as below:

Code:
control00 45EF0E4228F59D60521D071AC55E4BAC
...
control30 F93F0F0BC36E597B28D325BD10F99C4B

the third file should looks like:

Code:
host 222222 "" 0,0 Closed control30/F93F0F0BC36E597B28D325BD10F99C4B Linux 0x0+0+0 "TCP"

Now you migth think piece of cake Smilie for you yes, for me no, I know that I need to read one record in the second file and the read all the records in the first one with this passwd in it and then generate the third and so on, but to be honest I don't know how to open, read & control each of them in awk or perl.

I already reviewed the archive with no luck cuz, there're only examples about comparing files.

Can you help me with this?, I'm lost and desperate.

TIA for your help. Raul. Smilie
Last edited by vgersh99; 10-22-2010 at 11:09 AM.. Reason: code tags, please!
# 2  
Old 10-22-2010
Code:
nawk '
  FNR==NR { f1[$1]=$2;next }
  (s=substr($6,1,length($6)-1)) in f1 {$6=$6 f1[s];print}
' file2 file1 > file3

This User Gave Thanks to vgersh99 For This Post:
ranrodrig
# 3  
Old 10-22-2010
Code:
#!/bin/bash
for i in $(sed 's/\(.*\) .*/\1/' file2)
 do
    x=$(sed -n "/$i/s/.* \(.*\)/\1/p" file2)
    sed -n "/$i/s/\(.*\)\/ \(.*\)/\1\/$x \2/p" file1 >> file3
 done
 more file3
 
host xxxxxx "" 0,0 Closed control00/45EF0E4228F59D60521D071AC55E4BAC SOLARIS 0x0+0+0 "TCP"
host yyyyyy "" 0,0 Closed control01/45EF0E4228F59D60521D071AC55E4BAX SOLARIS 0x0+0+0 "TCP"
host 222222 "" 0,0 Closed control30/F93F0F0BC36E597B28D325BD10F99C4B Linux 0x0+0+0 "TCP"

Last edited by ygemici; 10-22-2010 at 11:57 AM..
This User Gave Thanks to ygemici For This Post:
ranrodrig
# 4  
Old 10-22-2010
Code:
$
$
$ cat file1
host xxxxxx "" 0,0 Closed control00/ SOLARIS 0x0+0+0 "TCP"
host yyyyyy "" 0,0 Closed control01/ SOLARIS 0x0+0+0 "TCP"
host 222222 "" 0,0 Closed control30/ Linux 0x0+0+0 "TCP"
$
$ cat file2
control00 45EF0E4228F59D60521D071AC55E4BAC
control01 98XY1A5643Q78T93728A670BD29C9PJM
control30 F93F0F0BC36E597B28D325BD10F99C4B
$
$ perl -lane 'if ($ARGV eq "file2") {$x{$F[0]}=$F[1]} else {/^(.*) (.*?)(\/)(.*)$/ && print "$1 $2$3$x{$2}$4"}' file2 file1
host xxxxxx "" 0,0 Closed control00/45EF0E4228F59D60521D071AC55E4BAC SOLARIS 0x0+0+0 "TCP"
host yyyyyy "" 0,0 Closed control01/98XY1A5643Q78T93728A670BD29C9PJM SOLARIS 0x0+0+0 "TCP"
host 222222 "" 0,0 Closed control30/F93F0F0BC36E597B28D325BD10F99C4B Linux 0x0+0+0 "TCP"
$
$
$

tyler_durden
This User Gave Thanks to durden_tyler For This Post:
ranrodrig
# 5  
Old 10-22-2010
Folks, thanks a lot for your help all the solutions works great, you saved my soul.

Now I would like to understand the awk code, can you explain to me vgersh99?, I see that you assing one field in an array and you set it later, but I don't understand the rest of the code.

I appreciate too much your help & patience.

Have a nice weekend.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 files, awk maybe?

I have 2 files, file1: alfa numbers numbers vita numbers numbers gama numbers numbers delta numbers numbers epsilon numbers numbers zita numbers numbers ... file2: 'zita' keepnumbers keepnumbers keepnumbers 'gama' keepnumbers keepnumbers keepnumbers 'misc' ... (11 Replies)
Discussion started by: phaethon
11 Replies

2. Shell Programming and Scripting

perl Compare zone files in directory with what is listed in named.conf

I would really appreciate any assistance that I can get here. I am fairly new to perl. I am trying to rewrite my shell scripts to perl. Currently I have a shell script (using sed, awk, grep, etc) that gets a list of all of the zone files in a directory and then looks in named.conf for what... (0 Replies)
Discussion started by: brianjb
0 Replies

3. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

4. Shell Programming and Scripting

Compare intervals (columns) from two files (awk, grep, Perl?)

Hi dear users, I need to compare numeric columns in two files. These files have the following structure. K.txt (4 columns) A001 chr21 9805831 9846011 A002 chr21 9806202 9846263 A003 chr21 9887188 9988593 A003 chr21 9887188 ... (2 Replies)
Discussion started by: jcvivar
2 Replies

5. Shell Programming and Scripting

Perl: compare columns of two files

Hi I have file 1 like this: file 2 is like this: The files are tab separated. I want to search for the first column values of file 1 in the first column of file 2 and merge the 3rd column value of file 2 to the corresponding line on first file. so the desired output is; I tried following... (2 Replies)
Discussion started by: polsum
2 Replies

6. Shell Programming and Scripting

Need Perl script to compare two CSV files

Need perl script to compare the two CSV files and and give out put in CSV format File MsPMTP.csv File ProfileNames.csv MsPMTP.csv is having lines like below JBL_VIJ_A_A962/r01sr4sl12/port#01-#13-Au4P-4c-TMi-PMNETR15 JBL_VIJ_A_A962/r01sr4sl12/port#01-#13-Au4P-4c-TMi-PMFETR15... (9 Replies)
Discussion started by: sreedhargouda
9 Replies

7. Shell Programming and Scripting

Perl script to compare two files

hi, As such I am new to perl on google search I found a code for Perl script to compare two files and print differences between them and instead of prinintg I want to store the diff. in a outputfile so can sombody provide assistance upon this from where can I edit in script to store the diff in... (1 Reply)
Discussion started by: dinesh.4126
1 Replies

8. UNIX for Dummies Questions & Answers

How to compare csv files using perl

I need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Can other ideas .please respond asap... (2 Replies)
Discussion started by: kittu1979
2 Replies

9. UNIX for Dummies Questions & Answers

Compare 2 csv files in perl

need to compare 2 csv files and report should containg number of matching lines,different lines ,missing lines in one file using perl. I dont want to use read line by line and scan thru the second file for matching line ,as this logic was so time consuming .Any ideas.i need the soultion badly .... (2 Replies)
Discussion started by: kittu1979
2 Replies

10. Shell Programming and Scripting

compare 2 coloum of 2 diff files using perl script

Hi, i am new to perl scripting.. i am still learing it.. i am asked to write a perl script which should compare 2 coloums of 2 different files. if those 2 coloumn are same the script should store the both the lines in 2 diff files. these are files, file 1: 21767016 226112 char 19136520... (3 Replies)
Discussion started by: vasuki
3 Replies
Login or Register to Ask a Question