awk - compare records of 1 file with 3 files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - compare records of 1 file with 3 files
# 1  
Old 02-18-2014
awk - compare records of 1 file with 3 files

hi.. I want to compare records present in 1 file with those in 3 other files and print those records of file 1 which are not present in any of the files. for eg -

Code:
file1     file2      file3      file4
1         1           5          7
2         2           6          9
3
4
5
6
7
8
9

output should contain :
Code:
3
4
8

Code:
nawk 'NR==FNR{a[$0]++;next} !a[$0] or !a[$0] or !a[$0]' file1 file2 file3 file4

i am using this code but i dont think it is correct.. problem arised when the file which i wanted to compare was huge and nawk ran out of space giving out of space tostring error.. so now i have splitted the file into 3 parts and then comparing.. also can we have different names for arrays used in this script to avoid the similar situation. I am using solaris system..

Moderator's Comments:
Mod Comment code tags
# 2  
Old 02-18-2014
I noticed that the logic you used is wrong. You should compare the other way around:
Code:
awk 'FILENAME!="file1"{A[$1];next}!($1 in A)' file2 file3 file4 file1

# 3  
Old 02-18-2014
Thanks yoda.. Jist a query.. Will array size keep on growing for all files or wil it start from beginning when second file is searched??
# 4  
Old 02-18-2014
!a[$0] or !a[$0] or !a[$0] --> if you use these result will be unexpected, and uses more memory as well, Hope you are reading answers yesterday I answered you on the same here

Nawk Problem - nawk out of space in tostring on Akshay Hegde - Shell Programming and Scripting - Unix Linux Forums

Well in Yoda's solution Array will be created only when it reads first file, for remaining files it just checks for array index whether index is available in array or not.
This User Gave Thanks to Akshay Hegde For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare files to pull changed records only

Hi, I am using Sun Solaris - SunOS. I have two fixed width files shown below. I am trying to find the changes in the records in the Newfile.txt for the records where the key column matches. The first column is a key column (example: A123). If there are any new or deletion of records in the... (4 Replies)
Discussion started by: Saanvi1
4 Replies

2. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

3. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

4. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

5. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

6. Shell Programming and Scripting

Compare 2 files having different number of columns and records

Hi , My requirement is to Compare 2 files having different number of columns and records and get the ouptut containing all the non-matching records from File A(with all column values ) .Example data below : File A contains following : Aishvarya |1234... (4 Replies)
Discussion started by: aishvarya.singh
4 Replies

7. Shell Programming and Scripting

How to compare data from 2 zip files and capture the new records from file2 to a new file

I have 2 zip files which have about 20 million records in each file. file 2 will have additional records than file 1. I want to compare the records in both the files and capture the new records from file 2 into another file file3. Please help me with a command/script which provides me the desired... (8 Replies)
Discussion started by: koneru
8 Replies

8. Shell Programming and Scripting

Compare Records between to files and extract it

I am not an expert in awk, SED, etc... but I really hope there is a way to do this, because I don't want to have to right a program. I am using C shell. FILE 1 FILE 2 H0000000 H0000000 MA1 MA1 CA1DDDDDD CA1AAAAAA MA2 ... (2 Replies)
Discussion started by: jclanc8
2 Replies

9. Shell Programming and Scripting

(awk) compare files in dir with an index file

Using awk I have an index file which has been seperated into 5 fields. The first field contains file names. What I need to do is check to see if a file exists in my current directory that is not in the first field of my index file. If its not i print out a message. Please help! (4 Replies)
Discussion started by: xthexonex
4 Replies

10. Shell Programming and Scripting

Awk Compare Files w/Multiline Records

I'm trying to compare the first column values in two different files that use a numerical value as the key and output the more meaningful value found in the second column of file1 in front of the matching line(s) in file2. My problem is that file2 has multiple records. For example given: FILE1... (4 Replies)
Discussion started by: RacerX
4 Replies
Login or Register to Ask a Question