Sponsored Content
Top Forums Shell Programming and Scripting awk - compare records of 1 file with 3 files Post 302888984 by Abhiraj Singh on Tuesday 18th of February 2014 08:36:14 AM
Old 02-18-2014
awk - compare records of 1 file with 3 files

hi.. I want to compare records present in 1 file with those in 3 other files and print those records of file 1 which are not present in any of the files. for eg -

Code:
file1     file2      file3      file4
1         1           5          7
2         2           6          9
3
4
5
6
7
8
9

output should contain :
Code:
3
4
8

Code:
nawk 'NR==FNR{a[$0]++;next} !a[$0] or !a[$0] or !a[$0]' file1 file2 file3 file4

i am using this code but i dont think it is correct.. problem arised when the file which i wanted to compare was huge and nawk ran out of space giving out of space tostring error.. so now i have splitted the file into 3 parts and then comparing.. also can we have different names for arrays used in this script to avoid the similar situation. I am using solaris system..

Moderator's Comments:
Mod Comment code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk Compare Files w/Multiline Records

I'm trying to compare the first column values in two different files that use a numerical value as the key and output the more meaningful value found in the second column of file1 in front of the matching line(s) in file2. My problem is that file2 has multiple records. For example given: FILE1... (4 Replies)
Discussion started by: RacerX
4 Replies

2. Shell Programming and Scripting

(awk) compare files in dir with an index file

Using awk I have an index file which has been seperated into 5 fields. The first field contains file names. What I need to do is check to see if a file exists in my current directory that is not in the first field of my index file. If its not i print out a message. Please help! (4 Replies)
Discussion started by: xthexonex
4 Replies

3. Shell Programming and Scripting

Compare Records between to files and extract it

I am not an expert in awk, SED, etc... but I really hope there is a way to do this, because I don't want to have to right a program. I am using C shell. FILE 1 FILE 2 H0000000 H0000000 MA1 MA1 CA1DDDDDD CA1AAAAAA MA2 ... (2 Replies)
Discussion started by: jclanc8
2 Replies

4. Shell Programming and Scripting

How to compare data from 2 zip files and capture the new records from file2 to a new file

I have 2 zip files which have about 20 million records in each file. file 2 will have additional records than file 1. I want to compare the records in both the files and capture the new records from file 2 into another file file3. Please help me with a command/script which provides me the desired... (8 Replies)
Discussion started by: koneru
8 Replies

5. Shell Programming and Scripting

Compare 2 files having different number of columns and records

Hi , My requirement is to Compare 2 files having different number of columns and records and get the ouptut containing all the non-matching records from File A(with all column values ) .Example data below : File A contains following : Aishvarya |1234... (4 Replies)
Discussion started by: aishvarya.singh
4 Replies

6. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

7. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

8. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

9. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

10. Shell Programming and Scripting

Compare files to pull changed records only

Hi, I am using Sun Solaris - SunOS. I have two fixed width files shown below. I am trying to find the changes in the records in the Newfile.txt for the records where the key column matches. The first column is a key column (example: A123). If there are any new or deletion of records in the... (4 Replies)
Discussion started by: Saanvi1
4 Replies
VCF-COMPARE(1)							   User Commands						    VCF-COMPARE(1)

NAME
vcf-compare - compare bgzipped and tabix indexed VCF files SYNOPSIS
compare-vcf [OPTIONS] file1.vcf file2.vcf ... DESCRIPTION
About: Compare bgzipped and tabix indexed VCF files. (E.g. bgzip file.vcf; tabix -p vcf file.vcf.gz) OPTIONS
-c, --chromosomes <list|file> Same as -r, left for backward compatibility. Please do not use as it will be dropped in the future. -d, --debug Debugging information. Giving the option multiple times increases verbosity -H, --cmp-haplotypes Compare haplotypes, not only positions -m, --name-mapping <list|file> Use with -H when comparing files with differing column names. The argument to this options is a comma-separated list or one mapping per line in a file. The names are colon separated and must appear in the same order as the files on the command line. -R, --refseq <file> Compare the actual sequence, not just positions. Use with -w to compare indels. -r, --regions <list|file> Process the given regions (comma-separated list or one region per line in a file). -s, --samples <list> Process only the listed samples. Excluding unwanted samples may increase performance considerably. -w, --win <int> In repetitive sequences, the same indel can be called at different positions. Consider records this far apart as matching (be it a SNP or an indel). -h, -?, --help This help message. vcf-compare 0.1.5 July 2011 VCF-COMPARE(1)
All times are GMT -4. The time now is 10:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy