Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk command to compare a file with set of files in a directory using 'awk'
# 1  
Old 05-30-2012
awk command to compare a file with set of files in a directory using 'awk'
Hi,

I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files.

To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the directory will have many fields seperated by '~'.

I tried 'awk' command as below

Code:
awk -F"~" 'FILENAME=="file1.txt"{A[$1]=$1} FILENAME=="SBL_LOYALTY_SALE_TXNS*.txt"{if(A[$1]){print}}' file1.txt SBL_LOYALTY_SALE_TXNS*.txt > common.txt

This did not generate any output.

Please provide your valuable suggestions.If any modifications needed to be done in 'awk' or any other unix command will help.
Last edited by Franklin52; 05-30-2012 at 06:54 AM.. Reason: Please use code tags for data and code samples
# 2  
Old 05-30-2012
Hi

Hoping you want to print the matching first lines, and not the filename itself:

Code:
$ cat f1.txt
guru~abc~efg
ammi
abc

Code:
$ cat f2.txt
abc~guru~ammi
bcd
efg

Code:
$ cat f3.txt
guru~abc~ammi
bcd
efg


Code:
$ awk -F '~' 'NR==1 && NR==FNR{x=$1;next}FNR==1{if ($1==x)print;}' f1.txt f2.txt f3.txt
guru~abc~ammi

# 3  
Old 05-30-2012
Hi guruprasad,

Many thanks for the reply.

The requirement is to compare the first column, not first line of all the records in the first file(Txn_queued.txt) with the first column of the files in the
Code:
directory(SBL_LOYALTY_SALE_TXNS_20120405.txt,SBL_LOYALTY_SALE_TXNS_20120420.txt  ).

Output should be the matching records from the files in the directory.

I tried using the above mentioned command as
Code:
awk -F '~' 'NR==1 && NR==FNR{x=$1;next}FNR==1{if ($1==x)print;}
  ' Txn_queued.txt SBL_LOYALTY_SALE_TXNS_20120405.txt SBL_LOYALTY_SALE_TXNS_20120420.txt > new.txt

This did not give any output.
Last edited by Scott; 05-30-2012 at 07:39 AM.. Reason: Code tags
# 4  
Old 05-30-2012
Hi
Post your sample input files and output file.
# 5  
Old 05-30-2012
sample records in file1.txt

Code:
271:13-APR-12:1:6189:1
1183:13-APR-12:1:6689:1
1183:13-APR-12:2:8993:10
1183:13-APR-12:2:8993:11
1183:13-APR-12:2:8993:12

sample records from files in directory

Code:
1183:13-APR-12:1:6689:1~1148141~380392198~1183~04-13-12~0~1~1~12.49~0~0.00~S~~0.00~~1~12.49~~N~
1183:13-APR-12:2:8993:10~863432~380391909~1183~04-13-12~0~1~1~0.95~0~0.00~S~~0.00~~12~92.46~~N~
2769:14-APR-12:2:5385:1~669725~399469944~2769~04-14-12~0~1~1~21.99~0~0.00~S~~0.00~~12~128.88~~N~
2769:14-APR-12:2:5385:2~1352601~399469944~2769~04-14-12~0~1~1~10.99~0~0.00~S~~0.00~~12~128.88~~N~
2769:14-APR-12:2:5385:3~1035266~399469944~2769~04-14-12~0~1~1~9.99~0~0.00~S~~0.00~~12~128.88~~N~

Output file should contain values like

Code:
1183:13-APR-12:1:6689:1~1148141~380392198~1183~04-13-12~0~1~1~12.49~0~0.00~S~~0.00~~1~12.49~~N~
1183:13-APR-12:2:8993:10~863432~380391909~1183~04-13-12~0~1~1~0.95~0~0.00~S~~0.00~~12~92.46~~N~

Last edited by Scott; 05-30-2012 at 07:40 AM.. Reason: Please use code tags
# 6  
Old 05-30-2012
Hi


Code:
$ cat f1.txt
271:13-APR-12:1:6189:1
1183:13-APR-12:1:6689:1
1183:13-APR-12:2:8993:10
1183:13-APR-12:2:8993:11
1183:13-APR-12:2:8993:12

Code:
$ cat f2.txt
1183:13-APR-12:1:6689:1~1148141~380392198~1183~04-13-12~0~1~1~12.49~0~0.00~S~~0.00~~1~12.49~~N~
1183:13-APR-12:2:8993:10~863432~380391909~1183~04-13-12~0~1~1~0.95~0~0.00~S~~0.00~~12~92.46~~N~
2769:14-APR-12:2:5385:1~669725~399469944~2769~04-14-12~0~1~1~21.99~0~0.00~S~~0.00~~12~128.88~~N~
2769:14-APR-12:2:5385:2~1352601~399469944~2769~04-14-12~0~1~1~10.99~0~0.00~S~~0.00~~12~128.88~~N~
2769:14-APR-12:2:5385:3~1035266~399469944~2769~04-14-12~0~1~1~9.99~0~0.00~S~~0.00~~12~128.88~~N~


Code:
$ grep -f f1.txt f2.txt
1183:13-APR-12:1:6689:1~1148141~380392198~1183~04-13-12~0~1~1~12.49~0~0.00~S~~0.00~~1~12.49~~N~
1183:13-APR-12:2:8993:10~863432~380391909~1183~04-13-12~0~1~1~0.95~0~0.00~S~~0.00~~12~92.46~~N~

# 7  
Old 05-30-2012
Hi,

this is fine if we are comparing only two files.

But the requirement is to compare the first file with all the files in a directory.All those files in the directory will have records similar to the one in the second file.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Simple awk command to compare two files and print first difference

Hello, I have two text files, each with a single column, file 1: 124152970 123899868 123476854 54258288 123117283 file 2: 124152970 123899868 54258288 123117283 122108330 (5 Replies)
Discussion started by: LMHmedchem
5 Replies

2. UNIX for Dummies Questions & Answers

awk command to compare files by column

So I have this issue. I have 4 files. the first one is the master file who has all possible combinations: file 1 - a - b - c - d - e the other three have some of the letters and a number instead of - for example file 2 34 a 5 c file 3 10 b 12 ... (3 Replies)
Discussion started by: Quijotes
3 Replies

3. Shell Programming and Scripting

Compare two files using awk command recursively

I want to compare two files, 1) Compare Each query result. 2) Compare Only first row of the Query output 3) Compare Time (3rd column), First file time is lesser than 2nd file then print the PO_NUM else do nothing. File1: C:\script>call transaction 1OPOP C:\script>Select ID, PO_ID, TIME, DES... (3 Replies)
Discussion started by: Ragu14
3 Replies

4. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

5. Shell Programming and Scripting

awk - compare records of 1 file with 3 files

hi.. I want to compare records present in 1 file with those in 3 other files and print those records of file 1 which are not present in any of the files. for eg - file1 file2 file3 file4 1 1 5 7 2 2 6 9 3 4 5 6 7 8 9 ... (3 Replies)
Discussion started by: Abhiraj Singh
3 Replies

6. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

7. Shell Programming and Scripting

Compare two files and set a third one using awk or perl

Folks I need your help cuz I've a file with 100,000 records that need to be compared against a passwd file (300) and then create a third one with the data in the first one and the passwd from the second one set in it. The format of the first file is: host xxxxxx "" 0,0 Closed control00/... (4 Replies)
Discussion started by: ranrodrig
4 Replies

8. Shell Programming and Scripting

awk to compare flat files and print output to another file

Hello, I am strugling from quite a some time to compare flat files with over 1 million records could anyone please help me. I want to compare two pipe delimited flat files, file1 with file2 and output the unmatched rows from file2 in file3 Sample File1: ... (9 Replies)
Discussion started by: suhaeb
9 Replies

9. Shell Programming and Scripting

(awk) compare files in dir with an index file

Using awk I have an index file which has been seperated into 5 fields. The first field contains file names. What I need to do is check to see if a file exists in my current directory that is not in the first field of my index file. If its not i print out a message. Please help! (4 Replies)
Discussion started by: xthexonex
4 Replies

10. Shell Programming and Scripting

awk command to find the count of files ina directory

hi Gurus, can anyone provide a awk command to get teh count of number of file sin a specific directory. appreciate any kind of information.. thanks (11 Replies)
Discussion started by: sish78
11 Replies
Login or Register to Ask a Question