File comparision line by line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File comparision line by line
# 15  
Old 07-03-2013
Thanks Don for your help...

I am using "grep -vF -f *.txt" command to compare,

but here is some issue, comparison is not working correctly...i want to compare file1 with file2, only column 2/3 in file2 should compare, if there is any difference should get output in file3.

---------- Post updated at 01:43 PM ---------- Previous update was at 01:37 PM ----------

see example i am expecting...
Code:
ACCOUNT|497|1108000-new|Jun|2045.00
ACCOUNT|497|Mmnfy-new|Jun|1903.00
ACCOUNT|497|922857-new|Jun|2045.00

Aggr_New_06_19_2013_1944.txt has below records
Code:
ACCOUNT|497|1108000-new|Jun|2045.00
ACCOUNT|497|Mmnfy-new|Jun|1903.00
ACCOUNT|497|922857-new|Jul|2045.00
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

Right now i am getting output:
Code:
ACCOUNT|497|922857-new|Jul|2045.00
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

But Actual Output expecting should be like below...
Code:
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

Script should compare only 2/3 columns in file2...please help on it.

Last edited by Scott; 07-03-2013 at 05:24 PM.. Reason: Code tags
# 16  
Old 07-03-2013
I give up. After two weeks of wasting our time, you now completely change the requirements used to determine the output you want. And you still haven't answered the basic question of how we are supposed to know which of the two files in a directory is the "1st" file and which is the "2nd" file.

If you want help on this new problem, start a new thread. In that thread:
  1. Give a complete description of what your input files look like (expected sizes, field separators, sample contents, filename format).
  2. Give a complete description of the processing that needs to be performed to produce the results you want.
  3. Show sample output that should be produced when your input files are processed as you described.
# 17  
Old 07-03-2013
1.Give a complete description of what your input files look like (expected sizes, field separators, sample contents, filename format).

One of our unix script will get 2 files from windows server and place in unix folder1(Everyday it will clean the folder and place only 2 files). these 2 files names change daily as file names contains date timestamp(File names like 'Jun_Agg_06_19_2013_1944.txt'),file size depends on data we get daily, it varies from 5MB -60MB filesize. so now we need to compare these two files and get output in 3rd file.

Input files contains only 5 columns with '|' delimeter, column1, 4,5 can be same, but we used to get difference in input files in column2 and 3, based on column 2 and column 3, file1 should compare with file2 and get result in file3.

2.Give a complete description of the processing that needs to be performed to produce the results you want.

Explained complete process above

3.Show sample output that should be produced when your input files are processed as you described.

see example below..

File1:

Code:
ACCOUNT|497|1108000-new|Jun|2045.00
ACCOUNT|497|Mmnfy-new|Jun|1903.00
ACCOUNT|497|922857-new|Jun|2045.00

File2:

Code:
ACCOUNT|497|1108000-new|Jun|2045.00
ACCOUNT|497|Mmnfy-new|Jun|1903.00
ACCOUNT|497|922857-new|Jul|2045.00
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

Right now i am getting output:

Code:
ACCOUNT|497|922857-new|Jul|2045.00
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00


But Actual Output should be like below...

Code:
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

Files should compare based on 2nd and 3rd columns and get output in to file3.
Moderator's Comments:
Mod Comment Please use CODE tags when showing sample input and output files as well as when showing code samples.

Last edited by Don Cragun; 07-03-2013 at 06:11 PM.. Reason: CODE tags
# 18  
Old 07-03-2013
Code:
$ awk -F"|" 'FNR==NR{a[$2$3]++;next}!a[$2$3]' file1.txt file2.txt

Code:
ACCOUNT|497|922865-new|Jun|4509.00
ACCOUNT|497|922987-new|Jun|3249.00
ACCOUNT|497|1|Jun|867.00

This User Gave Thanks to mjf For This Post:
# 19  
Old 07-04-2013
Thanks mjf, its working perfectly for comparison, can do manually.

But my requirement is to automate that file comparison, how can we automate this by using above script as i can't hard coded the file names as daily file names will change with time stamp?? please help.
# 20  
Old 07-04-2013
Could you please let us know the example file names. provide the names of both files and the part of the filename which would change daily.
# 21  
Old 07-04-2013
Code:
Example today will get files like below...

Aggr_New_06_19_2013_1944.txt
Aggr_New_06_20_2013_1956.txt

next day again it will come like...

Aggr_New_06_20_2013_1956.txt
Aggr_New_06_21_2013_1952.txt

it will keep on change daily with date time stamp.

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies

2. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

3. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

4. Shell Programming and Scripting

perl: comparision of field line by line in two files

Hi everybody, First I apologize if my question seems demasiad you silly, but it really took 4 days struggling with this, I looked at books, forums ... And Also ask help to a friend that is software developer and he told me that it is a bad idea do it by perl... but this is my problem. I moved to... (8 Replies)
Discussion started by: Thelost
8 Replies

5. Shell Programming and Scripting

Read file line by line and process the line to generate another file

Hi, i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here) i need to read this file line by line and generate an output file like the one below i.e based on N value the number of MSISDNs will vary, if N=1 then the following... (14 Replies)
Discussion started by: aemunathan
14 Replies

6. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

7. Shell Programming and Scripting

Line by Line Comparision of 2 files and print only the difference

Hi, I am trying to find an alternative way to do tkdiff. In tkdiff the gui compares each line and highlights only the differences. for eg: John works at McDonalds s(test) He was playing guitar tywejk John works in McDonalds 9908 He was playing guitar I am... (1 Reply)
Discussion started by: naveen@
1 Replies

8. UNIX for Advanced & Expert Users

how do you parse 1 line at a time of file1 ie. line(n) each line into new file

File 1 <html>ta da....unique file name I want to give file=>343...</html> <html>da ta 234 </html> <html>pa da 542 </html> and so on... File 2 343 234 542 and so on, each line in File 1 one also corresponds with each line in File 2 I have tried several grep, sed, while .. read, do,... (4 Replies)
Discussion started by: web_developer
4 Replies

9. Shell Programming and Scripting

file comparision by line

i have two files and i want to compare these two it shoud print those lines which are not in 2nd file a.txt 1236,AB,0 2345,CD,1 5679,EF,1 9123,AA,1 9223,AA,0 b.txt 1234,AB,0 2345,CD,1 5678,EF,1 9123,AA,0 o/p 1236,AB,0 5679,EF,1 9123,AA,1 9223,AA,0 (6 Replies)
Discussion started by: aaysa123
6 Replies
Login or Register to Ask a Question