Compare content between two files


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare content between two files
# 8  
If I run both scripts on your sample:

Code:
$ cat file1
Milestone1
Milestone2
Milestone3
Milestone4
Milestone5
Milestone6
$ cat file2
Milestone1
Milestone12
Milestone13
Milestone14
Milestone5
Milestone16
$ grep -xFf file1 file2
Milestone1
Milestone5
$ awk 'NR==FNR{A[$1]; next}$1 in A' file1 file2
Milestone1
Milestone5

Perhaps you could cut and paste this back into test input files and see if it works then and to spot the difference with your actual sample?
This User Gave Thanks to Scrutinizer For This Post:
# 9  
You can also try like this

1st sort both the files

Code:
sort filename1 -o filename1

Code:
sort filename2 -o filename2

then try

Code:
comm -12 filename1 filename2


Last edited by Vikram_Tanwar12; 12-14-2012 at 05:46 PM..
This User Gave Thanks to Vikram_Tanwar12 For This Post:
# 10  
Hi Folks thanks for your prompt reply.

Now I am getting the same contain repeated in different files, but there is another problem raised with it.

If I run the command specified by Scrutinizer and Vikram. It is giving me output of every repeated files, but it is not a uniq name.

I believe I need to more precise to explain the scenario, I am changing a little in my file structure example.

Let me explain. If I do
Code:
$cat File1
Milestone1
Milestone1
Milestone2
Milestone3
Milestone4
Milestone5
Milestone6

Code:
$cat File2
Milestone1
Milestone12
Milestone13
Milestone14
Milestone5
Milestone1
Milestone16

Now if I run the code specified by Scrutinizer and Vikram it is giving me output as
Code:
File2:Milestone1
File2:Milestone1
File2:Milestone5

As I told that I am working with huge number of data. The repeat lines will create trouble for me.

What I require is the out put will be
Code:
File2:Milestone1 , repeated 2 times
File2:Milestone5 , repeated 1 times

is it possible to get the results like that?

Last edited by Scrutinizer; 12-15-2012 at 05:41 AM..
# 11  
To get unique results you could just run everything throught sort -u:
Code:
grep -xFf file1 file2 | sort -u

or
Code:
awk 'NR==FNR{A[$1]; next}$1 in A' file1 file2 | sort -u



--
Alternatively, one could make sure results only get printed once, for example:
Code:
awk 'NR==FNR{A[$1]++; next}$1 in A{delete A[$1]; print $1}' file1 file2


===

To get a count, you could use the -H opion you can run everything through | sort | uniq -c
Code:
grep -HxFf file1 file2 | sort | uniq -c

or to also specify the file name:
Code:
echo | grep -xFf infile1904 infile1904b - | sort | uniq -c



--
To get exactly the result you specified, it would need to be something like this:
Code:
awk '
  NR==FNR{
    A[$1]++;
    next
  }
  $1 in A{
    T[$1]++
  } 
  END{
    for(i in T)printf "%s: %s , repeated %d times\n", FILENAME,i,T[i]
  }
' file1 file2


Last edited by Scrutinizer; 12-15-2012 at 05:58 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 12  
Maybe you have created file1 and file2 in Windos?
Then you need to convert them to Unix style first:

Code:
dos2unix file1 file1
dos2unix file2 file2

and now retry the scripts. They should work.

If you want both the matches and the number of matches, go with awk:

Code:
awk 'NR==FNR {A[$1]; next} $1 in A {count++; print} END {print count,"matching records"}' file1 file2 > file3

Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #458
Difficulty: Medium
NTP uses a flat, peer-to-peer system of time sources.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with compare 2 column content and corrected/replaced word

Input File CGGCGCCTCGCNNNCGAGCG CGGCGCGCCGAATCCGTGCG TCGCNGC GCGCCGC ACGGCNNNNN ACGGCCTCGCG CGGCNGCCCGCCC CGGCGCGCCGTCC Desired Output File CGGCGCCTCGCNNNCGAGCG CGGCGCGCCGAATCCGTGCG CGGCGCCTCGCATCCGAGCG TCGCNGC GCGCCGC TCGCCGC ACGGCNNNNN ACGGCCTCGCG ACGGCTCGCG... (6 Replies)
Discussion started by: perl_beginner
6 Replies

2. Shell Programming and Scripting

Compare two folders and file content

Hi, I am having two folders where i need to compare the content of files and also to know if any new files been added and redirect the difference output in respective filename logs. For e.g.: Directory D1: f1 f2 f3 Directory D2: f1 f2 f3 f4 i Need to compare the directories... (25 Replies)
Discussion started by: rohit_shinez
25 Replies

3. Shell Programming and Scripting

Ksh: how compare content of a file with an other array

Hi, I created a skript in ksh which generate a file with semicolon as separator, this is an example of the file a created: example content file: hello;AAAA;2014-08-17 hello;BBBB;2014-08-17 hello;CCCC;2014-08-17 I would need to compare the content in of the second column of this file... (3 Replies)
Discussion started by: jmartin
3 Replies

4. Shell Programming and Scripting

To compare the content of two text files

I have two files, sec.txt(1st File) 3172 disp.txt(2nd file) the file name is *********** 45676 Now i want to compare the value in sec.txt file with disp.txt file Excatly i want to compare the value 3172 in first file and 45676 in second file. i want to compare the first line of... (11 Replies)
Discussion started by: rammm
11 Replies

5. Shell Programming and Scripting

compare 2 files and show count same content.

$ cat File1 Non HTTP response code:java.net.ConnectException225073X 000000005143329Load time: 402335410224 Non HTTP response code: ava.net.ConnectException206423X 000000005143330Load time: 402305687161 Non HTTP response code: ava.net.ConnectException290212X 000000005143331Load time:... (1 Reply)
Discussion started by: ooilinlove
1 Replies

6. Shell Programming and Scripting

Compare a content of variable to a database column

Hi have an array like this colarray="a" colarray="b" colarray="c" colarray="d" colarray="e" colarray="f" the arrayvariable is in unix sh file i want to check the content of the array to oracle database table. that is whether "a" is present in the table. (4 Replies)
Discussion started by: barani75
4 Replies

7. Shell Programming and Scripting

Help with Compare and search content in directory

Hello, How to search in directory by comparing some string with the content of directory. Ex: I want to compare abhi string with the content of backup directory. i.e want to check that is there any file in backup directory having name ... (3 Replies)
Discussion started by: AbhijitIT
3 Replies

8. Shell Programming and Scripting

compare logfile content

Hi folks I have some logfiles like this: ./2009_08_22_14_08_entire_backup_no_1.log . . . ./2009_08_22_14_34_entire_backup_no_14.log each one contains a timestamp from "date +'%s'" now i need function which finds the logfile with the greatest number in it and returns (echos) the... (7 Replies)
Discussion started by: latenite
7 Replies

9. UNIX for Dummies Questions & Answers

Compare the content of a variable with a string

Hello all: I'm new in Unix and here and I'am spanish so my english isn't so good to explain my doubt. Here it is. Very urgent: I need to compare the value of a variable with a string. Example is this. Imagine that the variable x1 contains the path and a file text and I need to compare... (2 Replies)
Discussion started by: robdai
2 Replies

10. Shell Programming and Scripting

Compare the content of 2 files

Hi Guys, What is the most effecient way to compare the content of 2 seperate files and extract the result of there is a match? We have 2 separate log files and we are trying to find the common errors from the 2 files. Thanks, Odogbolu98 :( (3 Replies)
Discussion started by: odogbolu98
3 Replies

Featured Tech Videos