Restricted File Comparison

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Restricted File Comparison
# 1  
Old 09-12-2011
Restricted File Comparison

Hey guys,

I've got a scripting problem that has been bugging me so thought I'd ask the wise people here! Basically I have two overlapping log files, and I want to get the newest lines from the new log file that aren't in the old log file - but not the old lines in the old log that aren't in the new log.

Say the old log file has this content:
1: Apple
2: Banana
3: Carrot
4: Dog

And the new log has this content:
3: Carrot
4: Dog
5: Elephant
6: Fish

I want my script to output:
5: Elephant
6: Fish

Essentially, I want a diff of the two files, but only include the lines that are unique to the second file - is this possible?

edit: It seems that I can do it with "grep -v -x -f file1 file2", but this has O(n^2) complexity so won't work nearly fast enough with files that are reasonably long. Any other ideas?

Thanks for any help!
# 2  
Old 09-12-2011
1. If awk can get the first file in its memory you can do something like:
awk '
NR == FNR {a[$2] = $1 }
NR != FNR && !($2 in a)' old new

2. If you don't mind about the order of the input you can use sort and uniq on both files.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Restricted shell with execution permission on a file

Hi, I've created a user which shell is the restricted one (/usr/lib/rsh), but I've realized that the user cannot execute a script !!! The scope was to allow the ssh access to a user that could not access to any other directories but there that user should be able to run a script that retrieves ... (2 Replies)
Discussion started by: mary0
2 Replies

2. Shell Programming and Scripting

File Comparison

HI, I have two files and contains many Fields with | (pipe) delimitor, wanted to compare both the files and get only unmatched perticular fields. this i wanted to use in shell scriting. ex: first.txt 111 |abc| 230| hbc231 |bbb |210 |bbd405 |ghc |555 |cgv second.txt 111 |abc |230 |hbc231... (1 Reply)
Discussion started by: prawinmca
1 Replies

3. Shell Programming and Scripting

File Comparison: Print Lines not present in another file

Hi, I have fileA.txt like this. B01B02 D0011718 B01B03 D0012540 B01B04 D0006145 B01B05 D0004815 B01B06 D0012069 B01B07 D0004064 B01B08 D0011988 B01B09 D0012071 B01B10 D0005596 B01B11 D0011351 B01B12 D0004814 B01C01 D0011804 I want to compare this against another file (fileB.txt)... (3 Replies)
Discussion started by: genehunter
3 Replies

4. Shell Programming and Scripting

file comparison

Dear All, I would really appreciate if you can help me to resolve this file comparison I have two files: file1: chr start end ID gene_name chr1 2020 3030 1 test1 chr1 900 5000 2 test1 chr2 5000 8000 3 test2 chr3 6000 12000 4 test3 chr3 6000 15000 5 test3 file2:... (2 Replies)
Discussion started by: paolo.kunder
2 Replies

5. Shell Programming and Scripting

two file comparison

now i have a different file zoo.txt with content 123|zoo 234|natan 456|don and file rick.txt with contents 123|dog|pie|pep 123|tail|see|newt 456|som|sin|sim 234|pay|rat|cat i want to look for lines in file zoo.txt column1 that has same corresponding lines in column 1 of... (6 Replies)
Discussion started by: dealerso
6 Replies

6. Filesystems, Disks and Memory

Restricted File Creation for particuler pattern

HI All, I have this wierd requirment. Any help will be appriciated. I need to restrict file creation with certain pattern. Suppose, i want my system not to allow creation of files with *.exe extension. The requirment is that file should not be created like it happen in windows you can't create... (0 Replies)
Discussion started by: r00t4u
0 Replies

7. Shell Programming and Scripting

File Comparison

Hi i have 2 csv files a.csv and b.csv with the same number of columns and a list of values in both of it. Each and every individual value in both the files need to compared and if it matches then print correct in a new csv file otherwise print Incorrect eg a.csv 1,12/27/2007,Reward,$10.00... (5 Replies)
Discussion started by: naveenn08
5 Replies

8. Shell Programming and Scripting

File Comparison- Need help

I have two text files which have records of thousand rows. Each row is having around 40 columns. Each column is tab delimited. Each row is delimited by newline character. My requirement is to find for each row i need to find whether any column is different between the two files. For each row i... (8 Replies)
Discussion started by: uihnybgte
8 Replies

9. Shell Programming and Scripting

file comparison

hi I have 2 files to comapre ,in file a sible column it is numbers,in file b2 numbers and other values with coma separated. i want compare numbers in file a with file b,and the out put put should be in C with numbers in both file a and b along with other columns of file b. i used folowing... (7 Replies)
Discussion started by: satish.res
7 Replies

10. Shell Programming and Scripting

File Comparison

I have to compare two text files, very few of the lines in these files will have some difference in some column. The files size is in GB. Sample lines are as below: 11111122222222333333aaaaaaaaaabbbbbbbbbccccccccdddddd 11111122222222333333aaaaaaaaaabbbbbbbbbccccccccddeddd So assuming these... (19 Replies)
Discussion started by: net_shree
19 Replies
Login or Register to Ask a Question