Compare 2 log files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare 2 log files
# 1  
Old 04-15-2009
Compare 2 log files

Hello,

I am new here, so first of all I want say hello to everyone.

I am newbie on script but you may be able to help me on this :

I am on solaris (ksh)
I need to compare 2 files (one_trash.log / two_arch.log)
On the first file I've got like 1000 entries files name
Ex of one_trash.log :
one.tar
two.tar ..etc

On the second file I've got the repertories and the a lots of files name ~ almost 5000 entries
Ex of two_arch.log :
Rep1/rep2/rep3/one.tar
Rep10/rep21/rep32/two.tar ..etc

Here is my problem :
I need to create a file (ex match.log) that will contain all entries found on the one_trash.log that match with the two_arch.log. This file must contain the repertories and the file name

ex : Rep1/rep2/rep3/one.tar

I need this to see if the file name given on the first file one_trash.log exist on the two_arch.log and where this file is exactly stored.

here is an example of the two files :

one_trash.log

NCBO1T1_RTSTAT_20001227_20010103_S20010117
UK_IPSWITCHPRE_20031217_1452_1.TAR
UK_LONDONPSSPRE_20020327_0731_1.TAR


two_arch.log

OPE/NCB1/RTSTAT/NCBO1T1_RTSTAT_20010307_20010314_S20010328
OPE/NCB1/RTSTAT/NCBO1T1_RTSTAT_20010620_20010627_S20010704


I am sorry about my english.

Thanks a lot for your help. I lost on this. PLEASE Smilie

Last edited by Aswex; 04-15-2009 at 11:42 AM..
# 2  
Old 04-17-2009
Try:
Code:
fgrep -f one_trash.log two_trash.log

# 3  
Old 04-18-2009
The suggested fgrep -f could incorrectly match a filename from one_trash.log to a directory name in two_arch.log.

Try this:
Code:
awk -f aswex.awk one_trash.log two_arch.log

where aswex.awk contains:
Code:
BEGIN { FS = "/"}
FNR == 1 { fn++ } # track the file number
fn == 1 {
  a[$0] = 1
  }
fn == 2 && a[$NF]

# 4  
Old 04-19-2009
Quote:
Originally Posted by colemar
The suggested fgrep -f could incorrectly match a filename from one_trash.log to a directory name in two_arch.log.
That's a good point.

I'll just note that the awk code could be a bit simpler and easily on the command line:
Code:
awk -F/ 'FNR == NR { a[$0]++; next } a[$NF]'  \
one_trash.log two_arch.log


Last edited by otheus; 04-19-2009 at 06:44 AM.. Reason: FNR should be NR not 1
# 5  
Old 04-22-2009
Dear Friends,

I am so sorry to be so late to reply to your posts. Thanks a lot for your help.

I am coming back to you because none of your script or command line seems to work for me.

For sure I am not good enough to make them work and I am unable to find a way to compare this 2 log files.

The error code is :

awk: syntax error near line 1
awk: bailing out near line 1

here is what I've tried :
from the shell "ksh" (I am on solaris 9)

awk -f/ 'FNR == NR { a[$0]++; next } a[$NF]' \
what else ? > one_trash.log two_arch.log

And i've got the errors.

If you need more info, just let me know.

Thanks again for all your help
# 6  
Old 04-22-2009
use 'nawk' (or /usr/xpg4/bin/awk) instead of 'awk' on Solaris.
# 7  
Old 04-22-2009
Hi,

do you mind tell me what i need to do ? Here is the error code I've got

nawk: can't open file FNR == NR { a[$0]++; next } a[$NF]
source line number 1
context is
>>> <<<


I've done this :

nawk -f/ 'FNR == NR { a[$0]++; next } a[$NF]' \
what else ? > one_trash.log two_arch.log
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare files and share output from both files

hi all, Thanks to all for your great help... I have a scenario that I have two files (file1 & file2). I need to compare two files entire row by row and share the output if any discrepancies within two files. File1: DB1|TB1|C1,C3 DB2|TB2|C1,C2 DB3|TB3|C1,C2,C3,C4 File2: ... (2 Replies)
Discussion started by: Selva_2507
2 Replies

2. Shell Programming and Scripting

Compare multiple files, and extract items that are common to ALL files only

I have this code awk 'NR==FNR{a=$1;next} a' file1 file2 which does what I need it to do, but for only two files. I want to make it so that I can have multiple files (for example 30) and the code will return only the items that are in every single one of those files and ignore the ones... (7 Replies)
Discussion started by: castrojc
7 Replies

3. Shell Programming and Scripting

Compare two files, then overwrite first file with only that in both files

I want to compare two files, and search for items that are in both. Then override the first file with that containing only elements which were in both files. I imagine something with diff, but not sure. File 1 One Two Three Four Five File 2 One Three Four Six Eight (2 Replies)
Discussion started by: castrojc
2 Replies

4. Shell Programming and Scripting

Compare different String in Log Files

Hi Guys , sorry for my first post but a newbie here need some help on my simple scripts. I have some scripts below that count the job started and the job finished and is the job started and job finished equal ..then all job was successfully run and finished on that day. but sometime the was... (3 Replies)
Discussion started by: thermometer
3 Replies

5. Shell Programming and Scripting

Require compare command to compare 4 files

I have four files, I need to compare these files together. As such i know "sdiff and comm" commands but these commands compare 2 files together. If I use sdiff command then i have to compare each file with other which will increase the codes. Please suggest if you know some commands whcih can... (6 Replies)
Discussion started by: nehashine
6 Replies

6. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

7. Shell Programming and Scripting

How to compare 2 files & get only few columns based on a condition related to both files?

Hiiiii friends I have 2 files which contains huge data & few lines of it are as shown below File1: b.dat(which has 21 columns) SSR 1976 8 12 13 10 44.00 39.0700 70.7800 7.0 0 0.00 0 2.78 0.00 0.00 0 0.00 2.78 0 NULL ISC 1976 8 12 22 32 37.39 36.2942 70.7338... (6 Replies)
Discussion started by: reva
6 Replies

8. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

9. Shell Programming and Scripting

Compare log files and get latest

I have a directory location where in some logs gets generated whenever some application build is triggered. I need to send the generated log as an email to end user. I will get files like abcyyyy_mm_dd_time.log and next file will have different time for same day. Need to run a cron and take the... (18 Replies)
Discussion started by: yogi90
18 Replies

10. Shell Programming and Scripting

compare two files and to remove the matching lines on both the files

I have two files and need to compare the two files and to remove the matching lines from both the files (4 Replies)
Discussion started by: shellscripter
4 Replies
Login or Register to Ask a Question