compare realtime


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare realtime
# 1  
Old 02-27-2007
compare realtime

I have two log files which keeps appending every sec...I want to extract a certain field from each file(using awk for extracting the data) and compare them in real time...

ex:
log1
122
234
567

log2
234
567

log3
122

i need a log3 which keeps appending the data found in log1 and not in log2..
# 2  
Old 02-27-2007
Could you show us a sample of two log files and the expected content for log3? Which field in the log1 and log2 you want to extract?
# 3  
Old 02-27-2007
#!/bin/bash
#assuming that there is only one field in file1 and file2

for i in $(cat file1)
do
if grep -q $i file2
then :
else
echo $i >>file3
fi
done
# 4  
Old 02-27-2007
hmmm...

log1 which keeps appending every sec
234,abc
678,def
345,fgh

awk -F"," '{print $1}' log1

will give me
234
678
345

log2 which keeps appending every sec
345, ghi
678, jkl


awk -F"," '{print $1}' log2

will give me
345
678


now i want a log3 which compares the output of the above two commands and keeps appending... ex: i should get 234 in log3 as its there in log1 and not in log2
# 5  
Old 02-27-2007
Code:
#! /opt/third-party/bin/perl

my($i, %fileHash_a, %fileHash_b);

for( ;1; sleep 5 ) {

%fileHash_a = ();
%fileHash_b = ();

open(FILE, "< a") || die "file a <$!>\n";

while( <FILE> ) {
  chomp;
  $fileHash_a{$_} = $i++;
}

close(FILE);

$i=0;
open(FILE, "< b") || die "file b <$!>\n";
while( <FILE> ) {
  chomp;
  $fileHash_b{$_} = $i++;
}

close(FILE);

  open(FILE, "> update") || die "update <$!>\n";

  foreach my $k ( keys %fileHash_a ) {
    if ( exists $fileHash_b{$k} ) {
    }
    else {
      print FILE $k . "\n";
    }
  }
  close(FILE);
}


exit 0

All the differences would be captured in the file 'update'
This is a continuously running script, run it as a background process
Refresh rate can be altered through the sleep value given in for loop
Minimal time taken in comparison and generating what is not available in first file with respect to the second file
# 6  
Old 02-27-2007
Code:
$cat tmp1
234
678
345
$cat tmp2
345
678

Code:
$sort tmp1 > tmp1.sort
$sort tmp2 > tmp2.sort
$comm -23 tmp1.sort tmp2.sort
234

comm -23 prints only lines in the first file but not in the second
or
Code:
$grep -v -f tmp2 tmp1
234

# 7  
Old 02-27-2007
hi

Thanks....but this doesnt give me a realtime log...by the time i do cat and compare i would have lost many entries in the meantime(log gets appended with data every sec) .I want a log which keeps comparing in real time and gets appended every time it finds a diff in file...like in tail -f
Login or Register to Ask a Question

Previous Thread | Next Thread

1 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copying files realtime

I have a primary server where certain files are created real time. These files have varying file sizes. I want to FTP or copy them over to a different server server as soon a file gets created. I have to ensure that only full file is copied. The receiving end process expects a FULL file. I am ok... (3 Replies)
Discussion started by: vskr72
3 Replies
Login or Register to Ask a Question