UNIX for Dummies Questions & Answers

Hi,

I need to compare todays file to yesterdays file to find deletes.
I cannot use comm -23 file.old file.new.

Because each record may have a small change in it but is not really a delete.
I have two delimited files. the first field in each file is static. All other fields may change. I want to know if the static data has disappeared from the file. If I use comm -23 I will get the lines that data in field 2 or 3 etc may have changed but the static data in filed 1 is the same.

I did search this site and found a reply to another post that I used to solve my problem. I wrote a smal script that does work. But the problem is it is very slow.

My old file has 7000 lines my new file has 4000 lines. I ran my script and 30 minutes later it still was not done. I ran it on a UNIX box that has a lot of memory and processors. It is not a hardware issue.

Any sugestions? Below is my code:

while read static
do
found="no"
data=`echo "$static" | cut -d'|' -f1`

while read line
do

echo "$line" | grep "$data" >/dev/null
if [ $? -eq 0 ]
then
found="yes"
break
fi
done < file.new
if [ $found = "no" ]
then
echo "$static"
fi
done < file.old

I am trying to keep all data on the deleted line. I could cut both files down filed 1 into smaller files and then run a comm -23 on the smaller files. But then I loose all data in the other fields.

Code:

awk -F"|" ' 
BEGIN { 
	while( getline < "file.new" ) 
	{ arr[$1]=1 }
}
arr[$1] != 1 { print } 
' file.old

anbu23,

Thank you very much. this works great. I assume with a fixed length file I could use awk's substr command?

I also have two files that are fixed length where the first 20 characters is static. SO I re-wrote you helpful code to the below it also works do you see any issues with what i did I am not that good with awk.

awk -F"|" '
BEGIN {
while( getline < "file.new" )
{ arr[substr($1,1,20)]=1 }
}
arr[substr($1,1,20)] != 1 { print }
' file.old