awking and grepping parts of files: the 'super diff'


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awking and grepping parts of files: the 'super diff'
# 1  
Old 07-17-2008
awking and grepping parts of files: the 'super diff'

OKAY----

Here's what I must do.
I have two files. I need to compare the two files such as with the diff command. I am adding FILENEW to FILEOLD

If fields $1, $2, $5, and 6 are the same, then I don't want to add FILENEW records to FILEOLD.
If they are not, then append the lines.

Is this possible in the UNIX scripting world? It seems that this could be done in AIX. I also have Perl as a last resort. Looking hard, wondered if anyone has done something like this before.

Smilie
# 2  
Old 07-17-2008
it may be sickboys entry is similar...on phone will check when I get off
# 3  
Old 07-17-2008
Does anybody want to take a crack at this?
My slow brain is simply not understanding.

If for example:

awk 'BEGIN{FS=OFS=","}
NR==FNR{a[$1]=$0; next}
a[$1]{$0=$0 RS a[$1]}{print}' file2 file1

Adds a line to file1 if the first field matches in both files.
How do you only print the line if say, fields1, 2, 5, and 6 match both files?
And of course, I would have to reverse that (i.e., only add the lines to file1 if fields 1, 2, 5, and 6 are NOT matched between file1 and file2)

Obviously the first FNR line is file2, a[$1] being the first field.
The second line then matches a[$1] and appends a record to the output of file1.Smilie

Thanks much for any assistance.
# 4  
Old 07-17-2008
File1:
----------------------------
apples,pears,bananas,oranges,grapefruit,watermelon,pineapple
epples,pears,bananas,oranges,grapefruit,watermelon,pineapple
epples,paars,bananas,oranges,grapefruit,watermelon,pineapple
graby,epples,bananas,oranges,grapefruit,watermelon,pineapple


File2:
----------------------------
pearsrs,apples,bananers,orangas,grapefruit,graby,pineapples
apples,pears,bandanas,orangatangs,grapefruit,watermelon,wineapple



Wanted output: (File1: appended: only add records when fields 1, 2, 5 and 6 DO NOT match)
-----------------------------
apples,pears,bananas,oranges,grapefruit,watermelon,pineapple
epples,pears,bananas,oranges,grapefruit,watermelon,pineapple
epples,paars,bananas,oranges,grapefruit,watermelon,pineapple
graby,epples,bananas,oranges,grapefruit,watermelon,pineapple
pearsrs,apples,bananers,orangas,grapefruit,graby,pineapples
# 5  
Old 07-17-2008
nawk -f jeff.awk FILENEW FILEOLD

jeff.awk:
Code:
{ idx = $1 SUBSEP $2 SUBSEP $5 SUBSEP $6 }
NR==FNR { fnew[ idx ] = $0; next }
{
   if ( !(idx in fnew) )
      $0 = $0 ORS fold[idx]
}
1


Last edited by vgersh99; 07-17-2008 at 04:30 PM..
# 6  
Old 07-17-2008
Basically what this is is a logfile that has alerts from a program.
The entries get dropped into a file continually. But some entries have the same date, transaction ID, location, alert code and keep getting dropped in over and over (like until the problem is fixed)

The awk filter is to create a 2nd file that just adds new messages if they are not the same date, ID, location, code, and run on a continual basis filtering these out.
# 7  
Old 07-17-2008
Thanks
But something is awry.

OKAY, If I do:
nawk -f jeffawk file2 file1
------------------------
apples,pears,bananas,oranges,grapefruit,watermelon,pineapple
epples,pears,bananas,oranges,grapefruit,watermelon,pineapple

epples,paars,bananas,oranges,grapefruit,watermelon,pineapple

graby,epples,bananas,oranges,grapefruit,watermelon,pineapple

This is basically a cat of file1.


If I do:
nawk -f jeffawk file1 file2
I get the following:
-----------------------
pearsrs,apples,bananers,orangas,grapefruit,graby,pineapples

apples,pears,bananas,oranges,grapefruit,watermelon,pineapple



The first line, 'pearsrs', is right on the money. But this is basically a cat of file2.


Not sure
? ? ? ?

Last edited by jeffpas; 07-17-2008 at 05:19 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split the all files in to 8 parts in a folder

Hi, I have different files and i need to split the files in that folder split in to 8 parts with equal number of lines....! any fastest way of doing this in awk. for an example i have a file called "BillingDetails_BaseFile.csv" with total line count 65536 and i need to split in to 8 parts... (1 Reply)
Discussion started by: Raghuram717
1 Replies

2. Shell Programming and Scripting

Grepping or awking multiple lines in a file - regex

data.txt: hellohellohello mellomello1mello tellotellotellotello bellobellowbellow vellow My attempts: egrep ".*mello1\n.*bellow" data.txt awk '/.*mello1.*\nbellow/' data.txt how can i search for patterns that are on different lines using simple egrep or awk? i only want the... (7 Replies)
Discussion started by: SkySmart
7 Replies

3. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

4. Shell Programming and Scripting

Incrementing parts of ten digits number by parts

I have number in file which contains date and serial number: 2013101000. The last two digits are serial number (00). So maximum of serial number is 100. After reaching 100 it becomes 00 with incrementing 10 which is day with max 31. after reaching 31 it becomes 00 and increments 10... (31 Replies)
Discussion started by: Natalie
31 Replies

5. Shell Programming and Scripting

Using bash to separate files files based on parts of a filename

Hey guys, Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have: (a_b_c.txt) WB34_2_SLA8.txt WB34_1_SLA8.txt WB34_1_DB10.txt WB34_2_DB10.txt WB34_1_SLA8.txt WB34_2_SLA8.txt 77_1_SLA8.txt... (1 Reply)
Discussion started by: Breentax
1 Replies

6. UNIX for Dummies Questions & Answers

grepping log files

I have a log file and I have two unique strings which represent the start and end of the text I want to obtain. How can I get all the text inbetween this start string and the end string? Thanks (2 Replies)
Discussion started by: chrisjones
2 Replies

7. UNIX for Dummies Questions & Answers

grepping between files

Hi I have two files File 1 alias HOME =.. alias DATA = ${DATA}/runtime1/test alias SQL = ${DATA}/find1dir/test alias SQL1 = ${HOME}/sql/orcl alias SQL2 =... (2 Replies)
Discussion started by: ssuresh1999
2 Replies

8. Shell Programming and Scripting

grepping many values from same files

Hi All, I am having a script in which I am greping some values and storing them from files with .err and .log extensions. I feel I can do it better.But How? Below is my piece of code. oneerrors=`egrep -i -n "one" *.err *.log` twoerrors=`egrep -i -n "two" *.err *.log` ... (2 Replies)
Discussion started by: Sreejith_VK
2 Replies

9. HP-UX

BAD SUPER BLOCK - Run fsck with alternate super block number

Error received when I tried to restore a blank disk with an 'auto recovery' DDS tape via HP-UX recovery system 2.0 onto a 1Gb SCSI. I assumed it would do the setup, wrong. Could someone tell me the procedure to initial disk for recovering files using cpio. The system is a HP-UX 9.04 version on a... (1 Reply)
Discussion started by: admin wanabee
1 Replies

10. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies
Login or Register to Ask a Question