awk to compare diff output by fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to compare diff output by fields
# 8  
Old 11-10-2010
Quote:
Originally Posted by ux4me
wow.. i think it's working per your tip. do you mind explaining the code as i am trying to learn how to fish?

Thanks again.
no problem - the 'fishing rod' is the posted code.
# 9  
Old 11-17-2010
how can i modify this so that i can "dosomething else" when no line is found? Please advise and thanking you in advance.

i've tried to use this (see below) but it doesn't handle multiple lines read Smilie

Code:
[[ ! -z $line ]] && { dosomething_else ; } || { dosomething ; }

# 10  
Old 11-17-2010
Quote:
Originally Posted by ux4me
how can i modify this so that i can "dosomething else" when no line is found? Please advise and thanking you in advance.

i've tried to use this (see below) but it doesn't handle multiple lines read Smilie

Code:
[[ ! -z $line ]] && { dosomething_else ; } || { dosomething ; }

Code:
if [[ ! -z $line ]]; then
     cmd1
     cmd2 
     ...
else
     cmd3
     cmd4
     ...
fi

# 11  
Old 11-17-2010
Quote:
Originally Posted by ux4me
how can i modify this so that i can "dosomething else" when no line is found? Please advise and thanking you in advance.

i've tried to use this (see below) but it doesn't handle multiple lines read Smilie

Code:
[[ ! -z $line ]] && { dosomething_else ; } || { dosomething ; }

Code:
#!/bin/ksh

nawk '
  # create an variable "idx" which is a concatenation of the first 5 fields in a record/line
  # as we read the file line by line
  {idx=$1 FS $2 FS $3 FS $4 FS $5}
  {
    # see if "idx" is in array "a" - array "a" is indexed by the value of "idx"
    if (!(idx in a))
      print 0 ":" $0
    else
      # if "idx" is already in "a", check if the stored value (a[idx]) is less than the
      # last field ($NF) of the current record/line. If it"s, we see the "increase" in value
      # If so, output the current line (print $0)
      if (a[idx]<$NF)
         print 1 ":" $0
    # store the last field ($NF) of the current record in aarray "a" indexed by "idx"
    a[idx]=$NF
  }' myDiffFile | while IFS=: read smth line
do
  # read the output of "nawk" and do "doSomething" with the read line
  if [ "${smth}" -eq 1 ]; then
     echo "doSomething with [$line]"
  else
     echo "doSomethingElse with [$line]"
  fi
done


Last edited by vgersh99; 11-17-2010 at 07:25 PM.. Reason: fixed the embeded '
# 12  
Old 11-17-2010
Thanks again.. but i couldn't get the "IFS=:" to work, and i ended up using a flag instead to track the lines, i.e.:

Code:
flag=0
...
.. | while read line
do
flag=1
echo "doSomething with [$line]"
done
if [ "$flag" -eq 0 ] ; then
   echo "doSomethingElse with [noline]"
fi

another question: how can i doSomething when it's the first occurrence in the diff file before the increment counter comparison?
# 13  
Old 11-17-2010
strange - I get the following for the sample file you provided:
Code:
doSomethingElse with [AAA BBB CCC DDD EEE 10]
doSomething with [AAA BBB CCC DDD EEE 11]
doSomethingElse with [PPP QQQ RRR SSS TTT 5]
doSomethingElse with [VVV WWW XXX YYY ZZZ 2]

---------- Post updated at 06:29 PM ---------- Previous update was at 06:26 PM ----------

Quote:
Originally Posted by ux4me
Thanks again.. but i couldn't get the "IFS=:" to work, and i ended up using a flag instead to track the lines, i.e.:

Code:
flag=0
...
.. | while read line
do
flag=1
echo "doSomething with [$line]"
done
if [ "$flag" -eq 0 ] ; then
   echo "doSomethingElse with [noline]"
fi

another question: how can i doSomething when it's the first occurrence in the diff file before the increment counter comparison?
something like this?
Code:
#!/bin/ksh

nawk '
  # create an variable "idx" which is a concatenation of the first 5 fields in a record/line
  # as we read the file line by line
  {idx=$1 FS $2 FS $3 FS $4 FS $5}
  {
    # see if "idx" is in array "a" - array "a" is indexed by the value of "idx"
    if (!(idx in a) && FNR>1)
      print 0 ":" $0
    else
      # if "idx" is already in "a", check if the stored value (a[idx]) is less than the
      # last field ($NF) of the current record/line. If it"s, we see the "increase" in value
      # If so, output the current line (print $0)
      if (a[idx]<$NF || FNR==1)
         print 1 ":" $0
    # store the last field ($NF) of the current record in aarray "a" indexed by "idx"
    a[idx]=$NF
  }' myDiffFile| while IFS=: read smth line
do
  # read the output of "nawk" and do "doSomething" with the read line
  if [ "${smth}" -eq 1 ]; then
     echo "doSomething with [$line]"
  else
     echo "doSomethingElse with [$line]"
  fi
done

# 14  
Old 11-23-2010
Attempted the revised awk script with the inputfile:

Input file:
Code:
< AAA BBB CCC DDD EEE 123
> PPP QQQ RRR SSS TTT 111
> VVV WWW XXX YYY ZZZ 333
> AAA BBB CCC DDD EEE 124

Results:
Code:
doSomethingElse with [< AAA BBB CCC DDD EEE 123]
doSomethingElse with [> PPP QQQ RRR SSS TTT 111]
doSomethingElse with [> VVV WWW XXX YYY ZZZ 333]
doSomethingElse with [> AAA BBB CCC DDD EEE 124]

Expecting:
Code:
doSomething with [> PPP QQQ RRR SSS TTT 111]  # first occurrence
doSomething with [> VVV WWW XXX YYY ZZZ 333]  # first occurrence
doSomething with [> AAA BBB CCC DDD EEE 124]  # last counter field increased from 123 to 124


Last edited by ux4me; 11-23-2010 at 02:21 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Trying to use diff output to compare to a separate file

I have two files: smw:/working/iso_testing # cat a QConvergeConsoleCLI-1.1.03-49.x86_64.rpm aaa_base-13.2+git20140911.61c1681-1.3.i586.rpm acpica-20140724-2.1.2.i586.rpm test.rpm smw:/working/iso_testing # cat b QConvergeConsoleCLI-1.1.03-49.x86_64.rpm... (12 Replies)
Discussion started by: jedlund21
12 Replies

2. Shell Programming and Scripting

Compare 2 files and find missing fields awk

Hello experts! I have 2 files. file1 is a list file containing uniquely names. e.g.: name1 number number name2 number number name5 number number name10 number number ... file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g. name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies

3. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of... (7 Replies)
Discussion started by: sljnk
7 Replies

4. UNIX for Dummies Questions & Answers

Compare values of fields from same column with awk

Hi all ! If there is only one single value in a column (e.g. column 1 below), then return this value in the same output column. If there are several values in the same column (e.g. column 2 below), then return the different values separated by "," in the output. pipe-separated input: ... (11 Replies)
Discussion started by: lucasvs
11 Replies

5. Shell Programming and Scripting

compare 2 CSV fields from same diff output

Attached is a file called diff.txt It is the output from this command: diff -y --suppress-common-lines --width=5000 1.txt 2.txt > diff.txt I have also attached 1.txt and 2.txt for your convenience. Both 1.txt and 2.txt contain one very long CSV string. File 1.txt is a CSV dump of... (0 Replies)
Discussion started by: gvolpini
0 Replies

6. Shell Programming and Scripting

AWK Compare files, different fields, output

Hi All, Looking for a quick AWK script to output some differences between two files. FILE1 device1 1.1.1.1 PINGS device1 2.2.2.2 PINGS FILE2 2862 SITE1 device1-prod 1.1.1.1 icmp - 0 ... (4 Replies)
Discussion started by: stacky69
4 Replies

7. Shell Programming and Scripting

Compare fields in 2 files using AWK

Hi unix gurus, I have a urgent requirement, I need to write a AWK script to compare each fields in 2 files using AWK. Basically my output should be like this. file1 row|num1|num2|num3 1|one|two|three 2|one|two|three file2 row|num1|num2|num3 1|one|two|three 2|one|two|four ... (5 Replies)
Discussion started by: rashmisb
5 Replies

8. Shell Programming and Scripting

Need awk script to compare 2 fields in fixed length file.

Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file. i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies

9. Shell Programming and Scripting

Awk - Compare fields and increment variables

Hi, My first post to this group... I have a need to to parse a source file which is a capture from a network analyser. I have two fields that need to be checked: - Field 7 represents the packet length (an integer), and Field 4 represents a network address (e.g. 192.168.25.3) - The... (10 Replies)
Discussion started by: mv652
10 Replies

10. Shell Programming and Scripting

Compare two files and output diff to third file

I have made several attempts to read two files of ip addresses and eliminate records from file1 that are in file2. My latest attempt follows. Everything works except my file3 is exactly the same as file1 and it should not be. # !/usr/bin/bash # # NoInterfaces # Utility will create a file... (8 Replies)
Discussion started by: altamaha
8 Replies
Login or Register to Ask a Question