Compare an item in one line of a file against an item in the next line of the same file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare an item in one line of a file against an item in the next line of the same file
# 1  
Old 04-17-2020
Compare an item in one line of a file against an item in the next line of the same file

Hi all,

I found the following old thread very useful and in particular the very helpful post by @frans
Code:
while read A # read line and put in variable A. If unreadable (EOF) the break the loop.
do
    read B # read next line and put in variable B
    ((i+=2)) # increment variable i by 2 (like i=i+2)
    # [ "$A" != "$B" ] && echo "Lines $((i-1)) and $i are different"
    # can be written
    if [ "$A" != "$B" ] # if values of variables A and B are different ('[ ]' stands for 'test')
    then echo "Lines $((i-1)) and $i are different" # OK ?
    fi
done < inputfile # tells the loop to read from inputfile (else, it would read from keyboard.)

However, I am looking to try and compare one item / word from the first line to see if that same item is part of the second line

So in a sense, I'm trying to say something like
if $A | grep "XXX" = $B | grep "XXX" then do ..........

I know the position of the word in the first line so I tried using if [ "$A" | awk {'print $5'} = "$B" | awk {'print $2'} ]
but no joy. It didn't like the syntax of that.

Last edited by championc; 04-17-2020 at 01:29 PM..
# 2  
Old 04-17-2020
could you provide a sample/representative snapshot of your input file inputfile and a desired output.
# 3  
Old 04-17-2020
Not clear. Do you want to compare every other line to its predecessor? So, operate on pairs of lines? Or, on every single line? And compare predecessor's field 5 to actual line's field 2? Is it possible that read B hits EOF? How do you handle that?
# 4  
Old 04-17-2020
Thanks for taking the time to reply

I am trying to work with a firewall ruleset. Every DROP rule should have a partner LOG rule. But from the following, you can see that two rules do not have a corresponding LOG rule. So I am trying to process the file and identify all rules I have a DROP but no LOG. The two offenders in this selection below are in BOLD

Code:
set security firewall name Private-to-AA rule 25 action 'drop'
set security firewall name Private-to-AA rule 9999 action 'drop'
set security firewall name Private-to-AA rule 9999 'log'
set security firewall name Private-to-BB default-action 'drop'
set security firewall name Private-to-BB 'default-log'
set security firewall name Private-to-CC default-action 'drop'
set security firewall name Private-to-CC 'default-log'
set security firewall name Private-to-CC rule 9999 action 'drop'
set security firewall name Private-to-CC rule 9999 'log'
set security firewall name Private-to-DD default-action 'drop'
set security firewall name Private-to-DD 'default-log'
set security firewall name Private-to-DD rule 9999 action 'drop'
set security firewall name Private-to-DD rule 9999 'log'
set security firewall name local default-action 'drop'
set security firewall name local rule 10 action 'drop'
set security firewall name local rule 10 'log'

# 5  
Old 04-18-2020
a bit verbose (assuming 'drop' comes first), but....
awk -f cham.awk ruleSetFile.txt where cham.awk is:
Code:
{
  idx=""
  slide=(/drop/)?2:1
  for(i=1;i<=NF-slide;i++)
     idx=(i==1)?$i:idx $i
}
/drop/{d[idx]=$0;next}
/log/ && (idx in d) {delete d[idx]}
END {
  for(i in d)
    print d[i]
}

results in:
Code:
set security firewall name Private-to-AA rule 25 action 'drop'
set security firewall name local default-action 'drop'


Last edited by vgersh99; 04-18-2020 at 08:31 AM..
# 6  
Old 04-18-2020
Great thanks, I'll give that a go. I'm a real BASH noob, so I'd never have managed to get that solution, or anything close to it I think.

Do I need to make a separate script with the contents bring the code in the code section, and then call it from the awk command ? Or can I put the whole lot into a single command that I can paste into a command prompt ?
# 7  
Old 04-18-2020
for simplicity sake.... create a file cham.awk with the content as indicated in the post and execute the command line as indicated in the post.
There're other ways to execute what's been provided, but for now let's stick with the provided paradigm.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to process a list of items and uncomment lines with that item in a second file

Hello, I have a src code file where I need to uncomment many lines. The lines I need to uncomment look like, C CALL l_r(DESNAME,DESOUT, 'Gmax', ESH(10), NO_APP, JJ) The comment is the "C" in the first column. This needs to be deleted so that there are 6 spaces preceding "CALL".... (7 Replies)
Discussion started by: LMHmedchem
7 Replies

2. UNIX for Beginners Questions & Answers

Zabbix item for last line of a log file

Dear all,Zabbix version : 2.4 (yes, I know, upgrading soon - honest) Server OS version : CentOS 6, 64-bit (CentOS 7 with the Zabbix upgrade)I've got a large log file that I would like to read by an external process. It's basically the same as reading the item value on a web-page. I have... (5 Replies)
Discussion started by: rbatte1
5 Replies

3. Shell Programming and Scripting

How to compare previous and current item in for loop in bash?

Hey, I am trying to compare formated login and logout dates from one user at a host which I have stored in a tmp directory in order to find out the total login time. I need to compare them in order to find overlapping intervals. At first I tried to store each log in and logo date in an array... (3 Replies)
Discussion started by: Mumu123
3 Replies

4. Shell Programming and Scripting

Read a lis, find items in a file from the list, change each item

Hello, I have some tab delimited text data, file: final_temp1 aname val NAME;r'(1,) 3.28584 r'(2,)<tab> NAME;r'(3,) 6.13003 NAME;r'(4,) 4.18037 r'(5,)<tab> You can see that the data is incomplete in some cases. There is a trailing tab after the first column for each incomplete row. I... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

5. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

6. Shell Programming and Scripting

how to compare file line by line with awk

im a newbee to unix. I have a requirement to compare two files with awk. file1.txt a b c d e file2.txt a b d e here i want to compare each line in file1 with corresponding line in file2 and prinf the line with difference. ie to check required result as shown below a=a (dont... (3 Replies)
Discussion started by: kiranps
3 Replies

7. Programming

Storing C++-struct in file - problem when adding new item in struct

Hi, I have received an application that stores some properties in a file. The existing struct looks like this: struct TData { UINT uSizeIncludingStrings; // copy of Telnet data struct UINT uSize; // basic properties: TCHAR szHost; //defined in Sshconfig UINT iPortNr; TCHAR... (2 Replies)
Discussion started by: Powerponken
2 Replies

8. Shell Programming and Scripting

Print new item in file with symbol

Dear all, I have encountered some problem here. I prompt the user for input and store it into a data file, eg. key in name and marks so the data file will look like this andrew 80 ben 75 and the next input is carine 90. So the problem here is i want to print... (2 Replies)
Discussion started by: branred
2 Replies

9. Shell Programming and Scripting

Reading each item from a formatted file

Hi, I have a file generated like this - 1. Fire SQL and store the formatted output in a temp file echo "select path, empid, age from emp_tbl" | /usr/sql emp_db 2 > count_file | grep vol > tempFile 2. The tempFile looks like this after the above statement /vol/emp1 0732 ... (9 Replies)
Discussion started by: angshuman_ag
9 Replies

10. Shell Programming and Scripting

Searching for file and stopping at first item found

Hello, I try to write a shell script that would list all files on a directory and stop when it finds the first item specified on a find or ls command. How can I tell to the find or ls command to stop when it finds the first ".doc" file for example ? Thank you (7 Replies)
Discussion started by: davchris
7 Replies
Login or Register to Ask a Question