How to read file line by line and compare subset of 1st line with 2nd?

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to read file line by line and compare subset of 1st line with 2nd?
# 1  
Old 11-24-2014
How to read file line by line and compare subset of 1st line with 2nd?

Hi all,

I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like:

What I want to do is compare first line with second, second with third and so on to see if url1 is same as url2, url2 is same as url3 and so on.

I am trying to do this by copying the details of Test.log in a file File1.txt (I am going to do this periodically, maybe few times a day).
Then read File1.txt line by line and get url1 in a variable say Var1 by using
cut -d '|' -f2 <line1 of File1.txt>

and somehow be able to compare it with variable Var2 which I am hoping to get by using
cut -d '|' -f2 <line2 of File1.txt>


Problem's I am facing here are -
1) Cut is treating each space separated word as a field while I try to use it while reading File1.txt line by line in a loop like:
 while i in `cat File1.txt`
         cut -d '|' -f2 $i


 while read i
         cut -d '|' -f2 $i
     done < File1.txt

2) I have no idea how to be able to compare url1 with url2, url2 with url3 and so on.

Can anyone please help me here if what I am trying is even possible and if it is then how can I do it.

Thanks in advance.

Last edited by Scrutinizer; 11-24-2014 at 05:18 PM.. Reason: Additional code tags
# 2  
Old 11-24-2014
Why don't you read the file, using "|" as the IFS character?
while IFS="|" read A B URL C
    do echo $URL
    done < file

In the loop, save URL to OLDURL, and compare e.g.
[ "$URL" != "$OLDURL" ] && echo different || echo equal

This User Gave Thanks to RudiC For This Post:
# 3  
Old 11-24-2014
If you want to compare the 3rd field of adjacent records:
awk FS="|" 'prev != $3 { print; } { prev = $3; }'

But result (output) is desired?
# 4  
Old 11-25-2014
Thanks for the replies RudiC and derekludwig.
I am trying both of your suggestions.

@derekludwig, result field from the log/file is just a field which is not important.
I just need to compare field url.

I am trying both the suggestions and will post the outcome here soon.

---------- Post updated at 01:11 PM ---------- Previous update was at 03:25 AM ----------

RudiC, I followed your suggestion and it worked perfect for me :-)
Thank you both RudiC and derekludwig
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Read two file and compare the line

Hello Guys I need to read and compare two file, one file contains hostname, and others contain hostname and IP@, and the objective is to read each file and compare if line in file1 equal to first word in the second file2 file1 file2 this is my first code fqdn_hosts=list.txt... (2 Replies)
Discussion started by: Abdellah
2 Replies

2. UNIX for Beginners Questions & Answers

Get an output of lines in pattern 1st line then 10th line then 11th line then 20th line and so on.

Input file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (6 Replies)
Discussion started by: Sagar Singh
6 Replies

3. UNIX for Beginners Questions & Answers

Compare 1st column from 2 file and if match print line from 1st file and append column 7 from 2nd

hi I have 2 file with more than 10 columns for both 1st file apple,0,0,0...... orange,1,2,3..... mango,2,4,5..... 2nd file apple,2,3,4,5,6,7... orange,2,3,4,5,6,8... watermerlon,2,3,4,5,6,abc... mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies

4. Shell Programming and Scripting

Shell script UNIX to read text file line by line

i have a text file as belows, it includes 2 columns, 1st is the column name, 2nd is the file_name data_file.txt column_name file_name col1 file1 col2 file2 col3 file1 col4 file1 col5 file2 now, i would like to... (4 Replies)
Discussion started by: tester111
4 Replies

5. Shell Programming and Scripting

Bash script to read a file from particular line till required line and process

Hi All, Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool. Wraper script am trying is to do with above metion 2 files. utility tool accepts : a. userinfo file : which contains username b. item file : which... (2 Replies)
Discussion started by: Optimus81
2 Replies

6. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

7. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

8. Shell Programming and Scripting

Shell script to read a text file line by line & process it...

Hi , I am trying to write an shell, which reads a text file (from a location) having a list of numbers of strictly 5 digits only ex: 33144 Now my script will check : 1) that each entry is only 5 digits & numeric only, no alphabets, & its not empty. 2)then it executes a shell script called... (8 Replies)
Discussion started by: new_to_shell
8 Replies

9. Shell Programming and Scripting

Read file line by line and process the line to generate another file

Hi, i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here) i need to read this file line by line and generate an output file like the one below i.e based on N value the number of MSISDNs will vary, if N=1 then the following... (14 Replies)
Discussion started by: aemunathan
14 Replies

10. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies
Login or Register to Ask a Question

Featured Tech Videos