Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Comparing multiple lines in same file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing multiple lines in same file
# 8  
Old 04-07-2016
Try:
Code:
awk '$NF~/\.[0-9]+\./{if(NF>1) {p=$2; s=$0; next} if(p && $1!=p) {print s RS $0; p=x}}' file

Last edited by Scrutinizer; 04-07-2016 at 02:03 PM..
This User Gave Thanks to Scrutinizer For This Post:
seekryts15
# 9  
Old 04-08-2016
Thank you again for all the help!

Is there a way to check if there are other duplicate entries in the entire file? For example line 1 and 2 numbers should match but if the same number appears anywhere else send an error message. Any help is greatly appreciated, thank you.
# 10  
Old 04-08-2016
Try
Code:
awk 'NR%2 {getline T;  X = $NF; sub (X "$", ""); if (X T !~ /[^0-9.]/) {if (IP[T]++) print "Dup:", T;  if (T != X) printf "%s %s\n%s\n", X, $0, T}}' file
Dup: 15.235.18.20
15.235.18.21 Nova 
15.234.18.21
Dup: 15.235.18.21
Dup: 15.235.18.21
18.66.20.17 Vermont 
18.66.2.17
Dup: 13.7.24.25
Dup: 18.55.7.25

# 11  
Old 04-08-2016
When I run the command in a small list I get the list of duplicates but if I run it in a larger list (same format) it doesn't return anything.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

2. UNIX for Dummies Questions & Answers

Comparing 2 column of a file with the lines in another file

Hi I have 2 files which are tab delimited. file1 chr1 5 10 P1 KR4 chr1 10 20 1H LA1 R0 TA1 file2 P1 G6 13.27 0.2425 P1 KR4 18.79 0.3060 P1 DND1 19.44 0.2833 N1 DH1 0.99 1.08 1H R0 0.9 NA LA1 R0 90 0.9 TA1 KR4 1.8 8.9 TA1 R0 9.7 99I want to check whether first 2 columns in file2 (column... (1 Reply)
Discussion started by: raj_k
1 Replies

3. Shell Programming and Scripting

Perl for comparing numbers from previous lines in a file?

Hi everyone I have a question for you, as I am trying to learn more about Perl and work with some weather data. I have an ascii file (shown below) that has 10 lines with different columns. What I would like is have Perl find an "anomalous" value by comparing a field with the values from the last... (2 Replies)
Discussion started by: lucshi09
2 Replies

4. Shell Programming and Scripting

Comparing 2 text files & downloading a file if the last lines are different

Hello I'm having a little difficulty in writing a shell script for a few simple tasks. First I have two files "file1.txt" and "file2.txt" and I want to read and compare the last line of each file. The files look like this. File1.txt File2.txt After comparing the two lines I would... (2 Replies)
Discussion started by: RustikGaming
2 Replies

5. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

7. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

8. Shell Programming and Scripting

comparing lines in file

i have 2 files and i want to compare i currently cat the files and awk print $1, $2 and doing if file1=file2 then fail, else exit 0 what i want to do is compare values, with column 1 being a reference i want to compare line by line and then still be able to do if then statement to see if worked... (1 Reply)
Discussion started by: sigh2010
1 Replies

9. Shell Programming and Scripting

Help with comparing 2 lines in 2 different file in shell bash

Hi guys i need help with comparing lines in 2 separate files. Both files contain the same amount of lines and i need to output the difference into the 2nd file. The 1st file is always correct. 1st file (Expected.e): Tuesday, 11 August 2009 Wednesday, 12 August 2009 Thursday, 13 August 2009... (2 Replies)
Discussion started by: kcrap
2 Replies

10. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies
Login or Register to Ask a Question