Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need the line number of failed records

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need the line number of failed records
# 1  
Old 12-19-2011
Need the line number of failed records
Hi awk Gurus,

I have file as below :

Code:
file1.txt

7000,2,1,6
7001,2,1,7
7002,2,1,6
7003,1,2,1

file2.txt

7000,john,2,0,0,1,6
7000,john,2,0,0,1,7
7000,john,2,0,0,1,8
7000,john,2,0,0,1,9
7001,elen,2,0,0,1,7
7002,sami,2,0,0,1,6
7003,mike,1,0,0,2,1
7003,mike,1,0,0,2,2
7003,mike,1,0,0,2,3
7003,mike,1,0,0,2,4
8001,nike,1,2,4,1,8
8002,paul,2,0,0,2,7

I needed the output such that, for the ones in file2 which is not present in file1 is displayed. I have the awk as below which performs the operation :
Code:
awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4])' OFS="," file1.txt file2.txt

the Output is as below

Code:
7000,john,2,0,0,1,7
7000,john,2,0,0,1,8
7000,john,2,0,0,1,9
7003,mike,1,0,0,2,2
7003,mike,1,0,0,2,3
7003,mike,1,0,0,2,4
8001,nike,1,2,4,1,8
8002,paul,2,0,0,2,7

Now, i need the output such that along with above, the line numbers of non matching lines of file2.txt is also displayed. I am struck here. Can someone please help ?

Thanks,
Arun
Last edited by zxmaus; 12-19-2011 at 11:15 AM.. Reason: added more tags
# 2  
Old 12-19-2011
try
Code:
awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4]); END{for(X in a) N++; print N }' OFS="," file1.txt file2.txt

# 3  
Old 12-19-2011
Can't you just add FNR to your output?

Code:
awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4]) {print FNR OFS $0}' OFS="," file1.txt file2.txt

Last edited by CarloM; 12-20-2011 at 06:20 AM..
This User Gave Thanks to CarloM For This Post:
arunshankar.c
# 4  
Old 12-19-2011
Code:
=> awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4]); END{for(X in a) N++; print N }' OFS=","  1.txt 2.txt
7000,john,2,0,0,1,6
7000,john,2,0,0,1,7
7000,john,2,0,0,1,8
7000,john,2,0,0,1,9
7001,elen,2,0,0,1,7
7002,sami,2,0,0,1,6
7003,mike,1,0,0,2,1
7003,mike,1,0,0,2,2
7003,mike,1,0,0,2,3
7003,mike,1,0,0,2,4
8001,nike,1,2,4,1,8
8002,paul,2,0,0,2,7
10

Above is the output, i.e. the file2.txt is getting displayed. I actually needed the line number of failed lines. Please suggest

---------- Post updated at 08:55 PM ---------- Previous update was at 08:44 PM ----------

Quote:
Originally Posted by CarloM
Can't you just add FNR to your output?

Code:
awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4]) {print FNR FS $0}' OFS="," file1.txt file2.txt

This will print all the lines :
Code:
=> awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;next} !(a[$1,$2,$3,$4]) {print FNR FS $0}' OFS="," 1.txt 2.txt
1,7000,john,2,0,0,1,6
2,7000,john,2,0,0,1,7
3,7000,john,2,0,0,1,8
4,7000,john,2,0,0,1,9
5,7001,elen,2,0,0,1,7
6,7002,sami,2,0,0,1,6
7,7003,mike,1,0,0,2,1
8,7003,mike,1,0,0,2,2
9,7003,mike,1,0,0,2,3
10,7003,mike,1,0,0,2,4
11,8001,nike,1,2,4,1,8
12,8002,paul,2,0,0,2,7

Last edited by Scott; 12-19-2011 at 11:37 AM.. Reason: Code tags
# 5  
Old 12-19-2011
Quote:
Originally Posted by arunshankar.c
This will print all the lines :
I only added FNR, the rest is your original code - which you said 'performs the operation', so I assumed it was correct Smilie.

EDIT: Shouldn't !(a[$1,$2,$3,$4]) be !(a[$1,$3,$6,$7)?

EDIT2: Minor bug in my addition - FS should really be OFS (although they're the same in this case).
Last edited by CarloM; 12-19-2011 at 11:36 AM..
This User Gave Thanks to CarloM For This Post:
arunshankar.c
# 6  
Old 12-19-2011
Quote:
Originally Posted by CarloM
I only added FNR, the rest is your original code - which you said 'performs the operation', so I assumed it was correct Smilie.

EDIT: Shouldn't !(a[$1,$2,$3,$4]) be !(a[$1,$3,$6,$7)?
I am sorry, my mistake.
Code:
=> awk -F',' -v file1="$1" -v file2="$2" 'NR == FNR {a[$1,$2,$3,$4]++;count++;next} !(a[$1,$3,$6,$7]){print FNR FS $0}' OFS="," 1.txt 2.txt
2,7000,john,2,0,0,1,7
3,7000,john,2,0,0,1,8
4,7000,john,2,0,0,1,9
8,7003,mike,1,0,0,2,2
9,7003,mike,1,0,0,2,3
10,7003,mike,1,0,0,2,4
11,8001,nike,1,2,4,1,8
12,8002,paul,2,0,0,2,7

It works great Smilie Thank you.
Last edited by Scott; 12-19-2011 at 11:37 AM.. Reason: Code tags, please...
# 7  
Old 12-19-2011
Optional code
Why not try a useful unix code use which could open up many options..
Play with this...
Code:
Cat $file2 | awk-vfile1="$file1" -vFS="," '{ sprintf(cmd,"grep %s %s",$1,file1); result=system(cmd); if (result != 0){print $0;}}'

Last edited by Franklin52; 12-20-2011 at 08:11 AM.. Reason: Please use code tags for code and data samples, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

2. Shell Programming and Scripting

adding line number to *end* of records in file

Given a file like this: abc def ghi I need to get to somestandardtext abc1 morestandardtext somestandardtext def2 morestandardtext somestandardtext ghi3 morestandardtext Notice that in addition to the standard text there is the line number added in as well. What I conceived is... (4 Replies)
Discussion started by: edstevens
4 Replies

3. Shell Programming and Scripting

AWK print number of records, divide this number

I would like to print the number of records of 2 files, and divide the two numbers awk '{print NR}' file1 > output1 awk '{print NR}' file2 > output2 paste output1 output2 > output awl '{print $1/$2}' output > output_2 is there a faster way? (8 Replies)
Discussion started by: programmerc
8 Replies

4. Shell Programming and Scripting

Write $line number into textfile and read from line number

Hello everyone, I don't really know anything about scripting, but I have to manage to make this script, out of necessity. #!/bin/bash while read -r line; do #I'm reading from a big wordlist instructions using $line done Is there a way to automatically write the $line number the script... (4 Replies)
Discussion started by: bobylapointe
4 Replies

5. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

6. UNIX for Dummies Questions & Answers

How to read contents of a file from a given line number upto line number again specified by user

Hello Everyone. I am trying to display contains of a file from a specific line to a specific line(let say, from line number 3 to line number 5). For this I got the shell script as shown below: if ; then if ; then tail +$1 $3 | head -n $2 else ... (5 Replies)
Discussion started by: grc
5 Replies

7. Shell Programming and Scripting

how to get the data from line number 1 to line number 100 of a file

Hi Everybody, I am trying to write a script that will get some perticuler data from a file and redirect to a file. My Question is, I have a Very huge file,In that file I have my required data is started from 25th line and it will ends in 100th line. I know the line numbers, I need to get all... (9 Replies)
Discussion started by: Anji
9 Replies

8. Shell Programming and Scripting

Adding a columnfrom a specifit line number to a specific line number

Hi, I have a huge file & I want to add a specific text in column. But I want to add this text from a specific line number to a specific line number & another text in to another range of line numbers. To be more specific: lets say my file has 1000 lines & 4 Columns. I want to add text "Hello"... (2 Replies)
Discussion started by: Ezy
2 Replies

9. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies

10. UNIX for Advanced & Expert Users

delete records using line number(NR)

Hai I have a flat file which contains more than 6 crore lines or records. I want to delete only one line, using line number. For example I want to delete 414556 th line . How to do this using sed or awk command. thanks (3 Replies)
Discussion started by: tkbharani
3 Replies
Login or Register to Ask a Question