Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk script: print line number n of another file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk script: print line number n of another file
# 1  
Old 02-10-2010
awk script: print line number n of another file
Hi,

I wrote an awk script to analyse file A.
I call the script with files A and B. File A has lines like:

Code:
000000033100001
000000036100001
000000039100001

The first 9 characters are interpreted as a line number;
for each line number found I want to output this line number of file B.

searching around my idea was to use sed to output the line number, and system call sed inside awk...but the script below does not work, the call to sed fails. maybe just because the line_nr variable isnt correctley passed.
Can someone help me how to correctely write the call to sed or solve the problem inside awk?

regards
kp

Code:
awk 'BEGIN {
    printf("line %5s result code %12s record_id\n","","")}
{
    line_nr=substr($1,0,9);
    res_code=substr($1,10,3);
    rec_id=substr($1,13,3);
    printf("%09i  ", line_nr);
    printf("%03i %-20s ", res_code, rescode[res_code+0]);
    printf("%03i %-40s\n", rec_id, recid[rec_id+0]); # +0 needed to convert string rec_id to number
    system("sed -n \"line_nr p\" $2");
}' $1


---------- Post updated at 10:26 AM ---------- Previous update was at 10:15 AM ----------

ok, shame on me for finding this too late. the passing of variables to the system call is done via sysstring; the second file i passed via "-v sourcefile=$2" to awk:
Code:
 sysstring=sprintf("sed -n \"%i p\" %s", line_nr, sourcefile);
 system(sysstring);

so, the modified awk script works. the question remains: possible without system call?
Last edited by Scott; 02-10-2010 at 11:33 AM.. Reason: Please use code tags
# 2  
Old 02-10-2010
it is more simpler if you use associative array below:-

Note:- in solaris use nawk instead of awk.

Code:
awk 'NR==FNR{a[ int( substr($0,1,9) )] ; next}(FNR in a)' fileA fileB

BR

SmilieSmilieSmilie
This User Gave Thanks to ahmad.diab For This Post:
yahyaaa
# 3  
Old 02-16-2010
Quote:
Originally Posted by ahmad.diab

Code:
awk 'NR==FNR{a[ int( substr($0,1,9) )] ; next}(FNR in a)' fileA fileB

SmilieSmilieSmilie
Thank you! This is indeed very cool awk - I am still struggling how it works, how to use it and add additional code.Smilie

How is the performance of this code? My files have about 2,5 million lines and about 2 GB size.
Thank you
# 4  
Old 02-16-2010
will handle any size of lines without any problem test it. I think awk only have limitation on number of columns depend on your system -not sure about this info.-


BR
SmilieSmilieSmilie
# 5  
Old 02-19-2010
ahmad.diab, I am sorry, but I do not understand how this code works. I need to modify it

Code:
awk 'BEGIN { ...set some variables
}

{ # main
# set variables based on substrings of $1, e.g 
line_nr=int(substr($1,0,9));

# print something
# and then want awk to print the "matching" line in fileB
}'


I tried a lot of integrating this with your code, but ended with syntax errors.
My approach with system calls works, but yours is performing a lot better,
Any help appreciated.

with no luck.
# 6  
Old 03-10-2010
Quote:
Originally Posted by kpg
ahmad.diab, I am sorry, but I do not understand how this code works. I need to modify it

Code:
awk 'BEGIN { ...set some variables
}

{ # main
# set variables based on substrings of $1, e.g 
line_nr=int(substr($1,0,9));

# print something
# and then want awk to print the "matching" line in fileB
}'


I tried a lot of integrating this with your code, but ended with syntax errors.
My approach with system calls works, but yours is performing a lot better,
Any help appreciated.

with no luck.

may be you mean below:-
first of all the first index of substr is starting from 1 not 0.I think this was your mistake...after your modification find below:-

Code:
gawk 'BEGIN{} NR==FNR{line_num=int( substr($0,1,9) ) ; a[line_num] ; next}(FNR in a)' fileA fileB

you can use nawk,/usr/xpg4/bin/awk on solaris or gawk on other systems.

SmilieSmilieSmilie
# 7  
Old 03-10-2010
Thank you for your reply. I'll try to explain what I want to do:

fileA is:
000000011111000
000000329100001
000000330100001

the contents indicate line numbers (position 0 to 8) to look for in fileB
fileB is:
dataline1
dataline2
...
dataline32
dataline33
...


Now the output of my awk should be for each line in fileA
  • prettyfied line from fileA. eg print "error on line" line_num
  • prettyfied matching line (dataline line_num in fileb)

I just do not understand how to modify your code to achieve this.Smilie
Code:
1 gawk '
2 BEGIN{} 
3 NR==FNR{
4   line_num=int( substr($0,1,9) ) ; 
5   a[line_num] ;
6   next}
7  (FNR in a)
8 ' fileA fileB

at line 2 i can insert my initialization stuff.
#3. builds the array?
#4 sets the line_num variable;
#5 no idea what this does
#6 a search said "next means nothing else is done with this line from the first file"
#7 no idea

Maybe you can answer my questions,please?
Is it possible to let awk print a mixed output from fileA and matching lines in fileB

Thank you very much!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to find number in a field then print the line and the number

Hi I want to use awk to match where field 3 contains a number within string - then print the line and just the number as a new field. The source file is pipe delimited and looks something like 1|net|ABC Letr1|1530||| 1|net|EXP_1040 ABC|1121||| 1|net|EXP_TG1224|1122||| 1|net|R_North|1123|||... (5 Replies)
Discussion started by: Mudshark
5 Replies

2. Shell Programming and Scripting

Match pattern and print the line number of occurence using awk

Hi, I have a simple problem but i guess stupid enough to figure it out. i have thousands rows of data. and i need to find match patterns of two columns and print the number of rows. for example: inputfile abd abp 123 abc abc 325 ndc ndc 451 mjk lkj... (3 Replies)
Discussion started by: redse171
3 Replies

3. Shell Programming and Scripting

awk script -print line when $2 > $2 of previous line

Hi all, From a while loop I am reading a sorted file where I want to print only the lines that have $1 match and $2 only when the difference from $2 from the previous line is > 30. Input would be like ... AN237 010 193019 0502 1 CSU Amoxycillin AN237 080 ... (2 Replies)
Discussion started by: gafoleyo73
2 Replies

4. Shell Programming and Scripting

find string nth occurrence in file and print line number

Hi I have requirement to find nth occurrence in a file and capture data from with in lines (between lines) Data in File. <QUOTE> <SESSION> <ATTRIBUTE NAME='Parameter Filename' VALUE='file1.parm'/> <ATTRIBUTE NAME='Service Name' VALUE='None'/> </SESSION> <SESSION> <ATTRIBUTE... (6 Replies)
Discussion started by: tmalik79
6 Replies

5. Shell Programming and Scripting

awk/sed script to print each line to a separate named file

I have a large 3479 line .csv file, the content of which looks likes this: 1;0;177;170;Guadeloupe;x 2;127;171;179;Antigua and Barbuda;x 3;170;144;2;Umpqua;x 4;170;126;162;Coos Bay;x ... 1205;46;2;244;Unmak Island;x 1206;47;2;248;Yunaska Island;x 1207;0;2;240;north sea;x... (5 Replies)
Discussion started by: kalelovil
5 Replies

6. Shell Programming and Scripting

Help on Sed/awk/getting line number from file

I Have file1 with below lines : #HostNameSelection=0 :NotUsed #HostNameSelection=1 :Automatic #HostNameSelection=3 :NotForced I have file2 which has similar lines but with different values I want to copy the changes from file1 to file2 ,line by line only if line begins with '#'. for... (7 Replies)
Discussion started by: mvr
7 Replies

7. Shell Programming and Scripting

awk script to return the middle line number

I need an awk script that returns the 1st field of the line in the middle of a file. For example, if a file I have has 6 lines, I want it to return the 1st field on line number 3. If a file has 7 lines, I want the script to return the 1st field on line number 4. File1: 3 214 4 219 5 226 2... (8 Replies)
Discussion started by: jontjioe
8 Replies

8. Shell Programming and Scripting

sed script - print the line number along with the line

Hi, I want to print the line number with the pattern of the line on a same line using multi-patterns in sed. But i don't know how to do it. For example, I have a file abc def ghi I want to print 1 abc 2 def 3 ghi I know how to write it one line code, but i don't know how to put... (11 Replies)
Discussion started by: ntpntp
11 Replies

9. Shell Programming and Scripting

awk/shell script to print each line to a file

Dear People, My query is: have a file, which looks likes this: 10 20 30 40 50 1 2 3 4 5 100 200 300 400 500 what i need is: "PRINT EACH LINE TO AN UNIQUE FILE" desired output: file 1 10 20 30 40 50 file 2 1 2 3 4 5 (3 Replies)
Discussion started by: saint2006
3 Replies

10. Shell Programming and Scripting

shell script(Preferably awk or sed) to print selected number of columns from each row

Hi Experts, The question may look very silly by seeing the title, but please have a look at it clearly. I have a text file where the first 5 columns in each row were supposed to be attributes of a sample(like sample name, number, status etc) and the next 25 columns are parameters on which... (3 Replies)
Discussion started by: ks_reddy
3 Replies
Login or Register to Ask a Question