Visit Our UNIX and Linux User Community


Cheapest way to get line X from file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Cheapest way to get line X from file
# 1  
Old 10-30-2009
Cheapest way to get line X from file

So, I have a couple of scripts that go through lots and lots of data-- sometimes thousands of smaller files, sometimes a few very large ones. It's very common that I need a particular line, by line number X. There are lots of ways to do this:

Code:
awk 'NR == X {print ; exit}' file
sed -n 'Xp' file
head -n X file | tail -n 1

There are others, too... My question is, what's the most economical way you know to do this? In my primitive tests (involving a large file and the time command), head/tail seem to be the winner. What is surprising is how badly sed did. On a file with 4237905 lines, here are the results:

Code:
awk                   sed                     head/tail
real    0m1.258s      real    0m22.469s       real    0m1.011s
user    0m1.010s      user    0m22.170s       user    0m0.550s
sys     0m0.250s      sys     0m0.280s        sys     0m0.730s

I actually expected sed to win. Strange... Anyway, are there any other methods I should consider? Are there improvements to these methods? Thanks.
# 2  
Old 10-30-2009
try these from sed1liners:
Code:
 # print line number 52
 sed -n '52p'                 # method 1
 sed '52!d'                   # method 2
 sed '52q;d'                  # method 3, efficient on large files

 

Previous Thread | Next Thread
Test Your Knowledge in Computers #221
Difficulty: Easy
Apple's OS X (macOS) installed base had grown to 80 million users as reported at the WWDC 2014.
True or False?

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

2. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

3. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

4. Shell Programming and Scripting

Read file line by line and process the line to generate another file

Hi, i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here) i need to read this file line by line and generate an output file like the one below i.e based on N value the number of MSISDNs will vary, if N=1 then the following... (14 Replies)
Discussion started by: aemunathan
14 Replies

5. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

Featured Tech Videos