fish out lines according to line number


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting fish out lines according to line number
# 1  
Old 08-04-2011
fish out lines according to line number

Dear all,

I've a situation here, I have a big file which contains about 20Million line as shown as below

Quote:
>aaaaaa LKJSALDJLKDAJDALKJ
>aaaaab LKJSALDJLKDAJDALKJ
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaae JSLJLDJDALKJAKLKALKSLKSAL
>aaaaaf KLJLKAJLKSJLKSJLKAJSLKS
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
>..... ...... (20 million th line)
I want to fish out some lines according to the line position of the big file, for example here line number 3,4,7 ...
so the expected output will be like this

Quote:
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
currently I manage to fish out the sequence by creating scripting using sed command
Quote:
for f in `cat line.number` ; do sed -n '$fp' bigfile >> output ;done # line.number is the file contain the line number should be fished out from the big file
However i have millions of lines to be fished out. Using sed command as above takes me days to get the job done. This is because the sed will read from start of the line until end and it continue for each search.

Is there any other way to make things done faster.

Thanks in advance

kamal
# 2  
Old 08-04-2011
Assuming your numbers are in the file "line.numbers" like so
3
4
7
Code:
perl -lne '                                                                                                                                               
BEGIN {
  open $fnums, "<" , shift or die $!;
  @nums =  <$fnums>;
  chomp @nums;
}
$. == $nums[0] && print && shift @nums;
' line.numbers INPUTFILE

This User Gave Thanks to yazu For This Post:
# 3  
Old 08-05-2011
PERFECT!!!!

Thanks a lot Yuzu. You are really great. Your script takes only minutes to finish the job.

Hope I can be as good as this.

Thanks again
# 4  
Old 08-05-2011
Could this help you?
Code:
exp="";for i in 3 4 7; do exp=$exp$i"p;"; done;sed -n "$exp" inputfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Want to count the number of lines after the first line

hi, How can i count the number of lines after the first line in a flat file in unix? Say i have a flat file with a header like: Student Name Student ID .... Tnx (7 Replies)
Discussion started by: reignangel2003
7 Replies

2. Shell Programming and Scripting

Read line with particular number of lines

Hi all, I have a file sample.txt abc asd adf daf adw add adv wdf I want to control the number of lines to read Like if i give input as ./script_name 2 5 required output asd adf daf (2 Replies)
Discussion started by: krux_rap
2 Replies

3. Shell Programming and Scripting

Get the number of lines till I get line

Hi All, I have a file as below: abc.txt ****************************** * HEADER DESCRIPTION ****************************** *Supplier: Prism Customer: MNI -NIGERIA Quantity: 2 Type: PLUG-IN Profile: 70.00 *Subscription: Generic... (5 Replies)
Discussion started by: arunshankar.c
5 Replies

4. Shell Programming and Scripting

print lines between line number

Hi, Anyone help me to print the lines from the flat file between 879th line number and 1424th line number. The 879 and 1424 should be passed as input to the shell script(It should be dynamic). Can any one give me using sed or awk? I tried using read, and print the lines..Its taking too... (3 Replies)
Discussion started by: senthil_is
3 Replies

5. Shell Programming and Scripting

How to delete several lines from file by line number?

Hi I am using the following command to delete a line from the file by line number: line_number=14 sed "${line_number}d" inputfilename > newfilename Is there a way to modify this command to specify the range of lines to be deleted, lets say from line 14 till line 5 ? I tried using the... (5 Replies)
Discussion started by: aoussenko
5 Replies

6. Shell Programming and Scripting

Delete lines based on line number

I have a file with ~200K lines, I need to delete 4K lines in it. There is no range. I do have the line numbers of the lines which I want to be deleted. I did tried using > cat del.lines sed '510d;12d;219d;......;3999d' file > source del.lines Word too long. I even tried... (2 Replies)
Discussion started by: novice_man
2 Replies

7. Shell Programming and Scripting

add number in lines line by line in different files

I have a set of log files that are in the following format ======= set_1 ======== counter : 315 counter2: 204597 counter3: 290582 ======= set_2 ======== counter : 315 counter2: 204597 counter3: 290582 ======= set_3 ======== counter : 315 counter2: 204597 counter3: 290582 Is... (6 Replies)
Discussion started by: grandguest
6 Replies

8. UNIX for Dummies Questions & Answers

delete multiple lines by line number

I have been googling, but cannot find that works for me. I have a text file tmp.out with contents: sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd sdfgsdfg asdfiojhsdf asdoludhflsdjfhskldjfhsdjdlfsjdhnlj h sdja ouahsdjdafkljsa oljhljh I have another file... (11 Replies)
Discussion started by: ChicagoBlues
11 Replies

9. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies

10. UNIX for Dummies Questions & Answers

display lines after a particular line number

I have a file that has 1k lines and i want to print all the lines after 900th line. an 2)I want to move files f1 ,f2,f3,f4 to p1,p2,p3,p4 Please give me the commands. Thanx in adv. (6 Replies)
Discussion started by: rajashekar.y
6 Replies
Login or Register to Ask a Question