Sponsored Content
Top Forums Shell Programming and Scripting fish out lines according to line number Post 302544497 by masterpiece on Thursday 4th of August 2011 03:59:40 AM
Old 08-04-2011
fish out lines according to line number

Dear all,

I've a situation here, I have a big file which contains about 20Million line as shown as below

Quote:
>aaaaaa LKJSALDJLKDAJDALKJ
>aaaaab LKJSALDJLKDAJDALKJ
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaae JSLJLDJDALKJAKLKALKSLKSAL
>aaaaaf KLJLKAJLKSJLKSJLKAJSLKS
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
>..... ...... (20 million th line)
I want to fish out some lines according to the line position of the big file, for example here line number 3,4,7 ...
so the expected output will be like this

Quote:
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
currently I manage to fish out the sequence by creating scripting using sed command
Quote:
for f in `cat line.number` ; do sed -n '$fp' bigfile >> output ;done # line.number is the file contain the line number should be fished out from the big file
However i have millions of lines to be fished out. Using sed command as above takes me days to get the job done. This is because the sed will read from start of the line until end and it continue for each search.

Is there any other way to make things done faster.

Thanks in advance

kamal
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

display lines after a particular line number

I have a file that has 1k lines and i want to print all the lines after 900th line. an 2)I want to move files f1 ,f2,f3,f4 to p1,p2,p3,p4 Please give me the commands. Thanx in adv. (6 Replies)
Discussion started by: rajashekar.y
6 Replies

2. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies

3. UNIX for Dummies Questions & Answers

delete multiple lines by line number

I have been googling, but cannot find that works for me. I have a text file tmp.out with contents: sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd sdfgsdfg asdfiojhsdf asdoludhflsdjfhskldjfhsdjdlfsjdhnlj h sdja ouahsdjdafkljsa oljhljh I have another file... (11 Replies)
Discussion started by: ChicagoBlues
11 Replies

4. Shell Programming and Scripting

add number in lines line by line in different files

I have a set of log files that are in the following format ======= set_1 ======== counter : 315 counter2: 204597 counter3: 290582 ======= set_2 ======== counter : 315 counter2: 204597 counter3: 290582 ======= set_3 ======== counter : 315 counter2: 204597 counter3: 290582 Is... (6 Replies)
Discussion started by: grandguest
6 Replies

5. Shell Programming and Scripting

Delete lines based on line number

I have a file with ~200K lines, I need to delete 4K lines in it. There is no range. I do have the line numbers of the lines which I want to be deleted. I did tried using > cat del.lines sed '510d;12d;219d;......;3999d' file > source del.lines Word too long. I even tried... (2 Replies)
Discussion started by: novice_man
2 Replies

6. Shell Programming and Scripting

How to delete several lines from file by line number?

Hi I am using the following command to delete a line from the file by line number: line_number=14 sed "${line_number}d" inputfilename > newfilename Is there a way to modify this command to specify the range of lines to be deleted, lets say from line 14 till line 5 ? I tried using the... (5 Replies)
Discussion started by: aoussenko
5 Replies

7. Shell Programming and Scripting

print lines between line number

Hi, Anyone help me to print the lines from the flat file between 879th line number and 1424th line number. The 879 and 1424 should be passed as input to the shell script(It should be dynamic). Can any one give me using sed or awk? I tried using read, and print the lines..Its taking too... (3 Replies)
Discussion started by: senthil_is
3 Replies

8. Shell Programming and Scripting

Get the number of lines till I get line

Hi All, I have a file as below: abc.txt ****************************** * HEADER DESCRIPTION ****************************** *Supplier: Prism Customer: MNI -NIGERIA Quantity: 2 Type: PLUG-IN Profile: 70.00 *Subscription: Generic... (5 Replies)
Discussion started by: arunshankar.c
5 Replies

9. Shell Programming and Scripting

Read line with particular number of lines

Hi all, I have a file sample.txt abc asd adf daf adw add adv wdf I want to control the number of lines to read Like if i give input as ./script_name 2 5 required output asd adf daf (2 Replies)
Discussion started by: krux_rap
2 Replies

10. Shell Programming and Scripting

Want to count the number of lines after the first line

hi, How can i count the number of lines after the first line in a flat file in unix? Say i have a flat file with a header like: Student Name Student ID .... Tnx (7 Replies)
Discussion started by: reignangel2003
7 Replies
hp(1)							      General Commands Manual							     hp(1)

NAME
hp - handle special functions of HP2640 and HP2621-series terminals SYNOPSIS
DESCRIPTION
supports special functions of the Hewlett-Packard HP 2640 and HP 2621 series of terminals, with the primary purpose of producing accurate representations of most output. A typical use is: Regardless of the hardware options on a given terminal, tries to do sensible things with underlining and reverse line-feeds. If the termi- nal has the ``display enhancements'' feature, subscripts and superscripts can be indicated in distinct ways. If it has the ``mathematical- symbol'' feature, Greek and other special characters can be displayed. Options recognizes the following options: Specify that your terminal has the ``display enhancements'' feature, to make maximal use of the added display modes. Overstruck characters are presented in the Underline mode. Superscripts are shown in Half-bright mode, and subscripts in Half-bright, Underlined mode. If this flag is omitted, assumes that your terminal lacks the ``display enhancements'' feature. In this case, all overstruck characters, subscripts, and super- scripts are displayed in Inverse Video mode; that is, dark-on-light, rather than light-on-dark. Request minimization of output by removing new-lines. Any contiguous sequence of 3 or more new-lines is converted into a sequence of only 2 new-lines; that is, any number of successive blank lines produces only a single blank output line. This allows you to retain more actual text on the screen. DIAGNOSTICS
The representation of a line exceeds 1,024 characters. RETURN VALUE
returns zero for normal termination, and 2 for all errors. WARNINGS
An ``overstriking sequence'' is defined as a printing character followed by a backspace followed by another printing character. In such sequences, if either printing character is an underscore, the other printing character is shown underlined or in Inverse Video; otherwise, only the first printing character is shown (again, underlined or in Inverse Video). Nothing special is done if a backspace is adjacent to an ASCII control character. Sequences of control characters (e.g., reverse line-feeds, backspaces) can make text ``disappear''; in partic- ular, tables generated by that contain vertical lines will often be missing the lines of text that contain the ``foot'' of a vertical line, unless the input to is piped through (see col(1)). Although some terminals do support numerical superscript characters, no attempt is made to display them. SEE ALSO
col(1), neqn(1), nroff(1), tbl(1). hp(1)
All times are GMT -4. The time now is 03:48 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy