08-04-2011
fish out lines according to line number
Dear all,
I've a situation here, I have a big file which contains about 20Million line as shown as below
Quote:
>aaaaaa LKJSALDJLKDAJDALKJ
>aaaaab LKJSALDJLKDAJDALKJ
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaae JSLJLDJDALKJAKLKALKSLKSAL
>aaaaaf KLJLKAJLKSJLKSJLKAJSLKS
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
>..... ...... (20 million th line)
I want to fish out some lines according to the line position of the big file, for example here line number 3,4,7 ...
so the expected output will be like this
Quote:
>aaaaac KLJLKAJDLKJAKLJDLL
>aaaaad LKJLKADJLJDKLJLKJADLK
>aaaaag KLJALJSKSALKSAJSLKAJSLKS
.
.
.
.
.
currently I manage to fish out the sequence by creating scripting using sed command
Quote:
for f in `cat line.number` ; do sed -n '$fp' bigfile >> output ;done # line.number is the file contain the line number should be fished out from the big file
However i have millions of lines to be fished out. Using sed command as above takes me days to get the job done. This is because the sed will read from start of the line until end and it continue for each search.
Is there any other way to make things done faster.
Thanks in advance
kamal
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have a file that has 1k lines and i want to print all the lines after 900th line.
an 2)I want to move files f1 ,f2,f3,f4 to p1,p2,p3,p4
Please give me the commands.
Thanx in adv. (6 Replies)
Discussion started by: rajashekar.y
6 Replies
2. Shell Programming and Scripting
Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me.
I have a file say:
abc
def
ccc
ddd
ffff
The output should be:
Instance1=abc
Instance2=def
Instance3=ccc
Instance4=ddd
Instance5=ffff
... (2 Replies)
Discussion started by: chiru_h
2 Replies
3. UNIX for Dummies Questions & Answers
I have been googling, but cannot find that works for me.
I have a text file tmp.out with contents:
sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd
sdfgsdfg
asdfiojhsdf asdoludhflsdjfhskldjfhsdjdlfsjdhnlj h sdja
ouahsdjdafkljsa
oljhljh
I have another file... (11 Replies)
Discussion started by: ChicagoBlues
11 Replies
4. Shell Programming and Scripting
I have a set of log files that are in the following format
======= set_1 ========
counter : 315
counter2: 204597
counter3: 290582
======= set_2 ========
counter : 315
counter2: 204597
counter3: 290582
======= set_3 ========
counter : 315
counter2: 204597
counter3: 290582
Is... (6 Replies)
Discussion started by: grandguest
6 Replies
5. Shell Programming and Scripting
I have a file with ~200K lines, I need to delete 4K lines in it. There is no range.
I do have the line numbers of the lines which I want to be deleted.
I did tried using
> cat del.lines
sed '510d;12d;219d;......;3999d' file
> source del.lines
Word too long.
I even tried... (2 Replies)
Discussion started by: novice_man
2 Replies
6. Shell Programming and Scripting
Hi
I am using the following command to delete a line from the file by line number:
line_number=14
sed "${line_number}d" inputfilename > newfilename
Is there a way to modify this command to specify the range of lines to be deleted, lets say from line 14 till line 5 ?
I tried using the... (5 Replies)
Discussion started by: aoussenko
5 Replies
7. Shell Programming and Scripting
Hi,
Anyone help me to print the lines from the flat file between 879th line number and 1424th line number.
The 879 and 1424 should be passed as input to the shell script(It should be dynamic).
Can any one give me using sed or awk?
I tried using read, and print the lines..Its taking too... (3 Replies)
Discussion started by: senthil_is
3 Replies
8. Shell Programming and Scripting
Hi All,
I have a file as below:
abc.txt
******************************
* HEADER DESCRIPTION
******************************
*Supplier: Prism
Customer: MNI -NIGERIA
Quantity: 2
Type: PLUG-IN
Profile: 70.00
*Subscription: Generic... (5 Replies)
Discussion started by: arunshankar.c
5 Replies
9. Shell Programming and Scripting
Hi all,
I have a file sample.txt
abc
asd
adf
daf
adw
add
adv
wdf
I want to control the number of lines to read
Like if i give input as ./script_name 2 5
required output
asd
adf
daf (2 Replies)
Discussion started by: krux_rap
2 Replies
10. Shell Programming and Scripting
hi,
How can i count the number of lines after the first line in a flat file in unix?
Say i have a flat file with a header like:
Student Name Student ID ....
Tnx (7 Replies)
Discussion started by: reignangel2003
7 Replies
hp(1) General Commands Manual hp(1)
NAME
hp - handle special functions of HP2640 and HP2621-series terminals
SYNOPSIS
DESCRIPTION
supports special functions of the Hewlett-Packard HP 2640 and HP 2621 series of terminals, with the primary purpose of producing accurate
representations of most output. A typical use is:
Regardless of the hardware options on a given terminal, tries to do sensible things with underlining and reverse line-feeds. If the termi-
nal has the ``display enhancements'' feature, subscripts and superscripts can be indicated in distinct ways. If it has the ``mathematical-
symbol'' feature, Greek and other special characters can be displayed.
Options
recognizes the following options:
Specify that your terminal has the ``display enhancements'' feature,
to make maximal use of the added display modes. Overstruck characters are presented in the Underline mode. Superscripts
are shown in Half-bright mode, and subscripts in Half-bright, Underlined mode. If this flag is omitted, assumes that your
terminal lacks the ``display enhancements'' feature. In this case, all overstruck characters, subscripts, and super-
scripts are displayed in Inverse Video mode; that is, dark-on-light, rather than light-on-dark.
Request minimization of output by removing new-lines.
Any contiguous sequence of 3 or more new-lines is converted into a sequence of only 2 new-lines; that is, any number of
successive blank lines produces only a single blank output line. This allows you to retain more actual text on the
screen.
DIAGNOSTICS
The representation of a line exceeds 1,024 characters.
RETURN VALUE
returns zero for normal termination, and 2 for all errors.
WARNINGS
An ``overstriking sequence'' is defined as a printing character followed by a backspace followed by another printing character. In such
sequences, if either printing character is an underscore, the other printing character is shown underlined or in Inverse Video; otherwise,
only the first printing character is shown (again, underlined or in Inverse Video). Nothing special is done if a backspace is adjacent to
an ASCII control character. Sequences of control characters (e.g., reverse line-feeds, backspaces) can make text ``disappear''; in partic-
ular, tables generated by that contain vertical lines will often be missing the lines of text that contain the ``foot'' of a vertical line,
unless the input to is piped through (see col(1)).
Although some terminals do support numerical superscript characters, no attempt is made to display them.
SEE ALSO
col(1), neqn(1), nroff(1), tbl(1).
hp(1)