delete multiple lines by line number


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers delete multiple lines by line number
# 8  
Old 10-24-2008
Hammer & Screwdriver Strange, but working unix command solution (no sed or awk)

Code:
> cat file305
sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd
sdfgsdfg
asdfiojhsdf asdoludhflsdjfhskldjfhsdjdlfsjdhnlj h sdja
ouahsdjdafkljsa
oljhljh

> cat file305x
5
3

> tr " " "~" <file305 | cat -n | tr -d " " | egrep -v -f file305x | tr "~" " " | tr "\t" "~" | cut -d"~" -f2
sadfsdf sdfosuidhfousdhof soduhf osdfu osudfhosudhfd
sdfgsdfg
ouahsdjdafkljsa

# 9  
Old 10-24-2008
So this works fine:

awk ' FILENAME=="tmp_memrecno.dat" { arr[$0]=1}
FILENAME=="test.out" {
if( FNR in arr){ continue}
print $0
} ' tmp_memrecno.dat test.out > tmp.newfile


but when I try to pass the files as variables, the code breaks. Could you show me the syntax for setting and calling tmp_memrecno.dat and test.out files as variables for this code?

Thanks,

- CB
# 10  
Old 10-26-2008
Thanks Joeyg... I love one-liners (I hope the performance is better than the awk code, because we are talking 800 million records from which around 2000+ records will be rejected.)

- CB
# 11  
Old 10-26-2008
Hi.

This makes a single pass over the data file:
Code:
#!/usr/bin/env sh

# @(#) s2       Demonstrate creation of sed script for single-pass, delete lines.

set -o nounset

## The shebang using "env" line is designed for portability. For
#  higher security, use:
#
#  #!/bin/sh -

## Use local command version for the commands in this demonstration.

echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version bash sed
echo

# Create the line number test file, can be in any order.

cat >numbers <<EOF
4
7
3
EOF

# Create the test data file.

cat >data1 <<EOF
one
two
three
four
five
six
seven
eight
nine
ten
EOF

# Create the sed script file with sed itself:

sed 's|$|d|' numbers >script

# Now efficiently run sed once instead of n times, display the
# data.

sed -f script data1

exit 0

Producing:
Code:
% ./s2
(Versions displayed with local utility "version")
GNU bash 2.05b.0
GNU sed version 4.1.2

one
two
five
six
eight
nine
ten

This is essentially the same as the script in post # 6 of thread https://www.unix.com/shell-programmin...#post302154933
except that the operation is delete instead of print ... cheers, drl
# 12  
Old 10-30-2008
drl, your solution is by far the most efficient.

- CB
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

2. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

3. UNIX for Dummies Questions & Answers

How to delete lines above a certin line number in bash shell

Hi, I have written a script that returns the line number of the pattern i want and i stored the line number in a variable.Now i want to delete all the lines in a file above this line number which is stored in a variable. i am using sed '1,$getlinenumberd' > file1.txt which is not working(wrog... (5 Replies)
Discussion started by: learninguser235
5 Replies

4. UNIX for Dummies Questions & Answers

How to delete lines above a certin line number in bash shell

Hi, I have written a script that returns the line number of the pattern i want and i stored the line number in a variable(getlinenumber).Now i want to delete all the lines in a file above this line number which is stored in a variable. i am using sed '1,$getlinenumberd' > file1.txt which is... (2 Replies)
Discussion started by: learninguser235
2 Replies

5. Shell Programming and Scripting

search and replace, when found, delete multiple lines, add new set of lines?

hey guys, I tried searching but most 'search and replace' questions are related to one liners. Say I have a file to be replaced that has the following: $ cat testing.txt TESTING AAA BBB CCC DDD EEE FFF GGG HHH ENDTESTING This is the input file: (3 Replies)
Discussion started by: DeuceLee
3 Replies

6. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

7. Shell Programming and Scripting

How to delete several lines from file by line number?

Hi I am using the following command to delete a line from the file by line number: line_number=14 sed "${line_number}d" inputfilename > newfilename Is there a way to modify this command to specify the range of lines to be deleted, lets say from line 14 till line 5 ? I tried using the... (5 Replies)
Discussion started by: aoussenko
5 Replies

8. Shell Programming and Scripting

Delete lines based on line number

I have a file with ~200K lines, I need to delete 4K lines in it. There is no range. I do have the line numbers of the lines which I want to be deleted. I did tried using > cat del.lines sed '510d;12d;219d;......;3999d' file > source del.lines Word too long. I even tried... (2 Replies)
Discussion started by: novice_man
2 Replies

9. Shell Programming and Scripting

delete multiple lines by line number

I have file with 10000 records and i need to delete the lines in single shot based on line number range say from 10 to 51 , 53 to 59 , 105 to 107, 311 to 592 etc... between range works fine for me but how to achive for above case? please help sed '10,51 {d}' infile > outfile (5 Replies)
Discussion started by: zooby
5 Replies

10. Shell Programming and Scripting

Appending line number to each line and getting total number of lines

Hello, I need help in appending the line number of each line to the file and also to get the total number of lines. Can somebody please help me. I have a file say: abc def ccc ddd ffff The output should be: Instance1=abc Instance2=def Instance3=ccc Instance4=ddd Instance5=ffff ... (2 Replies)
Discussion started by: chiru_h
2 Replies
Login or Register to Ask a Question