Visit Our UNIX and Linux User Community


Locate and remove lines with sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Locate and remove lines with sed
# 1  
Old 11-05-2012
Locate and remove lines with sed

Gents,

I would like to remove some lines from a big file ( file2).

The objetive is to remove all the lines in file2 containing a certain string which are in file data2delete..

file data2delete contens:

Code:
2573.0  7260.01
2893.0  7255.01
2903.0  7245.01
2897.0  7255.01
2561.0  7255.01



file2
Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64
A          2571.0  7260.01410 75   1  -2204470 77 41
A          2571.0  7260.014 6 75   2   5182570 77 37
A          2557.0  7255.013 9 75   1   5153071 77 67
A          2557.0  7255.013 5 75   1  -3162871 77 42
A          2895.0  7255.011 7 75   1  -3203071 77 41
A          2895.0  7255.011 1 75   1  -2174671 77 56
A          2573.0  7260.01410 75   1   2259968 77 38
A          2573.0  7260.014 6 75   2   5268468 77 35
A          2901.0  7245.01211 75   1   4184671 79 56
A          2901.0  7245.012 4 75   1   3195171 77 43
A          2559.0  7255.013 9 75   2   5255268 77 42
A          2559.0  7255.013 5 75   2   9309067 76 30
A          2897.0  7255.011 7 75   2   4227270 78 47
A          2897.0  7255.011 1 75   1  -2153470 76 53
A          2575.0  7260.014 6 75   2   5162670 76 40
A          2575.0  7260.01410 75   2   9279767 77 41
A          2899.0  7245.01211 75   1  -2152872 77 62
A          2899.0  7245.012 4 75   1  -2142771 78 45
A          2561.0  7255.013 9 75   3  21319165 77 46
A          2561.0  7255.013 5 75   2   9309967 78 35
A          2899.0  7255.011 7 75   1   4142771 77 55
A          2899.0  7255.011 1 75   1  -2214770 77 42

Thanks for your help
# 2  
Old 11-05-2012
I like sed a lot, but I see ugliness in trying to get values from one file and use them to delete values in another file using sed.

I think:
Code:
grep -vFf data2delete file2

may work well.

It returns the following:

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2571.0  7260.01410 75   1  -2204470 77 41
A          2571.0  7260.014 6 75   2   5182570 77 37
A          2557.0  7255.013 9 75   1   5153071 77 67
A          2557.0  7255.013 5 75   1  -3162871 77 42
A          2895.0  7255.011 7 75   1  -3203071 77 41
A          2895.0  7255.011 1 75   1  -2174671 77 56
A          2901.0  7245.01211 75   1   4184671 79 56
A          2901.0  7245.012 4 75   1   3195171 77 43
A          2559.0  7255.013 9 75   2   5255268 77 42
A          2559.0  7255.013 5 75   2   9309067 76 30
A          2575.0  7260.014 6 75   2   5162670 76 40
A          2575.0  7260.01410 75   2   9279767 77 41
A          2899.0  7245.01211 75   1  -2152872 77 62
A          2899.0  7245.012 4 75   1  -2142771 78 45
A          2899.0  7255.011 7 75   1   4142771 77 55
A          2899.0  7255.011 1 75   1  -2214770 77 42

This User Gave Thanks to rwuerth For This Post:
# 3  
Old 11-06-2012
Thanks for your help..

If you know a way using awk and sed ,, please let me know... I like to learn

---------- Post updated 11-06-12 at 02:47 AM ---------- Previous update was 11-05-12 at 01:38 PM ----------

Some one can help me with sed
# 4  
Old 11-06-2012
With awk:
Code:
awk 'FNR==NR{a[$0];next}{p=1;for(i in a) if($0~i) {p=0;break}}p' data2delete file2

These 2 Users Gave Thanks to elixir_sinari For This Post:
# 5  
Old 11-06-2012
Or:
Code:
awk 'FNR==NR{A[$0];next} !(substr($0,12,15) in A)' data2delete file2

This User Gave Thanks to Scrutinizer For This Post:
# 6  
Old 11-06-2012
Many Thanks to all

---------- Post updated at 06:30 AM ---------- Previous update was at 03:28 AM ----------

Dear alls,

All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines

Example

data2delete

Code:
2915.0  7325.01
2893.0  7255.01

file2

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

output

Code:
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

Thanks for your help

---------- Post updated at 06:30 AM ---------- Previous update was at 06:30 AM ----------

Dear alls,

All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines

Example

data2delete

Code:
2915.0  7325.01
2893.0  7255.01

file2

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

output

Code:
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

Thanks for your help
# 7  
Old 11-06-2012
Quote:
Originally Posted by jiam912
All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines
All the solution's provided works perfect.

but i prefer rwuerth's solution. grep -vf

Code:
$ cat file2
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

$ cat file1
2915.0  7325.01
2893.0  7255.01

$ grep -vf file1 file2
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

This User Gave Thanks to pamu For This Post:

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove repetitive lines in a file with sed?

Hello, My goal is the make all x times repeated lines into a single line. I need to attain the expected output with sed -i , I need to overwrite the MyFile MyFile: Hello World Welcome Hello World Welcome Back This is my test Expected output: Hello World Welcome Welcome Back This is... (6 Replies)
Discussion started by: baris35
6 Replies

2. Shell Programming and Scripting

sed to remove next 15 lines after pattern in Solaris

Team, I am trying to use sed to delete 15 lines, after pattern patch, which includes the pattern as well in Solaris. I used the below command, as we do it Linux, but it's not working as expected in Solaris. I am getting the error as "garbled".sed '/\/table/,+15d' status.html sed: command... (8 Replies)
Discussion started by: Nagaraj R
8 Replies

3. Shell Programming and Scripting

How to remove certain lines using sed?

Hi, I am new to unix and i started some scripting recently. Please go through the following script i wrote. #!/bin/sh file='path../tfile' file1='path../tfile1' rmfile='path../test2' C1=1 C2=1 exec 3< $file1 while read LINE1; do read LINE2 <&3 a=$LINE1 b=`expr $LINE2 - 1` ... (1 Reply)
Discussion started by: Subbu123
1 Replies

4. UNIX for Dummies Questions & Answers

How to remove certain lines using sed?

Hi I have the following kind of line sin my file . print ' this is first'. print ' this is firs and next ' ' line continuous '. -- this is entire print line. print ' this is first and next ' ' line continuous and' 'still there now over'. -- this 3lines together a single print line. ... (5 Replies)
Discussion started by: Sivajee
5 Replies

5. Shell Programming and Scripting

SED to remove a line above and lines below.

:confused:Hi All, I need help on removing lines in a text file. Sample file : When there is a match ip for IPAddress in my `cat ip.out`, proceed delete line above until string "Comp" is found. Thank you very much. ---------- Post updated at 12:56 AM ---------- Previous update was... (4 Replies)
Discussion started by: chiewming
4 Replies

6. Shell Programming and Scripting

What's the command to remove empty lines with sed?

3 10 20 10 100 100 10000 Output: 3 10 20 10 100 100 10000 ---------- Post updated at 07:59 AM ---------- Previous update was at 07:56 AM ---------- sed '/^$/d' file doesn't work. (8 Replies)
Discussion started by: cola
8 Replies

7. Shell Programming and Scripting

using sed to remove lines

Can somebody explain why my sed command is not working. I do the folloinwg: Generates a binary file to /tmp/x1.out /usr/lib/sa/sa2 -s 4:00 -e 8:00 -i 3600 -A -o /tmp/x1.out decodes the file (no problem so far) sar -f /tmp/x1.out When I do this it does not appear to delete the... (4 Replies)
Discussion started by: BeefStu
4 Replies

8. Shell Programming and Scripting

How to remove lines before and after with awk / sed ?

Hi guys, I need to remove the pattern (ID=180), one line before and four lines after. Thanks. (5 Replies)
Discussion started by: ashimada
5 Replies

9. Shell Programming and Scripting

Using sed to remove lines where field is empty

I was just looking at this post: https://www.unix.com/shell-programming-scripting/22893-delete-multiple-empty-lines.html. and I am looking to achieve the same with sed. So the idea is to delete lines from a file where a certain field has no value. Inputfile: EMID MMDDYY HOURS JOB EMNAME 0241... (4 Replies)
Discussion started by: figaro
4 Replies

10. Shell Programming and Scripting

Sed or Awk to remove specific lines

I have searched the forum for this - forgive me if I missed a previous post. I have the following file: blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah alter table "informix".esc_acct add constraint (foreign key (fi_id) references "informix".fi ... (5 Replies)
Discussion started by: Shoeless_Mike
5 Replies

Featured Tech Videos