Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Locate and remove lines with sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Locate and remove lines with sed
# 1  
Old 11-05-2012
Locate and remove lines with sed
Gents,

I would like to remove some lines from a big file ( file2).

The objetive is to remove all the lines in file2 containing a certain string which are in file data2delete..

file data2delete contens:

Code:
2573.0  7260.01
2893.0  7255.01
2903.0  7245.01
2897.0  7255.01
2561.0  7255.01



file2
Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64
A          2571.0  7260.01410 75   1  -2204470 77 41
A          2571.0  7260.014 6 75   2   5182570 77 37
A          2557.0  7255.013 9 75   1   5153071 77 67
A          2557.0  7255.013 5 75   1  -3162871 77 42
A          2895.0  7255.011 7 75   1  -3203071 77 41
A          2895.0  7255.011 1 75   1  -2174671 77 56
A          2573.0  7260.01410 75   1   2259968 77 38
A          2573.0  7260.014 6 75   2   5268468 77 35
A          2901.0  7245.01211 75   1   4184671 79 56
A          2901.0  7245.012 4 75   1   3195171 77 43
A          2559.0  7255.013 9 75   2   5255268 77 42
A          2559.0  7255.013 5 75   2   9309067 76 30
A          2897.0  7255.011 7 75   2   4227270 78 47
A          2897.0  7255.011 1 75   1  -2153470 76 53
A          2575.0  7260.014 6 75   2   5162670 76 40
A          2575.0  7260.01410 75   2   9279767 77 41
A          2899.0  7245.01211 75   1  -2152872 77 62
A          2899.0  7245.012 4 75   1  -2142771 78 45
A          2561.0  7255.013 9 75   3  21319165 77 46
A          2561.0  7255.013 5 75   2   9309967 78 35
A          2899.0  7255.011 7 75   1   4142771 77 55
A          2899.0  7255.011 1 75   1  -2214770 77 42

Thanks for your help
# 2  
Old 11-05-2012
I like sed a lot, but I see ugliness in trying to get values from one file and use them to delete values in another file using sed.

I think:
Code:
grep -vFf data2delete file2

may work well.

It returns the following:

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2571.0  7260.01410 75   1  -2204470 77 41
A          2571.0  7260.014 6 75   2   5182570 77 37
A          2557.0  7255.013 9 75   1   5153071 77 67
A          2557.0  7255.013 5 75   1  -3162871 77 42
A          2895.0  7255.011 7 75   1  -3203071 77 41
A          2895.0  7255.011 1 75   1  -2174671 77 56
A          2901.0  7245.01211 75   1   4184671 79 56
A          2901.0  7245.012 4 75   1   3195171 77 43
A          2559.0  7255.013 9 75   2   5255268 77 42
A          2559.0  7255.013 5 75   2   9309067 76 30
A          2575.0  7260.014 6 75   2   5162670 76 40
A          2575.0  7260.01410 75   2   9279767 77 41
A          2899.0  7245.01211 75   1  -2152872 77 62
A          2899.0  7245.012 4 75   1  -2142771 78 45
A          2899.0  7255.011 7 75   1   4142771 77 55
A          2899.0  7255.011 1 75   1  -2214770 77 42

This User Gave Thanks to rwuerth For This Post:
jiam912
# 3  
Old 11-06-2012
Thanks for your help..

If you know a way using awk and sed ,, please let me know... I like to learn

---------- Post updated 11-06-12 at 02:47 AM ---------- Previous update was 11-05-12 at 01:38 PM ----------

Some one can help me with sed
# 4  
Old 11-06-2012
With awk:
Code:
awk 'FNR==NR{a[$0];next}{p=1;for(i in a) if($0~i) {p=0;break}}p' data2delete file2

These 2 Users Gave Thanks to elixir_sinari For This Post:
jiam912 pamu
# 5  
Old 11-06-2012
Or:
Code:
awk 'FNR==NR{A[$0];next} !(substr($0,12,15) in A)' data2delete file2

This User Gave Thanks to Scrutinizer For This Post:
jiam912
# 6  
Old 11-06-2012
Many Thanks to all

---------- Post updated at 06:30 AM ---------- Previous update was at 03:28 AM ----------

Dear alls,

All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines

Example

data2delete

Code:
2915.0  7325.01
2893.0  7255.01

file2

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

output

Code:
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

Thanks for your help

---------- Post updated at 06:30 AM ---------- Previous update was at 06:30 AM ----------

Dear alls,

All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines

Example

data2delete

Code:
2915.0  7325.01
2893.0  7255.01

file2

Code:
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

output

Code:
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

Thanks for your help
# 7  
Old 11-06-2012
Quote:
Originally Posted by jiam912
All codes works fine, but I notice that they deleted only one file in file2. I mean one time the value found in file data2delete?.

I would like to delete all the lines not only one line?,

FYI, the file2 has 2 time the same value repetead.. for that I would like to delete both lines
All the solution's provided works perfect.

but i prefer rwuerth's solution. grep -vf

Code:
$ cat file2
A          2915.0  7325.01212 75   2   5248969 78 46
A          2915.0  7325.012 7 75   2   5216571 80 44
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2893.0  7255.011 7 75   2   6205870 77 54
A          2893.0  7255.011 1 75   1  -2142671 76 62
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

$ cat file1
2915.0  7325.01
2893.0  7255.01

$ grep -vf file1 file2
A          2555.0  7255.013 9 75   1   2163071 78 59
A          2555.0  7255.013 5 75   1  -3162871 77 52
A          2903.0  7245.01211 75   1   3227270 77 54
A          2903.0  7245.012 4 75   1   3185571 78 64

This User Gave Thanks to pamu For This Post:
jiam912
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove repetitive lines in a file with sed?

Hello, My goal is the make all x times repeated lines into a single line. I need to attain the expected output with sed -i , I need to overwrite the MyFile MyFile: Hello World Welcome Hello World Welcome Back This is my test Expected output: Hello World Welcome Welcome Back This is... (6 Replies)
Discussion started by: baris35
6 Replies

2. Shell Programming and Scripting

sed to remove next 15 lines after pattern in Solaris

Team, I am trying to use sed to delete 15 lines, after pattern patch, which includes the pattern as well in Solaris. I used the below command, as we do it Linux, but it's not working as expected in Solaris. I am getting the error as "garbled".sed '/\/table/,+15d' status.html sed: command... (8 Replies)
Discussion started by: Nagaraj R
8 Replies

3. Shell Programming and Scripting

How to remove certain lines using sed?

Hi, I am new to unix and i started some scripting recently. Please go through the following script i wrote. #!/bin/sh file='path../tfile' file1='path../tfile1' rmfile='path../test2' C1=1 C2=1 exec 3< $file1 while read LINE1; do read LINE2 <&3 a=$LINE1 b=`expr $LINE2 - 1` ... (1 Reply)
Discussion started by: Subbu123
1 Replies

4. UNIX for Dummies Questions & Answers

How to remove certain lines using sed?

Hi I have the following kind of line sin my file . print ' this is first'. print ' this is firs and next ' ' line continuous '. -- this is entire print line. print ' this is first and next ' ' line continuous and' 'still there now over'. -- this 3lines together a single print line. ... (5 Replies)
Discussion started by: Sivajee
5 Replies

5. Shell Programming and Scripting

SED to remove a line above and lines below.

:confused:Hi All, I need help on removing lines in a text file. Sample file : When there is a match ip for IPAddress in my `cat ip.out`, proceed delete line above until string "Comp" is found. Thank you very much. ---------- Post updated at 12:56 AM ---------- Previous update was... (4 Replies)
Discussion started by: chiewming
4 Replies

6. Shell Programming and Scripting

What's the command to remove empty lines with sed?

3 10 20 10 100 100 10000 Output: 3 10 20 10 100 100 10000 ---------- Post updated at 07:59 AM ---------- Previous update was at 07:56 AM ---------- sed '/^$/d' file doesn't work. (8 Replies)
Discussion started by: cola
8 Replies

7. Shell Programming and Scripting

using sed to remove lines

Can somebody explain why my sed command is not working. I do the folloinwg: Generates a binary file to /tmp/x1.out /usr/lib/sa/sa2 -s 4:00 -e 8:00 -i 3600 -A -o /tmp/x1.out decodes the file (no problem so far) sar -f /tmp/x1.out When I do this it does not appear to delete the... (4 Replies)
Discussion started by: BeefStu
4 Replies

8. Shell Programming and Scripting

How to remove lines before and after with awk / sed ?

Hi guys, I need to remove the pattern (ID=180), one line before and four lines after. Thanks. (5 Replies)
Discussion started by: ashimada
5 Replies

9. Shell Programming and Scripting

Using sed to remove lines where field is empty

I was just looking at this post: https://www.unix.com/shell-programming-scripting/22893-delete-multiple-empty-lines.html. and I am looking to achieve the same with sed. So the idea is to delete lines from a file where a certain field has no value. Inputfile: EMID MMDDYY HOURS JOB EMNAME 0241... (4 Replies)
Discussion started by: figaro
4 Replies

10. Shell Programming and Scripting

Sed or Awk to remove specific lines

I have searched the forum for this - forgive me if I missed a previous post. I have the following file: blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah alter table "informix".esc_acct add constraint (foreign key (fi_id) references "informix".fi ... (5 Replies)
Discussion started by: Shoeless_Mike
5 Replies
Login or Register to Ask a Question