Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete line with match and previous line quoting/escaping problem

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete line with match and previous line quoting/escaping problem
# 1  
Old 01-13-2012
Delete line with pattern + previous line - quoting/escaping problem
Hi folks,

I've list of LDAP records in this format:
Code:
cat cmmac.export.tmp2
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:13:11:36:a5:06
dn: deviceId=0a92746a62pbms4662299650015961cfa23,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1c:fa:23
dn: deviceId=0a92746a62sbms4662373470015961d6d7b,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1d:6d:7b
dn: deviceId=0a92746a64cbms466298898001596010372,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:01:03:72
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc

Some of these records have cmmac attribute duplicity, so I don't want to process them further. I have list of duplicate records, so I'm going through source file and deleting cmmac: MATCH and dn: above. I've tried to utilize solution I've found on these forums
Code:
grep -v "$(grep -B 1 "MATCH" cmmac.export.txt)" cmmac.export.txt > cmmac.export.filtered

Everything would be fine, but bash is developing the command into
Code:
grep -v '"$(grep -B 1 "MATCH" cmmac.export.txt)"' cmmac.export.txt > cmmac.export.filtered which is not producing desired result.

Thank you for any ideas how to solve this.
Tomas

#--------------------------------------------------------
#Filtering duplicities from device exports
#--------------------------------------------------------
Code:
cp cmmac.export.tmp cmmac.export.tmp2
for fil in `grep cmmac cmmac.export.tmp | sort | uniq -c | grep -v " 1 " | awk -F' ' '{print $3}'`
do
echo $fil
grep -v \""\$(grep -B 1 \" $fil\" cmmac.export.tmp2)"\" cmmac.export.tmp2 > cmmac.export.txt
cp cmmac.export.txt cmmac.export.tmp2
done

+ cp cmmac.export.tmp cmmac.export.tmp2
++ grep cmmac cmmac.export.tmp
++ sort
++ uniq -c
++ awk '-F ' '{print $3}'
++ grep -v ' 1 '
+ for fil in '`grep cmmac cmmac.export.tmp | sort | uniq -c | grep -v " 1 " | awk -F'\'' '\'' '\''{print $3}'\''`'
+ echo 00:15:d0:00:a5:e6
00:15:d0:00:a5:e6
+ grep -v '"$(grep -B 1 " 00:15:d0:00:a5:e6" cmmac.export.tmp2)"' cmmac.export.tmp2
++ bashtrap
++ echo 'CTRL+C Detected '

Moderator's Comments:
Mod Comment Please use code tags, check PM, thanks.


---------- Post updated 13-01-12 at 05:52 AM ---------- Previous update was 12-01-12 at 01:10 PM ----------

Somehow I've found what to do:
Grep doesn't like putting the variable from "for" cycle, so I've wrapped it into echo call in subshell. I've no idea why it works this way.

Code:
for fil in $( grep cmmac cmmac.export.tmp | sort | uniq -c | grep -v " 1 " | awk -F' ' '{print $3}' );
do      
#echo $fil
#Following line deletes line with pattern fil from file tmp2 and stores in txt
grep -v "$(grep -B 1 " `echo $fil`" cmmac.export.tmp2)" cmmac.export.tmp2 > cmmac.export.txt
cp cmmac.export.txt cmmac.export.tmp2
done

Last edited by tomas.polak; 01-13-2012 at 06:55 AM.. Reason: code tags, check PM
# 2  
Old 01-13-2012
Please give an exemple of the wanted output

---------- Post updated at 12:22 PM ---------- Previous update was at 12:04 PM ----------

Code:
awk '{y=x;x=$0}$0~/cmmac/&&!A[$2]++{print "This record will be processed :" RS y RS $0}' yourinputfile


Code:
$ cat tst
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:13:11:36:a5:06
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:13:11:36:a5:06
dn: deviceId=0a92746a62pbms4662299650015961cfa23,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1c:fa:23
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:13:11:36:a5:06
dn: deviceId=0a92746a62pbms4662299650015961cfa23,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1c:fa:23
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc
dn: deviceId=0a92746a62sbms4662373470015961d6d7b,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1d:6d:7b
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc

Code:
$ awk '{y=x;x=$0}$0~/cmmac/&&!A[$2]++{print "This record will be processed :" RS y RS $0}' tst
This record will be processed :
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:13:11:36:a5:06
This record will be processed :
dn: deviceId=0a92746a62pbms4662299650015961cfa23,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1c:fa:23
This record will be processed :
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:58:83:bc
This record will be processed :
dn: deviceId=0a92746a62sbms4662373470015961d6d7b,ou=devices,ou=customer,ou=nl,o=upc
cmmac: 00:15:96:1d:6d:7b

---------- Post updated at 12:33 PM ---------- Previous update was at 12:22 PM ----------

if you are just interested in extracting the cmmac with unicity, you can go with :

Code:
$ awk '/cmmac/&&!A[$2]++{print$2}' tst
00:13:11:36:a5:06
00:15:96:1c:fa:23
00:15:96:58:83:bc
00:15:96:1d:6d:7b

(just replace "tst" with the name of your input file)
Last edited by ctsgnb; 01-13-2012 at 07:28 AM..
This User Gave Thanks to ctsgnb For This Post:
tomas.polak
# 3  
Old 01-13-2012
Hi ctsgnb,

what I've tried to achieve was:
I have a list of MAC's with duplicity in file in format:
Code:
00:15:96:1c:fa:23
00:15:96:1c:fa:24

I have a list of DN's and MAC's in format:
Code:
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc 
cmmac: 00:13:11:36:a5:06 
dn: deviceId=0a92746a62pbms4662299650015961cfa23,ou=devices,ou=customer,ou=nl,o=upc 
cmmac: 00:15:96:1c:fa:23 
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc 
cmmac: 00:15:96:58:83:bc

So I need to search against MAC addresses in duplicity list, find it in list of DN and MAC and delete also the previous line with DN. My problem was construction of special grep command which has in "for .. in do .. done" cycle a fundamental flaw - it doesn't like just variable like %f, you need to utilize subshell - `echo %f ` to ensure that in expands into desired form. I've found a working solution
Code:
cp cmmac.export.tmp cmmac.export.tmp2 
for fil in $( grep cmmac cmmac.export.tmp | sort | uniq -c | grep -v " 1 " | awk -F' ' '{print $3}' ); do       
#echo $fil 
#Following line deletes line with pattern fil and preceding line from file tmp2 and stores in txt 
grep -v "$(grep -B 1 " `echo $fil`" cmmac.export.tmp2)" cmmac.export.tmp2 > cmmac.export.txt 
cp cmmac.export.txt cmmac.export.tmp2 
done

and it's output is:
Code:
dn: deviceId=0a92746a54tbmd34b05758900131136a506,ou=devices,ou=customer,ou=nl,o=upc 
cmmac: 00:13:11:36:a5:06 
dn: deviceId=0a92746a64jbmr22a2083670015965883bc,ou=devices,ou=customer,ou=nl,o=upc 
cmmac: 00:15:96:58:83:bc

Thanks for your interest Smilie
# 4  
Old 01-14-2012
You could try something like this:
Code:
awk 'NR==FNR{A[$1]=1;next}/cmmac/&&!A[$2]{print p RS $0}{p=$0}' file1 file2

or
Code:
awk 'NR==FNR{A[$1]=1;next}{p=$0;getline}!A[$2]{print p RS $0}' file1 file2

This User Gave Thanks to Scrutinizer For This Post:
tomas.polak
# 5  
Old 01-17-2012
Who has no idea what Mr. Scrutinizer wrote, see these articles:
Code:
unstableme.blogspot.com/search/label/awk%20FNR

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk or sed to print the character from the previous line after the regexp match

Hi All, I need to print the characters in the previous line just before the regular expression match Please have a look at the input file as attached I need to match the regular expression ^ with the character of the previous like and also the pin numbers and the output file should be like... (6 Replies)
Discussion started by: kshitij
6 Replies

2. Shell Programming and Scripting

How to delete the previous line after pattern match?

Team, I am writing a shell script to perform few health checks of the system, where I need to delete the previous line in the text file after pattern match using sed (or) awk. Could you please help me out on this? For example, <td> <td style=color:green align=center> </td> </tr>... (6 Replies)
Discussion started by: Nagaraj R
6 Replies

3. AIX

Print nth previous line after match

Please help me print nth line after match awk or sed one line command. (3 Replies)
Discussion started by: sushma123
3 Replies

4. UNIX for Advanced & Expert Users

How to find a string in a line in UNIX file and delete that line and previous 3 lines ?

Hi , i have a file with data as below.This is same file. But actual file contains to many rows. i want to search for a string "Field 039 00" and delete that line and previous 3 lines in that file.. Can some body suggested me how can i do using either sed or awk command ? Field 004... (7 Replies)
Discussion started by: vadlamudy
7 Replies

5. UNIX for Dummies Questions & Answers

Awk match on columns and delete line

Hi, I have a file like this a 1 2 b 2 2 c 2 3 d 4 5 f 5 6 output a 1 2 c 2 3 d 4 5 f 5 6 Basically, I want to delete the whole line if $2 and $3 are the same. Thanks (5 Replies)
Discussion started by: jacobs.smith
5 Replies

6. UNIX for Advanced & Expert Users

Delete the exact match line from file

Hello All, I have following line and text file line =08 * * 3 /data/reports/bin/xyz.ksh -o customreports.com -f abc.sql -m xyz.com -c -g file:- abc.crontab 08 * * 3 /data/reports/bin/xyz.ksh -o customreports.com -f abc.sql -m xyz.com -c -g 06 * * 3 /data/reports/bin/xyz.ksh -o... (3 Replies)
Discussion started by: anyera
3 Replies

7. Shell Programming and Scripting

Delete the last line if match the string

Hi, How to delete the last line if is match the below string else no action... String checking "END OF FILE. ROW COUNT: " 9f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|2 9f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|0 229f680174-cb87-4f71-887a-93b6f62fa5aa|20077337254|3 END OF... (2 Replies)
Discussion started by: bmk
2 Replies

8. UNIX for Dummies Questions & Answers

return previous line for pattern match

Hi, Need some idea on file processing, I have file like below, Processing al sources ... ...No value found : CHECK. Completed comparing all sources. Comparing schedulers... Processing al targets ... ...No value found : From above I need to extract the line where "No value... (4 Replies)
Discussion started by: braindrain
4 Replies

9. Shell Programming and Scripting

delete a line that does not match the pattern

hi, i am parsing a file, in that searching for lines those contains "$threadNo.Received message:" , if that line contains the required fields write them into a separate file other wise ignore them. i am using the following code,but it is printing all the lines , i dont want to rpint , please help... (3 Replies)
Discussion started by: Satyak
3 Replies

10. Shell Programming and Scripting

Need to delete previous line after successful seearch

i have a file in following pattreen A: Size = 10 B: Size = 0 C: Size = 220 D: Size = 0 i want to Display only Charecters which have which have Size = 0 The Out put Should be B: D: Can Some one Help (7 Replies)
Discussion started by: pbsrinivas
7 Replies
Login or Register to Ask a Question