Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Multile Pattern Search in a same line and delete

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Multile Pattern Search in a same line and delete
# 1  
Old 04-16-2008
Multile Pattern Search in a same line and delete
HI Gurus,

I need to delete a line from a syslog file, if it matches three conditions.

Say for ex., if the device name is device.name.com and if it contains the syslog message PAGP-5-PORTFROMSTP in between the time period 00:00:00 to 04:00:00, then the particular line has to be deleted from the syslog file. The file size is 550 MB and so vi cannot be used here. THe only option I see here is sed and tried the following,

sed -e '/[00-04]:..:../d;/device.anme.com/d;/PAGP-5-PORTFROMSTP/d' syslogfile > tempfile
but its not working as expected.

I have a few lines from the syslog file copied below,

SRAMDEL<fsp>2008-03-01 00:01:26<fsp>router.dev.com<fsp>SPANTREE-6-PORTFWD<fsp>6<fsp>Port 5/1 state in VLAN 469 changed to forwarding<fsp>router.dev.com<fsp>102430100217
SRAMDEL<fsp>2008-03-01 00:01:33<fsp>21.228.23.24<fsp>CDP-4-DUPLEX_MISMATCH<fsp>4<fsp>duplex mismatch discovered on FastEthernet0/1 (not half duplex), with yourname.yourdomain.com GigabitEthernet0/0 (half duplex).<fsp>12.18.127.216<fsp>102430100218
SRAMDEL<fsp>2008-03-01 00:01:34<fsp>router.dev.com<fsp>ETHC-5-PORTFROMSTP<fsp>5<fsp>Port 5/1 left bridge port 5/1<fsp>router.dev.com<fsp>102430100219
SRAMDEL<fsp>2008-03-01 00:01:35<fsp>rtrL-OSL1.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 2 not rotating<fsp>rL1.dev.com<fsp>102430100220
SRAMDEL<fsp>2008-03-01 00:01:36<fsp>router.dev.com<fsp>ETHC-5-PORTTOSTP<fsp>5<fsp>Port 5/1 joined bridge port 5/1<fsp>router.dev.com<fsp>102430100221
SRAMDEL<fsp>2008-03-01 00:01:36<fsp>rtrR1.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 1 not rotating<fsp>rR1.dev.com<fsp>102430100222
SRAMDEL<fsp>2008-03-01 00:01:37<fsp>router.dev.com<fsp>SPANTREE-6-PORTFWD<fsp>6<fsp>Port 5/1 state in VLAN 469 changed to forwarding<fsp>router.dev.com<fsp>102430100223
SRAMDEL<fsp>2008-03-01 00:02:05<fsp>rtrL1.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 2 not rotating<fsp>rL1.dev.com<fsp>102430100224
SRAMDEL<fsp>2008-03-01 00:02:05<fsp>rtrI2.dev.com<fsp>CDP-4-DUPLEX_MISMATCH<fsp>4<fsp>duplex mismatch discovered on FastEthernet0/0 (not half duplex), with Switch FastEthernet0/5 (half duplex).<fsp>rtAI2.dev.com<fsp>102430100225
SRAMDEL<fsp>2008-03-01 00:02:06<fsp>rtr1.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 1 not rotating<fsp>rtR1.dev.com<fsp>102430100226
SRAMDEL<fsp>2008-03-01 00:02:23<fsp>TR01.dev.com<fsp>CDP-4-DUPLEX_MISMATCH<fsp>4<fsp>duplex mismatch discovered on FastEthernet1/0/44 (not half duplex), with SEP001D457DD302 Port 1 (half duplex).<fsp>TR00.dev.com<fsp>102430100227
SRAMDEL<fsp>2008-03-01 00:02:24<fsp>AL05.dev.com<fsp>PAGP-5-PORTFROMSTP<fsp>5<fsp>Port 5/48 left bridge port 5/48<fsp>AL005.dev.com<fsp>102430100228
SRAMDEL<fsp>2008-03-01 00:02:26<fsp>AL05.dev.com<fsp>PAGP-5-PORTTOSTP<fsp>5<fsp>Port 5/48 joined bridge port 5/48<fsp>AL005.dev.com<fsp>102430100228
SRAMDEL<fsp>2008-03-01 00:02:27<fsp>AL05.dev.com<fsp>SPANTREE-6-PORTFWD<fsp>6<fsp>Port 5/48 state in VLAN 21 changed to forwarding<fsp>ALS5.dev.com<fsp>102430100230
SRAMDEL<fsp>2008-03-01 00:02:33<fsp>12.148.197.126<fsp>CDP-4-DUPLEX_MISMATCH<fsp>4<fsp>duplex mismatch discovered on FastEthernet0/1 (not half duplex), with yourname.yourdomain.com GigabitEthernet0/0 (half duplex).<fsp>12.198.17.26<fsp>102430100231
SRAMDEL<fsp>2008-03-01 00:02:35<fsp>rtrws.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 2 not rotating<fsp>rtL1.dev.com<fsp>102430100232
SRAMDEL<fsp>2008-03-01 00:02:36<fsp>rt1.dev.com<fsp>ENVMON-3-FAN_FAILED<fsp>3<fsp>Fan 1 not rotating<fsp>rtrR1.dev.com<fsp>102430100233
SRAMDEL<fsp>2008-03-01 00:02:42<fsp>router.dev.com<fsp>ETHC-5-PORTFROMSTP<fsp>5<fsp>Port 5/1 left bridge port 5/1<fsp>router.dev.com<fsp>102430100234

Can you pls help me in getting the script working.

THanks a lot for your help
# 2  
Old 04-16-2008
Quote:
Originally Posted by sasree76
sed -e '/[00-04]:..:../d;/device.anme.com/d;/PAGP-5-PORTFROMSTP/d' syslogfile > tempfile
You have the time regex wrong.

Code:
/0[0-4]:..:../d

However, the logic seems backwards, too. You want to delete lines where all three conditions match, correct? Your script will successively delete lines where any one condition matches.

If they are always in the same order, you could simply use grep -v.

Code:
grep -v '0[0-4]:..:.*device\.anme\.com.*PAGP-5-PORTFROMSTP' syslogfile >tempfile

I don't see any matches on "device.anme.com" or "device.name.com" so it's a bit hard to test on the snippet you posted, though. If I use the device AL05.dev.com I find one match.
Last edited by era; 04-16-2008 at 02:42 PM..
# 3  
Old 04-16-2008
Hello Era,

THanks a lot for the tip. It really works. Thanks once again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search and Delete pattern with Control M

Hi I have data file like below: NET_SALES^M ^M---- new fields for -- ^M ,YABA_FLAG^M ,DOO_FLAG^M My requirement is to search for atleast 2 -- and remove all the words till first ^M is encountered. In above case, output should be like NET_SALES^M ^M ,YABA_FLAG^M ,DOO_FLAG^M TIA... (4 Replies)
Discussion started by: dashing201
4 Replies

2. UNIX for Dummies Questions & Answers

How to delete blank line/s before and after a search pattern?

Hi, Test file x.txt below. This file is generated by a program that I unfortunately do not have control on how it gets presented/generated. create PACKAGE "XXX_INTERFACE_DEFECT_RPT_TEST" is TYPE refCursor IS REF CURSOR; Function queryRecords ( p_status varchar2, ... ... ... )... (4 Replies)
Discussion started by: newbie_01
4 Replies

3. Homework & Coursework Questions

sed Multiple Pattern search and delete the line

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I have file which has got the following content sam 123 LD 41 sam 234 kp sam LD 41 kam pu sam LD 61 Now... (1 Reply)
Discussion started by: muchyog
1 Replies

4. Shell Programming and Scripting

Make Multile line is one line using window Perl

Hi All I have a .csv file which is showing data as ESP Client,ESP Engagement,Misc_Projects_120101,DEFAULT,HA,Unknown,No,Unknown,201704,4.1,Unknown,AAA,Collected-Done,"she,joy.",200111,Unknown,Full Time,,Delivery_DONE AMO,Approved,2012-12-03,2012-12-06,2012-12-06,"Occupied Hours ... (7 Replies)
Discussion started by: adisky123
7 Replies

5. Shell Programming and Scripting

awk delete/remove rest of line on multiple search pattern

Need to remove rest of line after the equals sign on search pattern from the searchfile. Can anybody help. Couldn't find any similar example in the forum: infile: 64_1535: Delm. = 86 var, aaga 64_1535: Fran. = 57 ex. ccc 64_1639: Feb. = 26 (link). def 64_1817: mar. = 3/4. drz ... (7 Replies)
Discussion started by: sdf
7 Replies

6. Shell Programming and Scripting

find pattern, delete line with pattern and line above and line below

I have a file that will sometimes contain a pattern. The pattern is this: FRM CHK 0000 I want to find any lines with this pattern, delete those lines, and also delete the line above and the line below. (4 Replies)
Discussion started by: nickg
4 Replies

7. UNIX for Dummies Questions & Answers

find pattern delete line with pattern and line above and line below

I have a file that will sometimes contain a pattern. The pattern is this: W/D FRM CHK 00 I want to find any lines with this pattern, delete those lines, and also delete the line above and the line below. (1 Reply)
Discussion started by: nickg
1 Replies

8. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies

9. UNIX for Dummies Questions & Answers

modify a particular pattern starting from second line of the search pattern

Hi, I think you ppl did not get my question correctly, let me explain I have 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: ... (1 Reply)
Discussion started by: imas
1 Replies

10. UNIX for Dummies Questions & Answers

modify a particular pattern starting from second line of the search pattern

Hi, I am new to this forum and i would like to get help in this issue. I have a file 1.txt as shown: apple banana orange apple grapes banana orange grapes orange .... Now i would like to search for pattern say apple or orange and then put a # at the beginning of the pattern... (2 Replies)
Discussion started by: imas
2 Replies
Login or Register to Ask a Question