Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove unwanted data?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove unwanted data?
# 1  
Old 05-09-2007
Remove unwanted data?
Hi Can any one help me remove the unwanted data? I would want to remove the complete event id 4910 ( the type there is INFO), that means, I have to remove starting from 7th - 19th lines. can any one of you please help?

Thanks,
Quote:
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXXXX
Message :
Auto-purged 23 log entries.
Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
# 2  
Old 05-09-2007
Try...
Code:
$ awk 'BEGIN{RS="Event Id"}!/Type : INFO/{if(p)printf RS p;p=$0}END{printf RS p}' file1 > file2
$ sdiff -w 60 file1 file2|nl
     1  Event Id: 4909                  Event Id: 4909
     2  Time : Tue May 8 23:06:06 20    Time : Tue May 8 23:06:06 20
     3  Type : RESET                    Type : RESET
     4  User : XXXXXX                   User : XXXXXX
     5  Message :                       Message :
     6  Auto-purged 23 log entries.     Auto-purged 23 log entries.
     7  Event Id: 4910               <
     8  Time : Tue May 8 23:08:13 20 <
     9  Type : INFO                  <
    10  User : XXXXXX                <
    11  Message :                    <
    12  From previous run            <
    13  Unable to open ODBC trace fi <
    14                               <
    15  DataStage Job 1 Phantom 2629 <
    16  rm: RT_SC1/jpfile: No such f <
    17                               <
    18  5 record(s) selected to SELE <
    19  DataStage Phantom Finished.  <
    20  Event Id: 4911                  Event Id: 4911
    21  Time : Tue May 8 23:08:13 20    Time : Tue May 8 23:08:13 20
    22  Type : STARTED                  Type : STARTED
$

# 3  
Old 05-09-2007
Code:
sed '7,19d' file > tmp
mv tmp file

Code:
id=4910
sed "/Event Id: $id/,/Event Id:/d" file > tmp
mv tmp file

# 4  
Old 05-09-2007
I guess i posted it incorrectly. What i meant to say was i want remove the complete event where the type is INFO.

another example:
Quote:
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXXX
Message :
Auto-purged 23 log entries.
Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXXXXX
Message :
Starting Job XXXXXXX.
$APT_DB2INSTANCE_HOME = XXXXXX
$APT_CONFIG_FILE = XXXXXXXXXX
DB_ALIAS = XXXXXX
DB_INST = XXXXXXXX
DB_NAME = XXXXXXX
PASSWORD = XXXXXX
USER = XXXXXX
SERVER = XXXXXXXX
Event Id: 4912
Time : Tue May 8 23:08:14 2007
Type : INFO
User : XXXXXXXXXX
Message :
Environment variable settings:
_=/usr/bin/nohup
MANPATH=/opt/quest/man:
LANG=C
LOGIN=dsadm
APT_LINKER=/usr/vacpp/bin/xlC_r
APT_LINKOPT=-G
APT_MONITOR_MINTIME=10
APT_DB2INSTANCE_HOME=XXXXXXXXX
OSH_STDOUT_MSG=1
Event Id: 4918
Time : Tue May 8 23:08:16 2007
Type : WARNING
User : XXXXXXX
Message :
Sequential_File_1: When checking operator: When validating export schema: At field "XXXXXXXXX": Exporting nulla
ble field without null handling properties
Expected output:

Quote:
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXXX
Message :
Auto-purged 23 log entries.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXXXXX
Message :
Starting Job XXXXX.
$APT_DB2INSTANCE_HOME = XXXXXX
$APT_CONFIG_FILE = XXXXXXXXXX
DB_ALIAS = XXXXXX
DB_INST = XXXXXXXX
DB_NAME = XXXXXXX
PASSWORD = XXXXXX
USER = XXXXXX
SERVER = XXXXXXXX

Event Id: 4918
Time : Tue May 8 23:08:16 2007
Type : WARNING
User : XXXXXXX
Message :
Sequential_File_1: When checking operator: When validating export schema: At field "XXXXXXXXX": Exporting nullable field without null handling properties
So all the events with type INFO is deleted. in the example the Events 4910 and 4912 has to be deleted
# 5  
Old 05-09-2007
Code:
sed "/^Event Id/s/.*/\\
&/" file | awk -v RS="" ' !/Type : INFO/ '

# 6  
Old 05-09-2007
Quote:
Originally Posted by anbu23
Code:
sed "/^Event Id/s/.*/\\
&/" file | awk -v RS="" ' !/Type : INFO/ '

Well, its not removing the complete event what i mean is

Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Last edited by ahmedwaseem2000; 05-09-2007 at 04:22 AM..
# 7  
Old 05-09-2007
Quote:
Originally Posted by ahmedwaseem2000
Well, its not removing the complete event what i mean by event is

Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Yes it not removing the complete event
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

2. Shell Programming and Scripting

Remove unwanted white space

Hi, I have a very big file 25GB with information present in it like $ head ind_stats update index statistics pfirm001.dbo.Office using 200 values go ... (11 Replies)
Discussion started by: sam05121988
11 Replies

3. Shell Programming and Scripting

How to remove unwanted " from string...

I have this Input File with extra double quotes in the middle. Please suggest how to handle this condition. Input File: "123985","SAW CUT CONCRETE SLAB 20"THICK",,"98.57","","EACH","N" "204312","ARMAFLEX-1 3/8 X 3"",,"2.48","","PER FOOT","N" "205745","MISTING HEAD HOLLOW CONE "C"... (3 Replies)
Discussion started by: BICC
3 Replies

4. Shell Programming and Scripting

How to remove unwanted strings?

Hi Guys, Can someone give me a hand on how I can remove unwanted strings like "<Number>" and "</Number>" and retain only the numbers from the input file below. INPUT FILE: <Number>10050000</Number> <Number>1001340001</Number> <Number>1001750002</Number> <Number>100750003</Number>... (8 Replies)
Discussion started by: pinpe
8 Replies

5. Shell Programming and Scripting

remove unwanted text using perl

Hello..I have a text file that need to remove unwanted text. This is the original file. No. Time Source Destination Protocol Info 16 0.649949 10.1.1.101 209.225.11.237 HTTP POST /scripts/cms/xcms.asp HTTP/1.1 ... (9 Replies)
Discussion started by: taxi
9 Replies

6. Shell Programming and Scripting

Remove unwanted lines

I have a .xml file, where i need some output. The xml file is like: Code: <?******?></ddddd><sssss>234</dfdffsdf><sdhjh>534</dfdfa>......... /Code I need the output like: code 234 534 . . . /code How can i do it? (5 Replies)
Discussion started by: anupdas
5 Replies

7. Emergency UNIX and Linux Support

Remove Unwanted Libraries - optimizing

We have a huge makefile composing of inclusion of libraries, objects and system libraries to generate a binary. How do we find out that which of the libraries we can remove in the most efficient way? Doing hit and trial method is a waste of time and can during the linking with some post linking... (12 Replies)
Discussion started by: uunniixx
12 Replies

8. Solaris

Remove unwanted packages

I got a system which was installed with SUNWCXall cluster installed on it and i want remove unwanted software like GMNOME, Java Desktop System, Staroffice and numerous other softwares .. i want to do an automated removal of these packages where its uninstalled by itself ..from the is there any... (4 Replies)
Discussion started by: fugitive
4 Replies

9. UNIX for Advanced & Expert Users

How to Remove the unwanted Blank Lines

I have a file with the below data, i would like to remove the end blank lines with no data. I used the below commands but could not able to succeed, could you please shed some light. Commands Used: sed '/^$/d' input.txt > output.txt grep -v '^$' input.txt > output.txt input.txt file... (5 Replies)
Discussion started by: Ariean
5 Replies

10. Shell Programming and Scripting

Remove unwanted XML Tags

I have set of sources and the respective resolution. Please advice how to resolve the same using Unix shell scripting. Source 1: ======= <ext:ContactInfo xmlns:ext="urn:AOL.FLOWS.Extensions"> <ext:InternetEmailAddress>AOL@AOL.COM</ext:InternetEmailAddress> </ext:ContactInfo> Resoultion... (1 Reply)
Discussion started by: ambals123
1 Replies
Login or Register to Ask a Question