Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove unwanted data?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove unwanted data?
# 8  
Old 05-09-2007
Quote:
Originally Posted by ahmedwaseem2000
Well, its not removing the complete event what i mean is

Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
You are right.
Code:
sed -e "/^ *$/d" -e "/^Event Id/s/.*/\\
&/" file | awk -v RS="" ' !/Type : INFO/ '

# 9  
Old 05-09-2007
ANBU, thanks for the help. I am sorry, for the example it is working fine but not in the real scenario.

Quote:
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXX
Message :
Auto-purged 23 log entries.
Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXX
Message :
From previous run
Unable to open ODBC trace file ^Oâà/apps/Ascential/DataStage/branded_odbc/lib/odbctrac.so
DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory
5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXX
Message :
Starting Job vrc045005j.
$APT_DB2INSTANCE_HOME = /opt/usr/bin
$APT_CONFIG_FILE = /apps/Ascential/DataStage/Configurations/CONFIG.apt
DB_ALIAS = XXXX
DB_INST = XXXX
DB_NAME = XXXX
PASSWORD = XXXX
USER = XXXX
SERVER = XXXX
this is the exact values
# 10  
Old 05-09-2007
Code:
$ cat file
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXX
Message :
Auto-purged 23 log entries.
Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXX
Message :
From previous run
Unable to open ODBC trace file ^Oâà/apps/Ascential/DataStage/branded_odbc/lib/odbctrac.so
DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory
5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXX
Message :
Starting Job vrc045005j.
$APT_DB2INSTANCE_HOME = /opt/usr/bin
$APT_CONFIG_FILE = /apps/Ascential/DataStage/Configurations/CONFIG.apt
DB_ALIAS = XXXX
DB_INST = XXXX
DB_NAME = XXXX
PASSWORD = XXXX
USER = XXXX
SERVER = XXXX
$ sed -e "/^ *$/d" -e "/^Event Id/s/.*/\\
> &/" file | awk -v RS="" ' !/Type : INFO/ '
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXX
Message :
Auto-purged 23 log entries.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXX
Message :
Starting Job vrc045005j.
$APT_DB2INSTANCE_HOME = /opt/usr/bin
$APT_CONFIG_FILE = /apps/Ascential/DataStage/Configurations/CONFIG.apt
DB_ALIAS = XXXX
DB_INST = XXXX
DB_NAME = XXXX
PASSWORD = XXXX
USER = XXXX
SERVER = XXXX

# 11  
Old 05-09-2007
well, in the message section there are tabs at the first position which i am not able to display here.
# 12  
Old 05-09-2007
if you have Python, here's an alternative:
Code:
#!/usr/bin/python
d={}
for line in open("file"):    
    if line.startswith("Event Id") and not d.has_key(line):
        d[line] = [line]
        temp = line
        continue
    d[temp].append(line)
for key in sorted(d.keys()):
    if not "Type : INFO\n" in d[key]:
        print ''.join(d[key])

output:
Code:
# ./test.py
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXXX
Message :
Auto-purged 23 log entries.

Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXXXXX
Message :
Starting Job XXXXXXX.
$APT_DB2INSTANCE_HOME = XXXXXX
$APT_CONFIG_FILE = XXXXXXXXXX
DB_ALIAS = XXXXXX
DB_INST = XXXXXXXX
DB_NAME = XXXXXXX
PASSWORD = XXXXXX
USER = XXXXXX
SERVER = XXXXXXXX

Event Id: 4918
Time : Tue May 8 23:08:16 2007
Type : WARNING
User : XXXXXXX
Message :
Sequential_File_1: When checking operator: When validating export schema: At field "XXXXXXXXX": Exporting nulla
ble field without null handling properties

# 13  
Old 05-09-2007
I dont have python.

the initial tabs are removed when i try to copy paste in the forum. anyways, I have put the indent now(its tabs in the places of indents). THis gives more clear picture. is there a way i can attach file ????

Quote:
Event Id: 4909
Time : Tue May 8 23:06:06 2007
Type : RESET
User : XXXXX
Message :
Auto-purged 23 log entries.
Event Id: 4910
Time : Tue May 8 23:08:13 2007
Type : INFO
User : XXXXX
Message :
From previous run
Unable to open ODBC trace file XXXXX

DataStage Job 1 Phantom 26294
rm: RT_SC1/jpfile: No such file or directory

5 record(s) selected to SELECT list #1.
DataStage Phantom Finished.
Event Id: 4911
Time : Tue May 8 23:08:13 2007
Type : STARTED
User : XXXXXXX
Message :
Starting Job XXXXXXX.
$APT_DB2INSTANCE_HOME = XXXXXX
$APT_CONFIG_FILE = XXXXXXXXXX
DB_ALIAS = XXXXXX
DB_INST = XXXXXXXX
DB_NAME = XXXXXXX
PASSWORD = XXXXXX
USER = XXXXXX
SERVER = XXXXXXXX
Event Id: 4912
Time : Tue May 8 23:08:14 2007
Type : INFO
User : XXXXXXXXXX
Message :
Environment variable settings:
_=/usr/bin/nohup
MANPATH=/opt/quest/man:
LANG=C
LOGIN=dsadm
APT_LINKER=/usr/vacpp/bin/xlC_r
APT_LINKOPT=-G
APT_MONITOR_MINTIME=10
APT_DB2INSTANCE_HOME=XXXXXXXXX
OSH_STDOUT_MSG=1
Event Id: 4918
Time : Tue May 8 23:08:16 2007
Type : WARNING
User : XXXXXXX
Message :
Sequential_File_1: When checking operator: When validating export schema: At field "XXXXXXXXX": Exporting nulla
ble field without null handling properties
Last edited by hitmansilentass; 05-09-2007 at 06:54 AM..
# 14  
Old 05-09-2007
Code:
sed -n '
/Event Id:/ b block
H
$ b block
b
:block
x
/INFO/!p' infile
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

2. Shell Programming and Scripting

Remove unwanted white space

Hi, I have a very big file 25GB with information present in it like $ head ind_stats update index statistics pfirm001.dbo.Office using 200 values go ... (11 Replies)
Discussion started by: sam05121988
11 Replies

3. Shell Programming and Scripting

How to remove unwanted " from string...

I have this Input File with extra double quotes in the middle. Please suggest how to handle this condition. Input File: "123985","SAW CUT CONCRETE SLAB 20"THICK",,"98.57","","EACH","N" "204312","ARMAFLEX-1 3/8 X 3"",,"2.48","","PER FOOT","N" "205745","MISTING HEAD HOLLOW CONE "C"... (3 Replies)
Discussion started by: BICC
3 Replies

4. Shell Programming and Scripting

How to remove unwanted strings?

Hi Guys, Can someone give me a hand on how I can remove unwanted strings like "<Number>" and "</Number>" and retain only the numbers from the input file below. INPUT FILE: <Number>10050000</Number> <Number>1001340001</Number> <Number>1001750002</Number> <Number>100750003</Number>... (8 Replies)
Discussion started by: pinpe
8 Replies

5. Shell Programming and Scripting

remove unwanted text using perl

Hello..I have a text file that need to remove unwanted text. This is the original file. No. Time Source Destination Protocol Info 16 0.649949 10.1.1.101 209.225.11.237 HTTP POST /scripts/cms/xcms.asp HTTP/1.1 ... (9 Replies)
Discussion started by: taxi
9 Replies

6. Shell Programming and Scripting

Remove unwanted lines

I have a .xml file, where i need some output. The xml file is like: Code: <?******?></ddddd><sssss>234</dfdffsdf><sdhjh>534</dfdfa>......... /Code I need the output like: code 234 534 . . . /code How can i do it? (5 Replies)
Discussion started by: anupdas
5 Replies

7. Emergency UNIX and Linux Support

Remove Unwanted Libraries - optimizing

We have a huge makefile composing of inclusion of libraries, objects and system libraries to generate a binary. How do we find out that which of the libraries we can remove in the most efficient way? Doing hit and trial method is a waste of time and can during the linking with some post linking... (12 Replies)
Discussion started by: uunniixx
12 Replies

8. Solaris

Remove unwanted packages

I got a system which was installed with SUNWCXall cluster installed on it and i want remove unwanted software like GMNOME, Java Desktop System, Staroffice and numerous other softwares .. i want to do an automated removal of these packages where its uninstalled by itself ..from the is there any... (4 Replies)
Discussion started by: fugitive
4 Replies

9. UNIX for Advanced & Expert Users

How to Remove the unwanted Blank Lines

I have a file with the below data, i would like to remove the end blank lines with no data. I used the below commands but could not able to succeed, could you please shed some light. Commands Used: sed '/^$/d' input.txt > output.txt grep -v '^$' input.txt > output.txt input.txt file... (5 Replies)
Discussion started by: Ariean
5 Replies

10. Shell Programming and Scripting

Remove unwanted XML Tags

I have set of sources and the respective resolution. Please advice how to resolve the same using Unix shell scripting. Source 1: ======= <ext:ContactInfo xmlns:ext="urn:AOL.FLOWS.Extensions"> <ext:InternetEmailAddress>AOL@AOL.COM</ext:InternetEmailAddress> </ext:ContactInfo> Resoultion... (1 Reply)
Discussion started by: ambals123
1 Replies
Login or Register to Ask a Question