Script to extract fields


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to extract fields
# 1  
Old 12-17-2008
CPU & Memory Script to extract fields

Dear all,

please see the logs shown below:


12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields/><resultcode>999</resultcode>
<account>9773575450</account><transno>1317</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1</inresultcode><inmtransno>229051853686229</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><authorities/><records/></response></agiml>)

Now i want to write a script which will extract the fields like this

TOPUP,9773575450,1317,BPL,1

these are the fields shown in the different tags.

And one thing more the fields between the tag <agiml>............</agiml>,
it is taking it a single column of log.

example less file | grep "9773575450"
will display the whole line.
# 2  
Old 12-17-2008
Are those fields always the same you need?

Btw, better use
Code:
grep "9773575450" file
# instead of
less file | grep "9773575450"

since the less is needless.

Last edited by zaxxon; 12-18-2008 at 05:03 AM.. Reason: changed /quote to /code
# 3  
Old 12-17-2008
Code:
sed '
s/.*<agiml>//  # remove up to <agiml>
s|</agiml>.*|| # remove </agiml> to end of line
s/<[^>]*>/,/g  # convert tags to commas
s/,,*/,/g      # compress multiple commas to a single comma
s/^,//         # remove leading comma
s/,$//         # remove trailing comma
' FILE


Last edited by cfajohnson; 12-17-2008 at 02:42 PM..
# 4  
Old 12-18-2008
Thanks,
It works, but there is a little problem
the output is like this

999,9821376954,1143,BPL
,228889066557802


999,9773575450,1317,BPL
,229051853686229
but i want the second line should come in the first line followed by BPL

Best Regards,
Akhtar Bhat
# 5  
Old 12-18-2008
So "999" is wanted too? In your first post it is not wanted and TOPUP was wanted too. Please give a statement what you want as output.


Edit:

Just in case you want all values between > and <, here you go:
Code:
awk '$1=$1' RS="<[^>]*>" infile2| sed -e '1d; $d'| sed -e :a -e 'N; s/\n/,/; ba'
TOPUP,999,9773575450,1317,BPL,1,229051853686229

Tried cfajohnson's - it worked flawless with my sed too.

Last edited by zaxxon; 12-18-2008 at 09:06 AM..
# 6  
Old 12-18-2008
Dear Sir,
Thanks for the help.
The reply you made is good but it is not working properly example when new logs are coming in the log file, It is shown below

Example logs:-

10-12 11:34:30.452 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields
/><resultcode>999</resultcode><account>9821376954</account><transno>1143</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1<
/inresultcode><inmtransno>228889066557802</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><
authorities/><records/></response></agiml>)
12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields
/><resultcode>999</resultcode><account>9773575450</account><transno>1317</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1<
/inresultcode><inmtransno>229051853686229</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><
authorities/><records/></response></agiml>)



Output of your command :-

TOPUP,999,9821376954,1143,BPL,1,228889066557802,) 12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (,TOPUP,999,9773575450,1317,BPL,1,229051853686229



The output should be like this:-
TOPUP,999,9821376954,1143,BPL,1,228889066557802
TOPUP,999,9773575450,1317,BPL,1,229051853686229



Best Regards,
Akhtar Bhat
# 7  
Old 12-18-2008
cfajohnson's works flawless with me. Maybe try another version of sed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to extract fields from a CSV i.e comma separated where some of the fields having comma as value?

can anyone help me!!!! How to I parse the CSV file file name : abc.csv (csv file) The above file containing data like abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies

2. Shell Programming and Scripting

How to extract fields containing specific strings?

Hello I have a log file with thousands of lines like below Sep 21 13:02:52 lnxtst01 kernel: New TCP in: IN=eth0 OUT= MAC=00:1a:4b:50:b7:32:00:08:e3:ff:fc:04:08:00 SRC=10.184.46.4 DST=10.162.139.21 LEN=60 TOS=0x00 PREC=0x00 TTL=59 ID=52961 DF PROTO=TCP SPT=55688 DPT=22 WINDOW=5840 RES=0x00 SYN... (3 Replies)
Discussion started by: magnus29
3 Replies

3. UNIX for Dummies Questions & Answers

Extract fields before search pattern

Hi, I have below file structure and need to display hours, minutes and seconds as different fields. Incase hour or minute field is not there it should default to zero. *** Total elapsed time was 2 hours, 54 minutes and 40 seconds. *** Total elapsed time was 42 minutes and 36 seconds.... (7 Replies)
Discussion started by: fristyguy
7 Replies

4. Shell Programming and Scripting

Need to run an API from a script and extract fields from output of API

Hi, I need to call an API (GetUsageDetails)from inside a shell script which takes an input argument acct_nbr. The output of API will be like : <usageAccum accumId="450" accumCaptn="PM_125" inclUnits="1410.00" inclUnitsUsed="744.00" shared="true" pooled="false" prorated="false"... (1 Reply)
Discussion started by: rkrish
1 Replies

5. Shell Programming and Scripting

Extract fields from different rows.

Hi, I have data like below. SID=D6EB96CC0 HID=9C246D6 CSource=xya Cappe=1 Versionc=3670 MAR1=STL MARS2=STL REQ_BUFFER_ENCODING=UTF-8 REQ_BUFFER_ORIG_ENCODING=UTF-8 RESP_BODY_ENCODING=UTF-8 CON_ID=2713 I want to select CSource=xya (18 Replies)
Discussion started by: chetan.c
18 Replies

6. UNIX for Dummies Questions & Answers

How to extract fields from etc/passwd file?

Hi! i want to extract from /etc/passwd file,the user and user info fileds, to a another file.I've tried this: cut -d ':' -f1 ':' -f6 < file but cut can be used to extract olny one field and not two. maybe with awk is this possible? (4 Replies)
Discussion started by: strawhatluffy
4 Replies

7. Shell Programming and Scripting

Extract fields

Hi, I have a tmp file like below: <ADATA> ANUM=900 ADESC=Saving ATYP=0 TXREGD=0 </ADATA> <ADATA> ANUM=890 ADESC=Saving ATYP=0 ABAL=9000 TXREGD=1 </ADATA> <ADATA> (2 Replies)
Discussion started by: kunigirib
2 Replies

8. Shell Programming and Scripting

Awk Command to extract Fields

Hi I am newbie.My requirement is: I have start time :Wed Apr 30 3:30:45 End time :Wed Apr 30 3:35:45 which is stored in a file. I have to extract timestamp 3:30:45 using awk. And Subtract the same The script: ---------- #!/bin/ksh LOG_FILE=$1 START_TIME=`head -2 $LOG_FILE... (1 Reply)
Discussion started by: usshell
1 Replies

9. Shell Programming and Scripting

Extract fields from from this

thank youuuu (0 Replies)
Discussion started by: rnallamothu
0 Replies

10. Shell Programming and Scripting

how to extract fields for /etc/passwd

hi, i would like to extract the just name field from /etc/passwd file, and nothing else. ie grep pauline, and Pauline Fowler comes back. cheeers paul mc (6 Replies)
Discussion started by: mopimp
6 Replies
Login or Register to Ask a Question