Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Logfile Reading

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Logfile Reading
# 1  
Old 01-11-2013
Logfile Reading
I am reading through a log file that has multiple entries:
xx-xxxx-xxx-130111090001
<XML>
...
...
</XML>
...
...
...
xx-xxxx-xxx-130111100001
<XML>
...
...
</XML>
There are 2 parts:
1) Take the XML statement and parse out. I have that with a sed script
sed -n '/<XML>/,/XML>/p' (yes, I know the / is missing)
2) Take the number from the previous line to the <XML> and put it in the XML portion.

The final output would be:
<XML>
<XMLDate>130111090001</XMLDate>
<XMLDate>130111100001</XMLDate>
</XML>

TIA for help
# 2  
Old 01-11-2013
Post a fuller sample of the XML please, with confidential things blanked of course, but at least one good unabridged section, not prettied up. Otherwise we will make code that works for what you posted but not actually for what you want.
# 3  
Old 01-11-2013
Sample
This is a sample of what the file I am reading looks like. There will be miscellaneous log messages that pop up between the XMLStuff blocks.

Code:
XX-XXXXXXX-Prod-WebService-121108045629 (4051801.189)	MESSAGE	CHTTPServer	Got new connection	127.0.0.1:9999		
XX-XXXXXXX-Prod-WebService-121108045631 (4051803.189)	MESSAGE	CHTTPConnection - POST data:	<?xml version='1.0' encoding='UTF-8'?>
			<XMLStuff>
				<Input>
					<Field1></Field1>
					<Field2>123456</Field2>
					<Input2>
					  <Field3>1234</Field3>
						
					</Input2>
				</Input>
			</XMLStuff>			
XX-XXXXXXX-Prod-WebService-121108045631 (4051803.189)	MESSAGE	HTTPConnection: Extracted URL: 				
XX-XXXXXXX-Prod-WebService-121108045631 (4051803.189)	MESSAGE	HTTPConnection: Dispatching transaction	127.0.0.1:9999	XMLStuff
XX-XXXXXXX-Prod-WebService-121109045629 (4051801.189)	MESSAGE	CHTTPServer	Got new connection	127.0.0.1:9999		
XX-XXXXXXX-Prod-WebService-121109045631 (4051803.189)	MESSAGE	CHTTPConnection - POST data:	<?xml version='1.0' encoding='UTF-8'?>
			<XMLStuff>
				<Input>
					<Field1></Field1>
					<Field2>8910</Field2>
					<Input2>
					  <Field3>1234</Field3>
						
					</Input2>
				</Input>
			</XMLStuff>			
XX-XXXXXXX-Prod-WebService-121109045631 (4051803.189)	MESSAGE	HTTPConnection: Extracted URL: 				
XX-XXXXXXX-Prod-WebService-121109045631 (4051803.189)	MESSAGE	HTTPConnection: Dispatching transaction	127.0.0.1:9999	XMLStuff

Last edited by Corona688; 01-11-2013 at 01:46 PM..
# 4  
Old 01-11-2013
Code:
$ cat xmlstuff.awk

BEGIN { print "<xml>"; }
END { print "</xml>"; }

/<XMLStuff>/ {
        print "<xmldate>" D "</xmldate>";
        do      {       getline;        } while(!($0 ~ /XMLStuff>/))
}

{
        split($1, A, "-");
        D=A[5];
}
$ awk -f xmlstuff.awk data
<xml>
<xmldate>121108045631</xmldate>
<xmldate>121109045631</xmldate>
</xml>
$

# 5  
Old 01-11-2013
Sorry....put wrong output
Thanks for the quick response.

I put the incorrect output out there.
Here would be the output
Code:
<XMLStuff>
	<Input>
        <XMLDate>121108045631</XMLDate>
		<Field1></Field1>
					<Field2>123456</Field2>
					<Input2>
					  <Field3>1234</Field3>
						
					</Input2>
				</Input>
         <Input>
         <XMLDate>121109045631</XMLDate>				
					<Field1></Field1>
					<Field2>8910</Field2>
					<Input2>
					  <Field3>1234</Field3>
						
					</Input2>
				</Input>
			</XMLStuff>

Last edited by Corona688; 01-11-2013 at 04:59 PM..
# 6  
Old 01-11-2013
Code:
$ cat xmlstuff.awk
BEGIN { print "<XMLStuff>"; }
END { print "</XMLStuff>"; }

/<XMLStuff>/ {
        getline ; print
        print "<xmldate>" D "</xmldate>";
        getline;
        while(!($0 ~ /XMLStuff>/)) {print ; getline }
}

{
        split($1, A, "-");
        D=A[5];
}
$ awk -f xmlstuff.awk data
<XMLStuff>
                                <Input>
<xmldate>121108045631</xmldate>
                                        <Field1></Field1>
                                        <Field2>123456</Field2>
                                        <Input2>
                                          <Field3>1234</Field3>

                                        </Input2>
                                </Input>
                                <Input>
<xmldate>121109045631</xmldate>
                                        <Field1></Field1>
                                        <Field2>8910</Field2>
                                        <Input2>
                                          <Field3>1234</Field3>

                                        </Input2>
                                </Input>
</XMLStuff>
$

This User Gave Thanks to Corona688 For This Post:
SivaAfi
# 7  
Old 01-11-2013
That works perfectly! Thanks!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Logs from logfile

Hi Team, Have to write a shell script to pick only 1 hr logs from the generated logfile and send it to other logfile. Thanks & Regards, Indu (3 Replies)
Discussion started by: indira_s
3 Replies

3. Shell Programming and Scripting

Reading string and adding the values in the final logfile

Dear all, I have small script which seems to be working but seems to have some bug. It suppose to read commonTxt and then print the noOfLines in outputFile. It is working for most of the txt but unable to add some of the variables values. Can somebody please spend looking at the thread and... (2 Replies)
Discussion started by: emily
2 Replies

4. Shell Programming and Scripting

Parsing Logfile

Hi, I need to continuously monitor a logfile to get the log information between a process start and end. the logfile look like this abcdddddddddd cjjckkkkkkkkkkkk abc : Process started aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbb abc... (6 Replies)
Discussion started by: Byorg
6 Replies

5. Shell Programming and Scripting

Reading and removing lines from logfile

Hello everbody. I'm trying to write a ksh script to find errors inside a logfile. But the main problem I found is that the "Logfile.txt" will got new lines very often. The software generates about 1000 lines per hour. (yes, ~24000 lines/day) "Logfile.txt" (with just very few lines) #... (0 Replies)
Discussion started by: killuayoukai
0 Replies

6. UNIX for Dummies Questions & Answers

writing to a logfile

I cannot get anything to go to my log. This is what i see on my screen when i run my korn shell. cd ok, cwd=/export/home/tsp_inst/TSP pget: /nas4/edata/tsp/rawdata/test2.gz: File exists But i cant get it to write it to my log. cd /this/is/my/newdirectory lftp -e 'pget -c -n 4... (1 Reply)
Discussion started by: tyngsboro
1 Replies

7. Shell Programming and Scripting

looking for string in logfile

I have a shell script that used to look for a particular sting in the last line of a log file. Howeve this string now has moved to the 3rd or 4th line from bottom. Can anyone point me to how i easiest chage this one to look within the last frew lines (5 or so) for the particular string rather... (6 Replies)
Discussion started by: jjlinux
6 Replies

8. Shell Programming and Scripting

logfile parsing

I thought I was pretty handy with awk until I got this one. :) I'm trying to parse a log file where the events could have different delimiters (2 scripts is ok), the errors are spread over multiple lines, and I"m trying to figure out how to not read the same lines that have already been read. ... (1 Reply)
Discussion started by: linkslice
1 Replies

9. Shell Programming and Scripting

logfile

hi iam new of the ksh script.iwant in formation of how to call in logfile in ksh scripts. if the meaning in ksh. please help me thanks naveen.g (1 Reply)
Discussion started by: naveeng.81
1 Replies

10. Shell Programming and Scripting

sort a logfile

Hi experts, I need help to sort a big logfile. logfile: ------- 67712 dkjd jd jj jjjj ------ kjkj jhjh kkk yggg lll hhh gffgf jj -------- i kkk kllkkl ------- Now I want every think between the "------" in one line. Normaly with paste no problem but you can see that the... (7 Replies)
Discussion started by: joerg
7 Replies
Login or Register to Ask a Question