Help with log manipulation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with log manipulation
# 1  
Old 01-31-2006
Help with log manipulation

Hi there,

Scripting isnt my strong point, so any help is appreciated greatly. I have a large log file which contains details of user requests. My aim is to pull out the Time, IP address and source fields from every record but my problem is that when a request fails, the record is incomplete and does not contain an IP or Source field. So when I run my script to extract the required fields I have a mismatched number of Date/Time entries to IP or source entries

An example of a correct record is

JobID: 1885571813506
CallerID: FR1ox8xl
ConfirmationID:
Address: 10.251.86.174
Source: \\10.176.45.16\live\3x\mus\co\vid\NaturalB_SZGZNR_MP4_AAC_56_8_VQ_L0a_STR.3gp

Whereas an incomplete record looks like

JobID: 0
ModuleID: DMSPMSS1::Server
Time: 01/22/2006 14:07:23

So what I need to do is run a script or command on the log to say if JobID =0 then delete that line and the two lines following. That way every Date/Time field will have a corresponding IP & source. I have been told awk is my way forward with this but have never used it and am struggling to come up with the correct logic nevermind syntax!!

Thanks
# 2  
Old 01-31-2006
How about:
sed '/JobID: 0/{N;N;d;}' < data
# 3  
Old 01-31-2006
Assuming there are no blank lines in the logfile between records
Code:
sed -n '/^JobID: 0/ {N;N;d;}; p' steve.txt

EDIT: Posted at *exactly* the same time as Perderabo Smilie Except mine has a superfluous -n and p Smilie

Cheers
ZB
# 4  
Old 01-31-2006
I did not expect that quick a reply, thank you very very much. That worked a treat Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Data manipulation, Please help..

Hello, I have a huge set of data that needs to be reformatted. Here is a simple example to explain the process. I have number n=5 and a input with many numbers separated with comma: ... (11 Replies)
Discussion started by: liuzhencc
11 Replies

2. Shell Programming and Scripting

Log File Manipulation

I have a log file which keeps on updating. What I need to do is write a different script, which could moniter the log file continously, and email some stakeholders if some error has occured.. log file gets a message from various sources, and after getting/processing the message, prints an... (1 Reply)
Discussion started by: mukeshguliao
1 Replies

3. Shell Programming and Scripting

String manipulation

Hi, i am just gettin exposed to UNIX. Could anyone of u help me out with dis problem..? i have a variable 'act' which has the value as follows, echo $act gives -0- -0- -----0---- 2008-06-04 -0- -0- echo "$act" | awk '{print ($act)}' gives, -0- -0- -----0---- 2008-06-04 -0- -0- I... (2 Replies)
Discussion started by: jerrynimrod
2 Replies

4. UNIX for Dummies Questions & Answers

string manipulation

Hi, I have a file with rows of text like so : E100005568374098100000015667 D100005568374032000000112682 H100005228374060800000002430 I need to grab just the last digits(bolded) of each line without the proceeding text/numbers. Thanks (5 Replies)
Discussion started by: james6
5 Replies

5. Shell Programming and Scripting

need help in time manipulation

i have a script that gives the last time the data is inserted in a file. i want to check the time difference b/w the sytem time and the time in file. for e,g. if script runs at 3.30.So $a=3.30 and the time in file is 3.00.So $time =3.00 then the time difference should be 30 min...... ... (6 Replies)
Discussion started by: ali560045
6 Replies

6. Shell Programming and Scripting

Text Manipulation.

Hi I have only ever used awk and sed for basic requirements up until now. I have had to break a log down for multiple purposes. Using awk, sed and a date script. I am left with this: (message id, time of msg attempt, message id, domain name, time of msg completion) ... (4 Replies)
Discussion started by: Icepick
4 Replies

7. UNIX for Advanced & Expert Users

File Manipulation

i have a CSV file with 2 columns Fund,Shares 1,100 2,150 3,190 The file is named as AccountNumber.txt(14888.txt,189879.txt...) That files shows the funds in that account I want to create a flat file with accountnumber as one more column since i need accountnumber also.The accoutnnumber... (1 Reply)
Discussion started by: kris01752
1 Replies

8. Shell Programming and Scripting

File manipulation

To all Guru I have a file like test.txt 111,122,212,123 data11,date12, data13, data14...data1n data21,date22, data23, data24...data2n data31,date32, data33, data34...data3n ... ... .. datan1,daten2, datan3, datan4...datann END I have to load this data into oracle , where first... (1 Reply)
Discussion started by: u263066
1 Replies

9. Shell Programming and Scripting

More log manipulation

wasn't sure if this should have a new thread since I'm still working on the same job I asked a question about earlier. But it's a different problem/question so I guess it deserves a new thread.... I have a log with roughly 10000 lines in it, each one has a timestamp. The problem is, the time... (8 Replies)
Discussion started by: StevePace
8 Replies

10. UNIX for Dummies Questions & Answers

manipulation

hi, i have got a table with n number of rows and 2 columns..how can i get get 1 row at a time using any unix command without copying to any file? Thanks and Regards vivek.s (13 Replies)
Discussion started by: vivekshankar
13 Replies
Login or Register to Ask a Question