Sponsored Content
Full Discussion: Help with log manipulation
Top Forums Shell Programming and Scripting Help with log manipulation Post 97581 by StevePace on Tuesday 31st of January 2006 07:01:22 PM
Old 01-31-2006
Help with log manipulation

Hi there,

Scripting isnt my strong point, so any help is appreciated greatly. I have a large log file which contains details of user requests. My aim is to pull out the Time, IP address and source fields from every record but my problem is that when a request fails, the record is incomplete and does not contain an IP or Source field. So when I run my script to extract the required fields I have a mismatched number of Date/Time entries to IP or source entries

An example of a correct record is

JobID: 1885571813506
CallerID: FR1ox8xl
ConfirmationID:
Address: 10.251.86.174
Source: \\10.176.45.16\live\3x\mus\co\vid\NaturalB_SZGZNR_MP4_AAC_56_8_VQ_L0a_STR.3gp

Whereas an incomplete record looks like

JobID: 0
ModuleID: DMSPMSS1::Server
Time: 01/22/2006 14:07:23

So what I need to do is run a script or command on the log to say if JobID =0 then delete that line and the two lines following. That way every Date/Time field will have a corresponding IP & source. I have been told awk is my way forward with this but have never used it and am struggling to come up with the correct logic nevermind syntax!!

Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

manipulation

hi, i have got a table with n number of rows and 2 columns..how can i get get 1 row at a time using any unix command without copying to any file? Thanks and Regards vivek.s (13 Replies)
Discussion started by: vivekshankar
13 Replies

2. Shell Programming and Scripting

More log manipulation

wasn't sure if this should have a new thread since I'm still working on the same job I asked a question about earlier. But it's a different problem/question so I guess it deserves a new thread.... I have a log with roughly 10000 lines in it, each one has a timestamp. The problem is, the time... (8 Replies)
Discussion started by: StevePace
8 Replies

3. Shell Programming and Scripting

File manipulation

To all Guru I have a file like test.txt 111,122,212,123 data11,date12, data13, data14...data1n data21,date22, data23, data24...data2n data31,date32, data33, data34...data3n ... ... .. datan1,daten2, datan3, datan4...datann END I have to load this data into oracle , where first... (1 Reply)
Discussion started by: u263066
1 Replies

4. UNIX for Advanced & Expert Users

File Manipulation

i have a CSV file with 2 columns Fund,Shares 1,100 2,150 3,190 The file is named as AccountNumber.txt(14888.txt,189879.txt...) That files shows the funds in that account I want to create a flat file with accountnumber as one more column since i need accountnumber also.The accoutnnumber... (1 Reply)
Discussion started by: kris01752
1 Replies

5. Shell Programming and Scripting

Text Manipulation.

Hi I have only ever used awk and sed for basic requirements up until now. I have had to break a log down for multiple purposes. Using awk, sed and a date script. I am left with this: (message id, time of msg attempt, message id, domain name, time of msg completion) ... (4 Replies)
Discussion started by: Icepick
4 Replies

6. Shell Programming and Scripting

need help in time manipulation

i have a script that gives the last time the data is inserted in a file. i want to check the time difference b/w the sytem time and the time in file. for e,g. if script runs at 3.30.So $a=3.30 and the time in file is 3.00.So $time =3.00 then the time difference should be 30 min...... ... (6 Replies)
Discussion started by: ali560045
6 Replies

7. UNIX for Dummies Questions & Answers

string manipulation

Hi, I have a file with rows of text like so : E100005568374098100000015667 D100005568374032000000112682 H100005228374060800000002430 I need to grab just the last digits(bolded) of each line without the proceeding text/numbers. Thanks (5 Replies)
Discussion started by: james6
5 Replies

8. Shell Programming and Scripting

String manipulation

Hi, i am just gettin exposed to UNIX. Could anyone of u help me out with dis problem..? i have a variable 'act' which has the value as follows, echo $act gives -0- -0- -----0---- 2008-06-04 -0- -0- echo "$act" | awk '{print ($act)}' gives, -0- -0- -----0---- 2008-06-04 -0- -0- I... (2 Replies)
Discussion started by: jerrynimrod
2 Replies

9. Shell Programming and Scripting

Log File Manipulation

I have a log file which keeps on updating. What I need to do is write a different script, which could moniter the log file continously, and email some stakeholders if some error has occured.. log file gets a message from various sources, and after getting/processing the message, prints an... (1 Reply)
Discussion started by: mukeshguliao
1 Replies

10. Shell Programming and Scripting

Data manipulation, Please help..

Hello, I have a huge set of data that needs to be reformatted. Here is a simple example to explain the process. I have number n=5 and a input with many numbers separated with comma: ... (11 Replies)
Discussion started by: liuzhencc
11 Replies
CONTROL.CTL(5)							File Formats Manual						    CONTROL.CTL(5)

NAME
control.ctl - specify handling of Usenet control messages DESCRIPTION
The file /etc/news/control.ctl is used to determine what action is taken when a control message is received. It is read by the parsecon- trol script, which is called by all the control scripts. (For an explanation of how the control scripts are invoked, see innd(8).) The file consists of a series of lines; blank lines and lines beginning with a number sign (``#'') are ignored. All other lines consist of four fields separated by a colon: message:from:newsgroups:action The first field is the name of the message for which this line is valid. It should be either the name of the control message, or the word ``all'' to mean that it is valid for all messages. The second field is a shell-style pattern that matches the email address of the person posting the message. (The poster's address is first converted to lowercase.) The matching is done using the shell's case statement; see sh (1) for details. If the control message is ``newgroup'' or ``rmgroup'' then the third field specifies the shell-style pattern that must match the group being created or removed. If the control message is of a different type, then this field is ignored. The fourth field specifies what action to take if this line is selected for the message. The following actions are understood: doit The action requested by the control message should be performed. In most cases the control script will also send mail to usenet. doifarg If the control message has an argument, this is treated as a ``doit'' action. If no argument was given, it is treated as a ``mail'' entry. This is used in ``sendsys'' entries script so that a site can request its own newsfeeds(5) entry by posting a ``sendsys mysite'' article. On the other hand, sendsys ``bombs'' ask that the entire newsfeeds file be sent to a forged reply-to address; by using ``doifarg'' such messages will not be processed automatically. doit=file The action is performed, but a log entry is written to the specified log file, file. If file is the word ``mail'' then the record is mailed. A null string is equivalent to /dev/null. A pathname that starts with a slash is taken as the absolute filename to use as the log. All other pathnames are written to /var/log/news/file.log. The log is written by writelog (see newslog(8)). drop No action is taken; the message is ignored. log A one-line log notice is sent to standard error. Innd normally directs this to the file /var/log/news/errlog. log=file A log entry is written to the specified log file, file, which is interpreted as described above. mail A mail message is sent to the news administrator. Lines are matched in order; the last match found in the file is the one that is used. For example, with the following three lines: newgroup:*:*:drop newgroup:tale@*.uu.net:comp.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:doit newgroup:kre@munnari.oz.au:aus.*:mail A newgroup coming from ``tale'' at a UUNET machine will be honored if it is in the mainstream Usenet hierarchy. If ``kre'' posts a new- group message creating ``aus.foo'', then mail will be sent. All other newgroup messages are ignored. HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.8, dated 1996/09/06. SEE ALSO
innd(8), newsfeeds(5), scanlogs(8). CONTROL.CTL(5)
All times are GMT -4. The time now is 09:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy