Sponsored Content
Full Discussion: Help with log manipulation
Top Forums Shell Programming and Scripting Help with log manipulation Post 97581 by StevePace on Tuesday 31st of January 2006 07:01:22 PM
Old 01-31-2006
Help with log manipulation

Hi there,

Scripting isnt my strong point, so any help is appreciated greatly. I have a large log file which contains details of user requests. My aim is to pull out the Time, IP address and source fields from every record but my problem is that when a request fails, the record is incomplete and does not contain an IP or Source field. So when I run my script to extract the required fields I have a mismatched number of Date/Time entries to IP or source entries

An example of a correct record is

JobID: 1885571813506
CallerID: FR1ox8xl
ConfirmationID:
Address: 10.251.86.174
Source: \\10.176.45.16\live\3x\mus\co\vid\NaturalB_SZGZNR_MP4_AAC_56_8_VQ_L0a_STR.3gp

Whereas an incomplete record looks like

JobID: 0
ModuleID: DMSPMSS1::Server
Time: 01/22/2006 14:07:23

So what I need to do is run a script or command on the log to say if JobID =0 then delete that line and the two lines following. That way every Date/Time field will have a corresponding IP & source. I have been told awk is my way forward with this but have never used it and am struggling to come up with the correct logic nevermind syntax!!

Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

manipulation

hi, i have got a table with n number of rows and 2 columns..how can i get get 1 row at a time using any unix command without copying to any file? Thanks and Regards vivek.s (13 Replies)
Discussion started by: vivekshankar
13 Replies

2. Shell Programming and Scripting

More log manipulation

wasn't sure if this should have a new thread since I'm still working on the same job I asked a question about earlier. But it's a different problem/question so I guess it deserves a new thread.... I have a log with roughly 10000 lines in it, each one has a timestamp. The problem is, the time... (8 Replies)
Discussion started by: StevePace
8 Replies

3. Shell Programming and Scripting

File manipulation

To all Guru I have a file like test.txt 111,122,212,123 data11,date12, data13, data14...data1n data21,date22, data23, data24...data2n data31,date32, data33, data34...data3n ... ... .. datan1,daten2, datan3, datan4...datann END I have to load this data into oracle , where first... (1 Reply)
Discussion started by: u263066
1 Replies

4. UNIX for Advanced & Expert Users

File Manipulation

i have a CSV file with 2 columns Fund,Shares 1,100 2,150 3,190 The file is named as AccountNumber.txt(14888.txt,189879.txt...) That files shows the funds in that account I want to create a flat file with accountnumber as one more column since i need accountnumber also.The accoutnnumber... (1 Reply)
Discussion started by: kris01752
1 Replies

5. Shell Programming and Scripting

Text Manipulation.

Hi I have only ever used awk and sed for basic requirements up until now. I have had to break a log down for multiple purposes. Using awk, sed and a date script. I am left with this: (message id, time of msg attempt, message id, domain name, time of msg completion) ... (4 Replies)
Discussion started by: Icepick
4 Replies

6. Shell Programming and Scripting

need help in time manipulation

i have a script that gives the last time the data is inserted in a file. i want to check the time difference b/w the sytem time and the time in file. for e,g. if script runs at 3.30.So $a=3.30 and the time in file is 3.00.So $time =3.00 then the time difference should be 30 min...... ... (6 Replies)
Discussion started by: ali560045
6 Replies

7. UNIX for Dummies Questions & Answers

string manipulation

Hi, I have a file with rows of text like so : E100005568374098100000015667 D100005568374032000000112682 H100005228374060800000002430 I need to grab just the last digits(bolded) of each line without the proceeding text/numbers. Thanks (5 Replies)
Discussion started by: james6
5 Replies

8. Shell Programming and Scripting

String manipulation

Hi, i am just gettin exposed to UNIX. Could anyone of u help me out with dis problem..? i have a variable 'act' which has the value as follows, echo $act gives -0- -0- -----0---- 2008-06-04 -0- -0- echo "$act" | awk '{print ($act)}' gives, -0- -0- -----0---- 2008-06-04 -0- -0- I... (2 Replies)
Discussion started by: jerrynimrod
2 Replies

9. Shell Programming and Scripting

Log File Manipulation

I have a log file which keeps on updating. What I need to do is write a different script, which could moniter the log file continously, and email some stakeholders if some error has occured.. log file gets a message from various sources, and after getting/processing the message, prints an... (1 Reply)
Discussion started by: mukeshguliao
1 Replies

10. Shell Programming and Scripting

Data manipulation, Please help..

Hello, I have a huge set of data that needs to be reformatted. Here is a simple example to explain the process. I have number n=5 and a input with many numbers separated with comma: ... (11 Replies)
Discussion started by: liuzhencc
11 Replies
Address(3pm)						User Contributed Perl Documentation					      Address(3pm)

NAME
Palm::Address - Handler for Palm AddressBook databases SYNOPSIS
use Palm::Address; DESCRIPTION
The Address PDB handler is a helper class for the Palm::PDB package. It parses AddressBook databases. AppInfo block The AppInfo block begins with standard category support. See Palm::StdAppInfo for details. Other fields include: $pdb->{appinfo}{lastUniqueID} $pdb->{appinfo}{dirtyFields} I don't know what these are. $pdb->{appinfo}{fieldLabels}{name} $pdb->{appinfo}{fieldLabels}{firstName} $pdb->{appinfo}{fieldLabels}{company} $pdb->{appinfo}{fieldLabels}{phone1} $pdb->{appinfo}{fieldLabels}{phone2} $pdb->{appinfo}{fieldLabels}{phone3} $pdb->{appinfo}{fieldLabels}{phone4} $pdb->{appinfo}{fieldLabels}{phone5} $pdb->{appinfo}{fieldLabels}{phone6} $pdb->{appinfo}{fieldLabels}{phone7} $pdb->{appinfo}{fieldLabels}{phone8} $pdb->{appinfo}{fieldLabels}{address} $pdb->{appinfo}{fieldLabels}{city} $pdb->{appinfo}{fieldLabels}{state} $pdb->{appinfo}{fieldLabels}{zipCode} $pdb->{appinfo}{fieldLabels}{country} $pdb->{appinfo}{fieldLabels}{title} $pdb->{appinfo}{fieldLabels}{custom1} $pdb->{appinfo}{fieldLabels}{custom2} $pdb->{appinfo}{fieldLabels}{custom3} $pdb->{appinfo}{fieldLabels}{custom4} $pdb->{appinfo}{fieldLabels}{note} These are the names of the various fields in the address record. $pdb->{appinfo}{country} An integer: the code for the country for which these labels were designed. The country name is available as $Palm::Address::countries[$pdb->{appinfo}{country}]; $pdb->{appinfo}{misc} An integer. The least-significant bit is a flag that indicates whether the database should be sorted by company. The other bits are reserved. Sort block $pdb->{sort} This is a scalar, the raw data of the sort block. Records $record = $pdb->{records}[N]; $record->{fields}{name} $record->{fields}{firstName} $record->{fields}{company} $record->{fields}{phone1} $record->{fields}{phone2} $record->{fields}{phone3} $record->{fields}{phone4} $record->{fields}{phone5} $record->{fields}{address} $record->{fields}{city} $record->{fields}{state} $record->{fields}{zipCode} $record->{fields}{country} $record->{fields}{title} $record->{fields}{custom1} $record->{fields}{custom2} $record->{fields}{custom3} $record->{fields}{custom4} $record->{fields}{note} These are scalars, the values of the various address book fields. $record->{phoneLabel}{phone1} $record->{phoneLabel}{phone2} $record->{phoneLabel}{phone3} $record->{phoneLabel}{phone4} $record->{phoneLabel}{phone5} Most fields in an AddressBook record are straightforward: the "name" field always gives the person's last name. The "phoneN" fields, on the other hand, can mean different things in different records. There are five such fields in each record, each of which can take on one of eight different values: "Work", "Home", "Fax", "Other", "E-mail", "Main", "Pager" and "Mobile". The $record->{phoneLabel}{phone*} fields are integers. Each one is an index into @Palm::Address::phoneLabels, and indicates which particular type of phone number each of the $record->{phone*} fields represents. $record->{phoneLabel}{display} Like the phone* fields above, this is an index into @Palm::Address::phoneLabels. It indicates which of the phone* fields to display in the list view. $record->{phoneLabel}{reserved} I don't know what this is. METHODS
new $pdb = new Palm::Address; Create a new PDB, initialized with the various Palm::Address fields and an empty record list. Use this method if you're creating an Address PDB from scratch. new_Record $record = $pdb->new_Record; Creates a new Address record, with blank values for all of the fields. The AppInfo block will contain only an "Unfiled" category, with ID 0. "new_Record" does not add the new record to $pdb. For that, you want "$pdb->append_Record". SOURCE CONTROL
The source is in Github: http://github.com/briandfoy/p5-Palm/tree/master AUTHOR
Alessandro Zummo, "<a.zummo@towertech.it>" Currently maintained by brian d foy, "<bdfoy@cpan.org>" SEE ALSO
Palm::PDB(3) Palm::StdAppInfo(3) BUGS
The new() method initializes the AppInfo block with English labels and "United States" as the country. perl v5.10.1 2010-02-23 Address(3pm)
All times are GMT -4. The time now is 02:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy