More log manipulation


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting More log manipulation
# 1  
Old 02-01-2006
More log manipulation

wasn't sure if this should have a new thread since I'm still working on the same job I asked a question about earlier. But it's a different problem/question so I guess it deserves a new thread....

I have a log with roughly 10000 lines in it, each one has a timestamp. The problem is, the time is logged in GMT and I need to convert to GMT+10 so I can match the lines up to a corresponding entry in another log which has its timestamps set to EST (GMT+10). Anyone know a way to do this??

Thanks in advance
# 2  
Old 02-01-2006
What is the format of the timestamp ? Give us a sample of how the lines look.

What OS are you on ?
# 3  
Old 02-01-2006
example line from logfile set to GMT:

01/22/2006 05:41:51 10.224.200.235 \\10.176.45.16\live\3x\mtv\co\vid\AdultShop.3gp

Unfortunately the other log set to GMT+10 looks more like :

2006 Jan 23 23:34:09:129 GMT 10 DAACAdapter.DAACAdapter Debug [Adapter] H3G_DAAC
-00000 " Country :505" tracking=#z2oqrqhLxrWoL-Oqd5zzyOzUzzw#

So the date is in a different format (but I'm not worried about that.... at least not yet.) and the time stamps are 10 hours apart. (those two log extracts are not related by the way, they're just examples cut from the different logs.)

Cheers
# 4  
Old 02-01-2006
If your system has GNU date, how about....
Code:
$ sed 's/\\/\\\\/g' /var/tmp/logfile | while read date time rest; do
>  timestamp="${date} ${time}";
>  newdate=`date -d "${timestamp} + 10 hours" +"%Y %b %H:%M:%S:00 GMT 10"`;
>  echo "${newdate} ${rest}";
> done

EDIT: Edited to take account of backslashes in input file

Cheers
ZB

Last edited by zazzybob; 02-01-2006 at 12:41 AM..
# 5  
Old 02-01-2006
Cheers Z

How do I tell if I'm using GNU date?? The response when I run this is

date: illegal option -- d
usage: date [-u] mmddHHMM[[cc]yy][.SS]
date [-u] [+format]
date -a [-]sss[.fff]
10.239.214.212 \\10.176.45.16\live\3x\vws\un\vid\PlayboyM15PromoZGZNR_MP4_AAC_54_8_VQ_L0a_STR.3gp

I'm using SunOS 5.8. I also tried replacing date with /usr/xpg4/bin/date to no avail.
# 6  
Old 02-01-2006
Ah, that'll be no to GNU date then!

Easiest way would be to grab the coreutils package from http://www.sunfreeware.com which contains GNU date.

Or use another method (such as Pederabos datecalc script - search the forums).

Cheers
ZB
# 7  
Old 02-01-2006
Dont worry, I've transferred the logs to another machine and the output of the script looks good, just double checking it now.

Nice one, I hope you boys on here get paid a lot of money :-) You're definitely worth it!

Cheers

Steve
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Data manipulation, Please help..

Hello, I have a huge set of data that needs to be reformatted. Here is a simple example to explain the process. I have number n=5 and a input with many numbers separated with comma: ... (11 Replies)
Discussion started by: liuzhencc
11 Replies

2. Shell Programming and Scripting

Log File Manipulation

I have a log file which keeps on updating. What I need to do is write a different script, which could moniter the log file continously, and email some stakeholders if some error has occured.. log file gets a message from various sources, and after getting/processing the message, prints an... (1 Reply)
Discussion started by: mukeshguliao
1 Replies

3. Shell Programming and Scripting

String manipulation

Hi, i am just gettin exposed to UNIX. Could anyone of u help me out with dis problem..? i have a variable 'act' which has the value as follows, echo $act gives -0- -0- -----0---- 2008-06-04 -0- -0- echo "$act" | awk '{print ($act)}' gives, -0- -0- -----0---- 2008-06-04 -0- -0- I... (2 Replies)
Discussion started by: jerrynimrod
2 Replies

4. UNIX for Dummies Questions & Answers

string manipulation

Hi, I have a file with rows of text like so : E100005568374098100000015667 D100005568374032000000112682 H100005228374060800000002430 I need to grab just the last digits(bolded) of each line without the proceeding text/numbers. Thanks (5 Replies)
Discussion started by: james6
5 Replies

5. Shell Programming and Scripting

need help in time manipulation

i have a script that gives the last time the data is inserted in a file. i want to check the time difference b/w the sytem time and the time in file. for e,g. if script runs at 3.30.So $a=3.30 and the time in file is 3.00.So $time =3.00 then the time difference should be 30 min...... ... (6 Replies)
Discussion started by: ali560045
6 Replies

6. Shell Programming and Scripting

Text Manipulation.

Hi I have only ever used awk and sed for basic requirements up until now. I have had to break a log down for multiple purposes. Using awk, sed and a date script. I am left with this: (message id, time of msg attempt, message id, domain name, time of msg completion) ... (4 Replies)
Discussion started by: Icepick
4 Replies

7. UNIX for Advanced & Expert Users

File Manipulation

i have a CSV file with 2 columns Fund,Shares 1,100 2,150 3,190 The file is named as AccountNumber.txt(14888.txt,189879.txt...) That files shows the funds in that account I want to create a flat file with accountnumber as one more column since i need accountnumber also.The accoutnnumber... (1 Reply)
Discussion started by: kris01752
1 Replies

8. Shell Programming and Scripting

File manipulation

To all Guru I have a file like test.txt 111,122,212,123 data11,date12, data13, data14...data1n data21,date22, data23, data24...data2n data31,date32, data33, data34...data3n ... ... .. datan1,daten2, datan3, datan4...datann END I have to load this data into oracle , where first... (1 Reply)
Discussion started by: u263066
1 Replies

9. Shell Programming and Scripting

Help with log manipulation

Hi there, Scripting isnt my strong point, so any help is appreciated greatly. I have a large log file which contains details of user requests. My aim is to pull out the Time, IP address and source fields from every record but my problem is that when a request fails, the record is incomplete... (3 Replies)
Discussion started by: StevePace
3 Replies

10. UNIX for Dummies Questions & Answers

manipulation

hi, i have got a table with n number of rows and 2 columns..how can i get get 1 row at a time using any unix command without copying to any file? Thanks and Regards vivek.s (13 Replies)
Discussion started by: vivekshankar
13 Replies
Login or Register to Ask a Question