Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

[awk] combine and convert time from log files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [awk] combine and convert time from log files
# 1  
Old 08-14-2012
SOLVED - [awk] combine and convert time from log files
dear all,

an awk newbie need your help.... i have log files with this format:

mylog1a.log:
Code:
"08/10/2012","5:05 PM"
"Hostname","Device Address","Count"
"","10.10.10.18","10234"

mylog2a.log:
Code:
"08/11/2012","5:05 PM"
"Hostname","Device Address","Count"
"","10.10.10.18","12543"

mylog1b.log:
Code:
"08/10/2012","5:05 PM"
"Hostname","Device Address","Count"
"myserver","10.10.10.19","13467"

mylog2b.log:
Code:
"08/11/2012","5:05 PM"
"Hostname","Device Address","Count"
"myserver","10.10.10.19","9362"

and i want to combine them into one log file with this result:
Code:
start time,end time,hostname,device address,count
08/10/2012 17:05,08/11/2012 17:05,,10.10.10.18,12543
08/10/2012 17:05,08/11/2012 17:05,myserver,10.10.10.19,13467

explanation for combine log:
- start time is taken from date first log
- end time is taken from date on second log
- time need to be convert to 24 hours format
- all quote characters should be removed
Last edited by makan; 09-19-2012 at 03:57 AM..
# 2  
Old 08-14-2012
And from which log file does one take the values of Hostname, Device Address and Count? first or second? Your output seems to be confusing in this aspect.
# 3  
Old 08-14-2012
Quote:
Originally Posted by elixir_sinari
And from which log file does one take the values of Hostname, Device Address and Count? first or second? Your output seems to be confusing in this aspect.
hi elixir,

actually, they are daily logs. you can assume that mylog1a.log is day 1 log and mylog2a.log is day 2 log for device 10.10.10.18. and it same for mylog1b.log and mylog2b for device 10.10.10.19

mylog1a.log is related with mylog2a.log, so hostname is taken from these files. but in this case hostname is not specified on log files so it should be empty ",,"
# 4  
Old 08-14-2012
How do you get the values which have been highlighted in red and in bold face:
Code:
start time,end time,hostname,device address,count
08/10/2012 17:05,08/11/2012 17:05,,10.10.10.18,12543
08/10/2012 17:05,08/11/2012 17:05,myserver,10.10.10.19,13467

# 5  
Old 08-14-2012
Quote:
Originally Posted by elixir_sinari
How do you get the values which have been highlighted in red and in bold face:
Code:
start time,end time,hostname,device address,count
08/10/2012 17:05,08/11/2012 17:05,,10.10.10.18,12543
08/10/2012 17:05,08/11/2012 17:05,myserver,10.10.10.19,13467

they are just taken from log, no calculation needed.

the idea is pulling data from daily logs and put them into a log file that will be send to other application.
# 6  
Old 08-14-2012
You are not getting what I am asking:

Code:
last line of mylog1a.log --> "","10.10.10.18","10234"
last line of mylog2a.log --> "","10.10.10.18","12543"

How does your output, corresponding to this log, have 12543 and not 10234?

Code:
last line of mylog1b.log --> "myserver","10.10.10.19","13467"
last line of mylog2b.log --> "myserver","10.10.10.19","9362"

How does your output, corresponding to this log, have 13467 and not 9362?
# 7  
Old 08-14-2012
oh, ok...
when we execute the script on day 08/11 the result is:
Code:
08/10/2012 17:05,08/11/2012 17:05,,10.10.10.18,10234
08/10/2012 17:05,08/11/2012 17:05,myserver,10.10.10.19,13467

and when we execute the script on day 08/12
Code:
08/11/2012 17:05,08/12/2012 17:05,,10.10.10.18,12543
08/11/2012 17:05,08/12/2012 17:05,myserver,10.10.10.19,9362

where "08/12/2012 17:05" value is retrive from a log that created on 08/12/2012 17:05


but if it is difficult to solve, maybe we can remove/eliminate the end_time field so the result:

when we execute the script on day 08/11 the result is:
Code:
08/10/2012 17:05,,10.10.10.18,10234
08/10/2012 17:05,myserver,10.10.10.19,13467

and when we execute the script on day 08/12
Code:
08/11/2012 17:05,,10.10.10.18,12543
08/11/2012 17:05,myserver,10.10.10.19,9362


i hope this makes my question clear...
Last edited by makan; 08-14-2012 at 11:50 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert UTC time into current UNIX sever time zone

Hi guys thanks for the help for my previous posts.Now i have a requirement that i download a XMl file which has UTC time stamp.I need to convert UTC time into Unix server timezone. For ex if the time zone of unix server is CDT then i need to convert into CDT.whatever may be the system time... (5 Replies)
Discussion started by: mohanalakshmi
5 Replies

2. Shell Programming and Scripting

Combine multiple unique lines from event log text file into one line, use PERL or AWK?

I can't decide if I should use AWK or PERL after pouring over these forums for hours today I decided I'd post something and see if I couldn't get some advice. I've got a text file full of hundreds of events in this format: Record Number : 1 Records in Seq : ... (3 Replies)
Discussion started by: Mayday22
3 Replies

3. Shell Programming and Scripting

How to combine two files with awk?

Hi, everyone! I have two files, I want to combine them as follows: File1 AAAA 23 45 AAAB 44 56 AAAC 34 65 AAAD 34 87 File2 AAAA 34 54 AAAE 34 56 Combined file AAAA 23 45 34 54 AAAB 44 56 AAAC 34 65 AAAD 34 87 AAAE 34 56 (13 Replies)
Discussion started by: xshang
13 Replies

4. Shell Programming and Scripting

Using awk or nawk to convert epoch time to date format

Looking for some help and usually when I do a search this site comes up. Hopefully someone can give me a little direction as to how to use one of these two commands to achieve what I'm trying to do. What am I trying to do? I need to take the time value in epoch format returned from the... (5 Replies)
Discussion started by: minigts
5 Replies

5. UNIX Desktop Questions & Answers

Convert time with AWK

Hi All, I need your help with the following: My input shows the time in this format 1311547776493 I need to convert it (as part of my AWK script) into this format: 18-08-2011 09:35:11.072 Thanks a lot! Royi (4 Replies)
Discussion started by: royip
4 Replies

6. Shell Programming and Scripting

How can view log messages between two time frame from /var/log/message or any type of log files

How can view log messages between two time frame from /var/log/message or any type of log files. when logfiles are very big and especially many messages with in few minutes, I would like to display log messages between 5 minute interval. Could you pls give me the command? (1 Reply)
Discussion started by: johnveslin
1 Replies

7. Shell Programming and Scripting

Shell script to convert epoch time to real time

Dear experts, I have an epoch time input file such as : - 1302451209564 1302483698948 1302485231072 1302490805383 1302519244700 1302492787481 1302505299145 1302506557022 1302532112140 1302501033105 1302511536485 1302512669550 I need the epoch time above to be converted into real... (4 Replies)
Discussion started by: aismann
4 Replies

8. Shell Programming and Scripting

Find and Convert UTC Time to PST Time

Hello All - I have a script that grabs data from the net and outputs the following data 46029 46.144 -124.510 2010 07 26 22 50 320 4.0 6.0 2.2 9 6.8 311 1012.1 -0.9 13.3 13.5 13.3 - - 46041 47.353 -124.731 2010 07 26 22 50 250 2.0 3.0 1.6 8 6.4 - 1011.6 - ... (0 Replies)
Discussion started by: drexnefex
0 Replies

9. Shell Programming and Scripting

Convert Epoch Time to Standard Date and Time & Vice Versa

Hi guys, I know that this topic has been discuss numerous times, and I have search the net and this forum for it. However, non able to address the problem I faced so far. I am on Solaris Platform and unable to install additional packages like the GNU date and gawk to make use of their... (5 Replies)
Discussion started by: DrivesMeCrazy
5 Replies

10. Shell Programming and Scripting

awk convert seconds to time of day

Does anyone know of a way to convert "seconds" to time of day in "hh:mm:ss" ? Trying to do in awk with strftime but with no luck. Thanks (2 Replies)
Discussion started by: timj123
2 Replies
Login or Register to Ask a Question