I am writing just to share my appreciation for help I have received from this site in the past.
In a previous post Split File by Data Group I received a lot of help with a troublesome awk script to reformat some complicated data blocks. What I learned really came in hand recently when I needed to parse some data from my cluster management logs. We use LSF Batch System from Platform Computing to manage job execution. The logs from this system look like the following:
Code:
Job <302735>, Job Name <SOME_JOB_NAME_HERE>, User <hsimpson>, Proj
ect <default>, Command </path/to/job.sh -sysparm "254 CR66
" -log /path/to/logs/log.log -print /path/to/print.txt>
Tue Aug 24 06:56:48: Submitted from host <computer1>, to Queue <Queue_1>, CWD </
path/to/scripts>, Re-runnable;
Tue Aug 24 06:56:50: Dispatched to <myserver.com>;
Tue Aug 24 06:56:50: Starting (Pid 22582);
Tue Aug 24 06:56:50: Running with execution home </home/hsimpson>, Execution
CWD </path/to/scripts>, Execution Pid <22582>;
Tue Aug 24 06:56:52: Exited with exit code 1. The CPU time used is 0.5 seconds;
Summary of time in seconds spent in various states by Tue Aug 24 06:56:52
PEND PSUSP RUN USUSP SSUSP UNKWN TOTAL
2 0 2 0 0 0 4
------------------------------------------------------------------------------
Job <302742>, Job Name <THIS_IS_A_JOB_NAME_HERE>, User <hsimpson
>, Project <default>, Command </path/to/job.sh -sysparm
"254 CR66" -log /path/to/logs/log.log -print /path/to/
print.txt>
Tue Aug 24 06:57:59: Submitted from host <server4>, to Queue <Queue_2>, CWD </
path/to/scripts>, Re-runnable;
Tue Aug 24 06:58:05: Dispatched to <myserver.com>;
Tue Aug 24 06:58:05: Starting (Pid 28265);
Tue Aug 24 06:58:05: Running with execution home </home/hsimpson>, Execution
CWD </path/to/scripts>, Execution Pid <28265>;
Tue Aug 24 14:14:42: Done successfully. The CPU time used is 3.9 seconds;
Tue Aug 24 14:14:42: Post job process done successfully;
Summary of time in seconds spent in various states by Tue Aug 24 14:14:42
PEND PSUSP RUN USUSP SSUSP UNKWN TOTAL
6 0 26197 0 0 0 26203
------------------------------------------------------------------------------
Job <302845>, Job Name <JOB_NAME_GOES_HERE_HERE_HERE>, User <hsimpson>, Proj
ect <default>, Command </path/to/job.sh -sysparm "254 CR66
" -log /path/to/logs/log.log -print /path/to/print.txt>
Tue Aug 24 08:16:07: Submitted from host <myserver.com>, to Queue
<Queue_3>, CWD </tmp>, Re-runnable;
Tue Aug 24 08:16:11: Dispatched to <myserver.com>;
Tue Aug 24 08:16:11: Starting (Pid 11867);
Tue Aug 24 08:16:11: Running with execution home </home/hsimpson>, Execution CWD <
/tmp>, Execution Pid <11867>;
Tue Aug 24 08:38:50: Exited with exit code 1. The CPU time used is 0.6 seconds;
Summary of time in seconds spent in various states by Tue Aug 24 08:38:50
PEND PSUSP RUN USUSP SSUSP UNKWN TOTAL
4 0 1359 0 0 0 1363
From each of these blocks of log information there are only certain items that interest me, and I want to be able to import those into excel to build stupid pretty charts from.
This is what I want in my output:
Code:
302735,Tue Aug 24 06:56:48,Tue Aug 24 06:56:52,2,2,4,SOME,JOB
302742,Tue Aug 24 06:57:59,Tue Aug 24 14:14:42,6,26197,26203,THIS,IS
302845,Tue Aug 24 08:16:07,Tue Aug 24 08:38:50,4,1359,1363,JOB,NAME
This is the code I used to generate the desired output
I am trying to write an awk program to reformat a data table and convert the date to julian time. I have all the individual steps working, but I am having some issues joing them into one program. Can anyone help me out? Here is my code so far:
# This is an awk program to convert the dates from... (4 Replies)
I am new to Perl. I need to reformat a data file as the last part of a script I am working on. I am stuck on this.
Here is the current format:
CUSTOMER Filename 09/04/07-08:49
CUSTOMER Filename 09/04/07-08:52
CUSTOMER Filename 09/04/07-08:52
CUSTOMER2 Filename 09/04/07-08:49
CUSTOMER2... (3 Replies)
Hi all,
I'm having a little trouble solving a file split I need to get done.
I have the following data:
1. Light
1A. Light Soft
texture: it's soft
color: the color value is that of something light
vital statistics: srm: 23 og: 1.035 sp: 1.065
comment: this is nice if you like... (8 Replies)
I have a file which have data like
A.txt
a
1Jan I am in a1.
1Jan I was born.
2Jan I am here.
3Jan I am in a3.
b
1Jan I am in b1.
c
2Jan I am in c2.
d
2Jan I am in d2.
5jan I am in d5.
date in the file might be vary evertime. (9 Replies)
I have below file listing
] ls -1 *.txt
MISTradesReport_141105_d130240_VOLCKER_EMEA_LOANIQ_FEED_2013-12-24.txt
MISTradesReport_141106_d130240_VOLCKER_NA_LOANIQ_FEED_2013-12-24.txt
MISTradesReport_141107_d130240_VOLCKER_EMEA_CDS_CRDI_FEED_2013-12-24.txt... (4 Replies)