03-12-2017
Help in capturing Time from Autosys Logs
Hi Guys, I'm very new to Shell scripting and have to design a code which I'm not able to find a way to. I will try to explain the aim in detail and shall be obliged if anyone could help me with the coding snippet.
I have an input file who's every row has a few details about an autosys Job. I shall write down the first entry of the file so that should give an idea :-
1,,Task Name,Application Name,Frequency of Job,expected starting Date and Time(2017-03-14 01:00 AM), Box Job name, Command Job Name
So in each row of the file there is a command Job name, for which there is an expected starting date and time in the input file itself. These two things are given in the input file itself. I have to now extract the latest log of this command job, from it's header extract the start time and compare it with the expected start date and time in the input file, if the extracted date and time from the log are greater than the expected one, it is indeed the log for that day and not a previous log. Also, the time in the log is of 24 hour format and the one in the input file is 12 hour.
Furthermore, for every row, after doing the above analysis, I have to add two more columns which are completion time if it is has completed and the current status of the Job i.e Scheduled, Running, Completed and failed.
If the Job has completed or failed, the log's last line shows "++++COMPLETED++++" or "++++FAILED++++" so maybe that can be used.
Please help me in coding the above scenario. Any assistance shall be of great help. Please help guys!!
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi All,
I am working on a korn shell script.
i have a file such as:
DS.PETSCO.20060601203514.20060531.ctl_20060717124431
i have 2 problems here.
1) i have to capture the time stamp from
the above file i.e this number 20060717124431.
format of time stamp is YYYYMMDDHHMMSS.
can... (4 Replies)
Discussion started by: pavan_test
4 Replies
2. UNIX for Dummies Questions & Answers
Hi All,
I am working on a korn shell script.
I have a problem.i am calculating the next 27 hours
from the time stamp.
say TSTAMP=20060724000000
if i add 27 hours to the above time stamp,
i will get 20060724143000
this is the code:
YEAR=`echo $TSTAMP | awk... (0 Replies)
Discussion started by: pavan_test
0 Replies
3. UNIX for Dummies Questions & Answers
Newbie question:
I wrote korn shell script that lets me connect to a cisco switch thru telnet from sun server. I'm wodering if or what command i would use to capture info that is being sent to standard output when the script is running. Putting part of my script below and results.
#!/bin/ksh... (2 Replies)
Discussion started by: wisher115
2 Replies
4. Shell Programming and Scripting
Hi Guys,
I am trying to capture the FTP Logs in a log file.
I am using the below code.
ftp -d -n -v $Remote_Host << EOD > $Ftp_LOG;
Since iam running the script in debug mode, i am able to see that the ftp is done and the file has been transferred.
But the log file does not have... (7 Replies)
Discussion started by: mac4rfree
7 Replies
5. Emergency UNIX and Linux Support
Hi,
I have been trying to capture console logs from the init script.
When the ramfs is mounted, i check if usb is connected , if conncted, i mount it and redirected the console logs like so:
cat & /dev/ttyS1 >> /mnt/logs.txt
I'm getting
/bin/sh : /dev/ttyS1 :permission denied
... (8 Replies)
Discussion started by: xerox
8 Replies
6. Shell Programming and Scripting
I have two set of questions.
1) To skip killing some process automatically.
2) To kill other process and capture their log.
I have set of process, some needs to be killed gracefully and others should be skipped.
Listed are the process.
adm 1522... (1 Reply)
Discussion started by: murali1687
1 Replies
7. Shell Programming and Scripting
I have a file that is created via a perl script where the file is named like so: 01-07-2016_10:17:08. I am running a shell script that needs to take this file and print it. I can capture the date portion fine, but I am unsure how to capture the time stamp, since there will be a difference from what... (1 Reply)
Discussion started by: ldorsey
1 Replies
8. UNIX for Beginners Questions & Answers
Hi Guys, I'm very new to Shell scripting and have to design a code which I'm not able to find a way to. I will try to explain the aim in detail and shall be obliged if anyone could help me with the coding snippet.
I have an input file who's every row has a few details about an autosys Job. I shall... (1 Reply)
Discussion started by: Crusnik02
1 Replies
9. Shell Programming and Scripting
I have a script
#!/bin/bash
HOST=ftp.example.com
USER=ftpuser
PASSWORD=P@ssw0rd
ftp -inv $HOST <<EOF
user $USER $PASSWORD
cd /path/to/file
mput *.html
bye
EOF
the script executes sucessfully I need to capture the FTP logs to a logfile
should contain
FTP Login successful
... (1 Reply)
Discussion started by: rajeshas83
1 Replies
LEARN ABOUT DEBIAN
condor_userlog
condor_userlog(1) General Commands Manual condor_userlog(1)
Name
condor_userlog Display - and summarize job statistics from job log files.
Synopsis
condor_userlog [-help] [-total | -raw] [-debug] [-evict] [-j cluster | cluster.proc] [-all] [-hostname] logfile ...
Description
condor_userlog parses the information in job log files and displays summaries for each workstation allocation and for each job. See the
manual page for condor_submiton page for instructions for specifying that Condor write a log file for your jobs.
If -totalis not specified, condor_userlogwill first display a record for each workstation allocation, which includes the following informa-
tion:
Job
The cluster/process id of the Condor job.
Host
The host where the job ran. By default, the host's IP address is displayed. If -hostnameis specified, the host name will be displayed
instead.
Start Time
The time (month/day hour:minute) when the job began running on the host.
Evict Time
The time (month/day hour:minute) when the job was evicted from the host.
Wall Time
The time (days+hours:minutes) for which this workstation was allocated to the job.
Good Time
The allocated time (days+hours:min) which contributed to the completion of this job. If the job exited during the allocation, then this
value will equal ``Wall Time.'' If the job performed a checkpoint, then the value equals the work saved in the checkpoint during this
allocation. If the job did not exit or perform a checkpoint during this allocation, the value will be 0+00:00. This value can be greater
than 0 and less than ``Wall Time'' if the application completed a periodic checkpoint during the allocation but failed to checkpoint
when evicted.
CPU Usage
The CPU time (days+hours:min) which contributed to the completion of this job.
condor_userlogwill then display summary statistics per host:
Host/Job
The IP address or host name for the host.
Wall Time
The workstation time (days+hours:minutes) allocated by this host to the jobs specified in the query. By default, all jobs in the log are
included in the query.
Good Time
The time (days+hours:minutes) allocated on this host which contributed to the completion of the jobs specified in the query.
CPU Usage
The CPU time (days+hours:minutes) obtained from this host which contributed to the completion of the jobs specified in the query.
Avg Alloc
The average length of an allocation on this host (days+hours:minutes).
Avg Lost
The average amount of work lost (days+hours:minutes) when a job was evicted from this host without successfully performing a checkpoint.
Goodput
This percentage is computed as Good Time divided by Wall Time.
Util.
This percentage is computed as CPU Usage divided by Good Time.
condor_userlogwill then display summary statistics per job:
Host/Job
The cluster/process id of the Condor job.
Wall Time
The total workstation time (days+hours:minutes) allocated to this job.
Good Time
The total time (days+hours:minutes) allocated to this job which contributed to the job's completion.
CPU Usage
The total CPU time (days+hours:minutes) which contributed to this job's completion.
Avg Alloc
The average length of a workstation allocation obtained by this job in minutes (days+hours:minutes).
Avg Lost
The average amount of work lost (days+hours:minutes) when this job was evicted from a host without successfully performing a checkpoint.
Goodput
This percentage is computed as Good Time divided by Wall Time.
Util.
This percentage is computed as CPU Usage divided by Good Time.
Finally, condor_userlogwill display a summary for all hosts and jobs.
Options
-help
Get a brief description of the supported options
-total
Only display job totals
-raw
Display raw data only
-debug
Debug mode
-j
Select a specific cluster or cluster.proc
-evict
Select only allocations which ended due to eviction
-all
Select all clusters and all allocations
-hostname
Display host name instead of IP address
General Remarks
Since the Condor job log file format does not contain a year field in the timestamp, all entries are assumed to occur in the current year.
Allocations which begin in one year and end in the next will be silently ignored.
Exit Status
condor_userlogwill exit with a status value of 0 (zero) upon success, and it will exit with the value 1 (one) upon failure.
Author
Condor Team, University of Wisconsin-Madison
Copyright
Copyright (C) 1990-2012 Condor Team, Computer Sciences Department, University of Wisconsin-Madison, Madison, WI. All Rights Reserved.
Licensed under the Apache License, Version 2.0.
See the Condor Version 7.8.2 Manualor http://www.condorproject.org/licensefor additional notices. condor-admin@cs.wisc.edu
September 2012 condor_userlog(1)