10-31-2008
Awk Script for generating a report
Hi all,
I have a log file of the below format.
[asdfg@st9872] 20081016:000042 asdflasjdf asljfljs asdflasjf safjl
[kronos@jd0121] 20081016:000229 /lask/ajlsdf/askdfjsa
[cak416@po0121] 20081016:000229 /lashflas /askdfaslj hsfhsahf
[close@as0121] 20081016:000304 lasflasj ashfashd
[cak416@sadfjk0121] 20081016:000304 lajfasdf ashfashdfhs
I need to generate a report based on the log file based on the timestamp. I need to search for particular errror messages that were encountered on an hourly basis from the log file. The second field here is the timestamp which is of the form yyyymmdd:hhmmss eg:20081016:000042. I need to create hour by hour statistics from this log file.
like errors encountered between 20081016:000042 and 20081016:230000
I tried using awk inside a shell script, but it was crude way doing things and it depended on the user input for the time, but what I need the script to do is to capture the information automatically from the log file based an hour by our basis.
below is the script that I tried. I am new to awk and I am clueless of doing this.
the variables nmbr1 and nmbr2 are being asked from the user through a shell script.
filename="/opt/tmp/ca/fds/logs/proof.out_20"
filename2="${filename}${dt}"
awk -v nmbr1=${nmbr1} -v nmbr2="$nmbr2" -v outfile="$outfile" '
{
FS=":";
if ((($2-$6) > nmbr1) && (($2-$6) < nmbr2)){
print $2,$3 >>outfile;
}
}
END {}' $filename2
#/home/i4zuss/proof.out_20081016
result=`grep "returned 7" $outfile | wc -l`
echo "Failure Return code 7 =" $result >>/tmp/results.out;
result=`grep "returned 5" $outfile | wc -l`
echo "Failure Return code 5 =" $result >>/tmp/results.out;
result=`grep "Max=30" $outfile | grep "acm" | wc -l`
Let me know if you guys have any idea abt how to solve this.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
My aim is to generate a report using shell script. There are various formats fields coloumns etc.
I want to print in a single line (row) but in different coloumn as given below:
field1 field2 field3 field4
....... ....... ...... .......
The spacing... (1 Reply)
Discussion started by: jisha
1 Replies
2. Shell Programming and Scripting
My file will contain following(log.txt):
start testcase: config loading
......
error XXXX
.....
end testcase: config loading, result failed
start testcase: ping check
.....
error ZZZZZ
.....
error AAAAA
end testcase: Ping check, result failed
I am expecting below output. ... (4 Replies)
Discussion started by: shellscripter
4 Replies
3. Shell Programming and Scripting
Hi,
I want to generate a report for count mismatching.
Steps followed for creating a script
for file in
1). I have to fetch the file name from the checksum.out #customer_information_
2). Added Detail #customer_information_Detail
3). Check the file exist or not.
3.1.1)if the the file... (1 Reply)
Discussion started by: number10
1 Replies
4. Shell Programming and Scripting
Hi,
I am presently stuck in a csv file.
INPUT CSV
baseball,NULL,8798765,Most played
baseball,NULL,8928192,Most played
baseball,NULL,5678945,Most played
cricket,NOTNULL,125782,Usually played
cricket,NOTNULL,678921,Usually played
EXPECTED OUTPUT CSV
... (7 Replies)
Discussion started by: scripter12
7 Replies
5. Shell Programming and Scripting
Hello All,
I need to execute around 15 queries after which all data should come in Excel format.
Executing 15 queries is not a problem.
Problem is how to present/put data in excel.
kindly suggest me how to start, what to study or what should i do.
thanks, (1 Reply)
Discussion started by: shubham8787
1 Replies
6. Shell Programming and Scripting
Hi,
I'm trying to reformat a file to create a new columns reflecting the previous 2 over and over. By that I mean currently each observation has two columns and I want to create a third which has a value equal to 1 minus the sum of the previous two.
This is slightly complicated as 1) I... (6 Replies)
Discussion started by: reformatplink
6 Replies
7. Shell Programming and Scripting
Hi I recently joined a project where I have been asked to generate a report using shell script accessing UNIX box.
I have no idea on how to do it as I am a beginner and learning shell scripts.
Suppose I have a XML:
<XYZRequest>
<effectiveDate>someDate</effectiveDate>... (2 Replies)
Discussion started by: vat1kor
2 Replies
8. Shell Programming and Scripting
Hi All,
What am trying to do is generate the report by compating two files.
File A
-----------
111 22222 3333
222 55555 7777
File B
-----------
11A 22222 3333
333 55555 7778
Now the report should be as follows
Added:
333 55555 7778
Removed: (6 Replies)
Discussion started by: Prashantckc
6 Replies
9. Shell Programming and Scripting
I have a data file similar to this (but many millions of lines long). You can assume that it is totally unsorted but has no duplicate rows.
Date ,Tool_Type ,Tool_ID ,Time_Used
3/13/2014,Screwdriver,Screwdriver02, 6
3/13/2014,Screwdriver,Screwdriver02,20... (2 Replies)
Discussion started by: Michael Stora
2 Replies
10. UNIX for Dummies Questions & Answers
Input:
A|1
B|2
C|3
D|4
Output:
A+B|3
A+C|4
A+D|5
B+C|5
B+D|6
C+D|7
A+B+C|6
A+B+D|7
A+C+D|8
B+C+D|9
A+B+C+D|10
I only managed to get the output for pairs of $1 values (i.e. combination of length 2): (4 Replies)
Discussion started by: beca123456
4 Replies
BF_TAR(1) BF_TAR(1)
NAME
bf_tar - shell script to write a tar file of a bogofilter directory to stdout
SYNOPSIS
bf_tar [-r] [-R] bogofilter_directory
DESCRIPTION
bf_tar bundles a bogofilter working directory in tar format and copies it to standard output (your console, or where you redirect it, see
EXAMPLES below).
OPTIONS
The -r option causes bf_tar to remove inactive log files after the archive has been written successfully. The default is to leave log
files.
The -R option causes bf_tar to remove inactive log files before the archive is written. This may reduce chances that the resulting archive
is recoverable should it become damaged. The archive may be smaller though. The default is to leave log files.
EXIT STATUS
The script exits with status code 0 if everything went well, and nonzero if it encountered trouble.
EXAMPLES
o bf_tar ~/.bogofilter > outfile.tar
Writes a standard .tar file containing the essential files from ~/.bogofilter to outfile.tar.
o bf_tar ~/.bogofilter | gzip -9 -c > outfile.tar.gz
Writes a gzipped .tar.gz file containing the essential files from ~/.bogofilter to outfile.tar.gz.
o bf_tar `pwd`/mydirectory > outfile.tar
Prepend $(pwd)/ or `pwd`/ if you want to specify an absolute path instead of a relative path.
NOTES
This script is meant for use with Berkeley DB based bogofilter versions.
This script requires a SUSv2 compliant pax utility.
This script expects a SUSv2 compliant shell. Solaris systems should have the SUNWxcu4 package installed (when bogofilter is configured) so
that /usr/xpg4/bin/sh can be used.
07/23/2007 BF_TAR(1)