taking count of log files and den performing some actions


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting taking count of log files and den performing some actions
# 1  
Old 02-20-2009
taking count of log files and den performing some actions

hi,

I've some log files in a path /SYSTEM/custom/data/MNP/xmlerror as:

MNP_PORT_IN_P200902191347563283_20090220181630_err_1.xml
MNP_PORT_IN_P200902171717114365_20090220211814_err_2.xml
MNP_PORT_IN_P200902191349023233_20090220181923_err_1.xml
MNP_PORT_IN_P200902191613464167_20090220121403.xml
MNP_PORT_IN_P200902172005315413_20090220125338_err_2.xml
MNP_PORT_IN_P200902121649264151_20090220160804.xml
MNP_PORT_IN_P200902172005315413_20090220125338_err_3.xml

Now i want the count of only those log files who are not having _err_1, _err_2, etc in the file name ie in the above case its 2 (count of .xml files only without having _err_number) in a day ie in the above case date is 20th Feb 2009 and that to of every half a hr.

ie. in my script i've given a sleep time of 5 mins so i want the count of files from 19:00 to 19:25 and from 19:30 to 20:00 of 20th Feb 2009.

And if the count of files is greater than 25 then an sms should be sent and an script "stop_bulk_engine.sh" should be executed.

Plz help me in this regard.

Thanks.
# 2  
Old 02-20-2009
Considering the sample input you have given...

The below command will give you the xml's which have been generated in 1 day

Code:
$ find *.xml -mtime -1 -print >>out.xml
$cat out.xml
MNP_PORT_IN_P200902121649264151_20090220160804.xml
MNP_PORT_IN_P200902171717114365_20090220211814_err_2.xml
MNP_PORT_IN_P200902172005315413_20090220125338_err_2.xml
MNP_PORT_IN_P200902172005315413_20090220125338_err_3.xml
MNP_PORT_IN_P200902191347563283_20090220181630_err_1.xml
MNP_PORT_IN_P200902191349023233_20090220181923_err_1.xml
MNP_PORT_IN_P200902191613464167_20090220121403.xml

The below will give you the count of files not having _err_1,_err_2 etc...

Code:
$ awk -F "_" '{if ($6!="err") print }' out.xml | wc -l
2

# 3  
Old 02-20-2009
MySQL

Hi,

Hope this also can do that..

Code:
sed -n '/^[A-Z].*[a-z]_[0-9].xml$/!p' inputfile|wc -l

Code:
sed -n '/\([A-Z]\{3\}\)_\([A-Z]\{4\}\)_\([A-Z]\{2\}\)_P\([0-9]\{18\}\)_\([0-9]\{14\}\).xml/p' inputfile|wc -l

Thanks
Sha
# 4  
Old 02-22-2009
I want to extract the timestamp from the file name and then take the count of every half an hr giving a sleep time of 5 mins between every half an hr for a day and taking that count in a variable and then performing some actions.

for example:
for files of 23rd Feb 2009
from 10:00 am to 10:30 am
and from 10:35 am to 11:00 am
and so on till 12:00 am of 24th Feb 2009

and staring from the night 12 of 23rd Feb 2009.

Thanks
# 5  
Old 02-23-2009
hi

below is my script:

#!/bin/sh

smsto1=0132777094
timeout=300

while :
do

TIME=`date '+%H%M'`
START=0700
END=2300


if [ $TIME -ge $START ] && [ $TIME -le $END ]
then

rcvd_file_cnt=`ls -l /SYSTEM/custom/data/MNP/xmlerror | sed -n '/\([A-Z]\{3\}\)_\([A-Z]\{4\}\)_\([A-Z]\{2\}\)_P\([0-9]\{18\}\)_\([0-9]\{14\}\).xml/p' |wc -l`

echo "Count is" $rcvd_file_cnt

[ $rcvd_file_cnt -eq 0 ] && { sendsms_all.exe $smsto1 "(ALERT) Greater Than 25 files received today.";}

[ $rcvd_file_cnt -gt 0 -a $rcvd_file_cnt -le 2 ] && { sendsms_all.exe $smsto1 "(ALERT) Greater Than 25 files received.";}
fi

sleep $timeout

done

In this i want to I want to extract the timestamp from the file name and then take the count of every half an hr giving a sleep time of 5 mins between every half an hr for a day.

But with the above one im getting the count of all files including different days but i need it wrt 1 day and of every half an hr.

please help in the same regard.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Error files count while coping files from source to destination locaton as well count success full

hi All, Any one answer my requirement. I have source location src_dir="/home/oracle/arun/IRMS-CM" My Target location dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct" my source text files check with below example.text file content $fn "\t" $dc "\t" $pc "\t" ... (3 Replies)
Discussion started by: sravanreddy
3 Replies

2. Programming

Performing a while for two files

Hi, This is for Perl. I have a while loop, and would like to process two files, the formats are given below : access access.<previousday>-* How can perform a while loop on both logs, while creating the logic for access.<previousday>-* to get format like "access.20130615-124139" when... (11 Replies)
Discussion started by: susankoperna1
11 Replies

3. Shell Programming and Scripting

Taking the count of sundays between two date ?

Hi Am using unix Ksh Datecalc and --date functions are not working have two input variables as DATE=01/12/2012 DATE1=23/12/2012 Need output as no of sundays = 4 Can anyone help me pls :( (2 Replies)
Discussion started by: Venkatesh1
2 Replies

4. Shell Programming and Scripting

Need to count files & create log of that.

Hi Friends, I need some help. First look at my files hierchachy /<level-1>/<level-2>/<level-3>/*.tif eg. : /2010-07-01/AFFIDAVIT-OF-SERVICE---FOR-SC/001/Babylon2_20100701012049_1278004849.49892_000.tif... (2 Replies)
Discussion started by: paragnehete
2 Replies

5. Shell Programming and Scripting

Need help on Mulitple files mutliple actions

Hi all, I have mistkanely gzipped twice an entire folder and sub folders, and also renamed the files during that process. I am trying to undo this, and I need help to create the batch to work on it. All folders are under my images directory, I have a output.txt file that holds all the... (1 Reply)
Discussion started by: saariko
1 Replies

6. Solaris

troubleshooting log detailing symptoms/error msgs/fix actions for NIS+ client authent

summary found at bottom. to skip straight to action summary, ctrl+f for <summary> this initially started with trouble changing passwords due to client being unable to authenticate, this was further caused by missing client files. This was transparent to me, so this details the road I took,... (0 Replies)
Discussion started by: ProGrammar
0 Replies

7. UNIX for Advanced & Expert Users

Performing Script on Multiple Files

Dear All I have group of files named : CDR.1,CDR.2.,CDR.3,CDR.4,CDR.5,CDR.6,etc....... I am performing an awk command look like this : nawk -f script CDR.* What i want is that i want to perform this command on range of files not all of them. Instead of writing CDR.* i want to write... (3 Replies)
Discussion started by: zanetti321
3 Replies

8. AIX

Script to perform some actions on multiple files

I have this Korn script that I wrote (with some help) that is run by cron. I basically watches a file system for a specific filename to be uploaded (via FTP), checks to make sure that the file is no longer being uploaded (by checking the files size), then runs a series of other scripts. The... (2 Replies)
Discussion started by: heprox
2 Replies

9. UNIX for Dummies Questions & Answers

What are core files and how can I delete them when am performing system maintenance??

Help me How can I go about doing this. also, if you have any idea of other files I can delete and what I can do to improve the performance of a system thats running too slow (6 Replies)
Discussion started by: IMPORTANT
6 Replies
Login or Register to Ask a Question