Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep -c text processing of a log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep -c text processing of a log file
# 1  
Old 10-02-2015
Grep -c text processing of a log file
I have a log file with below format.

Log File:

Code:
1 started job on date & time JOB-A
2 started job on date & time JOB-B
3 completed job on data & time JOB-A
4 started job on date & time JOB-C
5 started job on date & time JOB-D
6 completed job on data & time JOB-B
7 started job on date & time JOB-E
8 started job on date & time JOB-F
9 completed job on data & time JOB-C
10 completed job on data & time JOB-D 
12 completed job on data & time JOB-F

As given above I could see that E is not completed and hence is missing from log file. I am trying to find those jobs which are like E. That is having one entry for started and no entry for completed. I could do a

Code:
grep -c "started" Logfile
grep -c "completed" Logfile

and match them. That doesn't help me to find the jobs which are not completed. I am sure there is a better way to do it. One more thing to add, I don't have the list of job names.
Last edited by Scrutinizer; 10-03-2015 at 08:03 PM.. Reason: quote tags of input file -> code tags
# 2  
Old 10-02-2015
Try:
Code:
while read _ _ _ _ _ _ _ JOBID
do
	count=$(grep "$JOBID" "./LOGFILE"|wc -l)
	[ $count -eq 2 ] && \
		echo "Job $JOBID: All good!" || \
		echo "Job $JOBID: Invalid amount of job entries ($count)..."
done<LOGFILE

hth
This User Gave Thanks to sea For This Post:
ctrld
# 3  
Old 10-02-2015
Try also
Code:
declare -A CNT 
while read _ _ _ _ _ _ _ JOBID
        do if ((++CNT[$JOBID] == 2))
                 then  unset CNT[$JOBID]
           fi
        done< file  
echo ${!CNT[@]}
JOB-E

This User Gave Thanks to RudiC For This Post:
ctrld
# 4  
Old 10-02-2015
Thanks sea and RudiC.. It was fast and elegant as usual.
# 5  
Old 10-03-2015
Getting error with :

Code:
bash -x script
+ declare -A CNT
+ read _ _ _ _ _ _ _ JOBID
+ (( ++CNT[JOB-A] == 2 ))
+ read _ _ _ _ _ _ _ JOBID
+ (( ++CNT[JOB-B] == 2 ))
+ read _ _ _ _ _ _ _ JOBID
+ (( ++CNT[JOB-A] == 2 ))
+ unset 'CNT['
./rfnc.v: line 4: unset: `CNT[': not a valid identifier
+ echo 'JOB-A]'
JOB-A]
+ read _ _ _ _ _ _ _ JOBID

The unset somhow complains but works!?
# 6  
Old 10-03-2015
When you wrote the code you introduced an extra space in then unset CNT[ $JOBID] make sure that it is as unset CNT[$JOBID] with no space between the first [ and the $
# 7  
Old 10-03-2015
Ala, thanks I had found my mistakem real close to what you write :
Code:
then unset CNT[$JOBIDl

'l' instead of ].
Must change the colors of my terminal session!
Thanks to all!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Text File with Binary Values processing

Hello all, I have a txt file containing millions of lines. Below is the example: {tx:be} head -50 file.txt Instr1: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Instr1:... (6 Replies)
Discussion started by: Zam_1234
6 Replies

2. Shell Programming and Scripting

How to grep a log file for words listed in separate text file?

Hello, I want to grep a log ("server.log") for words in a separate file ("white-list.txt") and generate a separate log file containing each line that uses a word from the "white-list.txt" file. Putting that in bullet points: Search through "server.log" for lines that contain any word... (15 Replies)
Discussion started by: nbsparks
15 Replies

3. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

4. UNIX for Dummies Questions & Answers

Take output of processing in text file

Hi ALL, I am presently using perl script mukesh.pl I just want to catch its output into another text file . So I am using > File.txt . I am getting output but i want the whole processing of the script into that file please let me know . Thanks in advance Cheers Mukesh (1 Reply)
Discussion started by: mumakhij
1 Replies

5. Shell Programming and Scripting

Text processing of file

I have a text file which is a dataset. and I need to convert it into a CSV format The file is as follows : First line : -1 3:1 11:1 14:1 19:1 39:1 42:1 55:1 64:1 67:1 73:1 75:1 76:1 80:1 83:1 Second line " +1 5:1 11:1 15:1 32:1 39:1 40:1 52:1 63:1 67:1 73:1 74:1 76:1 78:1 83:1 There are a... (6 Replies)
Discussion started by: ajayram
6 Replies

6. UNIX for Advanced & Expert Users

perl text file processing using hash

Hi Experts, I have this requirement to process large files (200MB+).Format of files is like: recordstart val1 1 val2 2 val3 4 recordstart val1 5 val2 6 val3 1 val4 1 recordstart val1 ... (4 Replies)
Discussion started by: mtomar
4 Replies

7. Shell Programming and Scripting

processing file names using text files

Hi, I have to perform an iterative function on a set of 10 files. After the first round the output files are named differently than the input files. examples input file name = xxxx1.yyy output file name = xxxx1_0001.yyy I need to rename all of the output files to the original input... (5 Replies)
Discussion started by: ligander
5 Replies

8. Shell Programming and Scripting

KSH script -text file processing NULL issues

I'm trying to strip any garbage that may be at the end of my text file and that part is working. The problem only seems to be with the really long lines in the file. When the head command is executed I am directing the output to a new file. The new file always get a null in the 4096 position but... (2 Replies)
Discussion started by: geauxsaints
2 Replies

9. UNIX for Dummies Questions & Answers

text file processing

Hello! There is a text file, that contains hierarchy of menues, like: Aaaaa->Bbbbb Aaaaa->Cccc Aaaaa-> {spaces} Ddddd (it means that the full path is Aaaaa->Cccc->Ddddd ) Aaaaa-> {more spaces} Eeeee (it means that the full path is Aaaaa->Cccc->Ddddd->Eeeee ) Fffffff->Ggggg... (1 Reply)
Discussion started by: alias47
1 Replies

10. UNIX for Dummies Questions & Answers

Processing a text file

A file contains one name per line, such as: john doe jack bruce nancy smith sam riley When I 'cat' the file, the white space is treated as a new line. For example list=`(cat /path/to/file.txt)` for items in $list do echo $items done I get: john doe (1 Reply)
Discussion started by: TheCrunge
1 Replies
Login or Register to Ask a Question