Help with scritp to count an specific word into a log


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with scritp to count an specific word into a log
# 1  
Old 10-22-2006
Help with scritp to count an specific word into a log

Hello my friends, I need to count how many words are into a log file, I'm using:
Code:
cat logfile | grep 'word' | wc -l

Cuz the 'word' appears once per line.

But my logfile grow faster and at the end ofthe day is really big, so how i can count the 'word' only from (by example) line 4000 of the logfile til the end, then get the last number of line (x) and in the next time count from 'x' line til the new end (x'), that has sence?

Thanx for your help.

Lestat
# 2  
Old 10-22-2006
what you can do is capture the line count to a file.
then whenever you run the script, get the value into a shell variable. with this variable, you can do your processing..
# 3  
Old 10-23-2006
Firstly, you don't need to use cat and wc -l while searching. You can simply use
Code:
 grep -c 'word'  logfile


For searching from a particular line number onwards, you can save the line number in a variable and pass it to gawk or nawk:
Code:
 awk -v num=${saved_line_no} -v word=${word} 'NR>num && $0~word{ count++} END{ print count}' logfile

# 4  
Old 10-23-2006
You can do something like this :
Code:
#!/usr/bin/ksh

#
# Define logile and searched word
#

log_file=logfile
searched_word=word
prv_line_count_file=${log_file}.prv_count

#
# Get previous line count
#

if [ -f ${prv_line_count_file} ]
then
   prv_line_count=$(<${prv_line_count_file})
else
   prv_line_count=0
fi

#
# Get actual line count and memorize it
#

line_count=$(awk 'END {print NR}' ${log_file})
echo ${line_count} > ${prv_line_count_file}

#
# If logfile is smaller, start searching at the first line
#

if [ ${prv_line_count} -gt ${line_count} ]
then
   prv_line_count=0
fi

#
# Search for word
#

awk -v num=${prv_line_count} \
    -v word="${searched_word}" \
    'NR>num && $0~word { count +=1 } END { print count+0 }' \
    ${log_file}

jean-Pierre.
# 5  
Old 09-07-2007
You could do something much simpler:

tail -1000 filename | grep 'word' | wc -l

This will count the the number of lines that word appears in the last 1000 lines of the file (or less if there are not 1000 lines in the file).

So, you could keep a track of the line that you last searched up to, and get the number of lines (wc -l filename) and stick the difference into the -1000 bit.
# 6  
Old 09-07-2007
you have revived an old thread. pls check before posting.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to detect specific string in a log file and count it

Hello, can someone guide me on this? I don't know what is the best approach, (awk script, shell script) I am using RedHat Linux version 6.5. There is a third party application deployed on that server. This app by default generates 5 log files and each file is 20MB. These log rollover... (5 Replies)
Discussion started by: ktisbest
5 Replies

2. UNIX for Beginners Questions & Answers

UNIX script to check word count of each word in file

I am trying to figure out to find word count of each word from my file sample file hi how are you hi are you ok sample out put hi 1 how 1 are 1 you 1 hi 1 are 1 you 1 ok 1 wc -l filename is not helping , i think we will have to split the lines and count and then print and also... (4 Replies)
Discussion started by: mirwasim
4 Replies

3. Shell Programming and Scripting

Search for a specific word and print only the word from the input file

Hi, I have a sample file as shown below, I am looking for sed or any command which prints the complete word only from the input file. Ex: $ cat "sample.log" I am searching for a word which is present in this file We can do a pattern search using grep but I need to cut only the word which... (1 Reply)
Discussion started by: mohan_kumarcs
1 Replies

4. UNIX for Dummies Questions & Answers

How do I count how many times a specific word appear in a file (ksh)?

Hi Please can you help how do I count the number of specific characters or words that appear in a file? (8 Replies)
Discussion started by: fretagi
8 Replies

5. UNIX for Dummies Questions & Answers

How to print line starts with specific word and contains specific word using sed?

Hi, I have gone through may posts and dint find exact solution for my requirement. I have file which consists below data and same file have lot of other data. <MAPPING DESCRIPTION ='' ISVALID ='YES' NAME='m_TASK_UPDATE' OBJECTVERSION ='1'> <MAPPING DESCRIPTION ='' ISVALID ='NO'... (11 Replies)
Discussion started by: tmalik79
11 Replies

6. UNIX for Advanced & Expert Users

Count specific word or character per line

Hi, I need help regarding counting specific word or character per line and validate it against a specific number i.e 10. And if number of character equals the specific number then that line will be part of the output. Specific number = 6 Specific word or char = || Sample data:... (1 Reply)
Discussion started by: janzper
1 Replies

7. Shell Programming and Scripting

Need script to count specific word and iterate over number of files.

Hi Experts, I want to know the count of specific word in a file. I have almost 600+ files. So I want to loop thru each file and get the count of the specific word. Please help me on achieving this... Many thanks (2 Replies)
Discussion started by: elamurugu
2 Replies

8. Shell Programming and Scripting

Extract portion of log info based on specific word

Hi Gurus, I'm using HP-UX B.11.23 operating system. I've been trying to extract a specific wording for example: "A tool used by tp produced warnings" from my below log data, but could not find a way to solve it. My intention is, if the log contain the word: "A tool used by tp produced... (9 Replies)
Discussion started by: superHonda123
9 Replies

9. Shell Programming and Scripting

Word count of lines ending with certain word

Hi all, I am trying to write a command that can help me count the number of lines in the /etc/passwd file ending in bash. I have read through other threads but am yet to find one indicating how to locate a specifc word at the end of a line. I know i will need to use the wc command but when i... (8 Replies)
Discussion started by: warlock129
8 Replies

10. UNIX for Dummies Questions & Answers

How to count the occurences of a specific word in a file in bash shell

Hello, I want to count the occurences of a specific word in a .txt file in bash shell. Can somebody help me pleaze?? Thanks!!! (2 Replies)
Discussion started by: mskart
2 Replies
Login or Register to Ask a Question