Help parsing logs maybe with menu and variables?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help parsing logs maybe with menu and variables?
# 1  
Old 09-17-2009
Help parsing logs maybe with menu and variables?

I would like to parse through some logs looking for things like exception or failed (grep -i failed). Ideal would be if it were in a menu format so someone without unix ability could just choose option 1 2 or 3 etc. If I could pass the hostname to a variable also that would be awesome, so someone could input the hostname and the script would know for hostname 'name' do the following stuff, look through the logs and find exceptions? I appreciate anyone's help, I'm not very good at scripts. Thanks.

this is some info on the server pwd. Oh, since there are so many logs, it would be nice if it could do it by day, maybe pass the date to a variable or something like that so someone could search by date?

Code:
archive]$ pwd
/home/username/current_release/logs/archive
 archive]$ ls -al

dassimulator.eif.2009_245_155007.381.log   demo-sdif.ops.2009_252_190302.853.log
demo-dsdm.ops.2009_260_123941.256.log      demo-snif.ops.2009_260_135053.210.log
demo-HMD-sdif.ops.2009_260_133644.200.log  demo-sve.ops.2009_260_135208.928.log
demo-sam.ops.2009_259_231215.066.log


---------- Post updated at 02:25 PM ---------- Previous update was at 11:32 AM ----------

Also, if you write a script and do this:

Quote:
cd /home/henry
pwd
ls -al *.log>/home/henry/logfiles
cat /home/henry/logfiles |awk '{print $9}
Output is a filename
If I want to take that filename and pass it to grep -i failure
so I can see which files had failures? How can I do that?
I tried |grep -i failure but it didn't work, also < grep -i failure didn't work.
~

Last edited by Neo; 09-21-2009 at 03:53 PM.. Reason: please use code tags
# 2  
Old 09-17-2009
I would probably send all the logs to a syslog server and then use splunk to parse through them. It's a more generic solution that's already been written.
# 3  
Old 09-17-2009
I'm sorry, I don't think we have a syslog server. This is a small operation. I just need a menu to parse through the logs, maybe pick up the date and hostname and put them in variables to use with the menu. Smilie
# 4  
Old 09-21-2009
Can anyone please help? Thanks.
# 5  
Old 09-21-2009
I'm not sure a syslog server is actually required. have you at least looked at splunk?
# 6  
Old 09-22-2009
OK, I now have a menu script I got here from someone else. I want to change the menu a bit though.

It looks like this:


$ cat menu.sh
#!/bin/bash
#set -vx

while :
do
clear
# Display menu
echo
echo "*************************************************************"
echo "Please choose from the following options; type the"
echo "option number and hit the <Enter> key."
echo
echo " 1) To list names of the log files in the current DIR"
echo " 2) Display today's date and time"
echo " 3) Display a sorted list of people currently logged on"
echo " 4) Display whether a file is a file or a DIR"
echo " 5) Create a backup for a file"
echo " 6) Find a user by First of Last name in /etc/passwd file"
echo " 7) Find the manual pages for a specific command"
echo " 8) Exit"
echo
echo "*************************************************************"
read option
case "$option" in
1) echo "The files in the current DIR are: "
ls -al
echo "Hit <Enter> to continue."
read ignore
;;
2) echo "The current date is: "
date
unset date
echo "Hit <Enter> to continue."
read ignore
;;

At this point, I'd like to change the menu to do this:

echo ---------------------------------
cd /var/log
# /var/log/secure section
cat /var/log/secure |grep -i 'password check failed'|awk '{print $1,$2,$3,$6,$7,
$8,$11}'|sort -u|more
cat /var/log/secure |grep -i 'password check failed'|wc -l |awk '{print $0}'
cat /var/log/secure |grep -i 'sudo'|awk '{print $1,$2,$3,$7,$8,$9,$10}'|sort -u
cat /var/log/secure |grep -i 'sudo'|wc -l |echo sudo used

cat /var/log/secure |grep -i 'su'|awk '{print $1,$2,$3,$5,$7,$8,$11}'|sort -u
cat /var/log/secure |grep -i 'authentication failure'|awk '{print $1,$2,$3,$13}'
|sort -u
cat /var/log/secure |grep -i 'rhost'|sort -u

cat /var/log/secure |grep -i 'could not identify password' |awk '{print $1,$2,$3
,$13}'|sort -u

esac
done

I want it to cat the files and grep for certain items? Do I need another read statement? What should it be? I'd like to get this done today, as my meeting is in 4 hours. Any help would be appreciated. I tried this and it didn't work:

4) echo "Display problems with /var/log/secure and messages"
# read fdname

# if [ ! -e $fdname ]; then

# echo "$fdname does not exist."

# elif [ -d $fdname ]; then

# echo "$fdname is a directory."

# elif [ -f $fdname ]; then

# echo "$fdname is a regular file."

# else

# echo "$fdname is something else."

# fi

# echo "Hit <Enter> to continue."

# read ignore

# ;;
echo ---------------------------------
cd /var/log
# /var/log/secure section
cat /var/log/secure |grep -i 'password check failed'|awk '{print $1,$2,$3,$6,$7,
$8,$11}'|sort -u|more
cat /var/log/secure |grep -i 'password check failed'|wc -l |awk '{print $0}'
cat /var/log/secure |grep -i 'sudo'|awk '{print $1,$2,$3,$7,$8,$9,$10}'|sort -u
cat /var/log/secure |grep -i 'sudo'|wc -l |echo sudo used

cat /var/log/secure |grep -i 'su'|awk '{print $1,$2,$3,$5,$7,$8,$11}'|sort -u
cat /var/log/secure |grep -i 'authentication failure'|awk '{print $1,$2,$3,$13}'
|sort -u
cat /var/log/secure |grep -i 'rhost'|sort -u

cat /var/log/secure |grep -i 'could not identify password' |awk '{print $1,$2,$3
,$13}'|sort -u

esac
done
8) echo "Have a nice day"
sleep 1.5
break
;;
$

Thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing fields into variables

A record contains 50 fields separated by "~". I need to assign each of these fields to different variables. Following is the shell script approach I tried. RECORD="FIELD1~FIELD2~FIELD3~FIELD4~FIELD5~...........~FIELD50" VAR1=$(echo ${RECORD} | cut -d"~" -f 1) VAR2=$(echo ${RECORD} | cut... (5 Replies)
Discussion started by: krishmaths
5 Replies

2. UNIX for Dummies Questions & Answers

Parsing alphanumeric variables

Hi All, I have files with a column which has values and ranges, for example colA colB ERD1 3456 ERD2 ERD3 4456 I want to have the following output colA colB colC ERD1 3456 3456 ERD2 526887 526890 ERD3 4456 4456 Being a newbie to... (2 Replies)
Discussion started by: alpesh
2 Replies

3. Shell Programming and Scripting

Assigning values to reference variables for a dynamic menu driven script.

How do I assign values to reference variables? I am assigning a variable name to --> $user_var Then I am trying to change its underlying variable value by $((user_var))=$user_value .. its failing,, Please let me know if there is a way to do this dynamically.. FileA.props... (5 Replies)
Discussion started by: kchinnam
5 Replies

4. UNIX and Linux Applications

Parsing Tuxedo Logs

Right now I am parsing Tuxedo logs to calculate response times for various services. I was hoping to find a log tool that had support for Tuxedo and would generate drill down html reports. ---------- Post updated at 02:35 PM ---------- Previous update was at 02:33 PM ---------- I just wanted... (0 Replies)
Discussion started by: Lurch
0 Replies

5. Shell Programming and Scripting

help using read in menu script to cat out lines in logs

What is wrong with my menu script? Do I need to continue with the read statements? All I want to do with option 4 is to cat some /var/log/files and awk out a few lines? How do I do that please? $ cat menu.sh ... (11 Replies)
Discussion started by: taekwondo
11 Replies

6. Shell Programming and Scripting

Parsing log files, displaying logs between specific dates

Sorry, couldn't really think of a simple subject/title. So, I have a log file, and the dates are displayed like so: 2009-03-05 02:49:44 So the first and second field are the date/time. I can change them into a unix timestamp easily with: date -d "2009-03-05 02:49:44" +%s However,... (17 Replies)
Discussion started by: Rhije
17 Replies

7. Shell Programming and Scripting

Parsing out the logs and generating report

My file will contain following(log.txt): start testcase: config loading ...... error XXXX ..... end testcase: config loading, result failed start testcase: ping check ..... error ZZZZZ ..... error AAAAA end testcase: Ping check, result failed I am expecting below output. ... (4 Replies)
Discussion started by: shellscripter
4 Replies

8. UNIX for Dummies Questions & Answers

Parsing Powerbroker Logs for SysAdmin Changes (SOX)

I need to identify a list of AIX command strings that can be used to parse Powerbroker logs for changes that are being made by Unix SysAdmins. Need to filter out (as much as possible) inquiry or routine maintenance activity and concentrate on software/security changes. This is for internal... (1 Reply)
Discussion started by: bcouchtx
1 Replies

9. Shell Programming and Scripting

Parsing and getting values of variables

suppose i have a file value where it returns 3 values a=1 b=2 c=4 when i run it. i am using this file in my shell script. how do i parse and get the value of a b and c? (3 Replies)
Discussion started by: Rekha
3 Replies

10. Shell Programming and Scripting

How to pass variables to 3rd party unix menu?

Hello, I was wondering if it is possible to pass data to a unix driven 3rd party menu. Changing the code is out of the question. I have a menu with various options and I would like a ksh to execute the menu and input the required fields. For example. Main menu 1. Company Name 2. blah... (3 Replies)
Discussion started by: ctcuser
3 Replies
Login or Register to Ask a Question