Selecting Lines on text file

oconmx · #1 (**permalink**) 08-20-2009

Hi All,

I am creating a script that sends log data from text files to a Database and I will like to read sugestions, as I think that there might be better ways to achive this than with my shell script; maybe perl or I don't know, but I will like to read some sugestions.

The log is from Nagios, here is an example of the data I am interested in:

Code:

[1230791427] EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;SACNT141;SQL Event Log;2;Application [error] [mssqlserver #17055]: 3041 : BACKUP failed to complete the command BACKUP DATABASE [Apple_InterchangeSQ] TO [Apple_InterchangeSQ_Full] WITH  INIT
[1230791545] HOST ALERT: FSERVWEB;DOWN;SOFT;1;10.141.8.40 is unreachable (loss: 100%)
[1230791685] SERVICE DOWNTIME ALERT: ZCREH000;SYSTEM-DISK;STOPPED; Service has exited from a period of scheduled downtime
[1230793487] HOST DOWNTIME ALERT: MILSTARS;STOPPED; Host has exited from a period of scheduled downtime
[1230793964] SERVICE DOWNTIME ALERT: TXSLRCAD1;DISK-USAGE-E-90-95;STOPPED; Service has exited from a period of scheduled downtime

As you can see, there is a Timestamp, the type of alert and the information separated by commas. The diferent log entries have different Columns, thus I separate them according to the alert type, and then arrange the data to fit into my Database.

What will be the best way to read line by line, and according to the Alert type; create a CSV line with the info to be sent to the database (or put it directly in case of perl or PHP)....

Any advise is welcome, as I am currently using a shell script with grep and awk... and I feel is not the best way to do it.

Regards,
oconmx

chipcmc · #2 (**permalink**) 08-20-2009

i hope help you:

Code:

while read line
do
        TypeAlert=`echo "$line" | awk -F":" '{ split ( $1,vect,"]");   print vect[2] }'|sed "s/^  *//"`
        if [ "$TypeAlert" = "EXTERNAL COMMAND" ]
        then
                echo "EXTERNAL COMMAND ALERT"
                echo $line >> file1
        else
         if [ "$TypeAlert" = "HOST ALERT" ]
         then
                echo "HOST ALERT"
                echo $line >> file2
         else
           if [ "$TypeAlert" = "SERVICE DOWNTIME ALERT" ]
           then
                echo "SERVICE DOWNTIME ALERT"
                echo $line >> file3
           else
              if [ "$TypeAlert" = "HOST DOWNTIME ALERT" ]
              then
                 echo "HOST DOWNTIME ALERT"
                echo $line >> file4
              else
                if [ "$TypeAlert" = "SERVICE DOWNTIME ALERT" ]
                then
                        echo "[SERVICE DOWNTIME ALERT]"
                        echo $line >> file5
                fi
              fi
          fi
         fi
        fi
done<file

oconmx · #3 (**permalink**) 08-20-2009

Let me try it...

Looks good and way cleaner than my script

Thank you!

danmero · #4 (**permalink**) 08-20-2009

We can use only awk:

Code:

awk -F'\]|:' '{line=$0;sub("^ ","",$2);sub(" ","_",$2);file=$2;print line >> file;close(file)}' file

oconmx · #5 (**permalink**) 08-20-2009

danmero,

I dont understand all your code, I cant imagine how it separates the alert types... besides that, it is giving me an error:

Code:

awk: (FILENAME=file FNR=196) fatal: expression for `>>' redirection has null string value

Thank you!
Oconmx

danmero · #6 (**permalink**) 08-20-2009

Can you post the source filename and the line 196

Code:

awk 'NR==196' file

oconmx · #7 (**permalink**) 08-20-2009

Hi danmero,

It was an empty line, the file has many... I already fixed a few and crop the file to try your script, it's good, I like the Idea. I will try to add some modifications and try the shell script with a CASE statement to see which is faster...

Is there anyway to make awk skip blank lines?

Thank you!
Carlos

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
how to get the no. of lines in a text file	vimalr	Shell Programming and Scripting	8	08-18-2009 02:42 PM
get only some lines from a text file	fadista	UNIX for Dummies Questions & Answers	1	01-28-2009 06:15 AM
selecting only few lines from many based on a common pattern	damansingh	Shell Programming and Scripting	2	05-28-2008 06:29 AM
command for selecting specific lines from a script	gardasgangadhar	UNIX for Dummies Questions & Answers	1	05-13-2008 05:29 AM
Help with selecting specific lines in a large file	tansha	UNIX for Dummies Questions & Answers	2	02-06-2008 08:26 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread: