Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help needed with log conversion script.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help needed with log conversion script.
# 8  
Old 03-30-2010
Quote:
Originally Posted by rdcwayx
Code:
$ cat logfile
tcp://1.2.3.4:80
tcp://1.2.3.4:*
tcp://1.2.3.4:80,8080
tcp://1.2.3.4:80-101

$ awk -F ":|/" '/\*$/ {printf "Logged this from host %s\n",$4;next}
              /,/ {split($5,a,",") ; for (i in a) printf "Logged this from host %s port %s\n", $4, a[i];next}
              /-/ {sub(/-/," ",$5) ; printf "Logged this from host %s range %s\n",$4,$5;next}
              {printf "Logged this from host %s port %s\n",$4,$5}' logfile

Logged this from host 1.2.3.4 port 80
Logged this from host 1.2.3.4
Logged this from host 1.2.3.4 port 80
Logged this from host 1.2.3.4 port 8080
Logged this from host 1.2.3.4 range 80 101

Hi rdcwayx, Thnx.

I tried adding this:

Code:
for i in $(cat c2.txt)
do
        #proto="$i" | cut -d ':' -f 1;
        proto=${i:0:3}
        echo $proto

<then above script here>

done

This is to take the first 3 chars, tcp in this case, in a variable - proto - and use this to customize the statements as -> Logged $proto request from host ...

My script snippet doesn't seem right though.

Update:
I know how to do it now. I used $1 to put the value 'tcp':

Code:
$ awk -F ":|/" '/\*$/ {printf "Logged "$1" from host %s\n",$4;next}
              /,/ {split($5,a,",") ; for (i in a) printf "Logged  "$1"  from host %s port %s\n", $4, a[i];next}
              /-/ {sub(/-/," ",$5) ; printf "Logged  "$1"  from host %s range %s\n",$4,$5;next}
              {printf "Logged  "$1"  from host %s port %s\n",$4,$5}' logfile

Logged tcp request from host 1.2.3.4 port 80
Logged tcp request from host 1.2.3.4
Logged tcp request from host 1.2.3.4 port 80
Logged tcp request from host 1.2.3.4 port 8080
Logged tcp request from host 1.2.3.4 range 80 101


Best Regards.

---------- Post updated at 09:37 AM ---------- Previous update was at 05:50 AM ----------

Hi,

In this script:

Code:
$ awk -F ":|/" '/\*$/ {printf "Logged "$1" from host %s\n",$4;next}
A 
/,/ {split($5,a,",") ; for (i in a) printf "Logged  "$1"  from host %s port %s\n", $4, a[i];next}

B 
/-/ {sub(/-/," ",$5) ; printf "Logged  "$1"  from host %s range %s\n",$4,$5;next}

              {printf "Logged  "$1"  from host %s port %s\n",$4,$5}' logfile

When there are several ports separated by comma (,), they are put as separate lines which is fine.

But I am now trying to check on A if the ports are in sequential order or not. For example,
Code:
tcp://1.2.3.4:20,21,22,23,100

. In this record, the ports are in sequence, and these should be placed as range, and the last port as a separate line i.e.

Code:
Logged tcp request from host 1.2.3.4 range 20 23
Logged tcp request from host 1.2.3.4 port 100

Can I use if statement in Line A in awk? or there is a better way to do this..

Best Regards..

---------- Post updated at 04:36 PM ---------- Previous update was at 09:35 AM ----------

---------- Post updated at 04:38 PM ---------- Previous update was at 04:36 PM ----------
Quote:
Originally Posted by frans
Reviewed script with important modifications :
Code:
#!/bin/bash
Split() {    # Function to make an array with a variable
    eval "$1=( \"$(echo "${!1}" | sed "s/$2/\" \"/g")\" )"
}
while read L
do
    Split L ':' # Makes an array ( 0:Protocol 1:IP 2:Port(s) )
    if [ ${L[0]} = tcp ]
    then
        IP="Logged this from host ${L[1]:2}" # :2 to remove the 2 slashes from host IP
        P=${L[2]}
        Split P ',' # makes an array with port numbers
        if [ "$P" = "*" ]    # case B
        then echo "$IP"
        elif I=$(expr index $P '-') # case D
        then    echo "$IP range ${P:0:$((I-1))} ${P:$I}"
        else # case A & C
            for ((i=0; i<${#P[@]}; i++))
            do
                echo "$IP port ${P[$i]}"
            done
        fi
    else
        echo "Other protocol : ${L[0]}"
    fi
done < infile

Superb! Smilie

rdcwayx, earlier issues are resolved now. I made one change to the script. This will put appropriate protocol type tcp / udp as per the log:

Code:
#!/bin/bash
Split() {    # Function to make an array with a variable
    eval "$1=( \"$(echo "${!1}" | sed "s/$2/\" \"/g")\" )"
}
while read L
do
    Split L ':' # Makes an array ( 0:Protocol 1:IP 2:Port(s) )
#    if [ ${L[0]} = tcp ]
#    then
        IP=" Logged "${L[0]}" request from host ${L[1]:2}" # :2 to remove the 2 slashes from host IP
        P=${L[2]}
        Split P ',' # makes an array with port numbers
        if [ "$P" = "*" ]    # case B
        then echo "$IP"
        elif I=$(expr index $P '-') # case D
        then    echo "$IP range ${P:0:$((I-1))} ${P:$I}"
         else # case A & C
            for ((i=0; i<${#P[@]}; i++))
            do
                echo "$IP port ${P[$i]}"
            done
        fi
#    else
#        echo "Other protocol : ${L[0]}"
#    fi
done < c2.txt

I noticed that when the ports are in sequence, for example -

Code:
tcp://1.2.3.4:20,21,22

or
in sequence but randomly placed, like -
Code:
tcp://1.2.3.4:20,22,21

these could be placed in a range statement, i.e.
right now they will be printed as

Code:
Logged from host 1.2.3.4 port 20
Logged from host 1.2.3.4 port 21
Logged from host 1.2.3.4 port 22

but how can we print them as -

Code:
Logged from host 1.2.3.4 range 20 22

.

Implying if the ports are in numerical sequence, separated by comma, and may or may not be in sequentially placed in log entry, they should be written by way of 'range' instead of on separate lines.

Best Regards..
Last edited by morningSunshine; 03-30-2010 at 08:35 AM..
# 9  
Old 03-30-2010
Quote:
Originally Posted by morningSunshine
(...)
Implying if the ports are in numerical sequence, separated by comma, and may or may not be in sequentially placed in log entry, they should be written by way of 'range' instead of on separate lines.
Smilie That would be nice !! Smilie
Congratulations to who will have a simple way to do that !

Good night, sweet dreams !
# 10  
Old 03-30-2010
Quote:
Originally Posted by frans
Smilie That would be nice !! Smilie
Congratulations to who will have a simple way to do that !

Good night, sweet dreams !
Smilie I understand n should take a break now, especially if this is coming from ya. It'd be hard to beat for sure.

Gracias frans for your time n sharing knowledge.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

powershell script to unix shell script conversion.

Here is a powershell script to use restful API to create ticket in our ticketing tool. Can anyone please convert it to a shell script sothat, I can run it in Unix servers, below is the code: $body = @{ Customer= ''test' Summary= 'test summary' Impact= '4-Minor/Localized' ... (2 Replies)
Discussion started by: pandeybhavesh18
2 Replies

2. Shell Programming and Scripting

Conversion of Perl Script to Shell Script..

Hi Guys I am having a perl script that fetches exclude list from a unix client and I trying it to convert it to shell script but I am having issues please help me... #!/usr/bin/perl use strict; use warnings; use Getopt::Std; # To turn on debuging (i.e. more information) specify... (29 Replies)
Discussion started by: Pawan Ramnani
29 Replies

3. Shell Programming and Scripting

Help needed: script for timely average from log file

Please repost your query: Help needed: script for timely average from log file - Thank you. (0 Replies)
Discussion started by: mkfs
0 Replies

4. Shell Programming and Scripting

Help needed on Perl Script to Handle Log files that are rotated using logrotate

Hello all, I am working on a perl script which will read weblogic logfile and send the error messages to Zenoss Monitoring tool. At present the script works and it can able to send the error messages to Zenoss. The problem comes when the logrotate has been applied to the weblogic log file. At... (3 Replies)
Discussion started by: kar_333
3 Replies

5. Shell Programming and Scripting

Most vexing: Sed or Awk scripting for date conversion needed

Hi, I have some files being sent to me that have dates in them in this format: from 1/8/2011 15:14:20 and I need the dates in this format (mysql date format) To 2011-01-08 15:14:20 all I have so far is the regexp that detects the format: sed -r -e 's@\1/\2/\3\4\5\6]::$@do... (7 Replies)
Discussion started by: Astrocloud
7 Replies

6. Shell Programming and Scripting

help needed - log file monitoring script

hi Gurus, Need to pick your brains on this minor script project. I would like to continuously monitor a log file with sample log messages as below, and if PSOldGen percentage is either 99% or 100% for consecutively 10 times, alert someone. {Heap before gc invocations=46516: PSYoungGen ... (6 Replies)
Discussion started by: kenchen722
6 Replies

7. Shell Programming and Scripting

Conversion of below Tabs Tex file into CSV format file : shell script needed

Request if some one could provide me shell script that converts the below "input file" to "CSV format file" given Name Domain Contact Phone Email Location ----------------------- ------------------------------------------------ ------- ----- ---------------------------------... (7 Replies)
Discussion started by: sreenath1037
7 Replies

8. Shell Programming and Scripting

shell or perl script needed for ldif file to text file conversion

This is the ldf file dn: sdcsmsisdn=1000000049,sdcsDatabase=subscriberCache,dc=example,dc=com objectClass: sdcsSubscriber objectClass: top postalCode: 29600 sdcsServiceLevel: 10 sdcsCustomerType: 14 givenName: Adelia sdcsBlackListAll: FALSE sdcsOwnerType: T-Mobile sn: Actionteam... (1 Reply)
Discussion started by: LinuxFriend
1 Replies

9. Shell Programming and Scripting

help needed with ASCII conversion

I have a file say "codefile" here ,contains data like this Hi! How are you? I need to covert this data into stram of equivalant ASCII values I wrote follwoing script. #!/bin/bash while read -n1 char do printf "%d" \'$char done < codefile this gives me output ... (4 Replies)
Discussion started by: sunilmenhdiratt
4 Replies

10. Shell Programming and Scripting

Script Conversion To Ubuntu 8.10

I have a ping script I use on an old Open Step box (I guess its closely related to Mac OS X) and it runs fine, but now I built a system as a backup with Ubuntu 8.10 client and the script needs to be adapted a bit. Can anyone see where or how this needs to be done? The script starts and assigns the... (14 Replies)
Discussion started by: gbxfan
14 Replies
Login or Register to Ask a Question