Script going upward and downward direction


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script going upward and downward direction
# 1  
Old 10-18-2013
Script going upward and downward direction

Hello,

I have a log file as shown below. Each line starts with instance nr.

Code:
[0000000362]: Session[29]: IP [80.11.22.33] Processing HTTP Socket [20] Command [GET ]
[0000000363]: Connecting User-Agent Media: [ Mozilla/4.0]
[0000000364]: NOT Rejected Media[ Mozilla/4.0]
[0000000365]: Session[29]: Username [myfriend] Request Channel [6.gate]
[0000000366]: Username [myfriend] : Check if he is in User Database
[0000000367]: Username [myfriend] : not in the User Database
[0000000368]: Session[34]: IP [90.11.22.44] Processing HTTP Socket [20] Command [GET ]
[0000000369]: Connecting User-Agent Media: [ Mozilla/4.0]
[0000000370]: NOT Rejected Media[ Mozilla/4.0]
[0000000371]: Session[34]: Username [myfriend2] Request Channel [6.gate]
[0000000372]: Username [myfriend2] : Check if he is in User Database
[0000000373]: Username [myfriend2] : found in User Database
[0000000374]: Session[45]: IP [100.11.22.44] Processing HTTP Socket [20] Command [GET ]
[0000000375]: Connecting User-Agent Media: [ Mozilla/4.0]
[0000000376]: NOT Rejected Media[ Mozilla/4.0]
[0000000377]: Session[45]: Username [myfriend3] Request Channel [6.gate]
[0000000378]: Username [myfriend3] : Check if he is in User Database
[0000000379]: Username [myfriend3] : not in the User Database


What I need to have is a script which will look up "not in the User Database" phrase at each line and when the criteria matches, the script will go five lines above (from 367th instance to 362nd instance for this log file) and grab the IP.

Output file should be:

Code:
myfriend  80.11.22.33 
myfriend3 100.11.22.44

Can anybody help me?

Thanks in advance
Boris
# 2  
Old 10-19-2013
You could try something like:
Code:
#!/bin/ksh
elog="${0##*/}.$$"
awk -v elog="$elog" '
/ IP [[]/ {
        if(match($0, /([[:digit:]]+[.]){3}[[:digit:]]+/))
                ip = substr($0, RSTART, RLENGTH)
        else {  printf("IP address not found in line \"%s\"\n", $0) > elog
                ec = 1
        }
        next
}       
/found in User Database/ {
        ip = ""
}
/not in the User Database/ {
        if(match($0, /Username [[][^]]*/) == 0) {
                printf("Username not found in line \"%s\"\n", $0) > elog
                ec = 1
        } else {un = substr($0, RSTART + 10, RLENGTH - 10)
                printf("%s %s\n", un, ip)
        }       
        ip = un = ""
}       
END {   exit ec
}' log
ec=$?
if [ $ec -ne 0 ]
then    cat "$elog" >&2
        rm "$elog"
fi      
exit $ec

I tested this using the Korn shell, but it should work with any POSIX-conforming shell. If you want to try this on a Solaris/SunOS system, replace awk on the 3rd line in this script with /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

With your sample input, this produces the output:
Code:
myfriend 80.11.22.33
myfriend3 100.11.22.44

which matches what you requested except that there is always one space between output fields (where your sample output had two spaces in the first line and one space in the second line) and there are no trailing spaces (where your sample output had one trailing space on the first line and no trailing spaces on the second output line).
This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 10-19-2013
Or not as robust, but to do what you literally asked for (5 lines above), you could try a circular buffer:
Code:
awk -F'[][]' '{F6[NR%6]=$6} /not in the User Database/{print $4, F6[(NR-5)%6]}' file

This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 10-19-2013
Or
Code:
awk     '               {gsub (/\]|\[/,"")}
         $3=="IP"       {IP=$4}
         /not in.*base/ {print $3, IP}
        ' file
myfriend 80.11.22.33
myfriend3 100.11.22.44

This User Gave Thanks to RudiC For This Post:
# 5  
Old 10-19-2013
Hello,
Many thanks for you all.
Now, I am able to extract ip addresses that should be banned.
My question is how I can communicate iplist file with fail2ban

At first I added below line into crontab:

Code:
*/10 * * * * root /var/bin/iplist.sh

my iplist.sh file with chmod 755 under /var/bin is :

Code:
#!/bin/bash
cd /usr/local/darwin
awk -F'[][]' '{F6[NR%6]=$6} /not in the User Database/{print $4, F6[(NR-5)%6]}' system_backup.log > fail2ban.txt
sleep 1
grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' fail2ban.txt > daemon.log
fi

My daemon.log file is like this:
Code:
15.41.150.134
15.41.10.134
15.41.30.134
15.41.50.134
15.41.60.134
16.41.80.134
16.41.90.134
17.41.44.134

Then I added below explanation into fail2ban:
Code:
[darwin]
enabled = true
port = 8000,1934,5544,7001,7002,7003
filter = darwin
logpath = /usr/local/darwin/daemon.log
maxretry = 1
bantime = 3600

The next step is I created a file named darwin.conf :
Code:
[Definition]
failregex = .* <HOST>
ignoreregex =

Could you please let me know what I should type instead of failregex = .* <HOST> in related file?

When I stop/start fail2ban, I can not see banned ip addresses under /var/log/fail2ban file.

Thanks in advance
Boris

Last edited by Scrutinizer; 10-20-2013 at 03:10 AM.. Reason: Additional code tags
# 6  
Old 10-20-2013
What file with which structure is used where to what purpose?
# 7  
Old 10-20-2013
Quote:
Originally Posted by RudiC
What file with which structure is used where to what purpose?
Hello,
I did not want you to do all the details. I did not want to make you busy with all details. For that reason I asked you a couple of question on where I could not have found the solution.

Operating system linux and there is a software which creates its own log file. I shared the log file sample in my first message body. The main aim is to send all IP addresses which are not found in database to fail2ban.

I am sorry as I did not explain it well in my first message.

Thanks in advance
Boris
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search and Destroy Script Direction Help

Being a beginner in scripting I am not sure the direction to take to accomplish the below task and would love suggestions. GOAL input file: domains.list Read input file, search in named.conf and find domain and delete entry for the purpose of cleanup activity. named.conf entry example zone... (8 Replies)
Discussion started by: djzah
8 Replies

2. AIX

New to aix virtualization direction ?

picked up a 9111-520 p5 hardware recently with 8 Gig of RAM, lots of internal disk space...6 x 140 gig had 7.1 pre-installed, and managed to upgrade the firmware to latest SF240_xxxx goal is to virtualize this and have LPARs running aix 7.1, 6.x, and Rhat for ppc .. need some... (13 Replies)
Discussion started by: ppchu99
13 Replies

3. Shell Programming and Scripting

Displays the text upward

I need to print out text from a file in the console up, you know someone like that? (8 Replies)
Discussion started by: gizmo16
8 Replies

4. Fedora

In need of some direction

Okay, so I'm not a complete newb when it comes to using Unix/Linux. I've been using Ubuntu for a few years now and I've dipped my toes into a few other distros but now I want to get a bit serious. I'm looking at becoming a sysadmin but the trouble is...I have no idea where to start. What I'm... (1 Reply)
Discussion started by: Tamachan87
1 Replies

5. Fedora

Need Direction for extra work ?

Hey , I have become pretty normal, using unix and what not and working around FEDORA 9 I was wondering does anyone have any IDEAS or have anything I should try to build or scripts to write , or possibly know any sites where I could practice some things just so I know I am writing them... (2 Replies)
Discussion started by: Producer
2 Replies

6. Programming

Daemon direction. Or, What do I need to watch for?

Hi, I'm writing my first daemon application. I need to make sure I cover my bases as far as correct procedures, etc... I've tried to do my own legwork by reading as much as I could on daemonizing programs, etc... There are so many different examples, some include this but not that, etc...... (3 Replies)
Discussion started by: mph
3 Replies

7. Shell Programming and Scripting

re-direction

Say I have a single bin directory with Linux and SunOS executables, like this: bin/myprog_lnx bin/myprog_sun Assume these programs read from stdin and write to stdout and, thus, are meant to be run like this: myprog_lnx < filein > fileout My users may log in from a Linux or Solaris... (3 Replies)
Discussion started by: gsal
3 Replies

8. Programming

In what direction should I take computer programming?

I'm a senior in high school trying to start getting into computer programming. All I've done so far is picked up a book on C for beginners and started to teach myself. There aren't really any courses at my high school for introductory programming, so it looks like I'll have to wait for college to... (7 Replies)
Discussion started by: Fritzz
7 Replies

9. UNIX for Dummies Questions & Answers

New & could use some direction!

First, I just rebuilt/installed my custom kernel & I don't know how to check if it ran properly (I'm fairly sure it did, but I'm looking for reassurance that it loaded the new kernel file). Second, I'd love to get into programming, scripting, whatever, I want my imagination to be the builder &... (2 Replies)
Discussion started by: LazySpoon
2 Replies
Login or Register to Ask a Question