Read file, grab ip with fail2ban


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read file, grab ip with fail2ban
# 1  
Old 03-13-2014
Read file, grab ip with fail2ban

Solved with iptables.
Many thanks...


Hello,

Objective:
What I would like to accomplish is :
- To read file1 line by line and search each word in file2.
- To grab corresponding ip addresses found in file2
- To send related ip addresses to fail2ban (not iptables)

By this way, when I want to block any username, I will enter only his username into filtered.txt file and then his connection will be terminated when he tries to login to the system even when he changes his ip address.

Detail:
I have a txt file under /var/text folder and it consists of usernames:

filtered.txt

Code:
Code:
mark
angela
dimitriou
anna
michelle

What I need to do is:

1) Search each username (mark, angela, dimitriou, anna, michelle, etc..) in syslog file (it is under /var/log) and print it to a file (iplist)

I already created this file by below command:


Code:
Code:
grep -wf /var/test/filtered.txt /var/log/syslog > /var/log/iplist


2) A new fail2ban configuration file will be set. It will read iplist and

I created below conf file:

iplist.conf under /etc/fail2ban/filter.d/

Code:
Code:
[Definition]
failregex = mypc_regex_code Network: user .* login attempt from <HOST>
ignoreregex =

mypc_regex_code: It's as shown in syslog file.

I also added below lines into jail.conf in after [ssh] under /etc/fail2ban/


Code:
Code:
[iplist]
enabled = true
port = 34567,34789,35890
filter = iplist
logpath = /var/log/iplist
maxretry = 1
bantime = 10800

I stopped and started fail2ban but related ip addresses were not shown in fail2ban.log file. (I am sure that other fail2ban rules are working)

Can anybody give an advice about how to accomplish this objective?
May I add related usernames into deny.hosts file to block the connection?

Thanks in advance
Boris

Last edited by baris35; 03-13-2014 at 11:29 AM..
# 2  
Old 03-13-2014
Do you want to repel users or ip- addresses? What if any of those users logs in from a different node?
# 3  
Old 03-13-2014
Quote:
Originally Posted by RudiC
Do you want to repel users or ip- addresses? What if any of those users logs in from a different node?
Hello,
I solved this issue. I wanted to block a list of users. At first, the script kicks their original ip. When they changed their ip, cron will detect this and ban them once again.

I defined and limited each user's login date and hour and minute.
database.txt (##year_month_day_hour_minute)
Code:
sophie.bextor ##201403011017
dido ##201404251049
ray.charles ##201404290159
freddie.mercury ##201404012200
madonna ##201405050900

If he is not allowed to login anymore, the script sends his username to filtered.txt file and then mentioned above scenario starts.

filter_accts.pl
Code:
#!/usr/bin/perl

use Time::Local;
open outf, "> filtered.txt";

while (<>) {
    @f = split /#/, $_;
    $yr = substr($f[2], 0, 4);
    $mo = substr($f[2], 4, 2);
    $dy = substr($f[2], 6, 2);
    $hr = substr($f[2], 8, 2);
    $mi = substr($f[2], 10, 2);
    print outf if time > timelocal(0, $mi, $hr, $dy, $mo-1, $yr-1900);
}

When you run below code:

Code:
./filter_accts.pl database.txt

Output will be
filtered.txt:
Code:
sophie.bextor ##201403011017

to remove ## signs, i created filtered2.txt:
Code:
sophie.bextor

Looks up sophie.bextor's ip in syslog file:
Code:
grep -wf filtered2.txt /var/log/syslog > new.txt

to remove unneccessary lines:
Code:
sed -n '/login/p' /var/test/new.txt > /var/test/login

Then to be able to extract list of ip addresses:
Code:
awk '/^[0-9.]+[.][0-9]+$/{if(!a[$0]++)print $0}' RS="[ :/\n]" login > login-ip.txt

The last step is add prefix ( iptables -A INPUT -s ) to the beginning of each line and add -j DROP to the end of each line with below script.

replace.sh
Code:
#!/bin/bash
prefix="iptables -A INPUT -s "
file="login-ip.txt"
while read -r line
do
 echo "${prefix}$line"
done <$file > fail2ban_ip
#mv fail2ban_ip $file
sed -e 's/$/ -j DROP/' -i fail2ban_ip

when you open fail2ban_ip file, it will show something like this:
Code:
iptables -A INPUT -s 21.166.112.177 -j DROP
iptables -A INPUT -s 21.177.112.142 -j DROP
iptables -A INPUT -s 22.156.112.155 -j DROP
iptables -A INPUT -s 23.146.112.172 -j DROP

then, chmod 755 & ./fail2ban_ip under terminal

I will get those codes together into a script and set cronjob now .

As the problem has been sorted out by means of your valuable support, I would like to thank you all.

Regards
Boris

Last edited by baris35; 03-13-2014 at 12:07 PM..
# 4  
Old 03-13-2014
If I got you right, your setup allows one access to users having changed their IP address. I'm not sure this is what you want. Why not block access for the entire range of IP addresses and explicitely allow single users' nodes? Or, why not block/allow them in /etc/passwd?
# 5  
Old 03-13-2014
If you want to block a list of users, what are they to be prevented from doing?
  • Login - change their shell to be /bin/false
  • FTP . - Add them to /etc/ftpusers
  • SFTP - Add them to the file referred to in sshd.conf
  • Everything - Change their encrypted password manually to a simple short string, so the encryption at login will never match it.

Does that help?


Robin
# 6  
Old 03-13-2014
Quote:
Originally Posted by rbatte1
If you want to block a list of users, what are they to be prevented from doing?
  • Login - change their shell to be /bin/false
  • FTP . - Add them to /etc/ftpusers
  • SFTP - Add them to the file referred to in sshd.conf
  • Everything - Change their encrypted password manually to a simple short string, so the encryption at login will never match it.

Does that help?
Robin
They cant get login to system. I am not familiar with ftp or sftp but you can run below script, add a your username into database.txt file and check what he can not do.

test.sh under /var/bin

Code:
#!/bin/bash
cd /var/test
./release_ip
./filter_accts.pl database.txt
awk '{print $1}' filtered.txt > filtered2.txt
grep -wf filtered2.txt /var/log/syslog > new.txt
sed -n '/login/p' /var/test/new.txt > /var/test/login
awk '/^[0-9.]+[.][0-9]+$/{if(!a[$0]++)print $0}' RS="[ :/\n]" login > login-ip.txt
./replace.sh
cp -p fail2ban_ip /var/test/release_ip2
sed 's/-A/-D/' release_ip2 > release_ip
chmod 755 fail2ban_ip
chmod 755 release_ip
./fail2ban_ip
iptables-save

PS: You can find filter_accts.pl file in above posts.

System time is 17:09, today is 20140313 and defined username in database.txt will be banned one hour later.
database.txt under /var/test
Code:
a_username_to_be_banned ##201403131809


under /etc/crontab
Code:
*/10 * * * * root /var/bin/test.sh

Those are what i edited by other board members' help. Not my own codes.

Regards
Boris

Last edited by baris35; 03-13-2014 at 01:16 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

2. Shell Programming and Scripting

Script year_month_day_hour_minute with fail2ban

Hello, What I would like to do is a shell script which will read a database file, then it will compare the current date/hour/minute in each line existing in the database file. Today is 20140305 (year_month_day) & assume that the time is 15:11 at the moment. under /var/log/ database.txt ... (5 Replies)
Discussion started by: baris35
5 Replies

3. Shell Programming and Scripting

Grab from file with sed

Hello All I have a file with this type of records: =LDR 01157nas a22003011a 4500 =001 vtls000000013 =003 VRT =005 20111020150800.0 =008 100128c19699999sp\a|||||\||||0\\\||spa| =037 \\$a1327$i090$j090$k03 =039 ... (14 Replies)
Discussion started by: ldiaz2106
14 Replies

4. Shell Programming and Scripting

Grab 2 pieces of data within a file

I am a newbie and what I have is a captured file of content. I want to be able to grab 2 pieces of data, multiple times and print them to the screen. DataFile owner: locke user: fun data size: 60 location: Anaheim owner: david user: work data size: 80 location: Orange my script... (2 Replies)
Discussion started by: greglocke
2 Replies

5. Shell Programming and Scripting

Help me grab a passage out of this text file?

Hey guys, I need a command that grabs only this part of the .txt file (that is attached), and outputs it to another .txt file with only these contents below. Thanks in advanced brothers: Disk: Local Disk (C:), NTFS Disk Defragmentation Summary Disk Size 230.85 GB Free Space Size... (4 Replies)
Discussion started by: aabbasi
4 Replies

6. Emergency UNIX and Linux Support

Grab total page read from webalizer display in html

Hi, I need a way to grab the total combines since inception, total pages read from webalizer on my centos server or any other location (as long as since inception) and display the result live on my website So with each visit it would be increasing, or perhaps live (ajax) not sure But can... (0 Replies)
Discussion started by: lawstudent
0 Replies

7. Shell Programming and Scripting

Grab from the file in one command

Dear All, I have a file in which there are 54 fields, i want to grab the all the lines and send to a new file where filed18 has lenght greater than 14. How can i do it without if condition and faster way: currently i am reading file line by line and comparing the length read fileLine... (9 Replies)
Discussion started by: bilalghazi
9 Replies

8. UNIX for Dummies Questions & Answers

How to Grab the latest file

I have been trying to use the find command to grab the latest file in a directory and move it to another area. I can't seem to get only that file, I end up getting everything for the day. Any ideas? Thank you (1 Reply)
Discussion started by: n9ninchd
1 Replies
Login or Register to Ask a Question