The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Special Forums > IP Networking
.
google unix.com



IP Networking Learn TCP/IP, Internet Protocol, Routing, Routers, Network protocols in this UNIX and Linux forum.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Three reasons why GNU/Linux is better for Web servers than OS X iBot UNIX and Linux RSS News 0 06-30-2008 04:20 PM
cron job not working for some reasons ragha81 Shell Programming and Scripting 14 05-28-2008 08:04 AM
Eight Reasons NOT to Use Linux in the Enterprise - CIO iBot UNIX and Linux RSS News 0 08-02-2007 01:00 PM
To find out the reasons it crash sandeepkv SUN Solaris 3 03-09-2007 11:56 PM
Sun: High kernel usage & very high load averages lorrainenineill UNIX for Advanced & Expert Users 4 02-06-2006 12:32 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 11-18-2008
PWSwebmaster PWSwebmaster is offline
Registered User
  
 

Join Date: Feb 2006
Location: Canada
Posts: 33
netstat - possible reasons for high IP count ???

One of my servers started getting heavily loaded a few weeks ago for a few hours, so I did some studying and wrote a script to use netstat to get the IP addresses connected and the count. I put a new chain in iptables and if an IP is using more than 40 connections, it gets added to that chain which is then flushed every hour just to make sure no legitimate IP is blocked forever. If an IP is connected more than 100 times, it gets added directly to the INPUT chain and therefore is permanent until manually removed.

I'm mainly trying to figure out if those counts are good limits. Can there be legitimate reasons for an IP to be using more than 40 connections at a time? I tested going to a web page with 200 thumbnail images, and even then my IP was only listed a few times.
  #2 (permalink)  
Old 11-19-2008
joquendo joquendo is offline
Registered User
  
 

Join Date: Nov 2008
Location: New England
Posts: 5
Have you isolated what programs were being accessed by those addresses? Depending on what exactly your script is doing, it may not be a practical idea. Large companies often have gateways which will make it seem as if one person (one IP) is accessing hundreds of instances of a page. So imagine say 200 people at Cisco viewing your site. While you see it as one address, the fact is, there are many people viewing the site.

What is the overall purpose of your script. Considering I can spoof addresses, if your firewall is misconfigured, your provider isn't doing BCP filtering, as an attack vector, I can make your server ignore everything via 0.0.0.0 spoofing.
  #3 (permalink)  
Old 11-19-2008
PWSwebmaster PWSwebmaster is offline
Registered User
  
 

Join Date: Feb 2006
Location: Canada
Posts: 33
It's been different files and scripts, but mostly just files, being accessed at a high count of one IP like that.

Here's an example from log files for one case:

Quote:
60.50.105.33 - - [18/Nov/2008:08:45:04 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 91022 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:06 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 227228 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:13 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 115944 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:14 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 157814 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:17 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 156708 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:17 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 203672 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
60.50.105.33 - - [18/Nov/2008:08:45:17 -0500] "GET /uploads/2476/2008_TEB_-11-_Alban_Preaubert_FS.avi HTTP/1.1" 206 198600 "http://www.skatingvideoclips.com/uploads/2476" "Mozilla/4.0 (compatible; MSIE 5.00; Windows 98)"
Netstat showed that IP many times, so it wasn't their computer downloading a video a bit at a time. It looked more like they were downloading the same video a high number of times at once. There are about 400 videos there, so it wasn't 50 different people with the same OS and same gateway downloading the same video at the same time.

That IP is from Malaysia.
The host is: 33.105.50.60.klj03-home.tm.net.my and is probably from MY (MALAYSIA)

Most of the high count IPs have been from Malaysia, Taiwan, Poland, Japan and China.

When the server first started having high load trouble, I found the high number of connections to one file and renamed the file, then minutes later the same IP would have a high number of connections to a different file, then I blocked the IP from that site and minutes later the same file was being accessed a high number of times from a different IP. I wrote a little script to block IPs from that site automatically, then the IP would just keep changing and show as being from different countries. The script would just block access to the one site which meant giving a 403 page each time. Next thing I knew, the volume was climbing and they were just getting the 403 page 100 times a second. Definitely looked to me like someone was trying to crash the server, so I had to look into blocking them from the whole server.

Since I started running my auto iptables script a week ago, the server load has pretty much quit spiking.

The odds of many people from one company on the same router going to a site at the same time is quite slim, but later on I can adjust my script to check the log files to see if the IPs are all accessing the same file and using the same browser which would help prevent them from being blocked.
  #4 (permalink)  
Old 11-25-2008
techlinux techlinux is offline
Registered User
  
 

Join Date: Jul 2008
Posts: 86
You may want to consider either blocking or bandwidth throttling.
There are a number of ways to set bandwidth limits depending on your configuration.
Sponsored Links
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 08:00 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0