Sponsored Content
Contact Us Post Here to Contact Site Administrators and Moderators Duplicate postings when following up on a thread Post 302220172 by Neo on Thursday 31st of July 2008 04:13:42 AM
Old 07-31-2008
I spend many hours hunting down hungry, rogue spiders in the log files. I found many spiders not obeying robots.txt, so I blocked their network completely.

Maybe we will see some improvement before the upgrade, but I am not confident we will see much, since the bot traffic is very small compared to the Google search referral traffic.

Also, I spend many hours trying various MySQL and Apache configurations and tuning parameters, but that did not result in any tangible improvements and, in some cases, trying to add or allocate more resources just made it worse. For example, we were getting some errors because Apache needed more than 250 concurrent servers. I tried changing this to 350 and the load average when to over 350 (!!) during peak time, crushing the server.

Obviously, when we upgrade, traffic will greatly increase again. This is great, thanks to all of your super posts, great expertise, helping others help themselves.

Thank you for your patience during this phase of our growing pains!

As a side note, I am amazed at the number of spiders that roam the internet these days; and from all over the world. It seems everyone wants to be the next Google. I even noticed that two of the spiders were operating out of the new Amazon Elastic Computing Cloud (EC2).

Now I have a new security topic to write about, something like:

The Attack of the Cyberspiders from the Clouds !!!

You have to love life on the net Smilie
 

4 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sending output to a file - Merged from duplicate thread

Hi All, read dif echo `date +%Y%m%d`|./add $dif|./fmtdt %mon%dd The above script is for adding days to current date to find the new date. This script divides the current date into 20060220(YYYYMMDD) format and pass this output to add script. The add script will add the days to the... (2 Replies)
Discussion started by: muthu_nix
2 Replies

2. Forum Support Area for Unregistered Users & Account Problems

How to do postings

Hi All, Can you pls help me out on how to do new posting on this forum. (1 Reply)
Discussion started by: SGReddy
1 Replies

3. Post Here to Contact Site Administrators and Moderators

Change in display of date of postings

I would like to propose to change the display of dates in the postings, which is currently mm-dd-yy. To avoid ambiguity, I would like to propose dd-Mmm-yyyy. Thank you for your anticipated consideration. (5 Replies)
Discussion started by: figaro
5 Replies

4. Post Here to Contact Site Administrators and Moderators

Date stamp on thread postings.

Is it possible to have the actual date shown, instead of phrases like "1 Week Ago", on threads list of postings? If possible, how do I enable that? (0 Replies)
Discussion started by: Svenus
0 Replies
VNSTATI(1)							   User Manuals 							VNSTATI(1)

NAME
vnStati - png image output support for vnStat SYNOPSIS
vnstati [ -cdhimostv? ] [ --cache time ] [ --config file ] [ --days ] [ --dbdir directory ] [ --help ] [ --hours ] [ -hs ] [ --hsummary ] [ -i interface ] [ --iface interface ] [ --locale locale ] [ --months ] [ -ne ] [ -nh ] [ --noedge ] [ --noheader ] [ --output file ] [ -ru ] [ --rateunit ] [ --style number ] [ --summary ] [ --top10 ] [ --transparent ] [ --version ] [ -vs ] [ --vsummary ] DESCRIPTION
The purpose of vnstati is to provide image output support for statistics collected using vnstat(1). The image file format is limited to png. All basic outputs of vnStat are supported excluding live traffic features. The image can be outputted either to a file or to standard output. OPTIONS
-c, --cache time Update output file only if at least time minutes have passed since the previous file update. This option is ignored if stdout is used as output. --config file Use file as config file instead of using normal config file search function. -d, --days Output traffic for days. --dbdir directory Use directory as database directory instead of using the directory specified in the configuration file or the hardcoded default if no configuration file is available. -h, --hours Output traffic for the last 24 hours. -hs, --hsummary Output traffic summary including hourly data using a horizontal layout. -i, --iface interface Use interface instead of default or configured interface. --locale locale Use locale instead of using the locale setting specified in the configuration file or the system default if no configuration file is available. -m, --months Output traffic for months. -ne, --noedge Remove darker edges from around the image. -nh, --noheader Remove header containing title and update time. Time of the previous update will still be visible in the lower right corner using a less visible color. -o, --output file Write png image to file and exit. Output can be directed to stdout by giving "-" as filename. -ru, --rateunit Swap the configured rate unit. If rate has been configured to be shown in bytes then rate will be shown in bits if this option is present. In the same way, if rate has been configured to be shown in bits then rate will be shown in bytes when this option is present. Alternatively 0 or 1 can be given as parameter for this option in order to select between bytes (0) and bits (1) regardless of the configuration file setting. --style number Modify the content and style of outputs. Setting number to 3 will show average traffic rate in all outputs where it is supported. Other values will show bar graphics instead. -s, --summary Output traffic summary. -t, --top10 Output all time top10 traffic days. --transparent Toggle background color transparency depending of the TransparentBg setting in the configuration file. Alternatively 0 or 1 can be given as parameter for this option in order to either disable (0) or enable (1) transparency regardless of the configuration file setting. -v, --version Show current version. -vs, --vsummary Output traffic summary including hourly data using a vertical layout. -?, --help Show a command summary. FILES
/var/lib/vnstat/ Default database directory. Files are named according to the monitored interfaces. /etc/vnstat.conf Config file that will be used unless $HOME/.vnstatrc exists. See vnstat.conf(5) for more information. EXAMPLES
vnstati -s -i eth0 -o /tmp/vnstat.png Output traffic summary for interface eth0 to file /tmp/vnstat.png. vnstati -vs -i eth0+eth1+eth2 -o /tmp/vnstat.png Output traffic summary with hourly data under the normal summary for a merge of interfaces eth0, eth1 and eth2 to file /tmp/vnstat.png. vnstati -h -c 15 -o /tmp/vnstat_h.png Output hourly traffic statistics for default interface to file /tmp/vnstat_h.png if the file has not been updated within the last 15 minutes. vnstati -d -ne -nh -o - Output daily traffic statistics without displaying the header section and edges for default interface to standard output (stdout). vnstati -m --config /home/me/vnstat.cfg -i -o - Output monthly traffic statistics for default interface specified in configuration file /home/me/vnstat.cfg to standard output (std- out). AUTHOR
Teemu Toivola <tst at iki dot fi> SEE ALSO
vnstat(1), vnstatd(1), vnstat.conf(5), units(7) version 1.11 JUNE 2011 VNSTATI(1)
All times are GMT -4. The time now is 05:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy