04-22-2018
In general doing this manually would be folly but I know at least that there are efforts at blocking all "bad" bots (bots with recognisable user agents)
One example is: Apache Ultimate Bad Bot Blocker (find on github).
It uses BrowserMatchNoCase or similar to match user agents and put them in a list (set an environment variable for it) which then, as a whole, is denied.
There's little point in blocking known URLs your server doesn't have, as opposed to the bots that try to access the ones you *do* have.
So blocking the URLs is pointless (ineffective if you don't have them, and disruptive if you do have them), you will have to block the activity itself.
Many bots (most bots) do not actually identify as a common user agent, even the hacker-type bots will just use something recognisable.
Alternatively, when a known URL gets hit that requires password authentication fail2ban is often employed to block individual IPs.
Last edited by dryden; 04-22-2018 at 08:38 AM..
Reason: Automatic merging not acceptable
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello, how to hide the full addres url, in apache web server. eg, www.example.org/www/pub/index.html, the address in browser only www.example.org .
Thank You. (2 Replies)
Discussion started by: blesets
2 Replies
2. UNIX for Advanced & Expert Users
Hi all,
How can I enable encoding of special characters present in URL?
eg
If the URL is
http://127.0.0.1/test.cgi?param1=test & test co
it shouldbe encoded to
http://127.0.0.1/test.cgi?param1=test%20%26%20test%20co
Thanks and Reagards,
uttam hoode (3 Replies)
Discussion started by: uttamhoode
3 Replies
3. Cybersecurity
Hello.
I have scenario where a Client send request to Server1.
Server1 send request to Server2.
Request are xmlHTTPRequest - need to get data (XML) from Server2 back to client.
Trying to use APACHE proxy...
Anyone can help?
What to download / configure / ...?
Thank you for your help. (1 Reply)
Discussion started by: ampo
1 Replies
4. Web Development
Hello.
I have scenario where a Client send request to Server1.
Server1 send request to Server2.
Request are xmlHTTPRequest - need to get data (XML) from Server2 back to client.
Trying to use APACHE proxy...
Anyone can help?
What to download / configure / ...?
Thank you for your... (2 Replies)
Discussion started by: ampo
2 Replies
5. Shell Programming and Scripting
i need help on redirecting apache conf file
i want redirect everything to www.example.com/home
example if i type a url
www.example.com/home/text1
i need that redirected to www.example.com/home (0 Replies)
Discussion started by: shehzad_m
0 Replies
6. Web Development
Hi All,
i'am facing a problem with urls that don't have a filestructure under DocumentRoot.
A URL like http://mydomain.com/applicationrew/Structure1/Structure2/some?parameter=key¶meter1=key1
Should be rewritet to something else.
Now i defined a Location like
<Location ~... (3 Replies)
Discussion started by: wuschelz
3 Replies
7. UNIX for Advanced & Expert Users
I need help in apache url redirection:
I have added the below command in httpd.conf and it is working fine.
Redirect http://xyz.com/site/homehttp://abc.com/site/home
Can we set a rule such that http://xyz.com/site/* -> http://abc.com/site/* is applied
For... (0 Replies)
Discussion started by: raghur77
0 Replies
8. Web Development
I need help in apache url redirection:
I have added the below command in httpd.conf and it is working fine.
Redirect http://xyz.com/site/homehttp://abc.com/site/home
Can we set a rule such that http://xyz.com/site/* -> http://abc.com/site/* is applied
For... (0 Replies)
Discussion started by: raghur77
0 Replies
9. Web Development
I am attempting to control redirections on my site using the Apache vhosts config.
I have two-letter regions setup, such as /fr, /de, /es, which I am currently ignoring as you will see from my vhosts file below. However I also have a 301 permanent redirect setup to redirect /cm (and /fr/cm,... (3 Replies)
Discussion started by: crmpicco
3 Replies
10. Red Hat
Hi folks,
I am trying to configure Apache webserver and also a virtual host inside this webserver.
For Global server config: /var/www/html/index.html
For virtual host config: /var/www/virtual/index.html
Both client10 & www10 are pointing to 192.168.122.10 IP address.
BUT, MY... (1 Reply)
Discussion started by: freebird8z
1 Replies