Ban Bad Bots by IP

In the Internet age we live in, it’s not uncommon for web servers to be hit with Unintentional, not so Distributed, Denial of Service (DoS) Attacks. The attacks itself is intentional, but it’s not trying to bring down the server. It’s just some stupid ‘bot running a probe as fast as it can and, as a side effect, bogging down your server.

Bots are evil, but at least smart bots rate limit themselves to keep off the radar.

The typical stupid bot is running a password probe against an admin login. If you run any WordPress sites, you see this. All. The. Time. There are many good options to prevent these attacks, including renaming well known URLs, IP filtering, and automatic IP banning software, like Fail2ban.

However, sometime you just need to make the attack go away now. For that, I have a couple of Bash functions/tcsh alias.

Finding the Bad Guy

The first step is to find the source of the attack. It works by pulling all of the IP addresses out of the web server log and sorting them by frequency. If an IP is hitting you hard enough to bog down your server then it’s going to appear often in the log.

The IP address making the request is the first field in most common web server log formats, so we pull it out with awk '{ print $1 }' access_log. Next we sort all of the the IPs and pass them to uniq -c which counts up the unique lines in it’s input. Finally, we pass the output of uniq in to sort -nr, which sorts it numerically, and pipe that in to head to give us the first 20 lines.

All together it looks like:

awk '{print $1 }' access_log | sort | uniq -c | sort -nr | head -20

And outputs is a list of IPs preceded by the number of times they appear in the log.

5076 10.112.144.129
1953 192.168.30.7
1496 10.161.138.191
1471 10.211.158.44
1458 192.168.168.180
1081 192.168.34.67
1057 10.169.27.153
 952 10.56.27.175
 842 192.168.158.86
 807 192.168.250.140

It’s pretty safe to say that an IP that appears thousands of times isn’t legit (WARNING: this is not true, and this post is not useful if your server is behind a proxy. You’ll just be seeing the proxy’s IP(s) for everything). Found a bad guy? Let’s block ‘em.

Blocking IPs on Linux

On Linux, the route command can be use with the reject option cause routing for a IP or network to fail. For an IP, it’s:

/sbin/route add -host 10.10.10.1 reject

For a network, it’s:

/sbin/route add -net 10.10.10.0 netmask 255.255.255.0 reject

Blocking IPs on FreeBSD

FreeBSD has a blackhole option to route that will drop traffic from an IP or network on the floor. For a single IP it’s:

/sbin/route add 10.10.10.1 127.0.0.1 -blackhole

For a network it’s:

/sbin/route add 10.10.10.0/24 127.0.0.1 -blackhole

Removing a block

Typically, I leave blocked things blocked. It’s not worth my time to clean them up and the block will be removed by the next reboot. However, if you find you’ve blocked something legitimate or just like things tidy, route will remove blocks as well.

Linux:

/sbin/route delete 10.10.10.1 reject

FreeBSD:

/sbin/route delete 10.10.10.1

Putting it All Together

Here are the Bash functions:

top-ips() { awk '{print $1 }' $@ | sort | uniq -c | sort -nr | head -20; }
ban-ip() { /sbin/route add -host $@ reject; }
bsd-ban-ip() { /sbin/route add $@ 127.0.0.1 -blackhole'; }

and the tcsh aliases:

alias top-ips 'awk '\''{print $1 }'\'' \!^ | sort | uniq -c | sort -nr | head -20'
alias ban-ip '/sbin/route add -host \!^ reject'
alias bsd-ban '/sbin/route add \!^ 127.0.0.1 -blackhole'

Use them as follows:

% top-ips /var/log/apache/access.log
5076 10.112.144.129
1953 192.168.30.7
1496 10.161.138.191
1471 10.211.158.44
1458 192.168.168.180
1081 192.168.34.67
1057 10.169.27.153
 952 10.56.27.175
 842 192.168.158.86
 807 192.168.250.140
% ban-ip 10.112.144.129

As I said at the beginning of this post, there are better long term, automated solutions, but when you need the stupid bot gone and your server back now, these aliases will get the job done.