Using Fail2Ban to stop an attack

Had two attacks this week on different WordPress sites. One seems like a DDoS and the other an enumeration attack. Fail2Ban stopped both by quickly adding new jails.

Attack one, DDoS maybe

The first attack had been going on for a while. It wasn't enough to take the server down (or many that wasn't the poing) I'm not sure. Either way, it was clearly not user traffic.

Watching the server logs there was tons of HEAD & GET requests going to this page. With different sets of values for tag_ids and cat_ids.

/training/webinars/action~week/tag_ids~388,601,614,273,315,521,390,311,131,348,314,195,157,616,522,169,611,376,547,600,70,144,613,146,172,375,617,171,168,408,274,130,495,497,525,275,559,387,148,104,312,380,514/request_format~html/

That page does return a list of webinars, some of the tag_ids / cat_ids match, but I'm assuming they'd match on a lot of sites.

Given the above, I created the following filter.

filters.d/nginx-someclientdomainname.conf

[Definition]

failregex = ^ -.* "(GET|HEAD) /training/webinars/action~week/.*

ignoreregex =

And then added to the local jail config.

jail.local

#...

[nginx-someclientdomainname]
enabled = True
port = http,https
logpath = %(nginx_access_log)s
maxretry = 3
findtime = 60
bantime  = 172800
action   = iptables-multiport[name=nginx-shorter, port="http,https", protocol=tcp]

Don't use long names for the filters / jails. It does warn you, but it's also easy to miss, which I did. And that's why I needed the action config

Then restart Fail2Ban with fail2ban-client restart. I suggest tailing the Fail2Ban log, but it can be hard to see what's happening when there is a lot of bad traffic.

All the IP I checked from this attack where from China

Attack two, admin Ajax enumeration

This was just one IP in the US. They were POST'ing to /wp-admin/admin-ajax.php. I'm not logging the POST body, so not excatly sure what the point was. This was also enough traffic to take the site down a couple of times.

filters.d/nginx-wp-ajax.conf

[Definition]

failregex = ^ -.* "POST /+wp-admin/admin-ajax.php .*

ignoreregex =

jail.local

# ...

[nginx-wp-ajax]
enabled = True
port = http,https
logpath = %(nginx_access_log)s
maxretry = 20
findtime = 60
bantime  = 172800

Testing new filters

Fail2Ban has a tool for testing filters, which is awesome so you know if the filter is working.

I first grabbed some sample data.

tail -n 5000 /var/log/nginx/access.log > sample.log

Then you can test the filter with fail2ban-regex.

fail2ban-regex sample-2.log nginx-wp-ajax

Running tests
=============

Use   failregex filter file : nginx-wp-ajax, basedir: /etc/fail2ban
Use         log file : sample-2.log
Use         encoding : UTF-8


Results
=======

Failregex: 213 total
|-  #) [# of hits] regular expression
|   1) [213] ^ -.* "POST /+wp-admin/admin-ajax.php .*
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1000] Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
`-

Lines: 1000 lines, 0 ignored, 213 matched, 787 missed
[processed in 0.27 sec]

Missed line(s): too many to print.  Use --print-all-missed to print all 787 lines

Who's in jail

Here're the commands for seeing IPs that are blocked.

fail2ban-client banned will show all currently banned IPs.

fail2ban-client status <JAIL> will show details about a specific jail.

iptables -S | grep f2b show's the IPs that currently banned in the firewall.

Summarize the fail2ban logs

Reporting on today's activity:

grep "Ban " /var/log/fail2ban.log \
  | awk -F[\ \:] '{print $19,$17}' | sort | uniq -c | sort -n |

Group by IP address and Fail2Ban section:

grep "Ban " /var/log/fail2ban.log \
  | grep $(date +%Y-%m-%d) \
  | awk '{print $NF}' | sort \
  | awk '{print $1,"("$1")"}' \
  | uniq -c | sort -n

Grouping by Date and Fail2Ban section

zgrep -h "Ban " /var/log/fail2ban.log* \
  | awk '{print $6,$1}' \
  | sort | uniq -c

From: The Art of Web - Fail2Ban log