One of our customers recently was getting some massive 404 attacks, where attackers flood a website with requests for various pages, likely trying to find a vulnerability, and wanted to know if we had anything that could help reduce the attacks as they result in overloading servers and impacting performance for the site being attacked as well as others being hosted on the same server.
Cleavr installs and configures fail2ban for each server provisioned and can be a good resource to use to help prevent these types of attacks by blocking IPs that generate too many 404 errors.
Though, I do want to mention that you may not want to necessary block users who trigger one too many 404s as it may not be a malicious user, or you may also have some other processes running on your app where this may cause an interference. I just want to present it as an option for your consideration.
Step 1:
SSH into your server. View our guide on how to SSH into your server. Or, use your favorite SFTP client.
Step 2:
Add the following new file named nginx-4xx.conf to /etc/fail2ban/filter.d/
[Definition]
failregex = ^<HOST>.*"(GET|POST).*" (404|444|403|400) .*$
ignoreregex =
The above is a filter definition that tells fail2ban to look for errors marked 400, 403, 404, and 444.
Step 3:
Now, open /etc/fail2ban/jail.conf and add the following block of code to the end of the file.
[nginx-4xx]
enabled = true
port = http,https
filter = nginx-4xx
logpath = /var/log/nginx/access.log
bantime = 1800
findtime = 60
maxretry = 5
In the above, pay attention to bantime, maxretry, and findtime. This is saying if a user hits the 404 error 5 times over a 60 second period, then ban the user’s IP for 1800 seconds, which is 30 minutes.
Modify the values according to your needs - but, I recommend something that will deter attackers, who tend to probe many pages in less than a minutes, but that also will minimally impact legitimate visitors.
Step 4:
Now, restart process by running following command:
service fail2ban restart
To view status, including jailed IPs, you can run the following command:
fail2ban-client status nginx-4xx
It is a manual setup right now but we’ve added it to our backlog to make adding / managing additional fail2ban rules easy-peezy. 