Fail2ban
Naar navigatie springen
Naar zoeken springen
fail2ban is a tool designed to block suspicious server visits:
- Log files are real-time scanned for suspicious behaviour. (Apache, PHP, SSH, etc.)
- Scanning is done according to configurable filters. This is done through Python
- Selected lines are added to iptables - So, the actual blocking is done through iptables, which seems like an efficient approach to blocking
- Duration of these blocking rules is configurable. E.g., an hour for repeated inlog failures, or a month for an known unwanted crawler.
Overview
What it does
- Monitors Log Files: Fail2ban scans log files for specific patterns that indicate suspicious activity, such as failed login attempts or other anomalies
- Blocks IP Addresses: When it detects repeated failed attempts from a particular IP address, it can temporarily or permanently block that IP address using firewall rules (such as with iptables, nftables, or firewalld)
- Configurable: You can customize Fail2ban to monitor different services and define what constitutes "bad behavior." It can protect various services like SSH, FTP, HTTP, and more.
How it works
- Jails: Fail2ban uses "jails," which are configurations specifying which log files to monitor, what patterns to look for, and what actions to take. Each jail is associated with a particular service or application
- Filters: Filters are used within jails to define the patterns of failed attempts. These are often specified using regular expressions
- Actions: When a pattern is detected, Fail2ban can execute various actions, typically involving updating firewall rules to block the offending IP addresses.
Configuration
- Main Configuration File: /etc/fail2ban/jail.conf (though it's a good practice to create and modify /etc/fail2ban/jail.local to avoid overwriting the default settings)
- Filters Directory: /etc/fail2ban/filter.d/ contains filter definitions
- Actions Directory: /etc/fail2ban/action.d/ contains predefined actions for blocking IPs.
Benefits
- Flexible: Fail2ban can probably be configured for any network function that maintains a log file
- Makes WordPress security plugins superfluous? Would be nice if I can dump WordFence
- Efficient: Fail2ban has been written in Python, while the actual banning is (in our case) done by
iptables. Using a low-level tool like iptables for filtering, is much more efficient than using WordPress plugins like WordFence - Temporary blocking: It's really nice that blocks can be temporary, so in case someone genuinely makes 20 mistakes in a row, or if a hacked IP address eventually gets unhacked, that they regain access.
Disadvantages
Fail2ban basically does what it says on the package, but it does come with quite some challenges:
| Aspect | Remarks |
|---|---|
| Learning curve |
|
| Risky | The high learning curve makes it risky to deploy fail2ban: New suprises might lurk everywhere |
| High maintenance |
|
Alternatives
| Alternative | Remarks |
|---|---|
| CrowdSec | Probably the most obvious alternative to Fail2ban
|
| Own scripts | Have own scripts that parse Apache logs and derives firewall rules - We actually use this concurrency with Fail2ban right now
|
| Manual blocks |
|
Additional measures
In follow-up to the alternatives mentioned before: Some ideas about additional measures, as it is never a good idea to rely on just one tool.
| Measure | Remarks |
|---|---|
| Apache user agent blocks |
|
| Robots.txt |
|
| External firewalls |
Use external firewalls like Cloudflare - This could be an excellent solution:
|
Note: Existing rules flushed on restart
By default, when fail2ban is restarted, it flushes the existing iptables rules that it manages. This happens because fail2ban is designed to reinitialize its own firewall rules upon starting or restarting. Specifically, it clears and rebuilds the rules in the chains it manages (typically named f2b-*), which means any existing bans applied by fail2ban will be removed and need to be re-added as fail2ban reads the log files and re-applies bans.
Note that fail2ban probably doesn't flush rules outside the f2b-* chains: Those might get flushed when restarting a webserver, as iptables' rules are not persistent across restarts of iptables.
Start & stop
Basic commands
sudo systemctl stop fail2ban
sudo systemctl disable fail2ban
sudo systemctl enable fail2ban
Restart
To restart Fail2Ban on a systemd-based system like Ubuntu Server:
sudo systemctl restart fail2ban
To verify that it restarted correctly:
sudo systemctl status fail2ban
Or check logs in real-time:
sudo journalctl -u fail2ban -f
Restarting Fail2ban, has quite some consequences:
- The Fail2Ban daemon (fail2ban-server) stops and restarts
- For all jails, the stop actions that are defined within them, are called
- All f2b chains are flushed and deleted. Also jumps from INPUT to these chains, get deleted
- It reloads its configuration files: fail2ban.conf, jail.conf, jail.local, and any custom filters in filter.d/.
- All jails are re-initialized:
f2b-chains are newly created - Jumps will be inserted to these
f2b-chains, typically in table INPUT, by default usingiptables -I INPUT 1, which might be quite a problem. This is usually defined throughiptables-common.confbut can be overriden or replaced, etc. - See elsewhere - All IPs previously banned by Fail2Ban are unbanned, unless
banactionhas been configured to include external persistence.
Reload
You can reload Fail2Ban's configuration without a full restart using
sudo fail2ban-client reload
This is much lighter than a full systemctl restart fail2ban. Here's what it does:
- Re-reads all configuration files (jail.conf, jail.local, jail.d/*.conf, etc.)
- Reloads all jails with the new config
- Preserves currently banned IPs; In-memory retry counters; Running log monitors
- Rebuilds the f2b-* chains if the jail’s action config changed
Use reload when:
- You've modified jail/filter config files
- You want the changes to take effect
- But you don’t want to drop active bans or trigger a full restart.
Reload individual jails
Reload a single jail:
sudo fail2ban-client reload apache-badbots
Status & overview
There are different ways to check status and to get an overview of how Fail2ban is doing.
Summary:
sudo systemctl status fail2ban sudo fail2ban-client -d sudo fail2ban-client status sudo fail2ban-client status apache-badbots sudo fail2ban-client get apache-badbots logpath sudo fail2ban-client get apache-badbots filter sudo fail2ban-client get apache-badbots action sudo fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-badbots-custom.conf
Check daemon status
Example of an active instance:
$ sudo systemctl status fail2ban
● fail2ban.service - Fail2Ban Service
Loaded: loaded (/usr/lib/systemd/system/fail2ban.service; enabled; preset: enabled)
Active: active (running) since Wed 2024-09-18 04:05:35 CEST; 3 weeks 2 days ago
Docs: man:fail2ban(1)
Main PID: 2350 (fail2ban-server)
Tasks: 15 (limit: 115387)
Memory: 30.4M (peak: 49.2M swap: 2.3M swap peak: 2.3M)
CPU: 2h 48min 56.177s
CGroup: /system.slice/fail2ban.service
└─2350 /usr/bin/python3 /usr/bin/fail2ban-server -xf start
Notice: journal has been rotated since unit was started, output may be incomplete.
Example of a paused instance:
$ sudo systemctl status fail2ban
○ fail2ban.service - Fail2Ban Service
Loaded: loaded (/usr/lib/systemd/system/fail2ban.service; enabled; preset: enabled)
Active: inactive (dead) since Fri 2024-10-11 12:41:25 CEST; 54s ago
Duration: 3w 2d 8h 35min 48.694s
Docs: man:fail2ban(1)
Process: 2350 ExecStart=/usr/bin/fail2ban-server -xf start (code=killed, signal=TERM)
Process: 2407962 ExecStop=/usr/bin/fail2ban-client stop (code=exited, status=0/SUCCESS)
Main PID: 2350 (code=killed, signal=TERM)
CPU: 2h 48min 56.608s
Oct 11 12:41:24 server2 systemd[1]: Stopping fail2ban.service - Fail2Ban Service...
Oct 11 12:41:25 server2 fail2ban-client[2407962]: Shutdown successful
Oct 11 12:41:25 server2 systemd[1]: fail2ban.service: Deactivated successfully.
Oct 11 12:41:25 server2 systemd[1]: Stopped fail2ban.service - Fail2Ban Service.
Oct 11 12:41:25 server2 systemd[1]: fail2ban.service: Consumed 2h 48min 56.608s CPU time, 49.2M memory peak, 2.3M memory >
Notice: journal has been rotated since unit was started, output may be incomplete.
Check & dump complete configuration
I quite like
sudo fail2ban-client -d
because it checks the syntax of the configuration files, prior to dump it in a Jason-like format.
What it can't do:
- Tell from which configuration files it got its information
- Limit output to a specific jail, but this seems to work fine using grep.
Example:
$ sudo fail2ban-client -d | grep apache-badbots
2025-05-11 20:28:19,673 fail2ban.configreader [1835052]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'
['add', 'apache-badbots', 'polling']
['set', 'apache-badbots', 'usedns', 'warn']
['set', 'apache-badbots', 'addfailregex', '^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"(?:AliyunSecBot|ALittle Client|AwarioBot|Axios|Barkrowler|BitSightBot|Bloglines|bne\\.es_bot|Bulid|Bytedance|CensysInspect|ClaudeBot|coccocbot|curl|Custom-AsyncHttpClient|DataForSeoBot|DomainStatsBot|DotBot|Expanse|facebookcatalog|facebookexternalhit|Foregenix|Go-http-client|Google-Firebase|google-xrawler|GRequests|GuzzleHttp|GPTBot|Hello World|hello-world|heritrix|idealo-bot|ImagesiftBot|imagor|InternetMeasurement|ips-agent|ivre-masscan|Java|jorgee|Keydrop\\.io|KlarnaBot|Konqueror/4\\.2|l9explore|l9tcpid|LG-GC900|libwww-perl|Links \\(2\\.1|Linux Mozilla|masscan|meta-externalagent|MJ12bot|Moblie|ModatScanner|mozilla/2|mozilla/3|mozillia/4|Mozlila|MS Search 6|MSIE ?\\(?(4|5|6|7|8|9|10)|Nicecrawler|NT 11)|OI-Crawler|petalbot|PHP|presto|python-requests|reqwest|robertdavidgraham|Safari 1\\.2\\.4|SemrushBot|SEOkicks|serpstatbot|Slackbot|Slackware/13\\.0|sogou|tchelebi|VelenPublicWebCrawler|WebwikiBot|Wikistats|Win 9x|Win32|WinNT4|Windows (_?95|98|CE|ME|NT (4|5|6\\.0|6\\.1|6\\.2)|Mobile|Phone)|Windows NT (4|5|6\\.0|6\\.1|6\\.2)|yandex|yie9|YisouSpider|Zend_Http_Client|ZoominfoBot)"$']
['set', 'apache-badbots', 'datepattern', '^[^\\[]*\\[({DATE})\n{^LN-BEG}']
['set', 'apache-badbots', 'maxretry', 1]
['set', 'apache-badbots', 'maxmatches', 1]
['set', 'apache-badbots', 'findtime', '3600']
['set', 'apache-badbots', 'bantime', '2592000']
['set', 'apache-badbots', 'ignorecommand', '']
['set', 'apache-badbots', 'addignoreip', '127.0.0.1/8', '::1', '178.128.44.198', 'file:/etc/fail2ban/whitelist.conf']
['set', 'apache-badbots', 'logencoding', 'auto']
['set', 'apache-badbots', 'addlogpath', '/var/log/apache2/access.log', '\\']
['set', 'apache-badbots', 'addlogpath', '/var/log/apache2/example_com_access.log', '\\']
['set', 'apache-badbots', 'addlogpath', '/var/log/apache2/other_vhosts_access.log', 'head']
['set', 'apache-badbots', 'addaction', 'iptables-multiport']
['multi-set', 'apache-badbots', 'action', 'iptables-multiport', [['actionstart', "{ <iptables> -C f2b-apache-badbots -j RETURN >/dev/null 2>&1; } || { <iptables> -N f2b-apache-badbots || true; <iptables> -A f2b-apache-badbots -j RETURN; }\nfor proto in $(echo 'tcp' | sed 's/,/ /g'); do\n{ <iptables> -C INPUT -p $proto -m multiport --dports http,https -j f2b-apache-badbots >/dev/null 2>&1; } || { <iptables> -A INPUT -p $proto -m multiport --dports http,https -j f2b-apache-badbots; }\ndone"], ['actionstop', "for proto in $(echo 'tcp' | sed 's/,/ /g'); do\n<iptables> -D INPUT -p $proto -m multiport --dports http,https -j f2b-apache-badbots\ndone\n<iptables> -F f2b-apache-badbots\n<iptables> -X f2b-apache-badbots"], ['actionflush', '<iptables> -F f2b-apache-badbots'], ['actioncheck', "for proto in $(echo 'tcp' | sed 's/,/ /g'); do\n<iptables> -C INPUT -p $proto -m multiport --dports http,https -j f2b-apache-badbots\ndone"], ['actionban', '<iptables> -I f2b-apache-badbots 1 -s <ip> -j <blocktype>'], ['actionunban', '<iptables> -D f2b-apache-badbots -s <ip> -j <blocktype>'], ['port', 'http,https'], ['protocol', 'tcp'], ['chain', '<known/chain>'], ['name', 'apache-badbots'], ['actname', 'iptables-multiport'], ['blocktype', 'REJECT --reject-with icmp-port-unreachable'], ['returntype', 'RETURN'], ['lockingopt', '-w'], ['iptables', 'iptables <lockingopt>'], ['blocktype?family=inet6', 'REJECT --reject-with icmp6-port-unreachable'], ['iptables?family=inet6', 'ip6tables <lockingopt>']]]
['start', 'apache-badbots']
List active jails
fail2ban-client is basically a CLI to communicate with the fail2ban daemon (fail2ban-server), maybe similarly to how the CLI command mysql is effectively a tool to communicate with a MySQL-server.
What you can do with fail2ban-client:
- Check status (
fail2ban-client status) - Enable or disable jails
- Ban or unban IPs manually
- Reload or restart the server
- Test filters
- Debug configurations.
Example:
$ sudo fail2ban-client status Status |- Number of jail: 7 `- Jail list: apache-auth, apache-badbots, apache-get-dos, apache-noscript, apache-overflows, php-url-fopen, sshd
See detailed info per jail
Use
sudo fail2ban-client status <jailname>
without the f2b- prefix.
Example (1)
$ sudo fail2ban-client status apache-auth Status for the jail: apache-auth |- Filter | |- Currently failed: 0 | |- Total failed: 3017 | `- File list: /var/log/apache2/error.log `- Actions |- Currently banned: 44 |- Total banned: 2752 `- Banned IP list: 76.167.231.121 185.50.25.3 89.253.240.73 5.188.159.153 54.36.208.50 185.194.217.18 194.163.151.88 23.92.26.113 101.58.155.111 83.166.133.74 157.230.24.5 46.36.217.232 91.134.248.211 185.50.25.42 217.160.207.42 195.225.221.2 125.23.86.26 54.66.48.110 181.214.164.79 212.56.54.91 43.243.60.101 206.189.18.26 158.255.6.93 173.236.254.75 193.70.39.165 51.68.11.199 159.65.137.64 85.128.143.16 170.81.42.166 34.13.135.125 212.227.50.191 149.50.149.75 151.80.19.225 43.153.98.152 40.90.184.1 78.193.66.197 162.241.2.41 45.156.184.39 141.8.192.164 188.165.192.229 92.205.64.128 162.241.217.171 212.125.4.212 111.119.194.119
Example (2)
Yes, apache-badbots has 76.000 IP adresses. And it's only active for about 36 hours. Bans are for a month, hence the number of currently banned and total banned are the same.
$ sudo fail2ban-client status apache-badbots Status for the jail: apache-badbots |- Filter | |- Currently failed: 0 | |- Total failed: 109218 | `- File list: /var/log/apache2/access.log `- Actions |- Currently banned: 76452 |- Total banned: 76452 `- Banned IP list: 18.223.125.111 1.162.78.248 1.178.223.215 1.179.75.62 1.180.121.47 1.181.171.55 1.182.19.171 1.182.23.129 1.183.100.133 1.187.217.55 1.187.226.253 1.187.227.47 1.187.232.232 1.187.239.214 1.188.128.157 1.188.146.57 1.189.181.163 1.191.10.147...
More details per jail
Returns an array with IP addresses:
sudo fail2ban-client get apache-badbots banned
See what logpath is being used:
$ sudo fail2ban-client get apache-badbots logpath Current monitored log file(s): `- /var/log/apache2/access.log
Number of retries, findtime & bantime:
$ sudo fail2ban-client get apache-badbots maxretry 1 $ sudo fail2ban-client get apache-badbots findtime 3600 $ sudo fail2ban-client get apache-badbots bantime 2592000
This approach doesn't work to retrieve the value for filter, enabled or port.
Check fail2ban log
Fail2ban maintains a log file, typically to be found at /var/log/fail2ban.log. Probably my favourite way to check this log:
sudo tail -f /var/log/fail2ban.log
or
sudo tail -n 50 /var/log/fail2ban.log
An example of the output of the first command:
$ sudo tail -f /var/log/fail2ban.log 2025-05-08 12:02:00,644 fail2ban.filter [2341]: INFO [apache-auth] Found 111.119.194.119 - 2025-05-08 12:02:00 2025-05-08 12:02:00,856 fail2ban.actions [2341]: NOTICE [apache-auth] Ban 111.119.194.119 2025-05-08 12:03:11,944 fail2ban.filter [2341]: INFO [apache-get-dos] Found 20.236.227.53 - 2025-05-08 12:03:11 2025-05-08 12:06:00,122 fail2ban.actions [2341]: NOTICE [apache-auth] Unban 76.167.231.121 2025-05-08 12:06:00,146 fail2ban.actions [2341]: NOTICE [apache-auth] Unban 185.50.25.3
Appearantly, tail -f automatically shows the most recent 10 lines and subsequently adds lines in real-time when they appear.
f2s
f2s, an abbreviation of Fail2ban Status is an own script, that displays info about all inidividual jails. It's included in the usual Git account - not an alias in .bashrc
System of configuration files
It probably helps to have a good overview of the system of configuration files, to know where to look for stuff. This is just an impression:
| File | Remarks |
|---|---|
paths-common.conf
|
|
paths-debian.conf
|
Similar to paths-common.conf but less used
|
jail.conf
|
Example of a jail default - Note that it doesn't specify a banaction, but resorts to the default jail-independent value: # [apache-badbots] # Ban hosts which agent identifies spammer robots crawling the web # for email addresses. The mail outputs are buffered. # # Commented-out - 2025.05.09 # # port = http,https # logpath = %(apache_access_log)s # bantime = 48h # maxretry = 1 |
jail.local
|
|
jail.d/(directory) |
My preferred location for jail configuration files + commenting-out earlier definitions in jail.conf and jail.local
|
jail.d/custom.conf
|
Contains default values for bantime, findtime and banaction. I'm not sure if this file is actually used
|
filter.d/(directory) |
|
action.d/(directory) |
Contains some 64 default actions |
action.d/iptables.conf
|
Within this file, specific actions are defined (in addition to other things) - and customized:
|
Jails
A jail is a combination of three things:
| Thing | Remarks |
|---|---|
| log | The log file to be read |
| filter |
|
| action | What to do when a rule is triggered. Usually banning with a certain bantime |
Appearantly, there is no such thing as a default set of jails. The list of jails at the server I'm currently looking at:
- apache-auth
- apache-badbots
- apache-get-dos
- apache-noscript
- apache-overflows
- php-url-fopen
- sshd
Jail configuration options
Jails can be configured at different levels:
| Jail configuration file | Remarks |
|---|---|
/etc/fail2ban/jail.conf
|
Contains default configurations for a bunch of jails. However: Don't edit this file, as it might get overwritten with updates |
/etc/fail2ban/jail.local
|
Better: Incorporate jail configuration overrides in this file |
/etc/fail2ban/jail.d/ (directory)
|
Store jail overrides in this directory, with a separate file for each jail.
Example of the content of this directory (2025.04):
|
Jail configuration files
Of the various options mentioned above, I prefer to use separate files for each jail in directory /jail.d.
Examples of these files:
| File name | Content and/or remarks |
|---|---|
| apache-get-dos.conf |
[apache-get-dos] enabled = true port = http,https filter = apache-get-dos logpath = /var/log/apache2/*access.log datepattern = %%d/%%b/%%Y:%%H:%%M:%%S %%z maxretry = 300 findtime = 5m bantime = 1h |
| custom.conf |
[DEFAULT] bantime = 300 findtime = 300 banaction = iptables-allports |
| defaults-debian.conf | Empty |
| sshd.conf | slightly censored version of the actual file - Looks like a regular simple jail configuration file:
[sshd] enabled = true port: 20 logpath = %(sshd_log)s backend = %(sshd_backend)s bantime = 600 findtime = 650 maxretry = 5 |
Configuring filters
Default filters are available at /etc/fail2ban/filter.d/
Configuring iptables for fail2ban
Since April 2025, we use the folling structure for iptables' INPUT chain:
| Chain | Remarks |
|---|---|
WHITELIST
|
At the top: Stuff that always has to be accepted, starting with SSH access for administrators to the server |
BLOCKBOT
|
Ad-hoc fail2ban-like script that we're using while configuring fail2ban |
| f2b chains | E.g.: f2b-apache-auth
|
PUBLIC
|
|
Fail2ban creates its own chains for the INPUT table, if not present. It's quite picky with the exact name and characteristics of chains. If the exactly needed chain isn't present, fail2ban will create it, and there might be two chains with a seemingly identical name.
apache-badbots jail
For apache-badbots, see the separate chapter later on.
apache-nohome jail
apache-nohome jail is designed to catch bots or scanners that try to access non-existent user directories or homepages, especially those that follow patterns like:
GET /~admin/GET /~root/GET /~user/
These requests target legacy Apache behavior where user directories could be accessed via http://example.com/~username/. Most modern servers don’t support this anymore, so these requests almost always return 404.
apache-noscript jail
The apache-noscript jail in Fail2ban is designed to catch bots (or humans) trying to access script files that shouldn’t exist, like:
/login.php/wp-login.php/admin.asp/shell.cgi/xmlrpc.php/phpMyAdmin
These requests are typical of malicious crawlers, automated vulnerability scanners, or botnets looking for common CMS or admin panels to exploit.
Case: Configure apache-badbots (2025.05)
apache-badbots, is the usual jail for catching known unwanted bots and crawlers, based on their user-agent strings. They are immediately banned, as soon as they appear
| Part | Remarks |
|---|---|
| log | The log file to be read |
| filter | /etc/fail2ban/filter.d/apache-badbots.conf: Basically a list of user-agent strings
|
| action | What to do when a rule is triggered. Usually banning with a certain bantime |
We used to include these in AVHCFs, but probably better to include them in Fail2Ban:
- More flexible to configure them in fail2ban: One specification for the whole server, rather than for each individual AVHCF
- More intuitive to configure them in fail2ban, as this is a more obvious place to look for such things
- Lower treshhold for updating them - Again, because it's only one place where to do this
- Probably more CPU-efficient, as this results in iptable records, and those are surely more efficient than whatever Apache uses.