User-agent filtering

In Apache logs, the column that contains the "description" of the visiting agent (usually the web browser or other client software) is called the user-agent. It is part of the common log format or combined log format and provides information about the software making the request, such as the browser type, version, and operating system.

In the Apache access log, it typically appears in the following format (indentation mine):

"GET /index.html HTTP/1.1" 200 2326 "http://example.com/previous"
"
 Mozilla/5.0 
 (Windows NT 10.0; Win64; x64)
 AppleWebKit/537.36 (KHTML, like Gecko)
 Chrome/91.0.4472.124 Safari/537.36
"

It would be very useful to be able to filter incoming traffic, based on this user-agent, and this is not what a firewall is for, as this is OSI Layer 7-stuff, rather than layer 3 or 4 stuff.

Inventory

This is the stuff typically done by reverse proxy servers, application firewalls or web application firewalls (WAFs)

Inventory of tools for user-agent filtering

Tool	Remarks
HAProxy
Nginx
Mod_rewrite	Add-on for Apache that can rewrite URLs based on visitor's user-agent
Mod_security	Add-on for Apache and Nginx - Probably the most suitable tool fo
Mod_setenvif	Add-on for Apache to block traffic based on user-agent
Squid	?

See also

• Apache log files
• Apache mod_security
• iptables