How To Block User Agent in Apache

Sometimes you may need to block unwanted user agents, requests and referrers to your website. They might make you believe that your website gets a lot of traffic but they may not add any business to your website. They may not be bad but simply fill your web analytics data and server logs. In this article, we will look at how to block user agent in Apache web server. Although there are multiple ways to block user agents using network devices, firewalls, CDNs, and load balancers, we will look at how to block user agent in Apache server. This is very useful for most website administrators, since they may want to block these requests for their servers, websites, wordpress, etc.

How To Block User Agent in Apache

Here are the steps to block unwanted referrer and user agents in Apache. Before proceeding, you need to know which referrers and user agents you want to block. You can obtain their names from your web analytics tool like Google Analytics.

1. Enable mod_rewrite

Open terminal and run the following command to enable mod_rewrite. If it is already enabled, then you can skip this step.

$ sudo a2enmod rewrite

Also read : How to Install WordPress in NGINX

2. Block user-agents in Apache

Let us say you want to block user-agent whose name contains a string like “BingPreview”. In this case, add the following lines to your .htaccess file (at /var/www/html/.htaccess), apache configuration file (at /etc/httpd/conf/apache.conf or /etc/httpd/conf.d/apache.conf) or virtual host file, depending on your access.

RewriteEngine on 
RewriteCond %{HTTP_USER_AGENT}  ^.*BingPreview.*$ 
RewriteRule . - [R=403,L]

The first line enables mod_rewrite, the second line evaluates the user-agent value for each request and the third line blocks the ones with user-agent name containing “BingPreview”. In this case, Apache server does a pattern match.

If you know the exact user-agent or referrer names (e.g spambot, spamcrawler, etc) then you can add the following lines to block them.

RewriteEngine On 
RewriteCond %{HTTP_USER_AGENT} spamcrawler [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} spambot [NC,OR] 
RewriteRule . - [R=403,L]

The first line above enables mod_rewrite, while the next two lines check the exact user-agent value of each request and block the ones with user-agent values as spambot or spamcrawler. In this case, Apache server does an exact match.

Also read : How to Change Root Password in Ubuntu

If you want to block requests by referrer names (e.g blowfish, bluebot) add the following lines instead.

RewriteEngine on 
RewriteCond %{HTTP_REFERER} blowfish|bluebot [NC] 
RewriteRule . - [R=403,L]

In the above case, Apache will check the http_referer value of each request and block the ones matching blowfish or bluebot

Also read : How to Enable IPv6 in Apache

3. Restart Apache Web Server

Restart Apache web server to apply changes

$ sudo service apache2 restart

That’s it. Now Apache will start blocking requests based on user-agents and referrers.