How to block bad bots with .htaccess and php

Block Bad Bots

Block Bad Bots

In this tutorial, we’ll learn how to block bad bots and spiders from your website using .htaccess and PHP. We can save bandwidth and performance for customers, increase security, and prevent scrapers from putting duplicate content around the web.

Few days ago ,I was checking log files on my server and I found that a bot was hitting my site very badly .It was crawling pages of my website every second.I have checked the user agent of the bot and I found this.

Mozilla/5.0 (compatible; AhrefsBot/4.0; +

This bot was useless for me because so I had decided to block this bot by robots.txt .I have added few  extra line for the bot and block this bot  from  access my website.After 3 days,I  checked it back and found that it was still crawling my website avoiding rules of  robots.txt.Then I have decided to block this user agent by using .htaccess file.

So here is the tutorial to block bad bots with .htaccess .

1.Create a .htaccess file -If  .htaccess file  is not available on your server then create a new file and save as .htaccess.On windows you can’t directly create a file with this name because windows requires first name with extension and you are creating this file without first name.So use this method on open notepad and save as .htaccess.

2.Define rewrite base-Add these line to your .htaccessfile.

3.Block bad User Agents-Add these lines just after below RewriteEngine on.

NC -Not Case-sensitive



In my case after using this method that AhrefsBot has stopped to crawl my website.But some user agent may bypass this method by changing their user agent.So instead of blocking those bots by user agent you can block them by IP address.Add this line at the beginning of your .htaccess file.It will block all request from IP address .You can replace this IP with your bad bot’s IP address.

If you want block all request from IPs started with 127.0.0 you can add this line.

This code will also block all IPs from  to

Now come to PHP you can also do all above things with PHP.Here is a sample code to block a bot by user agent.

You can also block a bad bot by IPs.

You can save any of  above files as block.php and you can include this file to all your PHP files byadding a single line.