Disallows robots from indexing your site unless explicitly allowed.
Features:
Q: What is the purpose of this add-on?
A: Lots of robots crawling your forum will slow it down.
Q: I noticed a robot crawling my site but it's identified as a guest.
A: You need to add the name to the Blacklist field in the Options page.
Q: What happens if a robot which is not allowed comes to my site?
A: It receives a blank page.
Q: Why not use the robots.txt file to restrict robots?
A: Because robots ignore this file.
Database modifications:
This add-on will create a table called xf_andy_robots in your database.
Features:
- All phrases start with robots_ for your convenience.
- Download Andy-Robots-2.3.zip and unzip it.
- Copy the src/addons/Andy/Robots directory to your server.
- From the Admin Control Panel Install the add-on.
Q: What is the purpose of this add-on?
A: Lots of robots crawling your forum will slow it down.
Q: I noticed a robot crawling my site but it's identified as a guest.
A: You need to add the name to the Blacklist field in the Options page.
Q: What happens if a robot which is not allowed comes to my site?
A: It receives a blank page.
Q: Why not use the robots.txt file to restrict robots?
A: Because robots ignore this file.
Database modifications:
This add-on will create a table called xf_andy_robots in your database.