Robots | Digital Marketing Agency India

Robots

Get Hand-picked Digital Marketing News Inboxed Weekly

Robots.txt Features

  • No follow, No Index Tag Option for each page, by default it is yes and can be overridden manually
  • Robots.txt, just ensure your CMS doesn't stop robots.txt from rendering because of some settings, make a basic robots.txt in the root folder and ensure it is rendered properly. Download the basic robots.txt folder.
     

Manage page views to search engine via robot.txt

The CMS has an in-built module which makes effortless creation of important files like robot.txt possible with just a check. The image below is the section and the user just has to check the box beside "allow search engine to index this page" to generate an automatic robots file. It also allows page specific customization - it gives you the option to disallow search engine to crawl, index and prevent caching  the particular page.  Advance protocols like preventing description from appearing on search result can also be done.

Easy Creation Of Robots.txt

TechShu Consultancy Pvt. Ltd
22-05-2013
5/5 stars
Techshu is doing very good work for me. I am continuing to use them for this ongoing project