Robots txt Very Important Guidelines SEOFirstStep
A robots.txt file lives at the root of your site. So, for site www.example.com, the robots.txt file lives at www.example.com/robots.txt. robots.txt is a plain text file that follows the Robots Exclusion Standard. A robots.txt file consists of one or more rules. Each rule blocks (or or allows) access for a given crawler to a specified file path in that website.
Here is a simple robots.txt file with two rules, explained below
READ MORE...No.1 SEO Expert in India , Romil Tripathi SEO Consultant | Digital Marketing Consultant India Result Oriented
Comments
Post a Comment