What is Robots.txt?
Let me Explain robots.txt in Short Manner.Robots.txt is a few lines of text file with simple codes Which Instructs the Web Crawler on How to index and crawl your blog in search results.
How to Add custom Robots.txt in blogger?: I Explained on how to add custom robots.txt in blogger within a single picture below. If you don't See the Picture Clearly Click on it to enlarge the image.
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED
Replace: example.blogspot.com with your blogname. Advanced Robots.txt ( Includes Disallow Particular Post, Disallow Particular Page) : If you want to Exclude Particular post from indexing then Add the below code At the end of the custom code after giving the post year/Month/Post Url in the fields yyyy/mm/post-url.html.
"Disallow: /yyyy/mm/post-url.html"
For example: If you want to exclude "http://www.gobloggertricks.com/2014/06/list-of-101-high-pr-dofollow-comment.html" From google then you have to give the code as Disallow: /2014/06/list-of-101-high-pr-dofollow-comment.html . If you want to exclude particular page then you can add the below code at the custom robots.txt.
"Disallow: /p/page-url.html"
Replace Page-url.html with the url of your page. You can view your Robots.txt file by adding robots.txt at the end of your blog url for example talktraffik.blogspot.com/robots.txt If you have any doubt please comment below or mail us at: trafficandtweettip.com@gmail.com
Like the Post? Share with your Friends:-
.png)
nice blog.It identifies syntax errors, logic errors, mistyped words and also provides useful optimization tips.robots txt checker
ReplyDelete