How To Use Robots Text For Better Seo In Blogger

Robots text plays a main role in a blog's seo. With robots text we actually indicate search engines robots how to crawl our blogs and how to index it. Robots text is not the meta robot tag codes, its a text which is added in a blogs settings option which helps us to tell robots which pages to crawl and index and which do not. Previously I have created a post in which I have told how to add custom robot text in blogger, if you have already added robots text in your blog then you don't need to read that post but if you haven't added robots text in your blog yet then your first task is to read that post and after adding robots text in your blog you can follow this post to use robots text in proper way. In this post I will tell you how to allow or disallow robots to index pages or labels in search results.

Allowing or disallowing labels in search results:

I am starting from allowing or disallowing robots to index or crawl labels/categories of your blog e.g in-case of my blog Seo, Menus and Blogging etc. So first of all go to Blogger Dashboard > Settings > Search preferences > Crawlers and indexing > Custom robots.text and edit it. For instance see below image:

If you have read my post about adding custom robots text then your robots text will be similar to below code:

User-agent: Mediapartners-GoogleDisallow:

User-agent: *
Disallow: /search
Allow: /
Sitemap: http://Yourblog.blogspot.com/feeds/posts/default?orderby=UPDATED

In the about code I have added /search beside Disallow so now robots will not index any of your category, if your blog have sitelinks in search results and you wish to show only categories of your blog then remove /search and you are done!

How does it work?

It works in a very simple way how? lets see an example of link of a category opened in browser for details to learn how does it work. See below image:

In the above example I have opened my Seo label in browser, you can see the highlighted part of link, it lies in every category so to block a label in search results you have to add /search beside disallow. On more thing about adding /search is that is not only blocks labels to be indexed in results but also any queries made for search in blog is also not allowed to be indexed because if you take a look of link of a query made in blog it also has the same text(/search). See example below:

In the above example 101helper is a query made for search, so if you wish to block only labels and not the search queries you could you /label instead of /search. It is the proper way to block only labels.

Allowing or disallowing pages in search results:

Now a days every blog has pages like contact, about, copyright, sitemap etc, but nobody want to index pages like contact, about and copyright I am not sure about Sitemap :p. The reason not to index contact and about pages is they aren't useful for visitors and aren't interesting too so now you can control robots to index your pages or no but keep one thing in mind that you can't control a specific page to be indexed or not so if you add /p in front of Disallow: then every page of your blog won't be indexed. See example of link of a blogger page: