Add Custom robots.txt on blogger by labnole.com

Add Custom robots.txt on blogger

Today we will discuss Add Custom robots.txt on blogger. Blogger is known as "custom robots.txt", which means that you can now customize a file according to your preferences on the Blogger website.

Today we will discuss the benefits of custom robots.txt. So let's start the guide.

What is Custom robots.txt?

Search results on this website or blogger indicate how to index your blog and crawl the web crawler Custom robots.txt, This means that you can restrict web pages to your blog from the crawler website so that your block level page can point to search engines that are not important to index your demo page or pages.
Add Custom robots.txt on blogger
Which browser always scans the robots.txt file of Crawl before Crawler the web page and then shows results in search engines.

In this Custom robots.txt, you can ban any post search prohibition, special posts, prohibited pages.


Custom robots.txt Code:


User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap: http://your domain.com/feeds/posts/default?orderby=UPDATED
Past your domain name.
Explanation Custom robots.txt


User-agent: Mediapartners-Google
This code is for Google AdSense robots which helps in providing good advertising on the blog.



User-agent: *

( * ) With this star mark, our blog level links are limited to indexing search crawlers, which means that the web browser will not index our level links due to our code below. It is a very nice benefit.



Disallow: /search

The name of the connection that means connections will be ignored after the domain name. Take the example below to see the page link.
http://www.your domain.com/search/label/SEO



Allow: /

Our web page crawlers can crawl and index our blog's homepage.



Sitemap: http://your domain.com/feeds/posts/default?orderby=UPDATED
This is a custom sitemap URL. I would always say use custom sitemap. Because by default the default sitemap is 25 post indexes.

Example sitemap:-



Sitemap: http://your domain.com/atom.xml?redirect=false&start-index=1&max-results=500

Sitemap: http://your domain.com/atom.xml?redirect=false&start-index=500&max-results=1000


Custom robots.txt Explanation complete.

Set-up Custom robots.txt  to blogger

Click Settings >> Search Preferences ➪➪ Crawlers and indexing ➪➪ Custom robots.txt ➪➪ Edit ›› Yes.
Past all Custom robots.txt. Save Changes see Image.


Set-up Custom robots.txt  to blogger


If you like this tutorial,
Please subscribe to share and don't forget to comment.


Post a Comment

0 Comments