How to add robots.txt File in Blogger Get Best Idea

In this Lesson we will know about Robots.txt file with clear idea and then we can use for our website.

Step-1: 

• Actually You have a Robots.txt file in your website already. Chack it first >> 
• go to >> https://type-here-your-website-name.blogspot.com//robots.txt 
• Such as my website https://bloggerdesk-online.blogspot.com//robots.txt
• You can view like this....

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap: https://bloggerdesk-online.blogspot.com/sitemap.xml

This is default Robots.txt file for blogger. Google automatically added this file for blogger. We can changed it any time.

Step-2: 

• User-agent: *  means this section applies to all robots ( google, bing, yahoo etc. search engine)
• "Disallow: /  means the robot (google, bing, yahoo etc. ) should not visit any  post or pages in your site. For this your website will not index and nobody find your website from google, bing, yahoo....
• Disallow: /search means https://bloggerdesk-online.blogspot.com/search link will not index in robot ( google, bing, yahoo etc.)
• Allow: / means the robot that it should visit any  post or pages in your site.
• Sitemap: https://bloggerdesk-online.blogspot.com/sitemap.xml  means this Robot.txt file used for https://bloggerdesk-online.blogspot.com this website.

Step-3:

• If you don’t want to submit any post in google/ any search engine , For this you can add a robots.txt file 
• such as: I do not want to submit my this post https://bloggerdesk-online.blogspot.com/2020/03/how-to-make-a-site.html
• So now I will use this below file...

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Disallow: /2020/03/how-to-make-a-site.html

Allow: /

Sitemap: https://bloggerdesk-online.blogspot.com/sitemap.xml

Suppose I do not want to submite 3 post and 2 page in google/ another search engine. So I will use this code....

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Disallow: /2020/03/how-to-make-a-site.html

Disallow: /2020/04/how-to-create-new-template.html

Disallow: /2019/03/site.html

Disallow: /p/how.html

Disallow: /p/about-this.html

Allow: /

Sitemap: https://bloggerdesk-online.blogspot.com/sitemap.xml

Step-4: 

• If you not want to index your website now, you can use this code....

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /

Sitemap: https://bloggerdesk-online.blogspot.com/sitemap.xml

• If you use this code, your website should not index in search engine. 
• Finally, I will suggest to you that do not use Robot.txt without necessary.

Level One Lessons

<<Prev ① ② ③ ④ ⑤ ⑥ ⑦ ⑧ ⑨ Next>>

Level-2 Developer (Lesson 10 to 20)

Level-3 advance (Lesson 10 to 20)

Level-4 Super advance (Lesson 10 to 20)