what is robots.txt || How to add robot.txt on blogger

Today I am going to tell you about What is robots.text file if you  want to know about What is robots.text file , then read this article, Robots.txt File is very important for your blog website. The robot.txt file is like a robot that follows our commands and works, similarly the robot.txt file we give commands sends commands to the search engine and whatever search engine is updated in your blog. Indexes it on its server and accesses the logo. Robots.txt file is directly connected to indexing and crawl rate of your blog, so let's know what is Robot.txt file and how to use it in your blog.

What is robots.text file

 

What is robots.text file

Robots.txt File‌ A file that we add to our blog, we can hide its help from search engines like google, yahoo, bing. Whatever command we write under Robot.txt file, after that whenever a search engine comes to your blog to index the post, then a command will be given to you to index this blog website and all these things. Do not index The Robots.txt File में is in every post and page of yours and the Robots.txt File‌ which is the private part of your blog is hidden from search engines like Google, Yahoo, Bing and tells all these search engines which part index Please do not.

Why Robots.txt File is Important

If you do not add a Robots.txt file to your website, then whenever the Search Engine comes to your blog for indexing, it will index all your content and page and it also does not want to show what you want in Google Search. Will start showing but

if you put Robots.txt file in your website, whenever Search Engine comes to your website, then it will start indexing by following the text written in your Robots.txt file and what you do not want to show. Will not appear in Google Search.

 

What are the benefits of using a robots.txt file?

1. You can tell Search Engine by this that they index all these things from your website and do not index all these things
2. You can make your website's Duplicate File and Page noindex.
3. You can make Noindex the Category, Author Page and Comments by this.
4. You can make Noindex the video or image given in your website by this.
5. You can index your post in Search Engine soon by adding Sitemap to it.
6. You can secure your website
7. You can also SEO your website by blocking low quality post through this.

 

How to create robots.trust file

 

By the way, many websites that create Robots.txt file are available on the Internet, but you do not need to go to the internet to create a Robots.txt file and search for a website that creates robots.txt because today you have that robots.txt file here I am going to give the code that Blogger has mentioned and you can see it by going to Blogger help forum. If you want to see this, click here for it. 

You first understand the code of the robot.txt file given below.

User-agent: Mediapartners-Google
 Disallow:
 User-agent:
 Disallow: / search
 Allow: /
 Sitemap: https://saccessindia.blogspot.com/ sitemap.xml

1. User - agent: Mediapartners - 

Google: With this help you can hide the Google AdSense code from the user and if you do not use Google AdSense then this code is not useful for you. 

2. User - agent:   

By using this, you can show what you want to show to the user in your website and you can not show what you do not want to show, you should always keep this disallow because it is very much for the security of your website. Is important

3. Disallow: / search: 

If you do not want to show the keyword of your website, then you should always keep it Disallow and you can disallow any post that you do not want to show in Google by pasting the URL here.

4. Allow: 

 Here you can show what you want to show to the user and you should always allow it

5. Sitemap:  

Sitemap is very important for your blog because Sitemap gives information about the new post of your blog to Search Engine and Sitemap works more than this, like which post is popular, how much URL is in the post etc.