Should sitemap be disallowed in robots.txt? and robot.txt itself? [closed]

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 10 years ago.

Improve this question

This a very basic question, but I can't find a direct answer anywhere online. When searching for my website on google, sitemap.xml and robots.txt are returned as search results (amongst more useful results). To prevent this should I add the following lines to robots.txt?:

Disallow: /sitemap.xml
Disallow: /robots.txt

开发者_StackOverflow

This won't stop search engines accessing the sitemap or robots file?

Also/Instead should I use google's URL removal tool?

---

you won't stop the crawler from indexing robots.txt because its a chicken and the egg situation, however, if you aren't specifying google and other search engines to look directly at the sitemap, you could lose some indexing weight from denying your sitemap.xml. Is there a particular reason why you would want to not have users be able to see the sitemap? I actually do this which is specific just for the google crawler:

 Allow: /
 # Sitemap
 Sitemap: http://www.mysite.com/sitemap.xml