Remove page from Google Listings using robots.txt file
Is this the correct way to do this - below is my txt file, would this prevent Google from indexing my admin d开发者_JS百科irectory as well as oldpage.php?
User-agent: *
Allow: /
Disallow: /admin/
Disallow: http://www.mysite.com/oldpage.php
Yes you are absolutely correct except single file restriction.
User-agent: * : means for all crawler
Allow: / : allow access of full site
Disallow: /admin/ : restrict to admin directory
Disallow: /oldpage.php : restrict to oldpage.php
加载中,请稍侯......
精彩评论