robots.txt and serving images from a separate domain
I'm trying to find out if a robots.txt file set on domain A will still apply to an image served from domain B, but actually displayed on domain A.
Example:
Somewhere on the page on www.example.com
, there is an:
<img src="http://img.example.com/images/myimage.jpg" />`
In www.example.com/robots.txt
, it says:
User-agent: *
Disallow: /images/
So when the spider hits www.example.com and sees the image being served from img.example.开发者_如何学Ccom, will it index it?
robots.txt only applies to the hostname it actually exists on.
精彩评论