开发者

What is called first - robots.txt or mod_rewrite in htaccess

I need some help. I'm not sure about the order on request for mod开发者_运维问答_rewrite and robots.txt.

Some urls belong to a rewrite rule:

/index.php?id=123 to /home

Other urls don't have a rewrite:

/index.php?id=444

I made this entry to my robots.txt:

User-agent: *
Disallow: /index.php?id

Will the site with /home be indexed by search engines?


The robots.txt file is interpreted by the client (spider), and they don't know what rewrites you have in your system. Thus, spiders would not fetch URLs from your site if they look like the pattern in robots.txt but would if they found the same content through /home.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜