Disallow dynamic htaccess rewritten url
How can i disallow in robots.txt indexing of pages
http://example.net/something,category1.php
http://开发者_开发技巧example.net/something,category2.php
(...)
http://example.net/something,category152.php
I have try
Disallow: /something,*.php
But it say, i can't use wildcard (*) here.
You can change link format and pack all Disallow
pages into one url directory or write program/script to generate in robots.txt
a Disallow
all posible your Disallow
link looks.
精彩评论