301 redirects to all except robots.txt

We have moved our website to a new domain and want all pages of the old site to get removed from search engines. It's the same site, same content, just a new domain, so search engines are taking time because of duplicate content (maybe). We have added .htaccess 301 from our old site to new site as:

redirect 301 / http://new-domain.com/

Now, to remove our old site from search engines, we changed our robots.txt on the old site as:

User-agent: *
Disallow: /

The problem is,开发者_C百科 the search engines are fetching robots.txt from the new-domain.com because of .htaccess 301 redirect.

How do I restrict 301 redirect for robots.txt?

Remove the Redirect directive and try this mod_rewrite rule:

RewriteEngine on
RewriteRule !^robots\.txt$ http://other.example.com%{REQUEST_URI} [L,R=301]

This will redirect any request except /robots.txt.

I was trying different configurations and spent hours and finally I found this code which is working well and hope to get the desired results Here is the code for disallowing search engines.

User-agent: *
Disallow: /