Pretty URLs Vs. Duplicate Content

I'm trying to clear up a grey area about this much talked about topic...

Like most devs, I've made some pretty URLs with mod_rewrite. My sites internal links point to the pretty URLs and things are working nicely.

But, I can still access the old URL if I point to it directly.

Now, this is most certainly going to cause duplicate content issues so after doing some research it seems that 301 redirects are the way to go.

But.... and here's the grey bit...

If you are working on a site with thousands of URLs, what's best practice to achieve this? I don't wantto list 1k+ lines in .htaccess I thought of a regexp in my rewrite rule, but my pretty URLs have names from the database in them... and I can't access that from .htaccess :)

Have I hit a dead end? Is th开发者_运维问答ere a way around this? Would Google's canonical tag be a possibility??

Well, I don't know if this is the "definitive" answer, but I have a bunch of "functional" URLS like:

http://www.flipscript.com/product.aspx?cid=7&pid=42&ds=asdjlf8i7sdfkhsjfd978

but I remap the URLs, link to them and list them in my site map as:

http://www.flipscript.com/ambigram-ring.aspx

I haven't seen ANY evidence that identical URLS pointing to the same content within the same domain has any negative impact on SEO.

In fact, over the past year, I have climbed to the #1 position on Google with this in place for my primary keyword.

My theory about why this should be so is that Google applies the duplicate content penalty for entire "clone sites", not for just linking with different URLs to the same content within a single site.

A quick dirty way would be to re-route everything on the site via a PHP file that checks to see if the path is still valid, querying the database if necessary. Use a 301 redirect if the path has permanently moved. Soon enough these "grey urls" should hardly ever come across, and indexes should be updated across search engines. At which point you can remove the router.

If you could specify what your "grey url" looks like I may be able to suggest a better alternative.

"Would Google's canonical tag be a possibility??" -- Why not?

--> It automatically transfers page rank

--> Google recommends canonical tag even if the content differs slightly but is more or less similar.

--> Too many 301 redirects to pages within site are bad for SEO (my personal experience with Bing).

--> Too may 301 redirects increase the effective load time of content for your users (especially bad if the ping times from their location to your server is high).