Good whitelist for search terms

I'm implementing a sim开发者_运维百科ple search on a website, and right now I'm working on sanitizing the input. My plan is to make a whitelist of allowed characters. I'm using PHP, and so far I've got the current regex:

preg_replace('/[^a-z0-9 -]/i', '', $s);

So, I'm removing anything that's not alphanumeric or a space or a hyphen.

Is there a generally accepted whitelist for this sort of thing, or does it just depend on the application? I'm going to be searching on book titles, author names and book blurbs.

What about 2010 (A space odyssey)? What about Giscard d`Estaing's autobiography? ... This is really impossible to answer generally, it will depend on your application and data structures.

You want to look into the fulltext search functions of the database of your choice, or even specialized search appliances like Sphinx.

Clarify what engine you will use first to actually perform your search, and the rules on what you need to strip out will become much clearer.

Google has some pretty advanced rules for searches, but their basic rule is this:

Generally, punctuation is ignored, including @#$%^&*()=+[]\ and other special characters.

However, Google makes exceptions for common search terms, like C++, C#, or $100.

If you want a search as sophisticated as Google's, you can make rules against the above punctuation and have some exceptions. However, for a simple search, just ignore the characters that Google generally ignores.

There's not a generic regular expression to solve this problem. Your code strips out a lot of things you might want to keep, like commas, exclamation points, (semi-)colons, and non-English letters. If you have a full list of all of the titles in your database, you should be able to write a script that will construct a list of all characters found in all of your titles. If your regular expression strips out any of those characters, then you risk having problems (although passing this test doesn't mean that you won't run into problems).

Depending on how the rest of your search is implemented, you may be able to strip out valid characters and still return relevant search results. In this case, you would want your expression to allow non-English characters (since you don't want to split a word) but you might be able to remove all punctuation marks that aren't inside of a quote-delimited phrase. For example, searching for red haired should give you all of the results you would get from searching for red-haired plus a few extra.