Split only text on special characters, but not on numbers with decimal points

I'm thinking about a special regexp problem in PHP, but I can't find an solution. I try to split some text into terms to get simple words, numbers and web addresses.

So i decided to split on every non alphanumeric character ( \w ).

To work with different languages, I use \w with additional letters, like Ää éèÈ and so on.

Example:

20,000 15.20 This is at Text. Right?!
www.google.com Jean Béraud

Until now, i use the following regexp to split the text:

[^\w(äÄüÜöÖßèé)]

http://regexr.com?2uq98

Which works well in 80% of cases, but splits 20,000 into 20 and 00开发者_StackOverflow中文版0 also www.google.com into www google com

So i tried to hold the numbers together, but still split on points, like Text. to get Text

To match 15.20, the following works: (\d+\.\d+), but how do I combine the negation with the other regexp string? The following will not work: (\d+\.\d+)|[^\w(äÄüÜöÖßèé)]?

And: how do I handle the web address?

Something like this?

$result = preg_split('/\W*\s+/', $string, NULL, PREG_SPLIT_NO_EMPTY);

Demo, Result:

array(10) {
  [0]=>
  string(6) "20,000"
  [1]=>
  string(5) "15.20"
  [2]=>
  string(4) "This"
  [3]=>
  string(2) "is"
  [4]=>
  string(2) "at"
  [5]=>
  string(4) "Text"
  [6]=>
  string(5) "Right"
  [7]=>
  string(14) "www.google.com"
  [8]=>
  string(4) "Jean"
  [9]=>
  string(7) "Béraud"
}

Q: Why does \w matches é in my example?

A: That's based on the local of the system the PCRE library is used on, from the PHP Manual:

A "word" character is any letter or digit or the underscore character, that is, any character which can be part of a Perl "word". The definition of letters and digits is controlled by PCRE's character tables, and may vary if locale-specific matching is taking place. For example, in the "fr" (French) locale, some character codes greater than 128 are used for accented letters, and these are matched by \w.

Alternatively it might be helpful to specify the regex as working with UTF-8:

$result = preg_split('/\W*\s+/u', $string, NULL, PREG_SPLIT_NO_EMPTY);

Ensure $string is UTF-8 encoded. As UTF-8 is international, specific locale settings might not needed to be taken into account. Give it a try.

Based upon the answer from @hakre and your example, I believe you should combine the two expressions in this way:

$result = preg_split('/[^\wäÄüÜöÖßèé]*\s+[^\wäÄüÜöÖßèé]*/', $string, NULL, PREG_SPLIT_NO_EMPTY);

Your original expression successfully matched the appropriate characters (the [^\w(äÄüÜöÖßèé)] part).

The expression from @hakre added that there could be ANY NUMBER of these characters (even none of the characters) and should also be followed by one or more instances of whitespace (the \s+ part).

When you combine the two ideas, you get [^\wäÄüÜöÖßèé]*\s+[^\wäÄüÜöÖßèé]*, which basically translates to

any number of characters that are not in this character class, followed by at least one whitespace character, then possibly followed by any number of characters that are not in this character class

or

one or more whitespace characters, and any number of characters that are not in this character class, either before or after

As a test, I used this text sample (I assume you want to match French characters):

20,000 15.20 This is at Text. Right?!
www.google.com Comme son élève, où dois-je aller voir Jean Béraud?

and this regular expression: EDITED

/[^\wßàâäèéêëîïôöœùûüÿç]*(^|$|\s+)[^\wßàâäèéêëîïôöœùûüÿç]*/i

through the same site that @hakre used:

http://codepad.viper-7.com/MqztVt

END EDIT

This test showed that the expression will split even if the word begins or ends with one of the special characters. I added the /i switch at the end of the expression to make the match case-insensitive.