开发者

Regex: how to get \b to work with utf-8 charset?

I'm trying to remove words shorter than 3 characters from string, and came up with this regex: !\b[\pL\pN]{1,2}\b!u which works in general, but breaks on most of polish characters (ąćęłńśżź), which are treated as non letter symbols and word boundaries (and splits words in the middle if offending char happen to be there).

Sample code to demonstrate this phenomenon:

$text = " ää äaa aaäaaa ââ âaa aaâaa ąą ąaa aaąaaa àà áá ââ ãã åå ää ææ ßß ćć èè êê ëë ęę éé ìì íí îî ïï öö òò ôô õõ øø óó ùù úú ûû üü ąą ćć ęę łł ńń óó śś żż źź ";
echo "<h1>input</h1><p>$text</p>";
$text = preg_replace('!\b[\pL\pN]{1,2}\b!u', 'X', $text);
echo "<h1>output</h1><p>$text</p>";

Gives output (note aaąaaa being split):

input
ää äaa aaäaaa ââ âa开发者_高级运维a aaâaa ąą ąaa aaąaaa àà áá ââ ãã åå ää ææ ßß ćć èè êê ëë ęę éé ìì íí îî ïï öö òò ôô õõ øø óó ùù úú ûû üü ąą ćć ęę łł ńń óó śś żż źź 

output
X äaa aaäaaa X âaa aaâaa ąą ąX XXaaa X X X X X X X X ćć X X X ęę X X X X X X X X X X X X X X X ąą ćć ęę łł ńń X śś żż źź
                                 ^split

Setting different locale with setlocale() didn't help.

Using PHP 5.3.0 on Windows.


You cannot make \b work. It only matches ASCII word boundaries. http://www.mail-archive.com/pcre-dev@exim.org/msg00110.html

But you could try to use negative assertions (?<!\pL) and (?!\pL) in place of the \b test.


Try:

'~(?<=[\PL\PN]|^)[\pL\pN]{1,2}(?=[\PL\PN]|$)~u'

This matches any letter or number, which is

  • preceded by something that isn't a letter or number or by the start of the string
  • followed by something that isn't a letter or number or by the end of the string


I am using following regex.

$regExp = '(?<=[\s\p{P}\p{S}\p{C}]|^)('.$search.')(?=[\s\p{P}\p{S}\p{C}]|$)';

On my experience simple excluding ([^\p{L}\p{N}]) will not work, because some languages (like hindi/devanagari or japanese) have hidden symbols (vovels symbols), that shows as part of previous or next symbol.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜