开发者

How to read only English characters

I am reading a file that sometimes has Chinese and c开发者_运维问答haracters of languages other than English.

How can I write a regex that only reads English words/letters?

Should it just be /^[a-zA-Z]+/?

If I do the above, then words like "eété" will still be picked but I don't want that:

"été".match(/^[a-zA-Z]+/) => #nil  good I didn't want that word

"eété".match(/^[a-zA-Z]+/) => #not nil  tricked into picking something I did not want


The only truly English letter that comes to mind is wynn ƿ.

One could make an argument for eth ð and thorn þ, but it would be a much weaker argument than can be made for wynn.

Other than those, English typically uses the Latin alphabet, albeit with certain modifications. Wynn possibly excepted, there is no such thing as an English letter, only a Latin letter.

There certainly exist regular expressions that demand that base characters be in the Latin or Common scripts, such as for example

(?:[\p{Script=Latin}\p{Script=Common}]\pM*+)+

but as you haven’t specified whether you’re using a 7- or 8-bit version of Ruby or a 21-bit version, I’m not sure what to tell you.


You need $, which means end of line:

/^[a-zA-Z]+$/

Or if you use such filtration:

strings.select { |s| /^[a-zA-Z]+$/ =~ s }
# which is equal to strings.grep /^[a-zA-Z]+$/

you can use negative filtration method with slightly simplier regex:

strings.reject { |s| /[^a-zA-Z]/ =~ s }

where [^a-zA-Z] means any non-english character.


Sometimes it's useful to use the Iconv library to deal with non-ASCII:

require 'iconv'

utf8_to_latin1 = Iconv.new("LATIN1//TRANSLIT//IGNORE", "UTF8") # !> encoding option isn't portable: TRANSLIT//IGNORE
utf8_to_ascii_translit = Iconv.new("ASCII//TRANSLIT", "UTF8") # !> encoding option isn't portable: TRANSLIT
utf8_to_ascii_ignore = Iconv.new("ASCII//IGNORE", "UTF8") # !> encoding option isn't portable: IGNORE

resume = "Résumé"
utf8_to_latin1.iconv(resume) # => "R\xE9sum\xE9"
utf8_to_ascii_translit.iconv(resume) # => "R'esum'e"
utf8_to_ascii_ignore.iconv(resume) # => "Rsum"

Notice that Ruby is warning that the option choices are not portable. That means there might be some damage to the string being processed; The "//TRANSLIT" and "//IGNORE" options can degrade the string but for our purpose it's OK.

James Gray wrote a nice article about Encoding Conversion With iconv, which is useful for understanding what Iconv can do, along with dealing with UTF-8 and Unicode characters.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜