How make support cyrillic letters in JavaCC?
Can I make cyrillic tokens for *.jjt file?
For exampe tokens are given in jjt:
TOKEN : /* LITERALS */
{
< TEST: "тест" >
| < DEVELOP: "разработка" >
}
but the tokens in jj file look terrible:
TOKEN : /* LITERALS */
{
< TEST: "\u0421\u201a\u0420\u00b5\u0421\u0403\u0421\u201a" >
| < DEVELOP: "\u0421\u0402\u0420\u00b0\u0420\u00b7\u0421\u0402\u0420\u00b0\u0420\u00b1\u0420\u0455\u0421\u201a\u0420\u0454\u0420\u00b0" >
}
Use the following options:
options {
JDK_VERSION = "1.开发者_如何学C6";
UNICODE_INPUT = true;
JAVA_UNICODE_ESCAPE = false;
TRACK_TOKENS=true;
STATIC=false;
}
If I do Unicode with the help of her class:
class GetUnicode {
public static void main(String[] args) {
if (args.length < 1) return;
for (String input: args) {
for (int index = 0; index < input.length(); ++index) {
final char c = input.charAt(index);
final String s = String.format ("\\u%04x", (int)c);
System.out.print(s);
}
System.out.println();
}
}
}
After the start of the program:
$ java GetUnicode тест разработка
\u0442\u0435\u0441\u0442
\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430
As a result, I can see that the unicode strings do not match. For example:
"\u0442\u0435\u0441\u0442" != "\u0421\u201a\u0420\u00b5\u0421\u0403\u0421\u201a"
Do you have ideas why this happens?
PS: This bug occurs only under Windows OS.
The .jj file has been generated with the cyrillic characters escaped (Unicode). This is a good thing since they can then not be misread by using the wrong encoding. You should not worry about it; the token manager will work just fine.
精彩评论