how to detect WebPage charset,and get page content?
i开发者_JAVA技巧 use follows code to get page content:
URL url=new URL("http://www.google.com.hk/intl/zh-CN/privacy.html");
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openConnection().getInputStream()));
for(String line=reader.readLine();line!=null;line=reader.readLine()){
System.out.println(line);
}
reader.close();
page: http://www.google.com.hk/intl/zh-CN/privacy.html charset is "UTF-8",but my system default charset is "GBK",so, these code can't type right.
i know ,i can write a charsetname in InputStreamReader constructor:
new InputStreamReader(url.openConnection().getInputStream(),"UTF-8")
it's will be ok, but i want to know:
how to detect charset,and get page content? (not send two requests better)
any java library can do this? (get webpage content,and don't need set charsetname)
thanks for help :)
There is really no easy way to detect the proper charset. You can hope that the web page you are interested in declares the charset using a <meta charset="utf-8">
tag. When you detect that tag you could switch charset of your parsing.
There are also some libraries that make an effort to detect the charset, for example http://jchardet.sourceforge.net/.
精彩评论