What HTML parser to choose and why BeautifulSoup doesn't work?

I need to parse a HTML-page with windows-1251 charset (it's in russian).

The problem is that it is the web application and I have to use Python 2.4 without any opportunity to install modules on server. The only thing I tried to do was asking an administrator to install lxml module but nevertheless it wasn't built in the right way on 2.4 and attempt to import lxml.html fails.

Now I'm trying to select between BeautifulSoup and html5lib modules, but I didn't find any simple examples of using html5lib (I need just to extract some text from the certain div element with stripping all the other tags inside). In turn, BeautifulSoup returns an error 'junk characters in start tag: u'\u041f\u0440\u043e\u0434\u0430\u0436\u0430>' and any attempts to decode the source page from CP1251 to un开发者_高级运维icode or any other charset didn't make it.

What am i doing wrong? Or what parser I should use?

This question has a great answer on how to convert to UTF-8:

Python: how to convert from Windows 1251 to Unicode?

I love the BeautifulSoup library but I'm not familiar with odd character sets. If I were writing this I'd probably just try to sanitize the input to get me into unicode land where most libraries are able to play nicely.