Why my python mechanize script cannot work for this specific site
I need to simulate the process of inputting an item name and clicking the search button on http://ccclub.cmbchina.com/ccclubnew/.
if I inspect directly in HTML, the search form is described with name "searchKey"
<span class="se开发者_开发知识库archinput">
<input type="text" name="searchKey" id="searchKey" maxlength="25">
</span>
here below is the script:
import mechanize
import cookielib
# Browser
br = mechanize.Browser()
# Cookie Jar
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
# Browser options
br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
# Follows refresh 0 but not hangs on refresh > 0
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
# Want debugging messages?
br.set_debug_http(True)
br.set_debug_redirects(True)
br.set_debug_responses(True)
br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
br.open("http://ccclub.cmbchina.com/ccclubnew/")
I get
br.select_form('searchKey')
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "build\bdist.win32\egg\mechanize\_mechanize.py", line 524, in select_form
raise FormNotFoundError("no form matching "+description)
FormNotFoundError: no form matching name 'searchKey'
and the br.forms() is empty.
my question is: why mechanize cannot select the form which exists in html? what's the possible solution to handle this?
thanks
The input
with name searchKey itself is no form. A form comes with the <form>
tag, but honestly, this searchbox appears not to be part of a form; you'll have to simulate setting the text of the input and pressing it.
How about using lxml or BeatifulSoup?
精彩评论