web server returns "500 Internal Server Error" after sending this FormRequest using Scrapy
I construct the following FormRequest ac开发者_C百科cording to httpFox(Firefox addon)'s content. However, web server alway returns "500 Internal Server Error".
Could someone help me on this?
The original url is: http://www.intel.com/jobs/jobsearch/index_ne.htm?Location=200000008
Here is my spider's skeleton:
class IntelSpider(BaseSpider):
name = "intel.com"
allowed_domains = ["taleo.net"]
def start_requests(self):
req_china = FormRequest("https://intel.taleo.net/careersection/10020/moresearch.ajax",
formdata={
'iframemode': '1',
'ftlpageid': 'reqListAdvancedPage',
'ftlinterfaceid': 'advancedSearchFooterInterface',
'ftlcompid': 'SEARCH',
... # commentsThere are a lots of data here.#
'location1L2': '-1',
'dropListSize': '25',
'dropSortBy': '10'},
callback=self.test)
return [req_china]
def test(self, response):
print response.body
return
Your problem is from intel webpage, not from scrapy. but... Forms usually have some hidden fields,the best way to make POST request is like this:
def start_requests(self,response):
req_china = FormRequest.from_response(response=response,
formdata={
'iframemode': '1',
'ftlpageid': 'reqListAdvancedPage',
'ftlinterfaceid': 'advancedSearchFooterInterface',
'ftlcompid': 'SEARCH',
... # commentsThere are a lots of data here.#
'location1L2': '-1',
'dropListSize': '25',
'dropSortBy': '10'},
callback=self.test)
精彩评论