Problem with curl, xpath query
I need some help with my xpath query. I can get this code to work with just about every site I need to scrape except this small part of a particular site... I just get a blank page... Does anyone have an idea on how I can do this better?
//
$target_url = "http://www.teambuy.ca/vancouver/";
$userAgent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)';
// make the cURL request to $target_url
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT,$userAgent);
curl_setopt($ch, CURLOPT_URL,$target_url);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$html= curl_exec($ch);
if (!$html) {
echo "<br />cURL error number:" .curl_errno($ch);
echo "<br />cURL error:" . curl_error($ch);
exit;
}
// parse the h开发者_高级运维tml into a DOMDocument
$dom = new DOMDocument();
@$dom->loadHTML($html);
// grab all the on the page
$xpath = new DOMXPath($dom);
$hrefs = $xpath->evaluate("/html/body/div[@id='pagewrap']/div[@id='content']/div[@id='bottomSection']/div[@id='bottomRight']/div[@id='sideDeal']/div[2]/div/a/center/span");
foreach ($hrefs as $e) {
$e->nodeValue;
}
$insert = $e->nodeValue;
echo "$insert";
--EDIT--
No luck... Fatal error: Call to a member function loadHTMLfile() on a non-object in ... Line 4 //
$xpath_query = $dom->loadHTMLfile("http://www.teambuy.ca/vancouver/");
$hrefs = $xpath_query->evaluate("/html/body/div[7]/div[4]/div[3]/div[2]/div[1]/div[2]/div/a/center/span");
foreach ($hrefs as $e) {
echo $e->nodeValue;
}
$insert = $e->nodeValue;
echo "$insert";
don't use cURL. just use
$dom->loadHTMLFile("http://www.teambuy.ca/calgary/");
don't use
$xpath = new DOMXPath($dom);
just use
$href = $dom->xpath($xpath_query);
I imagine your xpath query could be simplified as well...
also,
foreach ($hrefs as $e) {
$e->nodeValue;
}
does nothing. might want to try this instead.
foreach ($hrefs as $e) {
echo $e->nodeValue;
}
精彩评论