php simple_html_dom not fetching content in proper order
I m back with another screen scraping problem. I m using simple_html_dom.php for screen scraping.I m pasting the code the,response text and 2nd response text. these responses are the result which simple_html_dom is providing me.it would look to u that response is ok,but its not. actually there will be a link..which my request is going and u can manually see what are the results(in ur browser) and what simple_html_dom is giving me.
So lets first see the code.
here se is search_engine ,$key is keyword(say digital signage)
function search_company_name( $com_id, $string, $c , $fetched )//----Function: search company name in resultant string
{
foreach($com_id AS $company => $id)
{
if(preg_match('/www\.'.$company.'/', $string)>0)
{ 开发者_JAVA百科
return array('id' => $id,'rank' => $c+1,'fetched' => $fetched+1);
break;
}
}
return NULL;
}
if($se === 'google.com' || $se === 'google.co.in' || $se === 'google.de' || $se === 'google.nl'|| $se === 'google.co.uk')
{
$count = 0; //---- Count the rank ----//
//----LOOP FOR 2 PAGES OF 100 RESULTS----//
for($num_of_results = 0; $num_of_results < 200; $num_of_results += 100)
{
var_dump('http://www.'.$se.'/search?q='.$key.'&start='.$num_of_results.'&num=100');// **check the link urself in ur browser**
$html = file_get_html('http://www.'.$se.'/search?q='.$key.'&start='.$num_of_results.'&num=10');
foreach($html->find('div[id=ires] ol li') as $row)
{
$string = $row->plaintext;
$temp = search_company_name($company_rank, $string, $count, $counter);
if(!is_null($temp))
{
reset($temp);
array_push($finalData, $temp);
foreach($company_rank as $company => $id)
{
if($temp[id] == $id)
{
unset($company_rank[$company]);break;
}
}
$counter++;
}
unset($temp);
if ($tot_company == $counter)
{
break;
}
$count++;
}
if($tot_company == $counter)
{
break;
}
$html->clear();
sleep(3);
}
}
code is fyn and works to.now pasting two responses.
<pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'http://www.google.co.in/search?q=Digital+Signage&start=0&num=10'</font> <i>(length=63)</i>
</pre><pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'Digital signage - Wikipedia, the free encyclopediaDigital signage is a form of electronic display that shows information, advertising and other messages. Digital signs (such as LCD, LED, plasma displays , ...Market and applications - Companies - Content - Technologyen.wikipedia.org/wiki/Digital_signage - Cached - Similar'</font> <i>(length=322)</i>
</pre><pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'Digital Signage | KiosksDynamic, enterprise digital signage software and hardware solutions for simple and complex digital out-of-home (DOOH) installations scaled to any sized ...www.digitalsignage.com/ - Cached - Similar'</font> <i>(length=222)</i>
as u can see there are two li response(check urself).these result are text of 1st li and 5li. so where 2-4 li hav gone??it is working correctly for 'digital signage software' in google. Plz help me.my project is near end and i got stucked in this silly prob.
second response may not lead me to post question.
Google returns differently structured results based on your user-agent. Find out your user-agent here, and then set your script to use the same.
Your IP address can determine which Google page is displayed, which may have different layouts.
精彩评论