开发者

php simple_html_dom not fetching content in proper order

I m back with another screen scraping problem. I m using simple_html_dom.php for screen scraping.I m pasting the code the,response text and 2nd response text. these responses are the result which simple_html_dom is providing me.it would look to u that response is ok,but its not. actually there will be a link..which my request is going and u can manually see what are the results(in ur browser) and what simple_html_dom is giving me.

So lets first see the code.

here se is search_engine ,$key is keyword(say digital signage)

function search_company_name( $com_id, $string, $c , $fetched )//----Function: search company name in resultant string
{
    foreach($com_id AS $company => $id)
    {
        if(preg_match('/www\.'.$company.'/', $string)>0)
        {     开发者_JAVA百科      
            return array('id' => $id,'rank' => $c+1,'fetched' => $fetched+1);
            break;            
        }        
    }    
    return NULL;   
}


if($se === 'google.com' || $se === 'google.co.in' || $se === 'google.de' || $se === 'google.nl'|| $se === 'google.co.uk')
{    
    $count = 0;       //---- Count the rank ----//
    //----LOOP FOR 2 PAGES OF 100 RESULTS----//
    for($num_of_results = 0; $num_of_results < 200; $num_of_results += 100)
    {
        var_dump('http://www.'.$se.'/search?q='.$key.'&start='.$num_of_results.'&num=100');// **check the link urself in ur browser**
        $html = file_get_html('http://www.'.$se.'/search?q='.$key.'&start='.$num_of_results.'&num=10');

        foreach($html->find('div[id=ires] ol li') as $row)
        {            
            $string = $row->plaintext;
            $temp = search_company_name($company_rank, $string, $count, $counter);            
            if(!is_null($temp))
            {
                reset($temp);
                array_push($finalData, $temp);
                foreach($company_rank as $company => $id)
                {
                    if($temp[id] == $id)
                    {
                        unset($company_rank[$company]);break;
                    }
                }
                $counter++;
            }
            unset($temp);
            if ($tot_company == $counter)
            {
                break;
            }
            $count++;
        }
        if($tot_company == $counter)
        {
            break;
        }
        $html->clear();
        sleep(3);
    }
}

code is fyn and works to.now pasting two responses.

<pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'http://www.google.co.in/search?q=Digital+Signage&amp;start=0&amp;num=10'</font> <i>(length=63)</i>
</pre><pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'Digital signage - Wikipedia, the free encyclopediaDigital signage is a form of electronic display that shows information, advertising and other messages. Digital signs (such as LCD, LED, plasma displays , ...Market and applications - Companies - Content - Technologyen.wikipedia.org/wiki/Digital_signage - Cached - Similar'</font> <i>(length=322)</i>
</pre><pre class='xdebug-var-dump' dir='ltr'><small>string</small> <font color='#cc0000'>'Digital Signage | KiosksDynamic, enterprise digital signage software and hardware solutions for simple and complex digital out-of-home (DOOH) installations scaled to  any sized ...www.digitalsignage.com/ - Cached - Similar'</font> <i>(length=222)</i>

as u can see there are two li response(check urself).these result are text of 1st li and 5li. so where 2-4 li hav gone??it is working correctly for 'digital signage software' in google. Plz help me.my project is near end and i got stucked in this silly prob.

second response may not lead me to post question.


Google returns differently structured results based on your user-agent. Find out your user-agent here, and then set your script to use the same.

Your IP address can determine which Google page is displayed, which may have different layouts.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜