PHP: fopen failed "HTTP Request Failed", but response header has a status code 200

I have a PHP script, that should connect to a proxy, chosen from a proxy list and download a file. Some of the proxies (out of 200-400 working ones) work perfectly, but others don't, and I cannot find out why.

Here the code that connects through the proxy:

    $proxy = determine_proxy ($proxyList);
 $proxyString = 'tcp://' . $proxy['ip'] . ':' . $proxy['port'];

 $userAgent = $userAgents [rand (0, $agentsCount - 1)];

 // set up our headers
 $hdrs = array( 'http' => array(
     'method' => "GET",
     'header'=> "Host: www.example.net\r\n" .
      // "User-A开发者_StackOverflow社区gent: $userAgent\r\n" .
      "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" .
      "Accept-Language: en-us,en;q=0.5\r\n" .
      "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n" . 
      "Keep-Alive: 115\r\n" .
      "Proxy-Connection: keep-alive\r\n" .
  "Referer: http://$url",     // Setting the http-referer
     'proxy' => "$proxyString",
     'request_fulluri' => true
     )
 );

 echo "Using proxy: "; print_r ($proxy); echo '<br>';

 $context = stream_context_create ($hdrs);  // set up the context

 $timeout = 3;
 $oldTimeout = ini_set('default_socket_timeout', $timeout);
 $oldAgent = ini_set ('user_agent', $userAgent);

 $fp = fopen ("http://www.example.net$file", 'r', false, $context); // open the file

 if (!$fp) {
  echo 'fopen failed! Skipping this proxy for now...<br>';
  print_r ($http_response_header); echo '<br />';
  unset ($http_response_header);
  flush(); @ob_flush();
  ini_set ('user_agent', $oldAgent);
  ini_set('default_socket_timeout', $oldTimeout);
  continue;
 }

 print_r ($http_response_header); echo '<br />';
 unset ($http_response_header);

The bizarre thing is that the response header for the failed tries is sometimes empty, and sometimes it's the following:

   Array (
 [0] => HTTP/1.0 200 OK
 [1] => Server: falcon
 [2] => Date: Sun, 16 Jan 2011 14:06:37 GMT
 [3] => Content-Type: application/x-bittorrent
 [4] => Cache-Control: must-revalidate, post-check=0, pre-check=0
 [5] => Content-Disposition: attachment; filename="example.torrent"
 [6] => Vary: Accept-Encoding,User-Agent
 [7] => Connection: close
)

And sometimes, it's this:

   Array ( 
 [0] => HTTP/1.0 200 OK 
 [1] => Server: falcon 
 [2] => Date: Sun, 16 Jan 2011 14:06:47 GMT 
 [3] => Content-Type: application/x-bittorrent 
 [4] => Cache-Control: must-revalidate, post-check=0, pre-check=0 
 [5] => Content-Disposition: attachment; filename="example2.torrent" 
 [6] => Vary: Accept-Encoding,User-Agent 
 [7] => X-Cache: MISS from proxy 
 [8] => Proxy-Connection: close
)

This is a response header from a successful attempt:

HTTP/1.0 200 OK
Server: falcon
Date: Fri, 21 Jan 2011 18:53:00 GMT
Content-Type: application/x-bittorrent
Cache-Control: must-revalidate, post-check=0, pre-check=0
Content-Disposition: attachment; filename="example3.torrent"
Vary: Accept-Encoding,User-Agent
X-Cache: MISS from www.example.com
X-Cache-Lookup: MISS from www.example.com:3128
Via: 1.0 www.example.com (squid/3.0.STABLE23-BZR)
Proxy-Connection: close

I am setting the user agent to be a valid user agent string, I have checked allow_url_fopen and it is set to On.

From RFC-2616, section 10:

200 OK

The request has succeeded. The information returned with the response is dependent on the method used in the request, for example:

GET an entity corresponding to the requested resource is sent in the response;

How is it possible, that the server via the proxy returns a status of 200, and still fopen fails? Does anybody have an idea about the problem and how to fix it?

The problem was the fact, that I was setting a socket timeout that in some cases was too low for fopen to manage and download all the data. After the timeout period has gone and fopen still hadn't downloaded the data, it returned FALSE and threw an "HTTP reqeust failed" error.

the server reported 200 OK , but the proxy still didn't know where to forward this data to , so you got a Request Failed ...

try using the VIA header