Get URL headers without the HTML

A bit of a strange question. Is th开发者_如何学Cere a way to ask a webserver to return only the headers and not the HTML itself ?

I want to ask a server for a URL and see if its valid (not 404/500/etc) and follow the redirections (if present) but not get the actual HTML content.

Thanks

• Preferably a way to do this in Ruby

use HEAD instead of GET or POST

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html Section 9.4

As suggested, check the Net::HTTP library..

require 'net/http'
Net::HTTP.new('www.twitter.com').request_head('/').class

This is exactly what HEAD HTTP method does.

For Ruby, there is a beautiful gem, much simpler than the low-level net/http that allows you to perform HEAD requests.

gem install rest-open-uri

then

irb> require 'rubygems'
=> true
irb> require 'rest-open-uri'
=> true
irb> sio = open("http://stackoverflow.com", :method => :head)
=> #
irb> sio.meta
=> {"expires"=>"Tue, 30 Nov 2010 18:08:47 GMT", "last-modified"=>"Tue, 30 Nov 2010 18:07:47 GMT", "content-type"=>"text/html; charset=utf-8", "date"=>"Tue, 30 Nov 2010 18:08:27 GMT", "content-length"=>"193779", "cache-control"=>"public, max-age=18", "vary"=>"*"}
irb> sio.status
=> ["200", "OK"]

It follows redirections. You have to rescue for SocketError when host doesn't exists or OpenURI::HTTPError if file doesn't exists.

If you want something more powerfull have a look at Mechanize or HTTParty.

Use Ruby's net/http and the HEAD method that Mak mentioned. Check ri Net::HTTP#head from the command line for info.

actually i had to fold pantulis' answer into my own. it seems like there are two kinds of urls neither fns worked alone so i did

module URI

  def self.online?(uri)
    URI.exists?(uri)
  end

  def self.exists?(uri)
    URI.exists_ver1?(uri)
  end

  def self.exists_ver1?(url)
    @url = url
    ["http://", "https://"].each do |prefix|
      url = url.gsub(prefix, "")
    end

    begin
      code = Net::HTTP.new(url).request_head('/').code
      [2,3].include?(code.to_i/100)
    rescue
      URI.exists_ver2?(@url)
    end
  end


  def self.exists_ver2?(url)
    url = "http://#{url}" if URI.parse(url).scheme.nil?
    return false unless URI.is_a?(url)
    uri = URI(url)
    begin
      request = Net::HTTP.new uri.host
      response= request.request_head uri.path
      #http status code 200s and 300s are ok, everything else is an error
      [2,3].include? response.code.to_i/100
    rescue
      false
    end
  end
end