Automatic form submission with PERL - debugging!

Our client has a simple setup.

Page A has a form that submits to page B which displays the query results. Unfortunately, there is no other API or DB access to get the data.

Since we need to do this query often, we decided to automate this submission with Perl.

I've determined the form key value pairs of Page A with a sniffer and replicated the code. However, on running the program page B is throwing a HTTP 500 error with no additional meaningful explanation.

Any pointers to debug this code? Code in itself is simple:

use strict;
  use w开发者_高级运维arnings;
  use LWP;
  my $browser = LWP::UserAgent->new;

  my $url = "targeturl.asp"
  my $response = $browser->post( $url,
    [
    "HisSort" => "1", 
    "RTsort" => "", 
    "chkHisRun" => "on", 
    "chkRTRun" => "on", 
    "optAdHoc" => "on", 
    "optHist" => "", 
    "optServer" => "servername", 
    "optStatus" => "", 
    "optWhat" => "H", 
    "txtEnd" => "", 
    "txtFields" => "1,0,10,17,11,18,24,19,21,25,1", 
    "txtHEnd" => "11/3/2010", 
    "txtHStart" => "11/1/2010", 
    "txtServer" => "", 
    "txtStart" => "",
    ]
  );

Note: I don't have access to the source of page A or page B

Firstly, I suggest looking at WWW::Mechanize which is a friendlier wrapper around LWP. Secondly, if your HTTP client is getting 500 errors, then there should be something more meaningful in the web server error logs. And finally, as Matthew has mentioned, you need to closely examine the request being sent by the browser and work out how it differs from the request that your Perl program is sending.

Team, This has been resolved. It eventually turned out that the problem was not with the headers but with the key value pairs I was sending. Page B wasn't doing validations on the fields and was plugging them into a query directly.

I had to try some brute force combinations (by testing with Page A) to get to what exactly page B was expecting.

Thanks to all who volunteered to help.