how to interact with website using HttpURLConnection?

Here is the story: There is a website with some form fields. After filling in the fields the user can click on search. By clicking the search button a list of links will be returned and shown underneath the form. If the number of links is more than 10 the user can click on next which is a javascript button.

A part of a java program of mine connects to this website using HttpUrlConnection and sends the form data through the established connection. Retrieving the first 10 links is not a problem; I mean the connection with website, sending the field data, and getting the first 10 links is not a problem at all. My problem is how to开发者_如何转开发 get the next 10 links.

Thanks for helping.

Take a look to HtmlUnit. This frameworks behaves like a browser and allows you to navigate programmatically between pages at your will. It executes Javascript embedded in pages, too.