Is there a good tutorial for figuring out what a website is doing so your program can do the same thing? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 9 years ago.

Is there a good guide or tutorial for people who need to programmatically interact with dynamic websites? There's been a rash of Perl questions about that lately, and I haven't found a good resource to point people toward. I'm asking not because I ne开发者_运维技巧ed one but because I don't want to waste my time writing it if it already exists. Although I'm most interested in Perl, the extra tools and techniques are mostly the same.

Typically, I see see these problems in people's questions:

• Handling, setting, and saving cookies
• Finding and interacting with forms
• Handling JavaScript inside your user-agent
  • especially things like onLoad, onSumbit, and Ajax
• Using HTTP sniffer tools
• Using Web developer plugins in interactive browsers
• Interacting with DOM, screen scraping, etc.

If there's no good tutorial, I'll add it to my list of things to do (unless someone else wants to do it). Along the way, if you don't have a suggestion for an existing tutorial, please suggest the things that you think should be in a new one, including links, your favorite tools, and your own user-agent development experiences. I don't care about the particular language you use.

---

The best I've seen is a Defcon presentation video.

---

Look at perl library of libraries. Some html parsing libraries should be made for talking to dynamic websites. Like: http://metacpan.org/pod/HTML::DOM

But do you want to use web-browser enhanced by perl. Or perl stand alone app?