Any open source browser GUI automation tool that will support clicking specific icons wherever they may be on the page? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance. Closed 10 years ago.

On either the Windows or Linux platform, is there a GUI automation tool that will open up a browser page (any brows开发者_运维知识库er, I don't care) and then click on every instance of a specific icon image, wherever it may be on the page and however many times it exists.

I seem to recall this ability in IBM's Rational Robot. Is there an open source alternative for the penniless ( oh yah... and the morally superior :-) )

EDIT:

I'd like to be able to click a specific bitmap or rectangle of pixels, where ever it is on the page, even if it is in a larger image or if it appears in a flash window. Where ever the image exists on the page, and whatever name the container or image has, click it.

---

Please take a look at sikuli.org.

Also see: sikuli related questions on SO

It uses image-based screenshot recognition and has an integrated script editor that allows you to insert target button images directly inline with the script.

It runs on Windows and Mac OS, and will work with any applications (browsers included).

The scripting language is Jython.

The library is open source with MIT license. Actually, it is developed at MIT.

---

Have you taken a look at Watir? It's a browser-based web application automation framework written in Ruby.

---

If you know the image url, use this selenium selector:

"dom:var elist=document.getElementsByTagName('img');var toret = null;
for (var i=0; i<elist.length; i++) { if (elist[i].src=='something.png') {
toret = elist[i]; } }; toret"

http://seleniumhq.org/docs/04_selenese_commands.html#locating-by-dom

---

AutoIt is another great tool for automation. You could check the User Defined Function (UDF) for IE, which comes as a part of default package.

It provides a whole lot automation IE Management, Windows System level management, Sound Management etc.,

You could start by downloading the no-install package http://www.autoitscript.com/cgi-bin/getfile.pl?autoit3/autoit-v3-sfx.exe and look into the examples of IE Management UDF in the UDFs3.chm help file.

---

If your image is identifiable on the page by a CSS id, you should be able to accomplish this using cucumber/webrat and/or cucumber/selenium (the latter if you also want to test javascript in the page). Not a GUI tool but very powerful.

---

The answer is NO: there is not an open source tool that does this. That is because this is a bad idea. Commercial tools like Rational tried to do this because they tried to be all things to all people (even if they didn't work very well). But selecting by bitmap image is easy to think about and hard to perform reliably so none of the modern popular tools even try. Instead they focus on how to be SUCCESSFUL.

The right thing to do, as others have already said here is to select images by CSS selector or by image file name. This can be done reliably by any modern GUI test tool: Watir, Selenium, Webrat ...