The TPTP gang were adjusting version info in an enhancement request I raised against their Automated GUI Recorder (AGR) and it reminded me I need to look into this issue again. I remember seeing a demo of the AGR back when I snuck into an Eclipse Architecture Council meeting a year or so ago. It looked pretty good and appeared to do what we've needed for the CDT since forever, i.e., something to automate testing the GUI.
The issue I have had with the AGR is that I want to integrate the test cases that AGR generates into our existing JUnit framework. Unfortunately, that was impossible, or more realistically, was a lot of work to do. And, also unfortunately, the word I hear through the grapevine is that the AGR really isn't staffed well enough to evolve it this significantly. It met the needs it was created for, i.e. test TPTP, and if it is to fulfill a larger role in Eclipse, it really needs more Eclipse developers contributing to it.
So if the need isn't there to trigger this investment, it got me wondering what people use to automate their GUI testing. Of course there are commercial products out there that do the trick and we've used those here at QNX. But for open source development, it would be nice if we had an even playing field so that anyone could contribute tests without having to purchase the "standard" tool, assuming we could even agree on which "standard" tool to use.
I heard of the Abbot GUI tester and the Costello GUI recorder, still the best names in open source IMHO, a number of years ago. While built for Swing/AWT, they started working on SWT support a long time ago but it has been slow going. I have heard of some people using it for their Eclipse testing, but I'd like to see an official release of it before committing to it. But it does what I want, though, i.e. integrate with normal JUnit.
So I'd like to hear people's opinions on this, especially on where they would like to see developers invest their time if they indeed wanted to contribute to an Eclipse open source GUI testing tool. I think the need is there, but that means "beans" if no one else does.
It's a case of where we need to really be eating our own dogfood if this is going to improve. If the Eclipse Releng team was convinced to use AGR for enhancing current tests, than maybe we would get more adoption of AGR (and community interest).
ReplyDeleteGreat point zx. The big question, though, is how to convince them to "feel the need" enough to do it...
ReplyDeleteIMHO, Dejan Glozic is the only person I know of in the Eclipse community who can make this automated GUI testing dream a reality.
ReplyDeleteTPTP's AGR was based on a framework he whipped together in a couple of days.
Imagine if he had a couple of weeks :)
Hi.
ReplyDeleteWhere I currently consult, we built our own framework for automating IE testing as JUnit tests. This is very successful and has a very nice API. It doesn't do recording because we would rather test drive the functionality.
When one of our apps was changed to a Swing GUI, we reworked the tool so that it could drive Swing as well (same API, which is nice).
I don't think it would take much to get it to provide a suitable adapter for Eclipse. We intend to open-source the IE and Swing versions, so if anyone is interested please let me know.
Can't get Google to display my public profile, so you can contact me at lance.walton at casualmiracles dot com
ReplyDeleteYou should take a look at our WindowTester tool. Currently part of RCP Developer, it is slated for standalone release at EclipseCon. WindowTester records UI actions to JUnit tests so they are easy to refactor and customize. The product supports SWT and Swing with the same API.
ReplyDeleteBEA used WindowTester to automate over 700 UI tests for their BEA Workshop product, and they are considering using it for their work on the WTP project. According to BEA, "Since we started using WindowTester, tests
that took 2–3 weeks to write previously can
now be done in 2–3 days" (they were using Abbot previously). Here's a link to the case study that resulted.
This comment has been removed by the author.
ReplyDeleteYou might be interested to know that GreenSocs is hosting quite a bit of eclipse activity for SystemC, and we hope to expand that... and the qemu link will (I hope) shortly be arriving too...
ReplyDeleteGet in touch if you'de like more info
ahh this has haunted me since the release of the AGR. We was quite fast in using this and i did a small trial sucessfully recorded AGR tests to test our internal clearquest schema with plugins from IBM. Allthough when i was trying to get this to run headless it all ended up in a mess.
ReplyDeleteSo we took a decision to drop the GUI testing for our internal plugins. I think one person in our team has gotten it to work, i am unsure if he actually hacked the AGR to get it to work or what. This has not been our top priority since we spent quite a lot of time when we tried this out and ran flat into the wall which was not so nice :-(
AGR is definitely on our wishlist. We made the decision to use it over some other tools some time ago, mainly because of it's cross platform support. (We need to test on more platforms than Win32 and linux!) The number of limitations it has is a shame, because if it is developed further it could be a great tool....
ReplyDelete