One issue we've always struggled with on the CDT is how to do GUI testing of the CDT's UI components. A number of us looked at different open-source solutions that would allow anyone contributing to the CDT to also contribute GUI tests. Some community members do use commercial tools for their internal testing. But we'd prefer something generally available for everyone.
Two solutions seemed to come out tops, albeit a very low tops. AGR from TPTP and the SWT port of Abbot. However each of these had two severe problems that prevented us from using them.
First, the Abbot/SWT port has been progressing very slowly to the point where we're not sure it's even alive. We do have one committer, though, that is looking at supplying them with patches and maybe that'll bring it back to life.
AGR also had one big stumbling block, it's dependence on the TPTP test framework. The CDT releng test run is a simple JUnit setup and we wanted to run our GUI tests as part of that without having to spend the effort of integrating the framework. Bug 138369 was raised to see if someone in the community could make AGR work in such a simple environment.
Well, today, we were notified that AGR has gone into an As-Is state (i.e. dead) and the bug was marked won't fix. I personally recommend against marking bugs in the CDT as won't fix, because, who's to say someone won't come along and try and fix it. Why shut the door like that. At any rate, it does point out that the community hasn't really rallied behind AGR as a GUI testing solution.
So what is the current state of open source GUI testing frameworks for Eclipse? Is there something else out there? Or is there still hope for either of these technologies? Is there an EclipseCon talk on this subject?
I didn't test these myself, but they were a topic of the last German Eclipse magazine:
ReplyDeletehttp://www.gerhardtinformatics.com/RCPFIT
http://www.gerhardtinformatics.com/RCPRobot
I have had a hard time myself finding good tools with which to test Eclipse apps. These two mentioned by bananeweizen look interesting, but I had trouble finding out much about them, like supported versions, license, etc.
ReplyDeleteInstantiations' commercial WindowTester (http://www.instantiations.com/windowtester/) product is free to Eclipse Committers or to anyone working on an open source project (http://www.instantiations.com/company/opensource.html). Eclipse member company BEA has over 800 WindowTester-based tests at last count (http://www.instantiations.com/PDFs/wt/BEA_CaseStudy.pdf).
ReplyDeleteWindowTester has very comprehensive SWT, JFace and RCP support, and we currently have GEF playback support in beta (demoed at EclipseWorld last month). Full GEF record and playback support will be available in Q1 of next year. Test are written (either by hand or by the WT recorder) in Java as JUnit tests.
I have heard good things about WindowTester. The show stopper for my situation was the licensing. The product I'm currently working on isn't open source. I have a requirement that our test suite should be able to be run by anyone in the company, although it will be used by on a few people 99% of the time. Last time I checked - a few months ago - it was prohibitively expensive to satisfy this requirement. Although, GEF support is very compelling, so I may pursue this again when it is supported. It seems as though WT is about the only game in town at this point.
ReplyDeleteAGR is dead for quite long time:(
ReplyDeleteAnd actually very few people use it.
The only one open source project use it i known is TPTP itself.But even tptp itself use it very little.(most of tptp's test are manul tptp tests.)
TPTP guys do have some plan to convent their maual tests into AGR tests.But very very slow. And i don't think they will finish it.
SWTBot(http://swtbot.sf.net) is upcoming for SWT UI Testing.
ReplyDeleteCheckout http://swtbot.org/ that I'm writing, while this is still in Beta, it looks promising (I'm writing it myself)
ReplyDeleteThere are some screencasts here:
http://swtbot.sourceforge.net/wiki/Screencasts
From the screencasts, WindowTester seems to do everything I want. But is a commercial product. While it is available for free to open source projects, I need to check the license closely.
ReplyDeleteswtbot, while showing promise is missing the one thing I would like to see, and that's a recorder. And it is pretty young.
At least there is choice out there.
The script for SWTBot is very simple to write. See:
ReplyDeletehttp://swtbot.svn.sourceforge.net/viewvc/swtbot/trunk/net.sf.swtbot.eclipse.spy/src/net/sf/swtbot/eclipse/spy/PlayBackThread.java?revision=142&view=markup
While SWTBot is still very young, I'd be more than happy to help out writing functional tests :)
We are planning to write RCP UI tests with GUIdancer. Maybe CDT is another good candidate.
ReplyDeleteI know commercial tools are not on your number one list, but we will
provide free licenses to Eclipse committers. Starting in January,
we will have one person to specify tests for the community.
Hopefully we can meet at the EclipseCon (maybe at the GUIdancer tutorial? ;)).
Alexandra
From David Orme's blog, "... Areas where we’ve had trouble is in getting the various #type() methods right. Currently, some versions of #type() work but some don’t, and ..."
ReplyDeleteAre there bug reports for this? Is anyone stuck on them?
Steve
i think i posted about this last year in your blog and the problems we faced then, we went to IBM rational and use their functional tester product for GUI testing now. Slowly we are getting more and more GUI tests.
ReplyDeleteUgh...why is it not apparent that this project is dead? This doc (http://www.eclipse.org/tptp/test/documents/userguides/Intro-Auto-GUI-4-4-0.html) should be removed or edited (see Future Goals) so that people looking for this functionality don't waste their time. What about other Eclipse "as-is" projects?
ReplyDelete