Cacio for UI testing
October 17, 2011 5 Comments
I became quite a fan of FEST recently. It’s a very good framework for unit testing user interfaces written in Java/Swing. However, I always struggled to get it to run reliably on a continuous build/test server. As far as I could figure out, this is due to the following reasons:
- UI tests are inherently using the computer’s desktop. If more than two test processes run in parallel or if a user is interacting with the same desktop, things get messed up.
- Many UI stacks are behaving asynchronously. I.e. when calling drawLine() and the call returns, there is no guarantee that the line has actually been drawn, the drawing command can still be waiting in paint command queue.
On Unix, both problems can be addressed to a certain degree by using Xvfb and enabling some debug flags on the X server to perform synchronous roundtrip communication with the X server. I tried that, and it still did not give me 100% isolation of tests as I wanted. On a Windows server, this is a complete nightmare (as is to be expected).
Last week we implemented a solution based on Cacio. This is a Java graphics backend that let’s Java not draw on the computer’s screen at all, and instead render into an offscreen buffer. Mouse and keyboard events are driven solely by java.awt.Robot (which is exactly what you need for testing UIs). Each testing process can have it’s own virtual screen (or even multiple virtual screens for testing multiscreen applications). The size of the screen can be configured. Even screenshots can be taken (using java.awt.Robot). Using this framework, each unit test runs fully isolated from other tests and from user or system interaction, thus making them perfectly reliable. The implementation is done 100% in Java, and comes as a JAR file that you simply plug into your classpath as any other testing framework.
Mario and I are in the process of setting up our own little company. We will extend the website soon, and when we’re at it, setup a project website for Cacio and IcedRobot as well. The Cacio testing framework (or if you prefer more buzz-fluffy words, Cacio Testing Cloud 😉 ) will be our first supported product. So if you are interested in using it in production environment and need support for doing so, you are welcome to contact us.