I am very excited about this, Red Hat is about the best company I can imagine working for. They stand for all the values that I stand for myself. Let me also document this important step in my life with a picture:

I took some free time to spend with my family for most of December, and now want to start through with cool new stuff. I am still not 100% sure which way to go. I am currently spending time to work on CacioWeb and want to turn it into a great product for LadyBug. We have some interested customers and could get some funding, so this is an obvious thing to do for the short term. We also have a bunch of other cool ideas in the pipeline. I am not sure if it can provide enough safe income for the long run to feed my family though, which is why I am also currently looking for an employment position in a nice company (if you happen to know a good one – here’s my CV – , let me know!).

To me that’s an exciting start into the new year. It will be interesting to see what will have happened one year from now

The first prototype

It all started when Mario, Robert and me were looking at JavaFX 2 for the first time a couple of weeks ago. We found that one of the big missing things is a Swing integration into JavaFX, to enable embedding/reusing work that has been done for Swing. Later that same day, Robert came to us and said, hey look what I’ve done, a SwingView to embed a Swing component into JavaFX. He showed us, and it worked indeed. It was a JButton in a JavaFX scene and when you pushed it, it changed it’s text. Very impressive! The way it worked was fairly simple but very smart at the same time: on the JavaFX view he registered a mouse clicked listener, and whenever that fires, he would call the button’s click() method and then paint the button to a BufferedImage, serialize it as PNG over ImageIO into a stream and read that stream from a JavaFX ImageView to show it on the JavaFX scene. Of course it was slow and inflexible (only handles specific events), but it was a good start and it got us working quickly.

Introducing the RepaintManager

One of the obvious shortcomings of the approach was that it only painted when the button got clicked. We needed to be more flexible to repaint more generic components whenever anything gets repainted (i.e. call to JComponent.repaint()). The only place in Swing that knows when to paint what is obviously the RepaintManager. So.. the first improvement was to install a custom RepaintManager, and keep track of dirty components by overriding addDirtyRegion() and then, when paintDirtyRegions() gets called, after all components have been painted by the superclass implementation, send the repainted components to JavaFX (using the same serialized BufferedImage approach as above). This is much more flexible and accurate than the original approach, it allowed us to implement full event handling (more on this in another post). It’s still not perfect though: whenever client code calls getGraphics() on the component to paint directly, it would blow up. On the way to implementing this RepaintManager solution, we needed to introduce a lot of infrastructure that is still around today (the RepaintManager is gone now as you will see in a bit), most importantly, we needed to provide the wrapped Swing component a Window and a WindowPeer, both of which we mocked up and have them delegate to JavaFX instead (stuff like getLocationOnScreen(), focus handling and other stuff) . In other words, the Swing component thinks it lives in its own window, while in reality the ProxyWindowPeer delegates most of its functionality to the JavaFX node that contains it (hence the name). We also got rid of the BufferedImage buffer that we introduced in the prototype, and now reuse the RepaintManager’s own backbuffer (otherwise we would get 3 layers of buffering).

Bye Bye RepaintManager

It soon became clear that the RepaintManager approach alone was not the ultimate solution. Most importantly, JFreeChart would eventually blow up since it calls getGraphics() on the Swing component directly to perform some sweet animation stuff. It was clear we need to support it. What we did was to implement a special Graphics2D subclass (ProxyGraphics2D) that wraps the BufferedImage buffer and delegates all drawing operations to the Graphics2D of that BufferedImage and copy that image to JavaFX from time to time. In that last bit lies the tricky part though. When to copy over to JavaFX? On Graphics2D.dispose() ? But what when code keeps the Graphics2D object and never disposes it? It still needs to paint something on screen. We could also trigger on certain events or on RepaintManager but none of this seems to be a fully working solution. What I ended up with was to paint whenever there was a couple of milliseconds (20-50 turned out to be good) of inactivity on the Graphics2D, or after it did not copy for too long, in case some code does a neverending drawing burst. This implementation fixes the NPE when code calls getGraphics(). And we soon realized that this obsoletes our own RepaintManager code above, since now all the painting can be done directly by the normal RepaintManager using the Graphics2D provided by the WindowPeer implementation (this also enables support for client side custom RepaintManagers). Also, after we have done that, we found that this new implementation failed on MacOS, and we tracked it down to the pointless use of SunGraphics2D in a couple of places in the Mac port where using plain Graphics2D is sufficient. Mario pushed a fix for this into the OpenJDK port.

Where we are now

The big remaining bottleneck now was the serialization of the BufferedImage over the stream. We contacted some folks at Oracle (Jonathan, Kevin, Richard: thanks a ton!) and they pointed us to some semi-internal API in JavaFX that can take a BufferedImage and turn it into a JavaFX Image. This solved the problem of serialization. We still have the problem that we always need to update the complete buffer, which takes too long for reasonable large Swing components. We hope to be able to resolve this as soon as the remaining parts of JavaFX get published. Also, Michael Paus pointed us to an interesting discussion that seems to indicate that the Oracle folks are working on a Canvas2D which provides Java2D access to a JavaFX node (you guys rock!), which is exactly what the SwingView needs for good performance.

In one of the next posts I will write something about the event handling, which is the other big/interesting part of the SwingView.

This has been the thing that brought me to GNU Classpath, back then I made GNU Classpath able to render this in Swing/Java2D

This component is slightly improved, it can renderer arbitrary tree structures and instead of displaying colorful rectangles, it can basically display any shape, control, image or whatever you can come up with as JavaFX node.

This will be included in ThingsFX shortly. I will also make a live demo of this, because a screenshot doesn’t really convey how cool the component is (dragging the tree around with the mouse pointer is sooo much fun ).

Update: Here’s the live demo.

Update: Here’s the source code.

You will need: Mercurial, Maven and JDK7

First clone the source code to your computer:

hg clone hg.openjdk.java.net/caciocavallo/ng/ cacio

Then build it using maven:

cd cacio
mvn clean install

Then in your project where you intend to use it, add this to the Maven dependencies:

    <dependency>
      <groupId>net.java.openjdk.cacio</groupId>
      <artifactId>cacio-tta</artifactId>
      <version>1.0-SNAPSHOT</version>
      <scope>test</scope>
    </dependency>

Finally, annotate your testcase with:

@RunWith(CacioTestRunner.class)
public class SimpleFESTTest {
...
}


Or if you used FEST’s GUITestRunner, you might want to use the equivalent:

@RunWith(CacioFESTRunner.class)
public class SimpleFESTTest {
...
}

Those steps should get you running with Cacio Testing. When you run a test with the above annotation, it should no more popup any windows or steal focus. I’d be happy if you could give it a try and report back how it goes, especially in case of problems.