Shenandoah GC: Brooks pointers

I didn’t write about Shenandoah in a while. We first needed to clear up some issues around it, which is done now. The project is not dead yet, quite the contrary, we’re working feverishly. Just now, we are about concurrent evacuation to work 🙂

Last time I wrote about concurrent marking. Before I carry on, I want to introduce a new concept: Brooks pointers. The idea is that each object on the heap has one additional reference field. This field either points to the object itself, or, as soon as the object gets copied to a new location, to that new location. This will enable us to evacuate objects concurrently with mutator threads (how exactly this is done is a topic for a separate post).

One problem of course is that as soon as we have two copies of an object (one in from-space, one in to-space), we need to be careful to maintain consistency. This means that any changes to objects must happen in the to-space copy. This is achieved by putting barriers in various places (everything that writes to objects, or runtime code that uses objects) that resolve the object before writing into it or using it. If we don’t do that, we might end up with some threads using the old copy, and some threads using the new copy, which is, obviously, a problem. The barrier simply reads the brooks pointer field and returns the forwarded object to any code that uses it. In terms of machine code instructions, this means one additional read instruction for each write operation into the heap (and some read operations by the runtime). Infact, we currently need two instructions, the reason for which I’ll explain later.

Eventually, when evacuation is done, we need to somehow update all references to objects to point to the to-space locations. We do that by piggy-backing on the concurrent marking phase. When we traverse the heap for marking, we see all live objects and references, and whenever we visit an object, we update all its object references to point to the new locations of the referents.

There’s two tradeoffs with using brooks pointers: we need more heap space (ideally, one word per object), and we need more instructions to read and write objects.

Next time, I’ll start explaining how concurrent evacuation works.

Because there have been many requests: yes, Shenandoah will be open source. Our plan is to propose a JEP as soon as we can, and make it an OpenJDK project if possible.

Shenandoah GC: Concurrent parallel marking

Yesterday, I gave a quick overview over the Shenandoah GC. (And Christine also started blogging about Shenandoah today, giving even more background information.) I mentioned that Shenandoah collects garbage in two main phases: marking and evacuation. All of that needs to happen concurrently with the mutator threads.

Let’s look at the marking phase first. We kick it off by briefly stopping the world in order to scan the GC roots (java stack references and such). Then we resume the world and start traversing the heap going from the objects we just marked. Concurrently with the mutator.

The problem when marking objects on the heap while running concurrently with the application/mutator threads is, of course, that the structure of the heap changes while we’re traversing it. There are a few solutions to this problem, and we did not invent anything new here. We decided that we’d go with the so-called snapshot-at-the-beginning (SATB). The idea is that we want to mark all the objects live that are live at the beginning of the marking phase (hence the name). We probably mark a few more objects as live (those that become live during marking) but we don’t care about those (they don’t hurt). The reason we want to do this is the following: suppose we have an object A that points to B. During marking, A becomes unreachable. But there comes a new object C into live that receives a reference to B. Now, it is possible that the concurrent marking misses B because it can’t reach it anymore through A, and it doesn’t reach it yet through C. We’d basically treat B as garbage even though it isn’t.

How does SATB work? During marking, we employ a write barrier, that is an additional hook that gets called whenever mutator threads write reference fields in objects (not to be confused with memory fences, which are also sometimes called barriers…). This write barriers logs the original contents of modified reference fields into a linked list. After the concurrent marking phase is done, we stop the world, and process this SATB-list by marking all objects in it, and traversing their references where necessary. When this is done, we resume the world, and we can be sure that all objects that have been live at the beginning of the marking phase are marked as live. And since we’re also counting liveness during marking, we then know how much live data each region contains.

I also said that we’d do the marking work using parallel GC threads. How does that work? Instead of having one thread traversing the graph of live objects in the heap, we use many threads. Each thread maintains a work-stealing-queue. Each item in the queue is a heap object. Initially, the queues get filled with objects referenced by the GC roots. Each GC thread then pops an object from the queue, marks it, increase the containing region’s liveness counter, then checks all reference fields of that object, and pushes each object that is referenced back to the thread’s queue. This happens in a loop. Eventually a thread will run out of objects. When that happens, the GC work thread attempts to steal from another thread’s queue. When all threads are done, i.e. no thread can steal from any other thread anymore, we’re done.

After the marking phase is done, we know for each region how much live data (and therefore garbage) it contains, and all reachable objects are marked as such. All of the above is already implemented in Shenandoah and works well. Next step is the actual evacuation of regions that have accumulated enough garbage. This is a topic for a followup post. Or more likely, several follow-up posts.


Shenandoah GC: An overview

Let me give you a quick overview how the Shenandoah GC works.

First of all, let’s look at the heap in Shenandoah. Shenandoah’s heap is basically (like all heaps) a big chunk of memory, and in Shenandoah it is divided into many regions. Consider a heap of 1 GB, and a region size of 1MB, then you would get 1000 regions. This is different from the classic mark-and-sweep collectors, that have no regions at all, or the classic copying collectors that usually divide the heap in 2, or only a few regions. The reason for doing this is that we want to selectively evacuate those regions with the most garbage with higher priority (you know this idea from Hotspot’s G1 collector). Additionally, it makes multithreading a little bit easier (you can have multiple threads working on separate regions without stepping on each other’s toes… well, in theory 😉 ).

Now how does it work now you ask? Shenandoah is a mark-copy collector. That already implies the two important phases of Shenandoah’s collection cycle. In the first phase, we mark all the live objects in the heap. Considering the ‘copy’ in its classification, this may seem unintuitive, because normally copying collectors don’t need a marking phase, that’s their point (they collect garbage by copying live objects from one region to another, and leave the garbage behind). The main reason for having a marking phase is that we need to calculate the amount of live objects in each region. This is most naturlally done by running a marking pass over the heap. Starting from the GC roots (stack maps, internal data structures, etc) we follow the object graph and mark each object that we encounter, and update the containing region’s live counter accordingly.

The second important phase is the evacuation phase. Here we first select the regions that we want to evacuate (from-regions) and corresponding regions that the objects are copied to (to-regions). This is what we need the liveness information for. We then simply scan the from-regions and copy all objects that are marked to appropriate (e.g. with enough empty space) to-regions. (Note that this is different than classic copying collectors, that traverse the heap like we do in the marking phase. Linear scanning of the heap is much more efficient because of more predictable memory access, and as an additional bonus it preserves the allocation order, which is often a good thing.) After evacuation, the from-regions only contain garbage objects and we can free them up for future allocations.

So far it all sounds very simple and straightforward, and in fact is not even new. What we do, however, (or plan to do) is to make all of the marking and evacuation concurrent with mutator threads (i.e. mark and evacuate regions while application threads are running and messing with the heap), and parallel (i.e. do marking and evacuation using potentially many GC threads). And this is where the complication begins. I will go into the details of the issues and how Shenandoah addresses them in the next posts of this little series. Stay tuned!

Shenandoah: A pauseless GC for OpenJDK

Update: It seems like this post gets frequently linked to, but is fairly old. For current information on Shenandoah GC, please refer to the Shenandoah Wiki.

It’s been quite a while that I did not post anything. The reason is that I was busy with a very interesting new project in the last weeks, Shenandoah, a new GC for OpenJDK. Christine, a collegue of mine in Red Hat’s Java team, set out to build a next generation garbage collector for OpenJDK. I jumped into this project a while ago, and I’m very excited about it.

I guess I will not go into technical details in this post, but instead outline a few of the problems of existing garbage collectors, and Shenandoah’s goals and some highlevel ideas how to achieve them.

  • Pause times: this problem existed since the invention of GCs, and hasn’t been fully solved since then. The good old stop-the-world garbage collection is probably still in everybody Java programmers mind. Nowadays we have more modern garbage collectors, and most of them drastically reduce pause times, but nevertheless, pause times still exist, be it for evacuation of regions, for book keeping of references, etc. Shenandoah aims to reduce pause times even more. Realistically speaking, there will still be pause times, but we aim to minimize them as much as possible. Our goal is to reduce them to less than 10ms per GC cycle.
  • Scalability. The larger the heap gets, and the more processors/threads are working, the more garbage is produced. This means more work for the garbage collector to find and free all this garbage. In other words, the garbage collector needs to be able to keep up with ever-growing memory demands and processor capability. We are shooting for TB-sized heaps.

The idea now is that, in order to reduce pause times, the GC needs to do as much work as possible concurrently to mutator threads (mutators are all threads that modify the heap, i.e. the actual application running). In order to be scalable, the GC needs to be able to utilize that ever-growing processing power by doing as much work as possible using multiple threads in parallel. (Notice that the terminology is a bit confusing here. In the Java GC world, ‘concurrent’ often means ‘concurrently with mutators’ while ‘parallel’ means ‘using parallel worker threads to do GC work’).

While that sounds conceptually simple, I can tell you it’s not. Boy, are there a lot of hairy issues to resolve here. I think I will explain the ideas one blog post at a time here. Stay tuned!


The last couple of years, I had to pass FOSDEM, for various reasons. I am very very happy that I finally sorted things out for this years FOSDEM, and I’m ready to go (hotel and trains booked). I will do a presentation about my Shark & Zero work on Saturday afternoon, and co-present Thermostat with Mario on Sunday noon, both in the Java dev room.

FOSDEM is one of those rare events in my life about which I can say that it had a very significant impact. I wouldn’t be where I am today, if I hadn’t attended it a few years ago. I am very happy that I can finally go there again, and meet lots of old and new friends and collegues. See you there, and cheers!

Quick Cacio-Web howto

Today we released Caciocavallo 1.3. The release announcement can be found here.

However, what is the more important news is that after the release, I fixed Cacio-Web to work with the latest Cacio build and enabled it in the default build so it doesn’t fall to the wayside again. On popular request I would like to summarize how to get Cacio-Web running. Note that this currently only works on Linux (patches to enable this on other platforms are welcome!)

First of all, check out the source code (the cacio-web changes are not yet released):

hg clone caciocavallo

Then build it (you need Maven!):

cd caciocavallo
mvn clean install

And finally you should be able to run with with something like this:

java -Dcacio.web.port=9091 -cp cacio-web/target/cacio-web-1.4-SNAPSHOT-jar-with-dependencies.jar:/home/rkennke/src/test/SwingSet2.jar

The -Dcacio.web.port parameter specifies on which port should Cacio-Web listen. Notice that the classpath needs to include your application (SwingSet2.jar in this case).

Then point your browser to a URL like this:


Where the parameter cls specifies the (fully qualified) name of the main class of the application to start.

Please let me know if you run into any problems.

The return of the Shark, part II (howto)

So, finally, after some back and forth, the Shark fixes landed in hotspot-comp (thanks Twisti for reviewing and pushing me). It took a little while to sort out the new atomic operations in LLVM. If you want to play with it, you first need LLVM 3.2 (not the latest 3.1 release!):

svn co llvm-3.2
cd llvm-3.2
./configure && make && make install

Then you need to check out hotspot-comp:

cd ..
hg clone
cd hotspot-comp

Finally, I recommend you use my build script for Shark: place it in the toplevel directoy of hotspot-comp and modify all the env variables to your needs. Most importantly, change LLVM_CONFIG to point to your $LLVM_INSTALL_DIR/bin/llvm-config. Enjoy the Shark! 🙂

The return of the Shark

During the last couple of days, I’ve been working on fixing Shark for OpenJDK8. Shark is an awesome compiler backend for OpenJDK’s Hotspot JIT, originally written by Gary Benson. The idea is that, instead of generating target machine code dircetly, Shark would generate an intermediate representation (IR) for LLVM, and let LLVM’s JIT compiler generate the target machine code. The advantage being that it’s much easier to port to new platforms than writing your own C1 and/or C2 compiler for HotSpot. Unfortunately, Gary left Red Hat’s Java team a while ago which basically left Shark unmaintained. In the time since, several significant changes happened both in HotSpot and LLVM. After having fixed Zero for OpenJDK8 I thought I’d give Shark a try and spend a little time on it. The most problematic changes in LLVM or Hotspot have been:

  • Hotspot’s internal oop class hierarchy has been split into two: oops (all objects) and metadata (classes, methods, etc). This caused some major headaches because Shark code was basically assuming everything’s an object, and it was not always clear what actually to use: a Klass* or its Java mirror (basically an instance of java_lang_Class)? Etc etc.
  • A bunch of intrinsics for atomic operations (compare-swap, memory barrier, etc) have been remove from LLVM, and in their place now is a direct representation in the IR. Which actually makes it much nicer and easier to use in Shark.
  • Several smaller little things, to many and probably to little to list here.

With this, I can now do a full build of OpenJDK/Shark on x86, which I am quite proud of. It’s already successfully running a lot of code, but at the same time I am still hitting some bugs. After I have ironed them out I will propose the changes for inclusion in OpenJDK8 of course.


Hotspot Zero & OpenJDK8 & DaVinci

During the last couple of weeks, I brought the Zero (i.e. no-assembly) interpreter of OpenJDK up-to-date with OpenJDK8 and (most of) the DaVinci project. Zero is not usually built in standard and developer builds of OpenJDK, and thus often falls by the wayside when new features are developed in Hotspot. Various fixes needed to be made in order to be able to build and run Zero with the latest developments in OpenJDK8 and the DaVinci project again:

  • The Makefiles needed to be updated for copying the .debuginfo and/or .diz files into the right places.
  • Various small and fairly obvious changes needed to be made like renamed or extended method names. (Some of them are not only needed for Zero, even the X86 CPP interpreter was broken.)

With those changes in place (with help from Chris Phillips), it was possible to build Zero with OpenJDK8. The next big step for me was to make Zero build with the DaVinci patch set, in particular with the meth-lazy-7023639.patch. This turned out to be a fairly large refactoring. This patch introduced the so-called lambda-forms for method handles, which moves most of the invocation logic into Java code. The method handles now generate synthetic methods to implement the invocation logic on-the-fly and call that, instead of implementing all of it in the VM. The most important changes that I made was:

  • Rewrote the interpreter handler for the invokedynamic instruction. It now basically resolves the call site and the lambda form to call, and calls it.
  • Implement a handler for the new invokehandle instruction. This is a JVM-internal bytecode that is generated for calls to the various intrinsics (see below), which pushes an optional appendix parameter on top of the interpreter stack, which is then consumed by those intrinsics and is basically a pointer to the target method to call.
  • Implement 5 new intrinsics for MethodHandle: invokeBasic(), linkToStatic(), linkToSpecial(), linkToVirtual() and linkToInterface(). The first one is used to call into a lambda form using a polymorphic signature. The latter 4 call out to the target method and are basically signature-polymorphic versions of the various invoke* bytecodes.

With those changes in place I can now build a Zero version of OpenJDK8 with the DaVinci (MLVM) patches and all jtreg tests for java/lang/invoke are passing now. As a very welcome side effect, I was able to throw out a *lot* of convoluted code for invokedynamic support in the interpreter. The new code is fairly simple and straightforward (and thus, more maintainable) instead. The code is available from a personal repository for now (it’s a forest-clone of the upstream hotspot-comp forest, with MQ patch queues in hotspot and jdk modules). In order to check it out, do this (you need the mq extension):


hg clone
cd hotspot-comp-zero
cd hotspot
hg qpush -a
cd ../jdk
hg qpush -a
cd ..

Then build with Zero enabled, see my build script to get an overview of the build variables.

Update: If you’re only interested in the patches themselves, have a look at the patch repository.

Hacking Hotspot in Eclipse

I recently started working on Hotspot, with the goal of getting the Zero port up-to-date with respect to he recent developments in MLVM. In order to work most efficiently, I set up a work environment in Eclipse, and thought it’d be useful to share a little HOW-TO.

First of all, you need to install Eclipse/CDT if you haven’t already. This gives you a very powerful C/C++ development environment in Eclipse.

You can then open an OpenJDK source tree as C/C++ project by selecting: File -> New -> Other and then ‘C/C++’ -> ‘Makefile project with existing code’. Enter project name, e.g. OpenJDK8, and the path to the existing location. Select the appropriate ‘Toolchain for indexer Settings’ below. Click Finish.

Then open the project properties by right-clicking on the project, and selecting ‘Properties’. There we need to setup a couple of things.Under ‘C/C++ Build’ -> Environment, enter all environment variables that you would normally set on the command line for building OpenJDK. At the very least, you need ‘ALT_BOOTDIR’ and LANG=C. Under ‘C/C++ Build’, click the tab ‘Refresh Policy’ and remove the only path that is there (otherwise the whole workspace will be refreshed after a build, which takes looooong’). Optionally, add any paths under ‘build’ that you are interested in. Under ‘C/C++ General’ -> ‘Paths and Symbols’, select the tab ‘Source Location’ and remove the toplevel project path, and enter any source paths you are working with (e.g. hotspot/src). This limits what is visible to the indexer, etc. In order to take full advantage of Eclipse for debugging, I also changed ‘C/C++ Build’, ‘Behavior’ tab, replace ‘all’ with ‘debug_build’ This will normally do a debug build of OpenJDK, which means that you get all the symbols and no compiler optimizations in the binaries. In order to be able to load the symbols in gdb, add ‘ZIP_DEBUGINFO_FILES=0’ into the environment variables.Then click ‘Apply’ and ‘OK’ to close the settings dialog. Select ‘Project -> Build Project’ to launch the first build of OpenJDK in Eclipse.

Debugging with Eclipse is similarily straightforward, open Debug Configurations, add a new C/C++ application, set up its properties for the binary, arguments and environment variables (make sure you use a debug-build binary) and run the thing! Being able to fully debug in Eclipse, navigating stack, inspecting variables, setting breakpoints and stepping through the code is so much more useful than doing the same in plain GDB:

Debug Hotspot Zero with Eclipse