Wednesday, August 26, 2009

The JNI Debug Problem

One thing I noticed the other day was that I'm doing a lot of JNI coding lately. In our Wind River Installer, based on p2 BTW, I have native code for doing a few things like compression and getting at the Windows registry. The CDT has native code for doing advanced process management, and getting at the Windows registry. I've played with Android native development which is all based on JNI. And I was thinking of hooking up the ALSA library to do Audio management with my prototype "Eclipse OS". That's a lot of Java/C coding. And, unfortunately, all without a debug solution :(.

JNI debugging was one of the early goals of the CDT. Unfortunately, it has never really materialized, at least not generically in a way we could incorporate into Eclipse. And given that I see a strong symbiotic relationship between Java and C/C++, I am getting more motivated to tackle this problem and see if we couldn't come up with a solution that we can integrate into the JDT and CDT (or maybe just the CDT).

But the first thing you run into when looking at the code, where do you start? I'd like to be able to step into native methods, hit breakpoints in native code and see up the stack all the way into the Java stack. And really have an integrated debug experience where you don't have to do much of a paradigm shift when going between the two worlds. And given the way the Debug platform is structured, that should be possible.

So there's different ways to slice the cat (not that I slice cats, I like cats). You could add an extension point to be able to plug in native handling into the JDT debugger. I'm not sure what the JDT gang feel of that, or whether they've hopefully thought of that. Another solution would be to build a new Eclipse Java debugger component, but base it on CDT's Debug Services Framework. That may make a more natural solution, but I fear it would be a lot of work and it would take a lot of investment to reach parity with JDT's debug solution.

I'd like to hear what the community thinks. What is the right solution. Hopefully we can come together, find the development resources, and finally reach this Holy Grail for the CDT.

11 comments:

  1. Hey Doug, wasn't there a GSOC project awhile ago that attempted to tackle this JNI/JDT/CDT problem?

    http://eclipse-soc-mariot.blogspot.com/2007/08/jni-plug-in-in-action-screencasts-part.html

    I don't know if it got anywhere after the GSOC term was over :(

    ReplyDelete
  2. I forgot about that. I have no idea where the code went, or even who the mentor was. I don't think any of the CDT committers were involved.

    ReplyDelete
  3. Ah, yes. Just looked at the code and the screenshots. Essentially all it did was launch the Java and C debuggers at the same time. And there was some magic to manage the step into. But it was a very superficial implementation and missed a lot of the big issues, like how to prevent the JVM from timing out. When gdb breaks, it breaks the whole process and the JVM stops responding to the JDT side and it gets ugly. There are some ideas to take from it, though, but it's maybe 5% of what needs to be done.

    ReplyDelete
  4. Hi Doug,

    After lot of JNI headaches, I recently tried JNA https://jna.dev.java.net and I've been quite satisfied with it so far

    Frederic

    ReplyDelete
  5. There is a similar issue with scripting languages based on Java. A lot of them generate byte code on the fly which is impossible to debug (since there isn't a bytecode level debugger; only a Java source code level one).

    I've been thinking about this issue, too, but on a more general scale. In almost all projects, we mix more than a single language. We have at least XML, Java and SQL. Sometimes, it's HTML and JavaScript, too.

    I wish there was something as "stop when someone accesses this XML element". Or "stop when the DB starts to send result and show them to me in a view before the program gets them".

    A. Digulla

    ReplyDelete
  6. Great Point, A. Digulla. I thought of this too in the context of GWT which has "native" JavaScript calls. It would be awesome to be able to debug into them as well.

    I'll definitely keep all multi-language scenarios in mind in this journey.

    ReplyDelete
  7. Great that you're interested in this. For the JS support in e4 we're beginning to think about some of these issues as we create a debugger for use with Rhino. We end up with a similar mixed stack to the JNI world where we have both Java and JavaScript present and it certainly would be nice to sort things out in a somewhat coherent way.

    The SOC work sounds interesting and similar as the current thinking is that we effectively launch two debug clients to connect to the debugged process and then merge things into one model. This stuff is still a ways away as we still need to get the vanilla JS debugger working but I think eventually we'll produce stuff or at least learn lessons valuable in sorting out multi-language scenarios.

    ReplyDelete
  8. Cool. Thanks Simon, that's good to hear. That's really confirming that we need this for more than just JNI.

    In the CDT, the Debug Services Framework, DSF, is a very flexible debug framework that should be able to handle these kind of things. Despite being in CDT, it is a general framework that we'd love to eventually get into the Debug Platform. Maybe this is the opportunity to start down that path.

    ReplyDelete
  9. Hi Doug,

    How's it going? We're also looking at this problem to be able to debug Java/Ada/C multilanguage programs. Please keep me in mind if something moves on this front.

    Thanks,

    Tom

    ReplyDelete
  10. Hey Tom! Will do. The first step is making the JDT debug model more flexible. Then we need to figure out how to get the debuggers all playing nicely together.

    ReplyDelete
  11. I know this is an old post of yours, but i thought i leave a comment here. I just tried debugging JNI code today with CDT and eclipse and it actually works really well. The only problem is that you have to start two debuggers, a JDT debugger and a CDT debugger. I didn't find that too cumbersome, but maybe i'm just happy with a lot less than other people :). I wrote a small tutorial on the process at http://apistudios.com/hosted/marzec/badlogic/wordpress/?p=202, hope you don't mind me linking to it here. In case you do just remove the link, no harm done.

    ReplyDelete