Fun with JTAG

As I mentioned previously, I am working on adding an officially supported CDT integration with gdb that can be used with JTAG hardware debugging devices. As a quick primer, JTAG devices allow you to have full control over the CPU and memory on a embedded computing board using a special connector that is now pretty much standard on all such boards. With debugging support, that means you can read and write memory and any memory mapped registers, read and write CPU registers, and set breakpoints. A lot of the JTAG vendors are starting to support integration of their devices with gdb as a front end to give developers a familiar interface, and for us on the CDT, allows us to leverage almost all of our existing gdb integration to provide an Eclipse UI interface.

JTAG debugging does have limitations. It's not overly fast, especially when compared to native debugging. Stepping through code takes around a second in the setup I have. And with most configurations, the JTAG debugger hardly works at all once virtual memory is turned on in the CPU, making process level debugging, as you normally do with OS's like Windows, Linux, QNX, etc, impossible. The biggest value of JTAG in the past has been for the initialization code that sets up the board and starts the operating system kernel. But that is starting to change though as JTAG debugger makers are figuring out how to do the virtual to physical and back translation and adding OS awareness in the debugger itself allowing for the full debug experience.

So I have the integration working now. With permission, I borrowed a lot of ideas from the Zylin Embedded CDT plug-in. Again, my hopes are to bring those guys and their customers on board to avoid the need for forking the CDT. It was pretty cool when I did my first debugger launch, and everything just worked. This is really the beauty of Eclipse and the CDT and the focus on extensibility, that makes adding new features a breeze.

Below is a picture of my set-up. I have a little TI OMAP board hooked up to a Abatron BDI2000 JTAG device hooked up to a network hub that eventually hooks up to my laptop. You can't see the screen, but trust me :), the CDT has reset the board, loaded in an image, started it up, and hit the breakpoint I had set. And you get all the CDT goodness like the variables, registers, and disassembly view. Tres cool!

My next step is to hook up qemu, the board emulator, with it's built in gdb remote stub, which works just like a JTAG device, to this whole thing so you can try it out without having to fork out money for real hardware...

F3, the CDT Wunder-Key

Unfortunately, I do most of my programming in Java (sorry Java-lovers, but I hate Java). But the JDT really makes it a breeze to write code for the CDT plug-ins. My favorite feature is F3, or Open Declaration. Whenever I'm investigating code, I like to go visit the implementation of some unknown method to see what it does. F3 lets me jump from class to class and get a quick overview of the system I'm trying to program against.

Well, I'm trying to do the same thing with the CDT. Before CDT 4, Open Declaration tended to be slow since it did a complete parse of the file you're viewing and all the files that it includes. With CDT 4, we're now only parsing the file and using the CDT index to get all the other declarations needed for that file.

As well, F3 tended to be hit and miss on whether it actually found anything. A lot of that had to do with the indexer's need for build information that is often hard to provide. Also a lot had to do with information that we hadn't collected yet, C++ template information for example.

With CDT 4, F3 promises to be a whole lot better. I'll be spending a bunch of my time as we start to wrap up CDT 4 development on making sure it finds as many definitions as it can so that it can be as useful to CDT users as it is for JDT users. The one I just added that made me think of blogging about it is #include statements. Wonder what's in that include file you #include'ing? Well move the cursor to the statement and hit F3. Bingo, there's the include file (at least as long as the index knows where it is). I look forward to adding more cool features like that.

Microsoft is making my Linux fonts ugly

I found this article mentioned on Slashdot. I've stated in the past that the main reason I don't like using Linux as my main development environment is that I find the fonts hard to read. My eyes are horrible, especially after long stints writing code. Windows for some reason just looks so much better, especially on LCD screens.

Well, the reason Windows looks better is their ClearType technology. After reading the article I tried turning it off, and sure enough, Windows sucked too.

Apparently the fuss over ClearType and FreeType, Linux's font technology, has to deal with patents that Microsoft holds on the techniques behind ClearType. With all the anti-Novell/Microsoft clauses in the GPLv3 dealing with patent protection and the essential prohibition on it, I've lost all hope. Despite what Richard Stallman may wish, Microsoft will likely never extend patent protection on ClearType to all of the Linux community which means they will have to pick the other route, i.e. to none of the community.

Which is really too bad. As much as I thank the FSF and GPL for giving us all those great GNU tools, I'm afraid that their conviction to ideals will also stunt the growth of open source, and especially Linux. The FSF may hate software patents, but they are a fact of life. And if the two worlds can't mix, then the poor user pays a price, one way or the other.

Oh, I hate Cygwin

I usually try not to blog while I'm angry. But I'm in the middle of trying to get the Firefox source set up in the CDT on my laptop, which, of course, is running Windows (XP mind you, still not brave enough to try Vista). And it has been a struggle.

First of all, it was a bit tricky to set up the build environment. I'm using Cygwin since Firefox would really rather be built on Linux, or in a convoluted environment that involves using Cygwin but the Microsoft tool chain, which the CDT doesn't really have support for yet. Luckily I found a web page that showed how to set it up to use the cygwin compilers. It's a little out of date but a few tweaks and I was able to get Firefox built.

Now I'm trying to get the build output into the CDT so that our cool Scanner Discovery feature can parse it and set up the include paths and symbols for the indexer. That's been tricky since the Firefox build wrapped all calls to gcc with a wrapper script which deals with converting paths, I guess. I've got that fixed, but now the source file paths used by Firefox uses the cygwin paths, i.e. /cygdrive/..., which our build output parser doesn't understand. So I'll have to introduce a new parser that does the cygpath conversion.

We've had a lot of bug reports lately on cygwin. Most of them have to do with the cygwin developers deciding not to support Windows path names any more, you know, the good ol' C:\blah. I guess understand their reasoning. Cygwin is meant to be a Linux emulation environment on Windows. I don't think that was their original intention since Cygwin actually predates all this Linux popularity, but that's what it has turned into (and even their web site now says so).

The issue for the CDT is that it isn't running under the cygwin environment. It can only deal with Windows paths. So whenever we see a cygwin path, we need to convert it. Not only that, when generating makefiles for cygwin make, we need to convert Windows paths to cygwin paths. Now, we can't do that for everything on Windows since tools like MinGW gcc does use Windows paths. It's pure evil (well maybe not that evil...)

So what this really means is that supporting Cygwin with the CDT is becoming a lot of work. This is one of the reasons I want to start promoting MinGW, a much more Windows friendly port of the gnu tool chain, as the gnu environment of choice on Windows. The problem, though, is that cygwin is very popular and easier to install and seems to have much more momentum than mingw. So we will need to continue to support both. But I'd sure like to see more of that momentum shift to MinGW. Which, in open source, that means I need to do more to help them.

Ada Project at Eclipse

In all the excitement of the conferences I've been to in the last couple of months, I've also been helping Aonix with their proposed Ada project at Eclipse, called ADT (of course). Ada, you say!?!?! Yes, Ada. Despite reports to the contrary, Ada is still being used in the industry, especially in the military and aerospace and especially in safety critical systems. And with the introduction of Ada 2005, the hope is to spread all that Ada goodness to the rest of the embedded industry.

So, why Ada? Funny enough, googling "Why Ada" returns a number of articles on that subject. This is something the Ada community has struggled with for quite a while. But from my point of view, it all comes down to writing safe code. Ada is a huge language and much of it has to do with making as hard as possible to write bad code. Code like "while (i++) *i = 42;" isn't possible. This is particularly of importance with embedded systems where flaws like this could be costly, very costly.

But Ada has also suffered from the stigma of bureaucracy. The fact that it was designed by committee for the U.S. Department of Defense (who has since dropped it as a requirement) doesn't help. Also the reference manual reads like a legal document, and there are few books out there to explain it to us regular Joe's.

From my perspective, though, I think the biggest problem with Ada has been the lack of an open source community around it. GNAT, the GNU Ada compiler has been around a while, but it isn't very straighforward to use with it's own build system. But gdb works fine with it for debug.

So, in the past, I've tried to drum up interest in Ada as an extension of the CDT at Eclipse. Aonix and AdaCore are two vendors that have done it commercially. I was thrilled when Aonix approached me about starting up such a project. And I really hope that other Ada tool vendors join in on the fun.

High quality open source solutions available to everyone really helps with growth at the grassroots level. Many universities and colleges are using the CDT for their coursework because it is freely available. The same could happen with the Eclipse ADT. But to ensure high quality, a community has to form around it to share in the work. Most Eclipse projects benefit from the spirit of "co-opetition", competitors working together for the common good of the market. This could grow the Ada market which would mean they could sell more professional tools and support. Such is the economic wheel that keeps Eclipse going and we'll see if the Ada market can do the same.

Final Thoughts on ESC

Doug Gaff raises a very good point about ESC. I think changes are in the wind. What they are, anyone and everyone can guess. There were certainly a lot of boards and devices on display, and all the big RTOS vendors were there. There were even a couple of tools vendors there. All the booths looked great and were staffed by good people. There was enough crazy stuff going on to build on the hype (get your picture taken with the Intel chopper, count the Marilyn Monroes - I still don't know what they were trying to sell other than awful blue bags). Being my first show, it all felt pretty cool, but pretty shallow.

The thing I found disappointing, though, was the profile and location of some of the companies I think are the innovators at the moment. Timesys was one. They were involved in the CDT in the early days. They've since changed their business model from a straight embedded Linux distribution vendor to a service organization for do-it-yourself'ers. I thought it was innovative, but their booth was tiny and really tucked away and hard to find. (Mind you not much smaller and tucked away than the Eclipse booth - just kidding Ian :)

The other was CodeSourcery. I ran across these guys on the Web when investigating gnu-based ARM toolchains to play with qemu. The more I ran into their name, the more interested in their business model I became. They contribute a lot to open source, especially with the gnu toolchain (gcc, gdb). I also see then contributing to qemu for the ARM and other target support. I don't know a lot about their internal structure, but they must be doing well. Their booth was still outside the main hall but I noticed every time I walked by they were talking to potential customers. Unfortunately, I didn't have time to drop by myself to find out more. On the surface, though, they really look like a great example of an "open source business". And, they call their developers "Sourcerers". How cool is that...

But as I've stated in the past. I really think the model for all embedded companies has to, and in a lot of cases already is moving to a more service oriented approach. As Doug Gaff mentioned, the problems embedded developers are trying to solve is getting harder and harder. It is just plain too hard and expensive to hire full-time experts in every area you need. We'll see over the years if this turns out to be true and we start finding those companies hidden outside the main hall inside with their own smiling Marilyn Monroes to scan your badge.

ESC is over, bring on gdb JTAG.

Well, my talk went pretty well. I think i had over 75 people there. I stopped counting at 60 and people were still filing in the door. You have to get your badge scanned to enter the room which took some time. But the audience was very attentive and had some great questions and feature ideas. Unfortunately I didn't capture them all, so if you were there and had an idea, please raise a bug for it :).

One thing a few people asked me about this week was the Zylin Embedded CDT plug-in that a few vendors are starting to use in their commercial products. It essentially supports debugging using JTAG devices that use gdb as their debug engine. Those who have been around for a while have heard of Øyvind from Zylin who has been trying to get his changes into the CDT proper. For various reasons, those changes have been rejected.

But I think it's time to take a serious look at this and see if we can address his issues while maintaining current functionality. Having forks of the CDT is bad for the community, and is really bad for those vendors that have to use them because it has the potential to limit further integration possibilities, especially if APIs are changed. The objective should be to make everyone happy. Sometimes that's not possible, but in this case I think it is. And I'm making it my mission for the rest of CDT 4 to do my best to make that happen.

Getting ready to go on stage

I got to the room early where I'm giving my presentation on the CDT for Embedded. It's going to be a busy day again so I'll just quickly talk about yesterday. One of the cool things I've found out about this area of San Jose is that there is free wireless everywhere as long as you don't mind a banner on top of every page. And it's a bit slow, but it's letting me do this, so I don't mind.

I had a couple of great conversations yesterday that will hopefully lead to CDT contributions. I'm still pleasantly surprised at all the people who drop by the booth who already use Eclipse or who know that they really need to be using it. That gives me a great feeling that the grassroots around Eclipse for embedded is already taking off. And I guess, when you look at the other choices, it probably should be of no surprise.

I also did a walk about yesterday trying to find booths that had Eclipse running. There were the regulars which had it front and center. And there were a few surprises. Ian found a few. And I was found a couple of more, including a surprise at the MIPS booth. This is happening to me quite often in the last year or so and it is really cool to see. It would be nice if these guys felt comfortable enough to let us know when they are using Eclipse. It is open source so they don't have to. But part of my job is to promote the CDT to get as many contributions as I can and I'd really like to show the momentum of the CDT in the industry. Which is pretty clear from my experience here at ESC, this it is growing quite well in the embedded space.

ESC Day 1

That was an interesting day. In case you missed it, I'm down at the Embedded Systems Conference in San Jose, California. Today, Doug Gaff, Ian Skerret and I met a number of press people. There's one who's already published the results of our meeting with him. It's interesting the diversity of knowledge these guys have about Eclipse. But these are the guys you need to talk to to get the message out and I think we did a good job of getting them all one equal footing.

I spent a few hours at the Eclipse booth. As I mentioned in my last entry, I was pretty disappointed in the size and the location of the booth. But I was pleasantly surprised at the number of people who walked by and said they were starting to use Eclipse. I think by my account, most people who did walk by had something to say, which was pretty cool.

A person from a trade journal from India stopped by and I had a nice talk with her. She mentioned that all of Bangalore was using the CDT. That's a lot of software developers! I also had a couple of people who were wondering about extending the CDT for the specific needs of their projects. And I had a guy walk by and gave us a woo-hoo. I guess he likes what we're doing.

Talking with Ian, it is clear we need to find a good way to get publicize everything and start building a community around embedded. Doug Gaff and the people participating in DSDP have done a great job at getting the big players in the market involved. But I'm a grassroots kind of guy and I think there is also opportunity to get the word out to the average Joe.

My idea is to revive a tutorial someone had written about using the CDT to build a tiny app for an ARM7 chip that had no operating system and just ran on the bare metal. I'm doing work in CDT 4 to support gdb debugging with JTAG so it's perfect timing. And this was affirmed by a couple of conversations I had with guys looking for exactly that. So I'll work with Ian to get a webinar or something together, so stay tuned...

Hello from ESC Silicon Valley

This is my first ESC (Embedded Systems Conference) since my days at ObjecTime, umpteen years ago and that was in Boston and was a smaller show. I just did a walk around to get myself oriented with the San Jose Convention Center and quickly went through the exhibit hall. It's quite a site. This is really my first "ultra-hype" trade show and the booths here are amazing. Some are set up as theatres with lots of chairs, some have multi-storing temporary structures (you won't catch me going up there!). It's pretty cool and I can't wait to see the action when it opens later today.

Now, I need to thank Ian Skerret for setting up the Eclipse booth and to those that helped set up meetings with members of the trade press. And no disrespect to the Eclipse Foundation, but the Eclipse booth is pretty tiny. We talked about this at EclipseCon in the Embedded panel and I again put forth the challenge...

Eclipse is an important force in the embedded industry. Unless my Google Alerts are making up stuff, more and more embedded vendors are taking advantage of the power and flexibility of Eclipse as the foundation of their tools offerings. But Eclipse is successful because of the vendors and developers that contribute to it. Eclipse had a huge marketing impact in the Enterprise market because of the huge vendors that promoted it.

We need to see the same thing in the embedded space. We need to take that ground swell in the grassroots and promote Eclipse as the phenomenon that it is. This will go a long way towards encouraging those that are using Eclipse to contribute back and help convince more vendors to join the community. Those of us who got in early, did so because we saw the vision of what Eclipse can do in the embedded space. It's time to start shouting about it!

Get your swim suit on

A colleague pointed this out to me and it really made my day. With the stress of getting CDT 4.0 M6 out the door while getting ready for a week at ESC, you need to have a laugh to help keep you going.

Here's what you do. Go to Google maps and click get directions. Start from somewhere in North America and end somewhere in Europe.

The directions will guide you to Long Wharf in Boston and then instruct you to "Swim across the Atlantic Ocean", which apparently is 5572 kilometers. You then get out of the water in La Havre, France, and continue on your way.

I love the sense of humour software developers have. I can imagine the design discussions around putting this feature in. I'm sure they're still laughing about it, just like we are now. Now, what can we do with the CDT, hmm...

GPLv3, freedom at what price?

I've got my fresh copy of the GPLv3 Third Discussion Draft Rationale and started to try to dissect it. It's a pretty intense legal document as it tries to build up a GPL that has no holes. I'll have to read it on the plane down to San Jose for next week's ESC to help me fall asleep.

But one thing that has struck me as I dig into it and read others' comments on it, is that the view of freedom put forth by Richard Stallman and the FSF may actually turn detrimental to the community instead of helping it.

There is no doubt that open source is successful today because of the GPL. The freedom for users to be able to download, possibly modify, build, and run their code is really what got a lot of developers involved in open source to begin with. And that momentum has grown and the qualify of the open source code has grown to become a serious force in the industry. But that all started in the day when most open source users were developers.

The issue I have is that the FSF does not differentiate between user and developer. But when it comes to critical systems, especially in the embedded space, I'm not sure users even want that freedom. While having GPL code in my TV is cool and I'm glad the manufacturer was able to take advantage of it, and theoretically ended up with good code cheaper than if they would have licensed something. And hopefully the developers contributed back to those open source projects to make them even better. But the last thing I want to do as a user is change that code, even though I know I could.

So while the GPLv3 tries everything in its power to ensure that the user can modify the code, my fear is that it will handcuff the developers working on this code. It's not free to them to make sure the user gets everything they need to do the modification. The manufactures can't charge for the code, but you can be sure device prices will go up if they need to ship an SDK and hardware with it.

GPL in embedded devices has always been a tricky subject, and GPLv3 seems to make it even trickier. I'm not sure the FSF is willing to listen to the concerns of embedded manufacturers, but I sure hope someone is making noise.

CDT 4, now 5 MB better

I was just checking last night's CDT build to make sure our source feature got generated correctly (it did, thanks Andrew!). One thing I noticed was that our run time downloads are now around 17 MBs. CDT 3.1.2 was around 12 MB. That's 5 MB more CDT! We haven't added any new plug-ins so it's all enhanced feature content and improved internal data structures that's contributing to it.

Our M6 is coming along nicely and should be released late Friday or over the weekend. It'll be a part of Europa M6 which I think is still a week or so away. The biggest thing you'll notice is a new New Project wizard that merges standard and managed make into a single experience. Standard make projects are now "Makefile" project types, something VS users will be familiar with, and allows us to associate tool chains with standard make projects to make it easier to set them up for indexing.

For our Windows users, you'll see a new MinGW tool chain which uses various tricks to find your MinGW installation so there is no more need to add this to your path. Also, it uses the CDT's internal builder to call the build tools directly so there is no need to install MSYS or Cygwin to get make and the other command line tools. This will feed into the EasyEclipse MinGW distribution that I have promised to deliver.

Aside from that, the index is more complete and has more information in it to drive all our parser based features. This also feeds into better performance for content assist and open declaration since we no longer do a full parse of the file and all it's includes. It's not often you get 'oo's and 'ah's over a content assist, but I did at our demo at EclipseCon.

I'll provide more details and a pointer to the New and Noteworthy on the CDT wiki once we wrap this thing up and get it out to you.

"Conflicts With"

So you know in Bugzilla you can mark a bug as a duplicate of another bug. I have no problem with people raising new bugs even if they end up being duplicates of another bug. That way we make sure we get them all.

But I've often run into cases where I'd like to mark that a bug "conflicts with" another bug. If you mark one of them FIXED, the other one automatically gets marked INVALID.

The tough part, though, is picking the one to fix...

Ever hear of EFS?

We're at a critical juncture in the evolution of the CDT. Everything has been going so good lately, with tons of indexer and build improvements, that I guess we were due for a bit of a crisis. The issue we are grappling with is how to use the CDT with projects that have source files sitting on a remote server, possibly across the Internet somewhere. It's a tough environment that will definitely stretch Eclipse and the CDT if we are ever successful in making it work well.

The solution I've been hoping to push is to use EFS to access those files and, in theory, the CDT should just work. What's EFS? Well, it's the Eclipse File System, which provides a layer of abstraction at the same level as java.io.File, but allows different implementations. Using the right IResource APIs and URIs instead of IPath, and everything should work. It seems designed to meet our needs.

I managed to build a FileSystem using FTPClient from Apache's common.net. My test is to run it against the ftp server on the CDT build machine at Eclipse. Saving files is a bit slow, and I guess that's to be expected and we can probably introduce some caching and worker threads to at least get it off the UI thread. I imagine RSE does this already and will need to take a closer look at their EFS implementation to see if it would work better.

The bigger issue is the number of instances where plug-ins are assuming IPath works and IResource.getLocation() works. And I kind of knew that going into it, that we'd have some work in the CDT to solve this, and that proved to be true. So I tried a General project and text files worked fine. So I though I'd try something more involved. How about a Ant build.xml file. Well, there again, massive NPEs because the build.xml editor and the AntModel assumes IPath to work. Not only that, they use java.io.File to get at the file. Ouch.

It is pretty clear that little work has been done to try out a real EFS implementation as backing for IResources, or at least not enough work. My feeling is that this remote project idea must transcend the CDT, and we've even thought about starting a new Eclipse project to focus on it. It's clear there is a lot of work to do and we need to start organizing if we really want to make EFS remote projects work.

Now that's Embedded!

I ran across this really cool project this morning. A guy built a little device that recorded light patterns, i.e. bright/dark, and then replayed them on an LED. He used a tiny Atmel micro controller and programmed it in C using, you guessed it, the CDT, with the WinAVR cross compiler toolchain.

It's projects like this that make working on the CDT so cool. We have this little device with 1K flash and 64 bytes, yes bytes!, of RAM and people can use the CDT to program it. Then we have the big iron supercomputers that the Parallel Tools Project gang work with, and you can use the CDT to program it as well. And, of course everything in between, from mobile multi-media devices, to Linux server applications, to the PC desktop, to gaming consoles. But this little project really made my day!

Mozilla Desktop Environment?

Snooping around the Internet news sites, I ran across a discussion on the Mozilla planning list about the potential of a desktop environment built out of Mozilla and their GUI language XUL (O.K. I found it through the trusty Slashdot, I'm sure you did too :).

I always thought the idea of building a desktop around a browser was intriguing and to me this is the promise of mobile tablet-like devices. But simply using Mozilla and it's plug-in architecture might be overkill. I think simply using the browser function with JavaScript would do the trick. Imagine, you fire up your tablet, the operating system boots and the window system comes up, and it could be any operating and windowing system, and the only thing that gets launched is the Browser. Small and fast. Perfect for mobile.

You'd probably have some dynamic HTML pages locally on the device to do desktop-y things like managing files, writing documents, playing media files, etc. Maybe that does require some help from Mozilla and its set of plug-ins. And add in Internet connectivity and AJAX and you can get a whole suite of applications that can leverage the power of the servers out on the Internet or at your office as well.

Once I get some time, I really want to dig deeper into the RAP (Rich AJAX Platform) project at Eclipse and see how their tools can be used to build such applications. And Mozilla probably is the platform of choice on the device since it is open source and readily available and works on many platforms. This all leads to some really interesting combinations of open source pieces to make something really different and fun for people to use. And Microsoft says there is no innovation in open source...

CDT Forever!

It's funny what Google Alerts will give you some times. When I got one pointed at http://www.cdtforever.com, I was pretty excited. A fan site for the CDT? Or just a well planned plot by my wife to buy her a new ring...

Ego-less Development

I remember early on in my CDT career running into an old boss and mentor. He asked how things were going and I told him great and that we were starting to build a C++ parser for the CDT. He was skeptical that we could do that and make it work well with the time and resources we had. But I was so excited to be building a C++ parser, I chose to put aside his wisdom. "It's not that hard."

Well, for years after, looking back now, I probably should have listened closer to him. The CDT's parser performance was abysmal. I really feel for the guys at QNX and others who were building products based on those early CDT releases. And, of course, you can guess what my first task at QNX was when I got there. Fix it!

Luckily I pulled a rabbit out of my, uh, hat. CDT 3.1's indexer is incredibly faster than our previous tries. I totally changed the approach we had taken and threw away the requirement of being 100% accurate that we had be preaching in futility since we started. Now, each file gets parsed only once and again only after it is saved. A lot of good work has gone into CDT 4.0 as well to add more information to the index so that we can use it for all parser based features making them incredibly faster too. It's not 100% accurate, but for 99% of CDT users, that's just fine.

But I still remember those years of pain as we fought hard to make the parser faster. Everything we tried resulted in only minor improvements. And every time we analyzed the issue, nothing really jumped out as to the cause. Unfortunately, it was probably our egos that kept us from giving up and trying a whole new approach. And if we had taken that approach earlier maybe we could have saved so many in the community the grief of an indexer stealing all their CPU on them.

"Ego-less development" is a mantra of mine and we should have followed it. Always question your design, try to understand the big picture and the impact of your design on the entire system. And if someone points out a flaw, listen and make sure you really are doing the right thing. I think this is especially true in open source with so many eyes on your work and so many hands trying to pull it in different directions. It's a huge challenge but if you are successful, the rewards are great, for everyone.

OpenKODE at GDC

You know, all my dreams will have come true if I am ever standing at a conference center attending a GDC, the Game Developers Conference. Interesting enough, this year, it was held the same week as EclipseCon, and in the same area of the world, so I was close. But it is just a dream, and we can probably save that one for another life time. At any rate, I am always interested to see if there is any big news that comes from there, especially in the tools/SDK front that might be of interest to CDT users.

One thing I did notice was that Khronos, the open media API people gave a number of presentations there and have nicely posted them to their site. The one I was most interested in was on OpenKODE, a core API that allows you to build portable applications. What I found most interesting in that presentation was a "State of the Union" of handheld multimedia applications. It shows why the industry needs to be interested in standard APIs to allow for growth of applications for these devices. And I firmly believe that and am glad Khronos is taking this on. They also confirmed what I believe about Java, that, while it really helps with cross platform development, if you need performance, like most 3D applications do, you really need to code natively, and that means C.

In theory, with OpenKode, you can build for one platform, say Windows, and with a simple recompile, run your application on another, say a handheld device running some other operating system. This is similar in purpose to SDL, but SDL has a patchwork architecture, where OpenKODE seems to be much cleaner. Acrodea has a sample implementation of it for Windows, and I wouldn't mind seeing someone do an open source licensed version of it so that we can pass it along in our upcoming EasyEclipse CDT distribution.

We can debate whether C code that requires a recompile to run on another platform is portable. However, if you look closely, to get any performance out of Java, you need a JIT which recompiles your Java anyway. With C, you just need to do it manually and ahead of time. As we build up a collection of portability libraries, I don't see that there is a big need to jump on the Java bandwagon.

A Visit to Google

Chris Recoskie and I had a couple of free hours on Thursday last week at EclipseCon and decided to accept a request to go visit to Google campus. One of their developers is a fan of the CDT and has produced some of our best bug reports on the indexer, so I felt it would be a great opportunity to learn more about what they were doing with the CDT.

It was a great visit and we all wished it could have been longer. It turns out that a few of them are using the CDT to work on some of their Linux C++ server applications. A lot of my early work on the CDT was meant to support this area, but I hadn't actually met anyone who was doing C++ server apps. So it was great to know that there is still a need there and an area where we need to keep in focus.

After our meeting, we went to their famous cafeteria for lunch. It was very cool. They had everything from hamburgers and hot dogs to gourmet pizzas and ethnic food. And, of course, it was free. The only problem we had was finding a place to sit since Google is really growing these days.

They are doing some cool stuff at Google and I am really honored that they are using the CDT there. It'll be a great story for both sides if we can solve any issues they have so that the CDT can be the IDE of choice for C++ server apps there.

EclipseCon The Day After

Here's three happy guys:



That's me on the left with Dave Inglis, the team lead for our Momentics IDE holding our award for best commercial Eclipse-based development tool, and Fred Plante, our fearless leader and manager of the QNX Tools team.

On our way home from EclipseCon, we stopped off to visited a good customer of ours. They like what we've done with Eclipse and had some good constructive criticism on areas where we could improve. They love the tools that are our value add with Momentics, but they confirmed what I always thought, that those tools don't matter much if the edit, build, debug cycle doesn't work as well as it can, or worse, hurts productivity.

But it really hit home coming the day after EclipseCon. It really showed how important our work on the CDT is to our customers. And judging from the passion of the people we work with on the CDT, this is true for all vendors that ship the CDT. I was very pleased that a lot our our customer's concerns have been addressed 3.1.2 and will be addressed in 4.0. I think we'll all have happy customers as CDT 4.0 hits the streets.

I also have one final thing to say about EclipseCon. Someone commented to me that they felt that there wasn't much buzz this year. Others though felt otherwise. What I think happened was that we saw much less marketing push from vendors and much more excitement from the engineers that were there. That's a different type of buzz and from an open source project perspective, much more important. Eclipse is driven by those who contribute to it and as someone else said, this truly was a "Community Party", and in my view, that's what will give Eclipse the biggest boost.

Another EclipseCon in the Books

Well, it's all over. What a great week. For me and the CDT community, it was the best EclipseCon ever. We had plenty of talks about the CDT and really got our message out. We had a lot of time to talk to each other. The CDT demo and project meeting went really well. I also had a lot of time to talk to others in the greater Eclipse community about possible future integrations and maybe even some new projects down the line.

The morning of the last day started with the panel I moderated on multi-language support in Eclipse. A lot of interesting scenarios came up where users want to work with multiple languages at the same time which really stretches the bounds of what we're doing. But it really shows that we need to work together more as Tools projects and the JDT to make sure that our plug-ins work together well and make it easier to create new language IDEs, maybe with the help of Safari.

The day concluded with the ending panel where Mike drilled the PMC leads for all the top level projects. I got to represent the Tools PMC and it was fun. One of the best suggestions I heard from the audience was that we really need to market the benefits for companies that put people to work on Eclipse projects. There are a lot of examples of companies that really do benefit financially by putting developers on Eclipse to make sure it meets their needs. We really need to showcase those examples to the world.

Anyway, it's been a long week and I'm pretty tired. Time for bed...

Time for a SAFARI?

I was pretty impressed by the SAFARI presentation by Bob Fuhrer from IBM Research. They have a lot of tooling that generates most of the cool features you get with an Eclipse language IDE, for any language that you specify. There are lots of wizards that generate Java plug-in code and most of it finds information based on a grammar you provide. A lot of it is pretty simplistic, but enough to get you going. At the very least, the generated code can serve as a great head start.

The good news is that they are close to bringing it to Eclipse for others to use. Of course we're building a multi-language framework with the CDT. So we'll have to figure out where we can leverage some of the cool things they have and where they can leverage some of the cool things we have. But this will be a great tool to help those poor souls who ask us how to build an IDE like the CDT for their own language.

CDT Morning at EclipseCon

My phone is in my room charging so I didn't get any pictures this morning. But it was cool to sit through the two CDT focused long talks and see the interest. We had over 50 people in each and a lot of attentive faces.

The first talk was by Phillipe Ombredanne from nexB and one of the leaders of the EasyEclipse open source Eclipse distribution. He presented a huge collection of plugins that extend the CDT in various useful ways. I was surprised at how many they were and thanked him for all the hard work he did to find them all and try them out. I am looking forward to working with him on EasyEclipse to build up an open source Ecilpse C/C++ distro that will be hugely useful for new users.

The second talk was by Chris Recoskie and Beth Tibbitts from IBM on how to use the CDT's parser data structures and the index to build static analysis utilities. This is a topic close to my hear since this is the area that got me into the CDT and where I've spent most of my development effort. But it is a pretty dry topic and I was very please to see people following closely what they were saying and we even had a couple of really good questions on the details of the DOM at the end. Maybe we'll see some cool features that can help users of the CDT write better code faster with the help of all the parser work we've done over the years.

EclipseCon Day 1

If you saw Ken Ryall's photo, here's from the other side of the room:

We had an overflow crowd for the CDT 4.0 demo. It's pretty cool how much interest we seem to get in what we're doing. The feedback was great and I think everyone was impressed with the work the team is doing. I'm totally excited about it and I hope that showed.

I also participated in Robert Day's panel on Eclipse in Embedded. Embedded continues to be one of the main growth areas with Eclipse and I think we concluded that we're just scratching the surface. We really need to raise the profile of Eclipse in this market. The engineers see the value of Eclipse as a platform for embedded tooling and it's unfortunate that isn't common knowledge. We have our work cut out for us, but it would be great to get a huge contingent of embedded tools developers interested in attending EclipseCon next year and to get a bigger voice in Embedded marketing events like ESC.

I'm also very tired. I don't think I'll stay in the bar until 2 a.m. this night...

Ouch my aiching head

I've got one of those "You better pace yourself, Doug" mornings. No worries, a couple of cups of coffee and I'll be good. But it was a good party last night as my colleagues and I from QNX celebrated our win in the best commercial developer tool category. It is quite an honour to be selected and, especially for the gang who have worked on Momentics since the beginning, long before I arrived on the scene, they should be proud of their achievements as one of the earliest third party to take Eclipse and make it a commercial success outside of Eclipse's traditional enterprise market.

Tutorial day went very good. I spent most of the day with these guys:

Chris Recoskie is on the right. He's Mr. CDT now at IBM. The three on the left are the main players in the Parallel Tools project. One of the coolest things about the CDT is how it is being used in some of the big iron supercomputing shops. We talked a lot about how the CDT can be used in an environment where you need to do your builds and debugging remotely in this environment. There's definitely some architectural challenges to overcome, but including HP who has similar needs, I am starting to thing remote development is reaching critical mass and we'll need to figure out how to organize the players.

I also went to the Remote System Explorer tutorial where I finally got the time to learn the gritty details of RSE straight from the guys who are working on it. I learned a lot in those two hours and actually we could of went even longer. And I think RSE has the potential to help the remote development people. We'll need to hook everyone up.

Well, I have a busy day of talks and the CDT 4.0 demo at 1:30. So I better get at it...

EclipseCon Day 0

Knock on wood, I still don't have horror stories from business travel. Today was another good day. We were delayed in Toronto for a couple hours due to a wind storm, but that just meant I missed the beers Sunday night. No biggy. I'm sure there will be plenty of that starting Monday.

On the flight from Ottawa to Toronto, I sat a couple of rows behind Joe Clark. Joe Who? (Canadians will get that one). He was a popular politician in Canada during the 80's and 90's. He's retired now and was traveling alone. He followed us through US customs so I guess he was traveling to the states to give a talk or something. Oh, by the way, he's a former prime minister of Canada and later the foreign minister. Only in Canada would we figure it isn't a big deal to let a former leader of our country travel around without security.

Then on the flight from Toronto to San Fran, we had an more important person on our flight, Mike Milinkovich himself. (Of course, Mike being Canadian would be the first to say he doesn't deserve such accolades). But he did get the royal treatment, much more than poor Joe Clark did our our little regional jet to Toronto. Mike got through the airport quickly and by the time we got to our car, he was long gone.

Anyway, I'm happy to be here and it's going to be a fun week. I already met some CDT people who were sitting in the bar when we got into the hotel. I hope a lot of the CDT community is here so I can meet them all and find out how they're doing. If you are here, feel free to stop me and say hi!

Oh, yeah, and it's a phone...

As part of my planning for next week, I recorded all the sessions I am thinking of attending in my Outlook calendar. Well, that's fine, but my laptop won't always be available during the week, especially given the rumoured power shortage down there.

Recently I switched mobile phone providers. I did this mainly because in my recent visits to Europe, I really missed having a phone. Luckily one of the major providers here in Canada has switched to GSM, so now I have a phone that should work over there, too.

I picked up one of the latest phones. I won't say brand names, but let's say CDT committer Ken Ryall would be proud ;). I wasn't really looking at features, I just wanted a phone that was really inexpensive, but not cheap, on a three year contract, and this was on the top of the list on my provider's web site.

The more I get into this thing, the less I am thinking of it as a phone. First of all, I learned that I qualified for a free 1 GB MicroSD stick (which unfortunately hasn't arrived yet for my EclipseCon trip). What would I use that for? Well, this thing is also an MP3 player. So that's kind of cool, I've been wanting one recently but didn't want to fork out $100+ for one (yes, call me cheap). And from my last blog entry, you can see it has the now standard camera feature, which I will probably use a lot next week. Playing with it, I see it is also a video camera too (now I see why I need a 1 GB stick). And of course, it does all the latest games, apps, what have you that run on mobile devices these days.

Now back to my original problem of accessing my calendar during EclipseCon, I was playing with the PC software that does various things with the phone over my laptop's Bluetooth connection, and I noticed a Synchronize app. I fired it up, and found out I can synchronize my Outlook calendar to my phone. (It just beeped with a reminder for a meeting I have in 5 minutes, I'd better wrap this up). So this thing is a PIM, too.

I'm not sure if I'll be come a mobile phone power user. I still hate that the screen is so tiny. But I have the chance now to give it a try, I guess...

Way ready for EclipseCon

I'm sitting here planning my schedule for EclipseCon next week and looked up to see a wicked winter storm a-happening. California here I come...

Getting ready for EclipseCon

Well, I just got my "EclipseCon" haircut and I'm starting my final preparations for next week's EclipseCon. If you've never been to one, these things are more than just a conference where people go to hook up with the Eclipse crowd, it's a celebration of everything we work hard for over the year. It's a lot of work to prepare for but it's always a week to remember.

This year, with the help of the growth of the CDT community, we've managed to get a higher profile in the program with a C/C++ Development track. Here's what you can expect to see next week.

Monday
Short Tutorial, Extending CDT To Support Your Compiler (Chris Recoskie, IBM). Learn how to make your own Managed Build integration.

Tuesday
Short Talk, CDT 4.0: easy to use and integrate (Mikhail Voronin, Intel). Get up to date on the new project model changes coming in CDT 4.0.

Demo, What's New in CDT 4.0 (Doug Schaefer, QNX and Markus Schorn, Wind River). Demo of all the cool new features that are coming in CDT 4.0, or at least the ones we have working right now.

Short Talk, Intelligent Command Line Processing for the CDT, (Chris Mead, ARM). Shows an integration with the Apache CLI library for adding cool tool integration features.

Wednesday
Long Talk, Assembling your open C and C++ workbench (Phillipe Ombredanne and Francois Grenade, nexb/EasyEclipse). A great look at how to leverage a number of open source components to create a complete C/C++ IDE.

Long Talk, C/C++ Source Code Introspection using the CDT (Chris Recoskie and Beth Tibbits, IBM). Show how you too can take advantage of the CDT's parser and code models to do some cool analysis of your code.

Short Talk, Multi-platform development with the CDT (Graeme Johnson and Gabriel Castro, IBM). Show how they use the CDT to build IBM's J9 for over 50 different configurations.

BOF, CDT Project Meeting (everyone!). We'll have our regular monthly meeting during BOF time. Everyone is invited to see what's going on in the CDT project and to provide input for any issues we need to discuss.

Thursday
Short Talk, Autotools Demo (Jeff Johnston, Red Hat). Shows the autotools integration with the CDT.

Poster (up all week - Poster Reception Wednesday evening)
Extending CDT Debugger to Support Device Software Development (Mikhail Khodjaiants, ARM). Shows how ARM extends the CDT to support their gnu toolchain, gdb, and JTAG.

Hello C#

Not to get anyone excited, especially a particular executive director of a popular foundation, but I've managed to enter, build, and run my first C# program using the CDT.

I had previously did a language extension for C# that provides syntax coloring for the C# keywords and associated the CEditor with .cs files. In the last hour (while trying to watch a poorly played curling game on TV - yes, they broadcast curling games on the national sports cable channel in Canada), I took everything I learned from my exercise of getting my other build integrations working this week and thought I'd give a build integration for the Microsoft C# compiler a try. After a little tweaking to handle this special case where there is only one tool that takes all .cs files and produces a .exe, I hit the build button and an exe came out.

Luckily for us, Microsoft has reused their PE executable format for .Net apps, so the CDT immediately recognized the end .exe file as an executable file, so I created a launch config for it and hit the run button. "Hello World" it said in the Console view. Tres cool!

Now there's a lot of functionality that's not there. There's no debugger and there's no parser to fill the outline view and index. So there's a long way to go. But it was cool to see.

Wild Week in CDT-land

Wow, it's Friday. I'm glad. This week, once M5 was out the door, Intel committed their long awaited rework of the CDT build model. The idea is to make much of the good things we've done with the Managed Build System build model available to all CDT builders, and also available to the rest of the CDT that could really use this information. The parsers really need to know what compilers and settings are being used to parse properly. I can also see a use by the debug system to default to the best debugger given the currently active build configuration/toolchain.

As well, we get an upgrade to the New Project Wizard. I think the old approach of forcing users to select a language, C or C++, and then a build system, standard versus managed, was pretty intimidating to new users. How are new users supposed to know what a builder is anyway? With the new New Project Wizard, the selection of builder is hidden at least. It's starting to look more like my previous favorite IDE...

What made the week busy was that to do this, a lot of the underlying architecture of both build systems changed. Of course, with any major architectural change, a lot of things broke. Mikhail S from Intel is working hard on all the bugs that are getting raised, and I was able to figure out how to get my MinGW and Windows SDK build integrations working in the new way, so we're getting there. All the other committers are also measuring the impact of the change.

I think the biggest challenge is to get vendors who depend on these systems to take a look at the changes as soon as they can. Mikhail has been talking about these changes for a long time and has been publishing patches for everyone to look at for over a month. And it was still a surprise to many how vast the changes were.

While we do want grow the functionality and the architecture of the CDT, it is important that we don't make too much work for those who are integrating. All I can do is encourage them to get in early and get in their feedback to cdt-dev and bugs in bugzilla as early as possible. I'm sure there are surprises in store for them, and for us on how they are integrating with the CDT in weird yet intriguing ways...

Watch what you blog

This has happened a few times now. I blog something and it ends up somewhere like here. Definitely makes me want to be careful what I say. But at least they are helping to spread the word.

The Teraflop 'CUDA

It's interesting how the big vendors play off each other when some cool new idea gets close to becoming product. The latest one was Intel with their 80 core research chip for highly parallel teraflop computing. Now we see NVidia has released their CUDA SDK and C-like compiler that does the same with their latest 8800 series video cards.

Apparently, you can get 520 gigaflops (billion floating point operations per second) with their top end 8800 card which features as many as 128 single precision floating point cores. Combine two such cards in an SLI configuration and you can hit the teraflop mark, theoretically. They have a compiler that compiles programs written in C with some CUDA specific extensions that gets downloaded to the card which runs as a co-processor to your main processor.

I can see the applications for such technology being pretty much limited to scientific simulation type things, not general purpose computing, at least not in the short term. The main issue is that not every will have these specific cards in their systems, and it really is NVidia specific. But it looks like they'll provide a huge amount of horsepower at a decent cost.

One of the reasons I blog about these things, other than I think they're cool, is to show that C/C++ development is still alive and well and growing. I remember seeing a study a couple of years back that showed it on the decline with everyone moving to Java or .Net. But there will always be applications that need to run as close to the hardware as possible to run efficiently as possible, either because the computing resources are limited such as with mobile devices, or because they are very specialized such as the NVidia processors. Languages such as C and C++ that compile directly to executable object code is still the best way to achieve these efficiencies, and why we see the CDT as popular as it is.

CDT 4.0 M5 Now Available

I am pleased to announce the availability of the first "public" milestone of CDT 4.0, M5. CDT 4.0 is going to be a huge release for us and bugzilla is telling me that we already have 265 bugs/enhancements that have been addressed (more than we did in all of CDT 3.1.0 BTW). With all the new committers working on it, it is important that we get these milestones out to the community for feedback and testing.

So feel free to give it a try. Instructions and links are available by following the CDT website, http://www.eclipse.org/cdt. Note that you require Eclipse 3.3 M5 to run it.

Unfortunately, we don't have a New & Noteworthy. But things to look out for include new Views like the Include Browser, Call Hierarchy View, Type Hierarchy View, and improvement in performance for Open Declaration and Content Assist. The managed build internal builder works better by getting dependency info out of the index. There hasn't been much change with debug and the standard builder.

New Face

For those who follow the Planet, I have a face finally. Thanks to Ian Bull for putting it together for me. For those that don't follow Planet Eclipse, boy are you missing out!

Also, I am honoured to be named as a finalist for the Top Ambassador award in this year's Eclipse Community Awards. It's funny since I had a hard time deciding between the other two finalists when I voted, Alex for is great work on EclipseZone and Chris zx for constantly hooking me up with cool things happening in the greater Eclipse community. Good luck to both of them!

Old news is good news

I have to admit, I am one of the worst committers when it comes to responding to newsgroups. I have a tough time just keeping up with cdt-dev and bugzillas and writing code and ... that I never seem to have time to go to the CDT newsgroup to see what people are talking about. Don't vote for me in the Top Committer category, that's for sure.

But I went there today to announce CDT 3.1.2. I guess this is the first time I did since I got my new laptop a couple of months ago (see!) and it downloaded all the messages since the beginning of time. It was interesting. There are over 12000 messages since the first one where John Duimovich launched it on December 10, 2001. The talk then was on today's CDT's predecessor from IBM which was very different (and some lives on today in the Remote Systems Explorer from DSDP/TM).

The first message after John's was from 'dominic' who started a thread of fans that were drewling over the idea of a C/C++ tool in Eclipse. There were a few posts from people who had great feature ideas, like ant support and UML modeling (which funny enough is what got me into the CDT). People were concerned that the team wasn't testing using the funky new GTK support on Linux available in early access form with Eclipse 2.0. And, of course, people were asking, if you can do C/C++ how about supporting Cobol and Objective C too.

It's pretty good reading and gives a great historical background on the evolution of the CDT. And it's a bit spooky since I hadn't even heard of the CDT at the time and all this activity was happening. But the coolest thing is the diversity of people that posted then and post today. It certainly is a great way to keep in touch with the community outside the cdt-dev walls. And I'm probably the last committer to see the value in participating, which is now my late new year's resolution...

CDT 3.1.2 Now Available

It's been a pretty busy week as we are getting two releases together pretty much at the same time. The first is the release of our latest maintenance release CDT 3.1.2. Bugzilla tells me there were 101 bugs fixed in this release, which is a pretty good number given that we have been very busy working on CDT 4.0 at the same time.

The biggest fix that I put in was to move away from using memory mapped files for the Index (also known in some circles as the PDOM). Memory mapped files worked well until we started running into very large workspaces where we started to run into limits on Windows. I've switched it to regular files that read in chunks at a time and use a least recently used (LRU) cache to make sure we don't take up too much memory. It did cost us some in performance, but I'm still within the targets I had set out for CDT 3.1, i.e. indexing Firefox in under 20 minutes.

More information can be found on the CDT Website.

Stay tuned for our first real milestone for 4.0 early next week...

OpenKODE: Cross Platform Mobile Gaming in C

I blogged a while back about an open source mobile gaming device, the GP2X. It's a pretty simple and inexpensive unit with a dual core ARM configuration. I think a lot of people are using it for simple 2D games and there are ports of the arcade machine simulator, MAME, to it. It's not a very powerful machine, but I thought the idea was pretty powerful.

It got me thinking about whether the industry would be interested in a similar platform but with 3D gaming support. And if not, why not. Maybe these things cost too much to build. Maybe the business case isn't there. I haven't seen too much interest from the big gaming houses on porting their titles to OpenGL ES, a standard 3D API for mobile. But then, there isn't really a high volume platform out there that supports it to the scale of the Sony PSP and Nintendo DS. Unless you count phones, but you can't do serious gaming on a 1" screen...

But it looks like the gang at Khonos.org are trying to do something about that. They recently ratified a draft specification for a common platform API called OpenKODE. It's a collection of C APIs, most of which they've already specified for video and audio. But it also has a OS abstraction layer so you can compile applications against any OS, or so goes the theory. The intent is to provide a platform for content providers that will allow them to hit as big a market as possible.

They are accepting comments from the industry and I see vendors starting to do press releases announcing support for it. My hope is that you'd start seeing mobile gaming platforms like you see with MP3 players, i.e., a lot of vendors making different types of devices but all supporting common standards. That would sure change the face of the industry a bit.

The CDT Can Help you See!

Well, not really, the NASA Vision Workbench can and this guy is using the CDT to work with it. After all the hard work we've put into the CDT, I'll take "Eclipse’s CDT plugin (which has definitely gotten alot better)" as a compliment.

What do you use for UI testing?

The TPTP gang were adjusting version info in an enhancement request I raised against their Automated GUI Recorder (AGR) and it reminded me I need to look into this issue again. I remember seeing a demo of the AGR back when I snuck into an Eclipse Architecture Council meeting a year or so ago. It looked pretty good and appeared to do what we've needed for the CDT since forever, i.e., something to automate testing the GUI.

The issue I have had with the AGR is that I want to integrate the test cases that AGR generates into our existing JUnit framework. Unfortunately, that was impossible, or more realistically, was a lot of work to do. And, also unfortunately, the word I hear through the grapevine is that the AGR really isn't staffed well enough to evolve it this significantly. It met the needs it was created for, i.e. test TPTP, and if it is to fulfill a larger role in Eclipse, it really needs more Eclipse developers contributing to it.

So if the need isn't there to trigger this investment, it got me wondering what people use to automate their GUI testing. Of course there are commercial products out there that do the trick and we've used those here at QNX. But for open source development, it would be nice if we had an even playing field so that anyone could contribute tests without having to purchase the "standard" tool, assuming we could even agree on which "standard" tool to use.

I heard of the Abbot GUI tester and the Costello GUI recorder, still the best names in open source IMHO, a number of years ago. While built for Swing/AWT, they started working on SWT support a long time ago but it has been slow going. I have heard of some people using it for their Eclipse testing, but I'd like to see an official release of it before committing to it. But it does what I want, though, i.e. integrate with normal JUnit.

So I'd like to hear people's opinions on this, especially on where they would like to see developers invest their time if they indeed wanted to contribute to an Eclipse open source GUI testing tool. I think the need is there, but that means "beans" if no one else does.

Now what am I supposed to do with 80 cores?

Yesterday, Intel announced a research project they've been working on to produce a core capable of reaching teraflop (a zillion floating point operations, or something...) performance. The chip they've come up with has 80 cores. Sounds like a lot, and it is. Now, this is a research chip and it is missing things like a memory controller which makes it impractical for anything. But it is serving it's purpose of helping us understand what such an environment of the future would be like. The benefits will be paradigm shifting, but we have a long way to go before we get there.

Of course, being a tools guy, I think the tools industry needs to jump on this. Without proper tools, developers just won't be able to deal with the complexity of the software that takes advantage of such scalability. This is one of the reasons I'm very interested in the Parallel Tools Platform (PTP) project at Eclipse. They are trying to deal with such environments today by providing tools that support developers working with supercomputers used for scientific research.

The challenges are enormous and include the whole edit/build/debug programming cycle. What libraries and programming language enhancements do you need to write an 80 thread or process application, how do you build and deploy such a thing, how do you debug it? I can still picture the problem of setting a breakpoint, having 10,000 threads hit that breakpoint and then trying to find the one that you are interested in. Sounds impossible. But there are some pretty smart people working on PTP and it'll be very interesting to see the solutions they come up with.

Open Source Translations?

Well, it was a busy week for conference calls. The latest was on an initiative to start an Eclipse project to take care of the translations for the Eclipse project. I went into it being mistaken that the initiative was simply trying to spread the translation work around for Europa so that IBM didn't have to do it all. It's a pretty expensive endeavour to translate all of Europa to 23 languages. But the intention is indeed to be more than that.

Now, for the CDT, translations haven't been a priority. We haven't released language packs for the CDT since the 2.x days. We still get a healthy 20% of our downloads to China and I haven't received feedback that they miss them. But I imagine we'd grow even more internationally if we had them.

Despite my original skepticism, I think a translations project is a great idea. Technically, making the translations isn't very complicated so you don't need to be a hard core developer to help with them. And, apparently there are some tools that have been made to help make it even easier. It would be cool for the users of Eclipse around the world to help out with this and would probably result in better quality as they actually use Eclipse.

My skepticism right now comes from the current focus on getting the Eclipse member companies to contribute rather than on building a community. I think if you focus too much on short term goals, like translating Europa, you won't solve much when the next release comes around. Hopefully, this project will find an inspired leader that break new ground in international community building. It certainly won't be an easy task. But it could be another great step forward for Eclipse.

CDT: An Open Culture

After today's CDT conference call, the gang here at QNX got to talking about how interesting these calls can be. Apparently, the CDT has something special going with how open we are.

The topic we were discussing was my mention of our plans with Momentics and CDT 3.1.2 and CDT 4.0. I did it just to let the community know that if I'm doing something weird, it's because I'm trying to get the CDT 3.1.2 into the next Momentics maintenance release and to set expectations on when we'd be trying to integrate our custom bits with CDT 4.0 to provide feedback on the new frameworks.

What I had forgotten was that in this highly competative industry, this is usually a no-no. Mentioning product plans could definitely cause a slap on the wrists. Product plans are high quality ammunition that the marketing teams use against each other. Not that I aspire to such tactics, mind you.

But with the CDT and even the DSDP, this hasn't really been a concern. We often drop hints on what we're doing commercially. It all comes back to "When is you problem my problem". If you at least know about my problem, you might make some effort to help me. But if I'm keeping my intentions secret, there is no trust, and I can't see how you can have any hope of co-operation from the community.

In the end, we concluded that the CDT was a pretty cool project to be a part of. There are a lot of projects at Eclipse now, and not all of them have realized the benefits that an open culture can bring them and their customers. And it's not just getting people to help you with your problems, either. It's just cool to be in touch with the rest of the industry and know what's going on out there.

Windows Vista versus OpenGL

I just finished reading Tom's Hardware's benchmarks comparing Vista and XP. I remember hearing that Microsoft was dropping OpenGL support and Tom's confirms it with their benchmark of OpenGL apps. I don't follow the graphics community as much as I'd like but this really has to be a blow for some of them. If not that, then it's a blow for them upgrading to Vista.

Back on my Digital Content Creation with Eclipse soap box, I guess this kind of spells doom for the OpenGL work that's been going on with Java and, by extension, SWT. I have always thought that it would be very cool to have a 3D modeler built on Eclipse and nicely integrated with the CDT to build games and such. Most of these modelers are built using OpenGL and probably don't work well on Vista.

So the question becomes, when will someone build DirectX support in Java in an SWT window?

It also provides a twist for the work I'm doing on Windows development. I've been torn between whether Windows developers should use the Windows SDK or MinGW with the CDT. I'm not sure how mature the DirectX support is with MinGW compared to their OpenGL support, so my feeling is that the Windows SDK with the DirectX SDK will be what developers will want to use. Which means I better get that Windows SDK integration done...

My Take on Eclipse PluginFest

Well, I'm finally home from my trip out to the Eclipse PluginFest. I made a side trip to Karlsbad, Germany to visit a customer, where it was pretty cool to see them adding their own plug-ins. Eclipse is indeed everywhere. And I was met by a very Canadian-like snow storm there. While it was fun to drive in, I missed out on a lot of sleep with the late arrival. But it was still warmer there than the -20C here.

Ian has a really nice summary of the PluginFest event so you can read that to get the general gist of what happened. From my seat, it went way beyond my expectations. I found everyone who attended to be very open and we all filled up our laptops with trial versions of each other's products and tried them all together. I was concerned that being a commercial event there would be a lot of protectionism happening. But it turned out that most of us already knew eachother from past Eclipse events, so were instantly comfortable with each other.

For the most part everything worked, but that is to be expected given the clean plug-in architecture of Eclipse. But it did point out an important requirement that everyone be on the latest version of the Eclipse plug-ins. Happily we made that decision at QNX last year and it paid off for everyone who did. It was cool to see other plug-ins work with our product and that should open the door for some pretty cool environments for our customers. And the sentiment was shared by the other vendors there as well.

But as Ian mentioned, the value of the event went well beyond testing our plug-ins. It was a great opportunity to talk to everyone who was there. I had a lot of people ask me CDT questions, including a couple asking how to get involved and contribute the CDT changes they have had to make for their products. I also got a chance to talk to a couple of the CDT committers that were there to go over design issues we need to resolve. Meeting face-to-face this way is pretty expensive, but it certainly has a lot of value for the people that get to go.

We definitely need to do this again, maybe after a year to give us all a chance to get onto the Europa platform. The gang at Symbian were great hosts, but I have a feeling a lot more vendors will be interested in this next time and we'll probably need a bigger venue. This was our first try at bringing vendors together for inter-operability testing and it really shows the value of the Eclipse architecture. And I'm sure all vendors building on Eclipse would be interested in this. I see big things in the future for the Eclipse PluginFest.

When is your problem my problem?

I just got off a conference call we hold semi regulary amongst the committers working on the CDT indexer. Now that we have people from four different organizations working on it, we need this level of co-ordination to ensure we don't trample on eachother. And it is a cool example of open source development at work.

Now when you get this many interests working on a common code base, you are likely to run into times when requirements start to conflict with each other, and you end up with conflict amongst the developers. This has started to happen with us and it is giving me an opportunity to really learn the right approach to take to make sure everyone walks away happy, or even whether it is possible for that to happen.

Taking a big step back, I think the first thing you have to do if you have a requirement you'd like the open source project to satisfy is to convince others in the community that it is their problem too. Now sometimes that isn't possible because the requirement really is particular to your customers. In that case, I think you need to convince the others that the changes you are proposing will have minimal impact on them and others who want to build on and use the project. There are times, especially with young projects, where people might bend over backwards to help you. But you certainly can't count on it, nor should you.

At the end of the day, what it really boils down to is that you have to convince people. Influencing is a critical skill that an open source developer must have to be successful working with open source. In a corporate environment, you have it easy since you can always go complain to management. But in open source, all the committers are peers, there are no managers, so they need to work out any conflicts with eachother. And that demands great people skills.

It is pretty interesting that our little indexing group has such great influencers and the discussions can become quite heated. But we do respect eachother and will continue to hammer out these issues and try to come up with the right solution for everyone.

And if this does work out for us, it'll make a great chapter for a book on working in open source. There may be a book out there already that I need to read, or maybe not and someone needs to write one. Because I see this happening on almost every open source project I see.

Eclipse is ... an Opportunity

Despite Doug Gaff's wish to have the last word on the Eclipse is You/Me/Who? issue, I do have one more thing to say on it (Sorry, Doug :). I was at the Device Debug meeting in Toronto last week, and for me it was deja-vu all over again. We had similar meetings on the CDT many times in the past, ones where a big number of people, 30-40 say, get together to talk about Eclipse projects, but in the end only 4 or 5 of them are actually planning on contributing anything to the project despite there being a need for many more.

Who are the rest? Well they are developers sent by their managers to make sure they keep tabs on what is going on. What it really tells me is that these companies don't really need anything from the project, not yet at least. But if something does come up that they can use, at least they know about it and can start making plans for it. But really, they are a long way off from having the compelling business reason that their managers need to allocate resources towards making any contributions back.

We've talked about the difficulty at making that leap from user to contributor. The biggest is just the expense of allocating developers to something that may not be their core competency. And, in fact, they may not have the Eclipse expertise on staff to actually do it. And it's really unfortunate, because without help, Eclipse projects have a hard time getting to the point where they would be useful enough to justify contribution. I feel Doug G's frustration at trying to start new projects in this environment.

So it got me wondering, looking at other examples in the industry, whether these companies would be willing to pool, not people, but money towards hiring shared developers. I know of a few small consulting companies that get paid to work on open source. I think this has the potential to solve some of the staffing problems that we're seeing on these projects. But I'm no expert on whether this is a good, or at least sustainable, business model. It would be interesting to see if people are willing to work together this way. And I encourage such consulting companies to ask around and see if they can make it work.

EasyEclipse for C and C++, half way there

In my previous entry, I yearned for a single install of Eclipse, CDT, MinGW, and some libraries to get you "Kickstart"ed using the CDT on Windows. As Chris, zx, quickly pointed out, EasyEclipse already has some of this done and hooked me up with Phillipe Ombrédanne from nexB who created EasyEclipse. Of course I already know Phillipe who led the Google Summer of Code project for Eclipse and who is scheduled to give a talk on the C/C++ Eclipse ecosystem (amonst many others I see) at this year's EclipseCon.

EasyEclipse for C and C++ is definitely a great start at what I wanted to acheive. And I see from the comments on the site, there are people already asking for MinGW to be included.

So I've altered may plan and will work with the gang at EasyEclipse to complete EasyEclipse as a full fledged IDE offering. They already have a thriving community so it will be cool to build on that and get more C/C++ users involved.

Time for a CDT Kickstart?

I subscribe to Google Alerts and get notifications whenever something new pops up mentioning the CDT. I've been seeing a lot of blog entries lately which is a cool sign that people are talking about the CDT and spreading the word. Unfortunately some of the comments deal with the difficulty of getting the CDT up and running, especially on Windows.

So I think it's about time to do something about it. One thing I've wanted to do for a while now is to create a Windows installer that bundles up everything you need to start building applications with the CDT. As a product manager I used to work with said, "How do you call it an IDE when you don't bundle a compiler with it?"

So my plan is to bundle the Eclipse Platform and the CDT run-times with the MinGW gcc build tools and gdb debugger. I will bundle in SDL, Simple Directmedia Layer, as a library to start building multimedia apps. I may look at bundling wxWidgets for building traditional GUI apps. I will also look at bundling a JRE so users can get Eclipse up and running with minimal effort, although I'm not sure if the Sun redistributable license will let me do this with an open source app and I'm not sure what other options I have.

Unfortunately a bunch of these components are GPL and LGPL so I won't be able to release it from Eclipse itself given the IP rules. I'll also need to figure out how GPL affects the package and it would be very disappointing if it prevents me from doing this. Sourceforge is probably the best place for this. I plan on using the Nullsoft Installer since it seems to have the most flexibility in any of the other free Windows installer technologies and is used by other open source projects such as MinGW itself.

So I'm interested in peoples' opinions on this, if this is the right set of components for example. Is it worth my time putting together? The idea is to keep it simple so it doesn't take me too much time to start and keep up to date. But I think it'll be a boost to help new users get a start with the CDT, a Kickstart if you will.

The D Programming Language

Well, if you are keeping score at home, you know that I hate Java even though I spend 95% of my coding time in Java, really like C# as Java done right, but not as much as I love C++ which though tough to learn does almost everything right, but can never forget C which is used by most of my QNX customers. And also not to forget Ada which has the best type safety system I've seen. But even with all these choices, I'm still searching for the ultimate programming language that maximizes quality and productivity when building complex systems.

So, of course, I had to check out the D programming language when I learned that it will be releasing in a 1.0 version "real soon now". From the dates on the spec, the idea has been around for a couple of years now. To sum up this new language, it essentially takes the best from C, C++, Java, C#, Python and Ruby in an attempt to make the uber systems programming language.

It is a valiant effort and does have my favorite language features such as garbage collection, delegates, strong typedefs, templates, overloading, etc. It also compiles to native code eliminating the need for a Virtual Machine, although given that it has garbage collection, it still needs an extensive runtime library. But with that and with a smooth integration with C and with Windows C libraries especially, it stays true to the systems programming paradigm.

But the language is huge, almost rivaling the size of Ada. I can see why it's taken quite a while to come up with a 1.0 quality compiler. And we really need to ask the question whether the world needs another systems programming language, especially when it doesn't really add anything new to the collective pool, other than collecting the pool.

I'll keep an eye on its progress as it starts to take on the world. At the end of the day, of course, it'll be the programmer community that decides whether D is so good that it is worth climbing the learning curve and adopting a new toolchain and maybe even a CDT integration.

Stream Computing

And now for something completely different...

Enough about I am Eclipse, Eclipse I am. I actually think Steve Northover really is Eclipse, or is it not Steve's Widget Toolkit? Or is it Crazy Doug's Tooling? Or something...

Anyway, the talk at Tom's Hardware today, or at least this guy's opinion piece, is on AMD's insistence that multi-core is dead, long live Accelerated Processing Units. Actually, it sounds like almost the same thing as multi-core just with fancier hardware as some of the cores. The Opteron architecture makes the array of possibilities interesting and feasible and it'll be cool to see what they come up with now that they've teamed up with my fellow Canadians at ATI.

One possibility I find intriguing is integrating Stream Processors into the concoction. From my Google searches I see stream processing has been around for a couple of years now and it has hit the streets as the technology behind the latest generation of graphics cards. Stream processing is essentially parallel processing units that perform like SIMD processors but on streams of data. In that sense it can handle larger volumes of data like a DSP but with multiple processing units.

So why is that interesting to me. Well, as I am always looking for the next great programming paradigm. When object-oriented first came along when I was in university, I was an early adopter because I could see the benefits it gave me when organizing my programs. I am always looking out for the next big improvement in programmer productivity and we've been stuck now for quite a while with objects and classes and methods and such.

I am positive the next big thing will be parallel programming. The hardware guys are making these great multi-core/multi-processing-thingy machines. The question is what is the right programming paradigm. Is stream processing it? Maybe. But I am thinking that the next big thing has to be multi-dimensional programming in some form or other. I still wonder back to Action Semantics from UML as a possibility, but this stream thing is interesting too...

Eclipse is You and Here's Why

There's been some interesting points on the Planet following the "Eclipse is You" post by Bjorn. It has always rubbed me the wrong way when people criticize the committers for not meeting their requirements. And it is not just with these posts, we get it sometimes on the cdt-dev list and bugzilla too. But as I mentioned in my last post, I do appreciate the feedback as it helps me understand what I need to do to grow the community.

But people keep forgetting one thing about the committers. They don't work for the Eclipse Foundation. They are not contractually obligated to do anything, really. I work for QNX Software Systems. They pay me to work on the CDT because it is a fundamental piece of our Momentics IDE. Any work I do beyond that is on my own initiative and if my time is needed elsewhere by my employer, I have to drop those things.

So when people say that the Foundation, or the Eclipse Board for that matter, should get the projects to do this or that, they can't. There is no mechanism in the governance model for Eclipse to make that happen. It just doesn't work that way.

That's why Eclipse is You. Because if you want something done in Eclipse and no one wants to, you have to do it. And, unfortunately, simply submitting patches doesn't work all the time. Because it requires committer time to apply and as I've mentioned, the committers are at the whim of their employers whether they have the time. Not only that, but you may have to persuade the committers that you are doing the right thing.

So we do the best we can and we try to go beyond the call of duty to make sure the community is happy. And most of the time, it works out. But sometimes it doesn't, and I understand the frustration. Remember though that Eclipse is a meritocracy. Submit a number of great patches and help the community out, i.e. go beyond the call of duty yourself, and a committer would be happy to nominate you in as one too.

It's all about You!

According to Bjorn, Eclipse is You. Being on the receiving end of many "CDT doesn't do , or CDT is too slow doing ", I couldn't agree more. Say those words and you earn instant membership in the CDT community. And I appreciate every one of them, I actually do. It means you care and have spent the time to contribute your guidance to our collective knowledge. And it's the first step down the path to contributing even more.

Now, Time Magazine has selected the Person of the Year, and it also happens to be You. Coincidence? I think not!

Migrating from Visual C++ to CDT

I've put the finishing touches on the CDT's managed build support for the Windows SDK. Well, at least there's enough there for people to try with our upcoming CDT 4.0 milestone (M4, but it's really our first for this release). It auto-detects where you've installed the compilers, header files, libraries, etc., by looking it up in the registry. I've also updated the error parsers to more accurately parse compile and link errors. It works pretty good and I'm using it to build the native code for the Windows debugger integration.

But, you know, I forgot about the standard builder. It's funny how you get tied up in solving the hard problems when the easy ones are there staring you in the face. I have to give a big thanks to three guys from IBM India who have written a tutorial on how to import Visual C++ projects into the CDT. The solution is elegant in its simplicity and really shows the flexibility of CDT's standard make projects.

All you need to do is get Visual Studio to generate the makefile for you. This is a feature that they've always had to support external builds (although in recent versions you can run Visual Studio headless to do builds as well). Then you create a CDT project at the root directory containing your source. Of course you'll have to change the make command to use nmake, Microsoft's own nasty version of make, but that's pretty easy to do and works well.

Combine that with this guy's perception that the CDT has certain features that Visual Studio users would like, and the discussions I've had with embedded developers using CDT but using Visual Studio for emulation on Windows, gives me a warm fuzzy that supporting the Windows SDK is the right thing for the CDT. There are Windows developers who are looking for a migration path to get into the Eclipse ecosystem.