CDT 4, What a difference a community makes

If you've ever been to a CDT meeting, chances are you would have heard me state one of my mantras, "Don't take a .0 version of an open source project and put it in a product". Certainly, in the past with the CDT this has been very true. And it certainly hasn't been the fault of the committers and contributors to the CDT who work on it with a passion and really want it to be the best C/C++ IDE it can be. A lot of it had to do with the lack of test pressure we'd get on these releases. And a lot had to do with the lack of feedback from the community until after the release. A lot of it had to do with a young community that was just starting to grow.

Over the last year, this has all changed. With the great bug reports we've been getting from the community and the volume of patches we've been getting from contributors, and the great work of our army of committers, I am happy to say that CDT 4.0.0, is the first .0 release of the CDT that I feel really good about. I'm sure everyone will be pleased with it from our open source community to the vendors that adopt it.

In the spirit of openness, I have to admit I have been very dissatisfied by our previous .0 releases. And I think I need to share what happened so that we can learn from it. Here are the "lowlights" of previous CDT .0 releases (BTW, the CDT 1.x.0 series actually went pretty well, so we'll start with 2):

• CDT 2.0.0 - first introduction of our new parser-based indexer and search. This was a dramatic change to the way we did indexing and the start of our performance woes. In fact I rewrote the CDT scanner (tokenizer) in 2.0.1 since it was brutally slow. We also introduced a new build model that wasn't really ready for prime time and really needed to be exercised with more tool chains.
• CDT 2.1.0 - This release was to meet one of the contributing vendor's product needs and was done while the other vendors were focused on the next release. It was a bit difficult to watch the extent of work they were doing mainly on their own. This really showed the need to standardize our releases (thanks Callisto, Europa, etc!) and how the model we had been following can't scale as the number of contributing, and consuming, vendors increase.
• CDT 3.0.0 - This was the big DOM release. Again, massive changes to the indexing and search framework done by one vendor. Even worse, that vendor later had to withdraw its resources on that massive bulk of code due to business reasons. This really showed us the need to have a diverse set of committers working on each component. Contributors will come and go, and you need to build a robust community to survive it.
• CDT 3.1.0 - Yet another new indexing framework. With the contributors to the original index gone, I took it upon myself to finally throw out our failing assumptions, 100% correctness as a requirement being the big one, and focus on a new strategy to address performance. At the end of the day, this new strategy is the right one and correctness was still pretty good. The problem was that I did it myself and bit off more than I can chew. It was an absolute necessity to get it done, though, but I did do a lot more work for 3.1.1 to make it product quality.

It's been a while ride over the last 5 years on the CDT (BTW, it was 5 years yesterday, I need to blog about that too!). I think we've learn a lot from it, probably enough to write a book about it. Being in the open, I hope others can learn from it too. At the end of it all, I think the most important aspect of any software platform project, commercial or open source, is that of community. And the most important job of a project is to build that community. And that there is probably nothing harder to do than build a community. As our industry goes through this shift, we really need more examples of what to do and what not to do. Then, hopefully, we won't have to always go through pain like this.

Some CDT Stats and Memories

In the rush to get numbers for Ian's Europa stats report, I got wind of a web site that gathers these stats for us. The CDT's numbers are here. There are a couple of items that caught my attention. First was that CDTs line of code count is only 646,651. I believe at one point we were actually more that 700,000 so we've really cleaned up a lot of dead features over the last year or so. It also shows the power of the Eclipse Platform that we can take it and offer a complete IDE for C/C++ in such little code.

The other number that caught my eye is the estimated 176 person years of effort to build all this. I'm not really sure how they got that number but looking at the breakdown by developer, it seems pretty reasonable. At a loaded labour rate of $150K US, that's about a $26M investment, which points back to the main reason I think the CDT is successful commercially. Not many companies could afford to do that on their own.

Taking a look at the contributors list, also reminded me of all the people that have worked on the CDT over the years, 32 in total. And although some have moved on, their contributions were huge and still pay a important role in the CDT today (thanks, John and Alain!). And we have new blood to take over and they are quickly catching up, especially the gang from Intel and Wind River, and the new team at IBM and Symbian are starting to role, and of course, our debug keepers from ARM, Nokia and Freescale.

On the eve of our biggest release, CDT 4.0, it was nice to sit back for a moment and think of all the great contributions we've received and the great people we've worked with over the last 5 years that play such a big part in getting us where we'll be tomorrow.

Linus on source control systems

I ran across this video on the weekend. It's Linus Torvalds giving a talk at Google headquarters about his thoughts on source control systems and why he likes the one he made the best. I thought I was overly opinionated, but wait until you see Linus. This talk was especially relevant to me as I am currently weighing the benefits and costs of moving from CVS to Subversion.

Despite his crass style, Linus does make a lot of points I have to agree with:

• CVS sucks. SVN tries to be a better CVS, but the problems are so fundamental and SVN doesn't make enough strides to make it much better than CVS.
• Merge is the key feature of any source control system. We've gotten used to CVS's weakness at merge and Eclipse's CVS support helps a lot. I actually think the SVN Eclipse plug-ins are a step backwards since I haven't seen a successful merge with them yet.
• Source control systems should allow everyone write access, at least on branches. I would love it if everyone who customized the CDT could do it in our main repository so we can see what they did and make it easier to incorporate their improvements. I'm not sure that is even possible in CVS and it requires central administration at least in SVN. I guess with Linus's GIT, allowing this is it's number one advantage.

I'm actually a big fan of Rational's ClearCase because of it's ability to manage branches and merges. It was a behemoth to manage, and a lot of people found it hard to work with, but I really appreciated the ability to work in a private branch and make occasional merges back to the main line. I'll definitely want to look at GIT and see how it compares. Of course, we would need an Eclipse plug-in to work with it...

BTW, when I visited Google during EclipseCon, I saw this stage. It's in their cafeteria and you can see people getting drinks in the background. I wonder how many other cool visitors they get like this.

Now there's a screen I can live with

I've just been reading a few notes on the web about Palm's new Foleo "Mobile Companion". If you haven't seen it yet, it's a mini-laptop type thing that's intended to work with Palm's Treo smartphone. But under the hood it's really a mini-laptop that has wi-fi and Bluetooth connectivity and a USB port and SD slot for expansion. And it runs Linux with an Opera browser. The price is also pretty reasonable at $600 US and that's with a 10" screen and full size keyboard, although there's no function keys.

So, I'm not promoting the product. Yes, I know people from Palm who are contributing to Eclipse and the CDT, which I'm sure they're using in conjunction with this product. But I think it could the be the start of a trend. Everyone loves smartphones and getting their mail in Blackberry's and such, but the size of the screen and keyboard on these devices really limits their usefulness beyond their mobility. People still need laptops to do their real work.

But I think there's room in the mobility market for devices like this one. The embedded system-on-a-chips are there now to do it. And I think you'll even see games on these things with the 3d capabilities of these chips. With solid state memory like SD cards getting bigger and cheaper, these could be really useful little machines. Palm was first and it fits their niche, but I wonder if anyone else will take the plunge and make a more generally useful "mini-laptop."

UML Action Semantics, Naturally Parallel

Earlier in my career, I had the honor of reviewing a part of the UML spec which involved a very surreal phone meeting with one of my heroes in this industry, Jim Rumbaugh. The area was on the UML's Action Semantics. At the time it was a separate spec but it is now intertwined in the Superstructure document as UML's Action behavior.

The idea according to Jim was to provide a sort of assembly language that all software behavior could map to. But I thought it provided a more powerful concept, that of the Action itself. An Action is a unit of behavior that has inputs, does some processing, and produces outputs. The outputs of one action feeds into the inputs of other actions. The "Ah-ha" is that all actions that have their inputs satisfied theoretically run in parallel.

This concept isn't new. Hardware designers have been thinking this way forever. I believe Petri nets present a similar idea in mathematical terms. But what struck me was this was a really powerful paradigm that can make it easier for programmers to write highly parallel programs. What was needed, though, was a good, 2-dimensional programming language that allowed programmers to create actions and hook up the inputs and outputs quickly and, of course, with minimal typing. But something like that really wasn't an objective for UML.

It's probably one of the reasons I'm keenly watching Eclipse's Modeling project. Aside from a great framework for creating domain specific languages, it has the capabilities that would be needed to build this "Action" language. And with a good back end that produced code for today's multi-core clusters, I really think this could be a good way to help programmers meet Intel's challenge that "Software has to double the amount of parallelism that it can support every two years" to catch up to the what the hardware guys are doing.

How Different Are Linux Distro's anyway?

In my years in software development since Linux has taken off in popularity enough to warrant commercial software vendors porting their wares to it, one thing I've seen vendors having trouble with is dealing with the massive number of different Linux flavours out there. Back when we were just Windows and Unix (commercial *nix's if you will), life was so much easier. The operating system vendors ensured that the releases were well defined so that we could easily put together a reasonable list of supported versions for our products.

With Linux, it really is next to impossible to do that. Novell and Red Hat do fill in that role as commercial Linux vendors that provide a stamp of approval over their versions of all the packages that go into a Linux distribution. But, really, none of the developers I know that are using Linux are using any of those commercial Linux'es. They're using Fedora, OpenSuSE, and more lately Ubuntu. It really is impossible to validate your products against all the possible combinations of Linux that your customers may want to use.

But, I then ask the question, so what? How different are these distributions anyway that makes it so hard to support Linux. Yes, you may have version differences in the packages, and things like the major versions of GTK can break under GUI applications like Eclipse. Also, it's pretty confusing the number of different ways to set up user's environment variables, but then applications shouldn't be relying on that anyway. I really wonder if there's much else that can affect most software products.

It bugs me every time someone tries to explain away a bug with, sorry, that version of Linux isn't a reference platform so we can't look at your problem, especially when the person is using a recent distro like Ubuntu. But it really does speak to the challenges that software vendors face with the fragmentation of the Linux market. But I guess it's part of the price we pay for "freedom".

cdt-dev is my office

Today is RC2 day for CDT 4. As we get closer to milestone build dates, I send out friendly reminders to the cdt-dev mailing list on what bugs are still open against that milestone. It's just a prod to get the developers to do something about them so that we have no open bugs on a milestone when we do the build. It's worked every time as we get the friendly but odd 'Zarro bugs found' message from my bugzilla query when we're ready.

RC2 was no different. This morning we had two left. Ken from Austin, Texas gave an update on his asking for feedback. I, from Ottawa, Canada gave some feedback for him to go ahead and fix it. Bala from London, England mentioned he had a patch ready for his and Mikhail from Russia replied saying he was looking at it. I'm confident we'll be ready in a couple of hours to fire off the build and get the RC out by the end of the day.

This happens regularly on the CDT and once in a while I stand back and think of what just happened (and I think I've probably blogged about this before too). We have a very effective development team working on the CDT, and the cdt-dev mailing list is the backbone of that collaboration. A lot of groups use different technologies such as instant messenger or IRC channels, but for us the cdt-dev mailing list works great. Bugzilla comes in at a close second. But then, we treat bugzillas as mini mailing lists anyway.

I think the biggest benefit of the cdt-dev list is that it's open to anyone. If you want to see what's happening with the CDT at a high level, that's the place to go. If you want more detail, then you really got to watch the bug reports and signing up to receive notifications on the cdt-*-inbox accounts are the best way to catch the train.

From my experience on the CDT, the most important tool you have to build a community is open communication, like mailing lists, forums, IRC. As your community grows, the only way to really talk to them all is via open communication, so it really forces you down that path and you end up doing it anyway. But in the early days, it was a hard habit to get into, especially when QNX was by itself, or even when I was at Rational and we started working with the QNX gang who were only a 5 minute drive from the Rational office. But open communication has really paid off in the end for the CDT and the reach of our cdt-dev mailing list impresses me time and time again.

Open Source Ripped?

Bjorn made me read this article by Howard Anderson. Well, he didn't make me, but it is a topic I'm very interested in as well: How does open source make sense in a commercial world?

It's actually a very interesting article. When I finished it, I had to remind myself of the title. In the end, I wasn't sure if he was for or against open source. His general thesis seems to be that open source is a tool used by small companies to gain market share against big companies. Yes, he's right. I've seen that. There are a lot of smaller companies shipping a world class IDE with their products making them more attractive. They leverage open source (i.e. Eclipse) to do it to lower costs since building a world class IDE is prohibitively expensive for most. I think it's a great business model. I guess he was just looking at it from the big proprietary company side.

There are a couple of areas where I have to disagree with Howard, though. He mentions open source is a "religion". Well in some circles, I guess open source participants see it that way. Certainly from the outside it looks like Richard Stallman is playing the part of a religious leader, and FSF is his church.

Howard also seems to believe that the people writing open source are doing so at night when they come home from their real jobs of working on proprietary software. But that's not what I do as an open source developer. Open source is my day job. The company I work for is one of those companies that is reaping the benefits of the open source business model, and is willing to invest in open source to help build a community where we can share the work with each other. And there are lots of developers like me from many companies. Open source is not a religion to us, but a business means to a business end.

So while it'll probably be impossible to shake the stigma of the open source "religion" from what we do, open source in the spirit of "co-opetition" (co-operating competitors) is a vital tool available to the commercial world. Some communities are set up for this to work well, like Eclipse, while others, not as much (and I won't name them unless over a beer :). But the ones that are, seem to be the ones that the big proprietary companies fear the most. Which means we must be on to something...

Bye-bye 32-bit Windows

I just read that Microsoft was putting an end to 32-bit support with their operating systems. I guess that shouldn't really be a surprise. It is a real struggle for device driver writers to support both and I think Windows 64-bit has really been hurt by that. The same is probably true for Linux as we don't see much demand for the 64-bit Linux version of the CDT either at 2.5% versus the 32-bit Linux at 20%.

Maybe this will finally trigger people to focus more on 64-bit and start writing programs with that in mind. The biggest change for C/C++ programmers is the size of pointers changes. I've seen a lot of code that assumes you can merrily cast a pointer into an int and back and everything is happy. One example of this is with Java native code where we like to stow away pointers in Java fields for later calls back into native-land. Well with 64-bit, while the size of pointers change, the size of int does not. I'm also hearing that there are different interpretations of the sizeof(long) where on some platforms it's 64-bits, but on others it stays as 32. Then there's long long (gcc) and int64 (msvc) which in the 32-bit world also means 64-bits.

Suffice it to say that the 64-bit world gets a little messy. We'll think back to the simple days of 32-bit with a sigh. But then, I think things will still be better than the now ancient 16-bit world was (now who's old enough to remember that?). Now they say that this will start after 2008, but given the length of time it took to get Vista out, people with 32-bit only machines shouldn't worry too much. They'll be ready for the dumpster by then anyway. And do we really need another operating system version beyond Vista? Microsoft hopes so, and I'm sure they'll use the 64-bit push as a marketing ploy to help you think so too.

Eclipse Wants You!

A few days ago I posted a blog entry worrying about the openness of the Eclipse Platform team. I thought it would generate a flood of comments saying I was wrong. And to a certain extent, I probably am wrong. But I think its a serious issue that people need to think about.

I guess the point I was really trying to make is that we as the Eclipse community outside of the Platform have depended too much on IBM/OTI's great contributions, to the point where we expect them to fix all of our problems. My experience with open source projects is that it just doesn't work that way.

Open source developers usually work on open source projects for a reason. They are trying to get something done for themselves, and really as a side affect they hope that others will find it useful as well and maybe come help out. Because open source software is free, I think people start to think it's more like a charity, but it isn't. And I think this is even a bigger factor with Eclipse since the vast majority of developers are employed to work on Eclipse projects. They respond to the community as much as they can, but at the end of the day, if their empoyer asks them to work on something else, that has to have priority.

So if you have a bug that isn't getting the attention you think it deserves, please think of the people at the other end. There's a good chance it's not because they think you're problem isn't important, but that they have probably been assigned work elsewhere and really just don't have the time. Do as much leg work as you can. Create a really good bug report that has a patch and a really good justification that shows you thought about the fix as much as the committer would have. Make it as easy for the committer to fix your problem as you can.

And if you find you really depend on certain functionality that isn't being provided or bugs that you really need fixed, and you do enough great patches, you can become a committer too. The more committers we get from different employers, the better off we'll all be. That kind of redundancy is important in open source and is something we've really learned to appreciate on the CDT project.

I love bugs from playstation.sony.com

I have no idea what this group is doing with the CDT. They've been on the outskirts of our community for quite a while and this bug was just marked VERIFIED to show they are still around. In fact, one of my first experiences as a CDT "dude" was at the first EclipseCon where someone from the Sony Playstation group stopped Sebastien, the first CDT project lead, and I and asked us for information about CDT extensibility. We thought it was pretty cool then, and it's still cool now.

Forget OO, C++ is a better C

I was working on bug 176353 trying to get interrupt signals sent to cygwin-based gdbs. Part of the magic for that in the CDT is something called the Spawner. Spawner subclasses Java's Process and adds the ability to send Unix type signals to them. It also implements some fancy I/O but that isn't supported on Windows. This class is a little Java and a lot JNI native code that interacts with the operating system to implement the signals as well as the other Process type things like starting up and waiting for completion of the process.

So, I guess this code was written originally with Visual Studio 6 many moons ago. However, continuing with my theme of using MinGW for Windows development, I've created makefiles to build the spawner DLL. What made this situation a little weird was that the original creators of the spawner were guys who didn't really know C++, so they did it in C. I always find it weird doing C in VS, but that's what these guys did. So when I wrote the makefile, I used make's default of running gcc on these files. Makes sense.

However, I was having trouble when I added a calls to a couple of Windows routines. I was getting undefined references at link time to the two calls I added. Weird, I didn't get any compile errors. When that happens it usually means I forgot to add the library. So I added it and still got the errors. What's going on here? Is it something broken in the MinGW port of these libraries? Was my code just getting tired and cranky?

Well for some reason, maybe I was getting tired and cranky, I wondered if it was because I was using C instead of C++. Part of my debugging technique is to start assuming the least likely cause and testing it out. This was really damn unlikely. But I tried it out. I changed the compiler to be g++ and ran it over the .c files. I thought it would have treated them as C files and nothing would really be different.

But to my surprise, g++ compiled the .c files as C++. And I got a ton of errors. And almost all the errors were for passing the wrong types to functions, especially since I was using UNICODE and not everything was really using wchar_t. Well, no wonder things weren't working. Of course the one thing it found were compile errors with the two functions I was calling. I had forgot to add the include to the header that defined them which specified the correct calling convention, WINAPI, which is why I was getting the link errors in C. But then C was happy to play along without having to see the declarations of the functions and made some bad assumptions. I wasted a good couple of hours trying to figure out what was wrong.

So, forget the object-oriented, templates, namespaces, operator overloading, and all the other cool features of C++, C++ at its core is just a much better C. It has proper type checking that helps you find those errors before link time, or worse, run time. It all feeds into helping you build better software faster, which is what this whole tools industry is all about. And if you've programmed in C++ for years and have to go back to C, don't forget to make that paradigm shift back to the 80's...

A lesson in release management

One of the first things I remember learning about managing projects came well before I even considered doing so. It came from the lore at the big telecomm company I worked at. They had an old telecomm switch that was doing quite well sales wise, but they had started working on their latest and greatest architecture that would pave the way to the future (which it did in the end). However, they were so excited, they started announcing it to their customers well before the release date.

Well, guess what happened. The customers got excited, too. They didn't want to buy the old switches anymore, they wanted the cool new one. Unfortunately, the dates ended up getting delayed and that spelled trouble since sales of the old switches were drying up. Lesson learned, though, and you notice a lot of companies holding back release information for that very reason.

Well, I think the same thing is happening to the CDT. For the first three months of this year, we've been hovering around the 65,000 downloads mark. It's not our biggest. That happened last October and November when we hit 85,000. But it was steady.

Well, I just did the numbers for April and found them at a disappointing 55,000. Maybe it's just a glitch. Maybe people are happy with getting the CDT from other places, like Linux distributions.

But it makes me wonder if this is a side-affect of CDT 4. We've been making a lot of noise about it, and we're finding that a lot of people are using the CDT 4 milestone builds, especially starting at M6 which just happened to be at the beginning of April. I haven't been counting the milestone builds in our figures.

We'll see how May's numbers are, but it would be interesting if we're seeing the pre-announcement affect in open source projects too. And I guess, why not?

gdb is a-calling

I've been working over the past week trying to get process interrupts working on Windows so that we can get suspend working again with gdb. The suspend call gets translated to a SIGINT signal raised on the process, which on Windows gets converted into a Control C event. For some reason it stopped working lately, mind you I don't remember ever actually trying it. It was fun, though. It got me back to my roots as a Windows developer and renewed my dislike for Hungarian notation.

Now that I got it working, though. I am finding that when you suspend gdb, both the cygwin version and the mingw version, that you end up in a pretty useless state. The stack trace has gdb totally confused. The addresses don't even look like real addresses. I thought for a moment that it was because of something I did in the CDT. But when I tried it from the command-line I got the same result. I did a Google search and looked at the cygwin and mingw forums and couldn't find anything useful. Maybe no one cares. Maybe it's my machine. Who knows (do you?).

At any rate, it has led me down a path to take a deeper look at how gdb is implemented and try and debug it myself. This is something that I've wanted to do for a while. We use gdb as our debugger in QNX Momentics so I may be able to help out our gdb guys too. It'll also give me something else to test the CDT with. I'm not sure we've tested Makefile projects enough in CDT 4 so this will give me a chance to do that. My focus will be on MinGW gdb and it would be cool to be able to contribute any fixes I come up with, or even a port to the latest gdb back to these guys.

AMD does big parallel, too.

Just read this at my favorite hardware site, Tom's Hardware (although I'm not so happy with their new look and feel). AMD is showing off their upcoming uber enthusiast platform which includes dual Agena FX quad-core processors and their (or ATI's, I'm still finding the new world order there weird) R600 multi-thread (320?) stream processor GPU based graphics card.

The demo was doing real-time face recognition on passers-by. It really shows that what used to be done as batch jobs can now be done in real-time with consumer massively parallel systems. We always wonder why we need more horse power - my web browser is working fine on my 3 year old system at home. But I think we're hitting a tipping point for a number of really cool applications that will change the way Joe Consumer looks at computers.

Which makes me even more excited to get into the raytracing hobby time project I've started to show off the CDT as an IDE for "really cool stuff". Now, to figure out how to get one of these machines :)

Need some faster floating point?

As I mentioned a couple of days ago, my new hobby is raytracing. I have a curious mind (in more ways than one :) and I always wonder how computers do the things they do. So when my sons got old enough to enjoy video games, we started down that path, and, of course, my curiosity lead me into the world of game programming. In order to make games look good, game programmers use a lot of tricks and play with optical allusions a lot. It's pretty complicated and every time I tried to get into it more, the real world would pull me out and make me deal with it.

In researching my recent blog entry on raytracing, I found a sweet elegance that I always look for in computer architectures. The algorithms are pretty straightforward, albeit pretty compute intensive, so the barrier to entry into this area seems low enough that I can work on it when I get a chance. It also looks to benefit immensely from parallel processing, another interest of mine, and will get me into that area as well. That, and the demos I saw showed some wicked shadow affects that really added to the realism of scenes, so it'll be cool to show off as a CDT demo as well (as opposed to the spinning polygons I use as an SDL/OpenGL demo right now that you may have seen at ESC).

My first step was to build a vector class that does math with 3D vectors, a critical component of all graphics programming. The sample I was looking at used regular C++ floating point arithmetic with a vector composed of a float for each of the three axis.

    class vector {
    public:
        vector(float _x, float _y, float _z)
            : x(_x), y(_y), z(_z) { }
        void operator +=(const vector & v) {
            x += v.x; y += v.y; z += v.z;
        }
    private:
        float x, y, z;
    };

Pretty basic. But this is the first example of an algorithm that can benefit from parallelism. Since I have a fairly new laptop, I wondered if I could leverage SSE, Streaming SIMD Extensions to implement this. I also wondered how well gcc and the MinGW variant I'm using handles SSE. So I gave it a try.

    class vector {
    public:
        vector(float _x, float _y, float _z) {
            float array[4] __attribute__((aligned(16)))
                = { x, y, z, 1 };
           xyz = _mm_load_ps(array);
        }
        void operator +=(const vector & v) {
            xyz += v.xyz;
       }
    private:
        __m128 xyz;
    }

The constructor is a bit more complicated. And with most things dealing with SSE, 16 byte alignment is critical for good performance. And looking at the generated assembly, I was pleased to see that gcc, after making sure I put the -msse2 option on the compile, worked hard at keeping the instances of __m128 aligned like that. The performance tests I ran with addition showed an O.K improvement in performance, especially as the number of math operations grew. But when I tried multiplication instead of addition, the performance gains were astronomical. Well worth the extra typing.

Now that I've got that under my belt, I can't wait to actually draw something...

They're great, but are they open enough?

This is something I normally only talk about after a few beers at EclipseCon. But I think I need to get it off my chest, especially after seeing a recent e-mail on the eclipse-dev list. I'll apologize in advance if I'm getting this wrong and feel free to correct me.

Now first of all, don't get me wrong, I am one of the biggest fans of the gang at IBM's OTI office (still more OTI than IBM they are) and a lot of them are good friends. The quality of the platform and the great extensibility it offers is what has made Eclipse what it is today. We'd all still be in the dark ages if it wasn't for their great work.

And, you know, I think they've come a long way as far as working in the open goes. When we started the CDT, it was really hard to know what they were doing and many times we were surprised by API and functionality changes in milestone builds that required us to scramble to fix up. I think on both sides, CDT and Platform, we've gone that extra mile to make sure we communicate better as committers. We've even received patches from the Platform team to make sure we didn't break when changes did occur.

The thing that has me concerned was highlighted in a mail that just came across on the eclipse-dev list from Kim Moir (sorry Kim, you're just the messenger). "I just talked to McQ regarding the plan. This is what he said... -Component leads can make rules more strict as they see fit..." And the mail goes on a bit longer to talk about the endgame rules for accepting changes.

Now it is my understanding that it's really up to the individual projects to decide what their development processes are, and I guess this is what the Eclipse project (or was it the Platform sub-project) has decided their processes to be. And you know, looking at the rules McQ has set out, they really are trying to give more power to the committers and my first was reaction was that this was a positive step in the right direction.

But, personally, in my role as the CDT Project Lead (which is also a sub-project which makes me a peer to McQ, who is IBM's Mike Wilson, BTW), the last thing I want to do is dictate to my committers what the endgame rules would be. Actually the last thing I want to do is dictate anything. Maybe it's just the way I am, but I feel the responsibility for the processes and guidelines falls with the committer group as a whole. If they don't agree, I'm actually powerless to stop them anyway so I'm really just a facilitator. Mind you, to make sure we have rules, I usually suggest something and if I get no feedback, which happens a lot, we assume everyone agrees. But in the end it should be the committers that decide.

Obviously the Eclipse team is set up differently. A lot of it is historical and due to the organization of the team, both as an Eclipse PMC as an an organization at IBM. But I would really like to see the Eclipse team open up their processes and decision making more and be a bit more transparent.

CDT 4.0 RC0 Now Available

It's been a crazy week or so as we worked feverishly to get our RC0 build together. This build marks our feature completion date where all new features have been implemented. We did this a bit earlier than the Platform (which is the end of this week, we actually locked down at the end of last week) to allow us more time for testing and bug fixing.

This is by far the biggest CDT release we've done since the first one as far as the number of new features go which, of course, introduced a lot of risk as far as introducing bugs as well. We also need to be careful about backwards compatibility too so that users can use their old workspaces with the new CDT. The best way to help us out right now is to download the RC0 build and give it a try and raise bugs on any problems you find. Visit the CDT web page to find out how.

I've signed up to do a webinar mid July to show off the new CDT. It'll probably take that long to learn it all, too...

Ubuntu + Dell, Significant? You betcha.

I just picked this up from ZDNet, another of my favorite sites for picking up industry news. Dell has signed a deal with Canonical, the makers of Ubuntu Linux, to provide support for Ubuntu on it's consumer PC line. This isn't the first time Dell has tried to enter the Linux fray, but word has it this time will be different.

I think this is actually a huge deal for Linux on the desktop and there are a couple of reasons why I think this will fly. First of all, Ubuntu is hugely popular in the Linux world, especially with people who are, not necessarily zealots, but really want to get into Linux with low overhead. And from what I can see, and apparently what Dell has found out, that's a lot of people. I'm probably in that category and a lot of the articles I've read about Ubuntu are written by people in that category too.

The second big reason why this is different is the support deal with Canonical. Essentially, you can use Ubuntu on a Dell system without worrying about whether the drivers will work with all the hardware you're buying. This has always been my major stumbling block with Linux (other than the crappy fonts :). I had Linux running on a Dell laptop five years ago and struggled to get things like suspend and the graphics display working correctly. This should address that. And the deal will ensure Canonical has a vested interest in making sure it works.

And vested they are. This has to be huge for Canonical. I always wondered how they intended on making money given all the investment they've made in Ubuntu to make it consumer friendly just to give it away for free. This is it, and I shouldn't be surprised. I've always thought that you can't make money on Linux by selling it in a box. You make money selling services or other products that leverage Linux, just as we do with Eclipse.

This has clearly been Canonical's strategy and they've started down a road where it'll pay off. Making Ubuntu consumer friendly has actually created a market for Linux PCs. Dell is just responding to the desires of that market. Mind you, it's not for the weak at heart and it really was a huge gamble, but luckily for Canonical, their CEO Mark Shuttleworth came with deep pockets to make it happen. But happening it is.

Zero to breakpoint in 10 seconds

We're getting close to our first release candidate, RC0, of CDT 4. One of the key objectives for CDT 4 was to simplify the new user experience. Thanks to some new features in the CDT like new project templates and in the Platform like contextual launching, we can now get you from a blank workspace to your first breakpoint in 10 seconds (faster if you type faster :).

Here's how:

1. File->New Project.
2. In the New Project Wizard, select C++ -> C++ project, click Next.
3. Select the Executable project type, clicking the + and selecting one of the Hello World templates and if more than one Toolchain is listed, pick one. Type in the Project name and click Next.
4. Fill in the form with information the template needs to generate your Hello World app, click Finish (or next to play with the build configurations, but the defaults are fine).
5. Click the Build button in the toolbar, wait for the build to finish.
6. Click the Debug button (one click debugging!) and accept the switch to the Debug perspective.

And you're done. The debugger hits the default breakpoint on main and you are set to go.

Give that we're not done yet with CDT 4, there is a caveat at the moment. The one click debugging only works with the MinGW integration. To set that up, simply run the MinGW compiler and gdb installers from www.mingw.org, or have MinGW installed in C:\MinGW. The CDT will automatically pick up the install location. We'll get the other ones (Cygwin, Linux, etc) working as by the time CDT 4 ships at the end of June.

You can't do that with Java

O.K., it's not that I hate Java. It's more, I run into times where I need to do something that I can't do in Java. Some of my favorite C++ features include operator overloading, crazy C++ templates like the ones you get with the Standard Template Library, and flexible memory management, not to mention the great job that C++ compilers do at optimizing all this magic into fast object code.

It's really the flexible memory management that I miss the most. Allocating memory out of the heap is expensive. That's pretty common knowledge. With Java, every Object gets allocated out of the heap. I remember the first time I ran into this early in my Java career. I had this little class that had a couple of fields that I used to store temporary information that got passed down to some other methods. I couldn't believe that I had to allocate it out of the heap. With C++ it had become second nature to declare an object and have it automatically allocated on the stack. And when the function I declared it in finished, whether due to a return or an exception, the destructor for the object gets called so you can clean up any mess. And using C++ pass by reference, I was able to do all that with minimal typing (my other mantra - I hate typing, especially with my sore finger right now).

The other cool feature of C++ is the ability to override the operator new to do your own memory management. That way you can allocate all instances of a class in a special memory pool. Or pass parameters to operator new to do anything you want. I've run into this as I've started looking closer at ray tracing algorithms (my new hobby). One of the speed ups they mentioned was allocating all contents of one of the structures in a given memory region to help leverage CPU data caches in an effort to squeeze every ounce of performance out of the machine as they can (which is really needed to get any resemblance of real-time ray tracing on today's machines). Now that's something you can't do in Java, at least not without some native code, which then isn't really Java.

Java has it's place and I love it for writing Eclipse plug-ins. But despite bold predictions by the IT industry, C/C++ will never go away as long as we continue to throw as much processing at these fancy new CPUs and GPUs as we are. For some reason, our appetite for speed continues to outstrip all that performance that the silicon vendors are working so hard to put in our hands.

Bug 160012 - The CDT Team at Work

There have been massive changes in the CDT's build system and the CDT's New Project wizard. This is all great work that the gang at Intel in Russia have been doing to clean up the long standing weirdness of making the user pick between the two competing build systems in the CDT: standard vs. managed. The user still picks, but it's much more subtle. Along with this, other components of the CDT can start getting more information about what the build system is doing so that we can do things like pick the default debugger based on the active tool chain.

Another component that we've been eagerly waiting for was the new project template support proposed by the gang at Symbian in London. This allows us to gather some information in the New Project wizard and generate source files and build settings based on a template. Now this proposal actually occurred pretty much in parallel with Intel's build system work, and given that, they didn't really take each other into account.

Well, once the build system was in place with M6 at the beginning of April, it was time to mesh them together. I am thrilled with how this has worked out. It was not an easy task as we had to undo assumptions that had been made. Not to mention the time frame was short with feature freeze being this weekend. But it was great to see how well the two groups worked together along with the odd input from us over here in North America. To see for yourself, check out the bug report for 160012 where the discussions happened. At last count, there were a 110 comments on it, some of them pretty lengthy.

I've done "around the world" development in a commercial setting but never at this level and never this successful. Every morning, I wake up and sift through a pile of bug updates that my friends in Europe and India have sent out. We then get a few hours where we're actually at work at the same time and the bug traffic is pretty heavy in the morning. But then tails off towards the end of the day. You always have to think about what time it is elsewhere (even though someone may still be working late - go to bed Mikhail S! :).

I think that it's a sign of a successful open source project when you have contributors from around the world with diverse needs but all fighting through the time differences to work together for the common good of the project. This is really the main reason I love working on the CDT. Helping create the world's best and hopefully soon, most popular, C/C++ IDE is pretty good too...

Subclipse withdraws? Someone tell Bjorn...

I just read Eugene's post about Subclipse withdrawing their project proposal from Eclipse. He points at a well written statement by Mark Phippard explaining why. I guess they have their reasons. But it would have been nice to see something posted on their project proposal newsgroup about this.

I got involved in this Subclipse versus Subversive debate when we were discussing moving to subversion on the cdt-dev mailing list. In the discussion of Eclipse clients, I mentioned that when I tried them I preferred Subversive. And actually, with some recent trials I did for work, I still prefer Subversive. Mark made a somewhat nasty reply to my comments. He made me feel bad for going against Subclipse. And in my searches for other peoples opinions, I more often than not saw him comment the same in support of Subclipse, and I'm sure he'll comment here. I certainly commend him for standing up for his project and I sometimes do the same for the CDT, but I try to be more polite about it.

So, I guess that means Subversive wins at Eclipse. From my seat, and others have a right to disagree, but I am talking about my seat not theirs, having one project is a good thing. In reality, I don't care who wins, but I do care that we produce a good subversion client for Eclipse and I don't see how two competing projects helps anyone. They do, or are intended to do, the exact same thing. In fact, they almost look identical. I almost had to check the features list to make sure which one I had.

But I think we have a long way to go to get subversion client support up to the same capabilities as CVS. Having one project that we can all work on will help make that happen. My intention is recommend moving the CDT to subversion over the summer, but only if the client meets our needs. That means we on the CDT have a vested interest in making that happen. And I know how to make patches and attach them to bugzillas, so I can't wait to get some time on it. And I will spend time on Subversive because it is an Eclipse project. So will others in the Eclipse community, because of that sense of community that is Eclipse. That's something I think the Subclipse guys forgot to take into consideration.

Ray tracing the future of Gaming

I love the Inq (the Inquirer). I'm not sure who their writers are but I usually seem to find out about new things there first. Last night I read this cool article about work going to support real-time ray tracing to render 3D graphics. After a Google search, I found the real article that this guy was referring to here on PC Perspective. It's based on research happening at Saarland University in Germany where they've developed an API called OpenRT, which of course is similar to OpenGL. They also have a prototype so that you can try it out.

I remember back in university almost 20 years ago now a couple of buddies of mine that were doing ray tracing for their graphics class. The images they produced were pretty cool and realistic for the time. But it took overnight to generate one frame. Mind you that was on good old Sun 3's, but you certainly wouldn't think of doing this in real time, even today.

The ray tracing demos you'll find in PC Perspective article and at the OpenRT site are amazing, though. From what I've read, doing shadows in current technologies like OpenGL or DirectX is very difficult and game developers almost always take short cuts, which leaves the scenes a bit unreal. But with ray tracing, it appears to be much easier and the scenes appear much more believable, which is the end goal for all 3D animation.

What's changing is the march towards many multi-core CPUs by Intel and AMD. One of the big advantages of ray tracing it the scalability of the algorithms to parallel threads. Each pixel is determined independently of the other pixels. All you need to do is partition the screen to the cores and you get almost linear scalability in performance.

Now, mind you, the demos I saw, especially the one from the OpenRT site, used a lot of cores, mainly 32 and one was even at 48. But I imagine there's opportunities for improvement given this early stage, and even for some hardware acceleration for parts of the algorithm. But if you were wondering what you were going to do with that quad-core furnace of a chip, here's one idea. And it's pretty interesting to see that Intel caught on to this idea a couple of years ago.

Ubuntu 7.04, has Linux's time come?

As I've ranted on my blog in the past, I'd love to use Linux, but I still find the user experience, especially the look and feel, to be a long way from cleanliness and professionalism of Windows XP. And from I've seen of Vista, it's nowhere even close.

But I have a buddy, Rodney, at work who swears by Linux, especially Ubuntu. So much so, he has it installed on his laptop as his main work environment. Of course, I keep bugging him about how ugly I think it all looks, and he fires back with the cool 3D/alpha blending environment of the latest experimental extensions to X and Gnome. It's all good fun, but at the end of the day I'm happy to walk back to my desk and sit at my Windows machine.

I've been playing with the beta release of VMware Workstation 6. I'm a big fan of vmware from way back and every new major release seems to bring something new that makes me like it even more. This release brings a new UI that makes running vmware full screen a lot easier to use and more Windows friendly. The performance seems to be a bit better too, but then lately it's been pretty good anyway. I use vmware to run the x86 target of our Neutrino RTOS for testing with a target. And, of course, I use it to experiment and test with a Linux host.

So to get up and running on the vmware beta, I downloaded the latest Ubuntu 7.04 release. Rodney's been raving about it so I had to give it a try. The Ubuntu install experience is the best I've seen with any Linux distro. You boot up into a full Linux/Gnome environment off the CD, and then double click the Install icon to launch the installer. Just coming up cleanly off the Install CD gives you confidence the real thing is going to work. After that, it's just a matter if making sure all the packages you need are there. This is still a pretty harsh task that's not intended for the weak. But the package manager helps install those things quickly (once you properly guess at the names of the packages you need, like sun-java6-bin ?).

The look is still not up to Windows standards, but it seems to get better every time I try a new distro. Maybe I'm just getting more open to the idea of using Linux. Certainly if you're an engineer who knows a lot about *nix already, like taking advantage of Linux's features for embedded development such as mounting files as disks, then I think you'd be happy with the latest Ubuntu. But if you're my Mom, sorry Mom, stay on Windows. At least for now...

Fun with JTAG

As I mentioned previously, I am working on adding an officially supported CDT integration with gdb that can be used with JTAG hardware debugging devices. As a quick primer, JTAG devices allow you to have full control over the CPU and memory on a embedded computing board using a special connector that is now pretty much standard on all such boards. With debugging support, that means you can read and write memory and any memory mapped registers, read and write CPU registers, and set breakpoints. A lot of the JTAG vendors are starting to support integration of their devices with gdb as a front end to give developers a familiar interface, and for us on the CDT, allows us to leverage almost all of our existing gdb integration to provide an Eclipse UI interface.

JTAG debugging does have limitations. It's not overly fast, especially when compared to native debugging. Stepping through code takes around a second in the setup I have. And with most configurations, the JTAG debugger hardly works at all once virtual memory is turned on in the CPU, making process level debugging, as you normally do with OS's like Windows, Linux, QNX, etc, impossible. The biggest value of JTAG in the past has been for the initialization code that sets up the board and starts the operating system kernel. But that is starting to change though as JTAG debugger makers are figuring out how to do the virtual to physical and back translation and adding OS awareness in the debugger itself allowing for the full debug experience.

So I have the integration working now. With permission, I borrowed a lot of ideas from the Zylin Embedded CDT plug-in. Again, my hopes are to bring those guys and their customers on board to avoid the need for forking the CDT. It was pretty cool when I did my first debugger launch, and everything just worked. This is really the beauty of Eclipse and the CDT and the focus on extensibility, that makes adding new features a breeze.

Below is a picture of my set-up. I have a little TI OMAP board hooked up to a Abatron BDI2000 JTAG device hooked up to a network hub that eventually hooks up to my laptop. You can't see the screen, but trust me :), the CDT has reset the board, loaded in an image, started it up, and hit the breakpoint I had set. And you get all the CDT goodness like the variables, registers, and disassembly view. Tres cool!

My next step is to hook up qemu, the board emulator, with it's built in gdb remote stub, which works just like a JTAG device, to this whole thing so you can try it out without having to fork out money for real hardware...

F3, the CDT Wunder-Key

Unfortunately, I do most of my programming in Java (sorry Java-lovers, but I hate Java). But the JDT really makes it a breeze to write code for the CDT plug-ins. My favorite feature is F3, or Open Declaration. Whenever I'm investigating code, I like to go visit the implementation of some unknown method to see what it does. F3 lets me jump from class to class and get a quick overview of the system I'm trying to program against.

Well, I'm trying to do the same thing with the CDT. Before CDT 4, Open Declaration tended to be slow since it did a complete parse of the file you're viewing and all the files that it includes. With CDT 4, we're now only parsing the file and using the CDT index to get all the other declarations needed for that file.

As well, F3 tended to be hit and miss on whether it actually found anything. A lot of that had to do with the indexer's need for build information that is often hard to provide. Also a lot had to do with information that we hadn't collected yet, C++ template information for example.

With CDT 4, F3 promises to be a whole lot better. I'll be spending a bunch of my time as we start to wrap up CDT 4 development on making sure it finds as many definitions as it can so that it can be as useful to CDT users as it is for JDT users. The one I just added that made me think of blogging about it is #include statements. Wonder what's in that include file you #include'ing? Well move the cursor to the statement and hit F3. Bingo, there's the include file (at least as long as the index knows where it is). I look forward to adding more cool features like that.

Microsoft is making my Linux fonts ugly

I found this article mentioned on Slashdot. I've stated in the past that the main reason I don't like using Linux as my main development environment is that I find the fonts hard to read. My eyes are horrible, especially after long stints writing code. Windows for some reason just looks so much better, especially on LCD screens.

Well, the reason Windows looks better is their ClearType technology. After reading the article I tried turning it off, and sure enough, Windows sucked too.

Apparently the fuss over ClearType and FreeType, Linux's font technology, has to deal with patents that Microsoft holds on the techniques behind ClearType. With all the anti-Novell/Microsoft clauses in the GPLv3 dealing with patent protection and the essential prohibition on it, I've lost all hope. Despite what Richard Stallman may wish, Microsoft will likely never extend patent protection on ClearType to all of the Linux community which means they will have to pick the other route, i.e. to none of the community.

Which is really too bad. As much as I thank the FSF and GPL for giving us all those great GNU tools, I'm afraid that their conviction to ideals will also stunt the growth of open source, and especially Linux. The FSF may hate software patents, but they are a fact of life. And if the two worlds can't mix, then the poor user pays a price, one way or the other.

Oh, I hate Cygwin

I usually try not to blog while I'm angry. But I'm in the middle of trying to get the Firefox source set up in the CDT on my laptop, which, of course, is running Windows (XP mind you, still not brave enough to try Vista). And it has been a struggle.

First of all, it was a bit tricky to set up the build environment. I'm using Cygwin since Firefox would really rather be built on Linux, or in a convoluted environment that involves using Cygwin but the Microsoft tool chain, which the CDT doesn't really have support for yet. Luckily I found a web page that showed how to set it up to use the cygwin compilers. It's a little out of date but a few tweaks and I was able to get Firefox built.

Now I'm trying to get the build output into the CDT so that our cool Scanner Discovery feature can parse it and set up the include paths and symbols for the indexer. That's been tricky since the Firefox build wrapped all calls to gcc with a wrapper script which deals with converting paths, I guess. I've got that fixed, but now the source file paths used by Firefox uses the cygwin paths, i.e. /cygdrive/..., which our build output parser doesn't understand. So I'll have to introduce a new parser that does the cygpath conversion.

We've had a lot of bug reports lately on cygwin. Most of them have to do with the cygwin developers deciding not to support Windows path names any more, you know, the good ol' C:\blah. I guess understand their reasoning. Cygwin is meant to be a Linux emulation environment on Windows. I don't think that was their original intention since Cygwin actually predates all this Linux popularity, but that's what it has turned into (and even their web site now says so).

The issue for the CDT is that it isn't running under the cygwin environment. It can only deal with Windows paths. So whenever we see a cygwin path, we need to convert it. Not only that, when generating makefiles for cygwin make, we need to convert Windows paths to cygwin paths. Now, we can't do that for everything on Windows since tools like MinGW gcc does use Windows paths. It's pure evil (well maybe not that evil...)

So what this really means is that supporting Cygwin with the CDT is becoming a lot of work. This is one of the reasons I want to start promoting MinGW, a much more Windows friendly port of the gnu tool chain, as the gnu environment of choice on Windows. The problem, though, is that cygwin is very popular and easier to install and seems to have much more momentum than mingw. So we will need to continue to support both. But I'd sure like to see more of that momentum shift to MinGW. Which, in open source, that means I need to do more to help them.

Ada Project at Eclipse

In all the excitement of the conferences I've been to in the last couple of months, I've also been helping Aonix with their proposed Ada project at Eclipse, called ADT (of course). Ada, you say!?!?! Yes, Ada. Despite reports to the contrary, Ada is still being used in the industry, especially in the military and aerospace and especially in safety critical systems. And with the introduction of Ada 2005, the hope is to spread all that Ada goodness to the rest of the embedded industry.

So, why Ada? Funny enough, googling "Why Ada" returns a number of articles on that subject. This is something the Ada community has struggled with for quite a while. But from my point of view, it all comes down to writing safe code. Ada is a huge language and much of it has to do with making as hard as possible to write bad code. Code like "while (i++) *i = 42;" isn't possible. This is particularly of importance with embedded systems where flaws like this could be costly, very costly.

But Ada has also suffered from the stigma of bureaucracy. The fact that it was designed by committee for the U.S. Department of Defense (who has since dropped it as a requirement) doesn't help. Also the reference manual reads like a legal document, and there are few books out there to explain it to us regular Joe's.

From my perspective, though, I think the biggest problem with Ada has been the lack of an open source community around it. GNAT, the GNU Ada compiler has been around a while, but it isn't very straighforward to use with it's own build system. But gdb works fine with it for debug.

So, in the past, I've tried to drum up interest in Ada as an extension of the CDT at Eclipse. Aonix and AdaCore are two vendors that have done it commercially. I was thrilled when Aonix approached me about starting up such a project. And I really hope that other Ada tool vendors join in on the fun.

High quality open source solutions available to everyone really helps with growth at the grassroots level. Many universities and colleges are using the CDT for their coursework because it is freely available. The same could happen with the Eclipse ADT. But to ensure high quality, a community has to form around it to share in the work. Most Eclipse projects benefit from the spirit of "co-opetition", competitors working together for the common good of the market. This could grow the Ada market which would mean they could sell more professional tools and support. Such is the economic wheel that keeps Eclipse going and we'll see if the Ada market can do the same.

Final Thoughts on ESC

Doug Gaff raises a very good point about ESC. I think changes are in the wind. What they are, anyone and everyone can guess. There were certainly a lot of boards and devices on display, and all the big RTOS vendors were there. There were even a couple of tools vendors there. All the booths looked great and were staffed by good people. There was enough crazy stuff going on to build on the hype (get your picture taken with the Intel chopper, count the Marilyn Monroes - I still don't know what they were trying to sell other than awful blue bags). Being my first show, it all felt pretty cool, but pretty shallow.

The thing I found disappointing, though, was the profile and location of some of the companies I think are the innovators at the moment. Timesys was one. They were involved in the CDT in the early days. They've since changed their business model from a straight embedded Linux distribution vendor to a service organization for do-it-yourself'ers. I thought it was innovative, but their booth was tiny and really tucked away and hard to find. (Mind you not much smaller and tucked away than the Eclipse booth - just kidding Ian :)

The other was CodeSourcery. I ran across these guys on the Web when investigating gnu-based ARM toolchains to play with qemu. The more I ran into their name, the more interested in their business model I became. They contribute a lot to open source, especially with the gnu toolchain (gcc, gdb). I also see then contributing to qemu for the ARM and other target support. I don't know a lot about their internal structure, but they must be doing well. Their booth was still outside the main hall but I noticed every time I walked by they were talking to potential customers. Unfortunately, I didn't have time to drop by myself to find out more. On the surface, though, they really look like a great example of an "open source business". And, they call their developers "Sourcerers". How cool is that...

But as I've stated in the past. I really think the model for all embedded companies has to, and in a lot of cases already is moving to a more service oriented approach. As Doug Gaff mentioned, the problems embedded developers are trying to solve is getting harder and harder. It is just plain too hard and expensive to hire full-time experts in every area you need. We'll see over the years if this turns out to be true and we start finding those companies hidden outside the main hall inside with their own smiling Marilyn Monroes to scan your badge.

ESC is over, bring on gdb JTAG.

Well, my talk went pretty well. I think i had over 75 people there. I stopped counting at 60 and people were still filing in the door. You have to get your badge scanned to enter the room which took some time. But the audience was very attentive and had some great questions and feature ideas. Unfortunately I didn't capture them all, so if you were there and had an idea, please raise a bug for it :).

One thing a few people asked me about this week was the Zylin Embedded CDT plug-in that a few vendors are starting to use in their commercial products. It essentially supports debugging using JTAG devices that use gdb as their debug engine. Those who have been around for a while have heard of Øyvind from Zylin who has been trying to get his changes into the CDT proper. For various reasons, those changes have been rejected.

But I think it's time to take a serious look at this and see if we can address his issues while maintaining current functionality. Having forks of the CDT is bad for the community, and is really bad for those vendors that have to use them because it has the potential to limit further integration possibilities, especially if APIs are changed. The objective should be to make everyone happy. Sometimes that's not possible, but in this case I think it is. And I'm making it my mission for the rest of CDT 4 to do my best to make that happen.

Getting ready to go on stage

I got to the room early where I'm giving my presentation on the CDT for Embedded. It's going to be a busy day again so I'll just quickly talk about yesterday. One of the cool things I've found out about this area of San Jose is that there is free wireless everywhere as long as you don't mind a banner on top of every page. And it's a bit slow, but it's letting me do this, so I don't mind.

I had a couple of great conversations yesterday that will hopefully lead to CDT contributions. I'm still pleasantly surprised at all the people who drop by the booth who already use Eclipse or who know that they really need to be using it. That gives me a great feeling that the grassroots around Eclipse for embedded is already taking off. And I guess, when you look at the other choices, it probably should be of no surprise.

I also did a walk about yesterday trying to find booths that had Eclipse running. There were the regulars which had it front and center. And there were a few surprises. Ian found a few. And I was found a couple of more, including a surprise at the MIPS booth. This is happening to me quite often in the last year or so and it is really cool to see. It would be nice if these guys felt comfortable enough to let us know when they are using Eclipse. It is open source so they don't have to. But part of my job is to promote the CDT to get as many contributions as I can and I'd really like to show the momentum of the CDT in the industry. Which is pretty clear from my experience here at ESC, this it is growing quite well in the embedded space.

ESC Day 1

That was an interesting day. In case you missed it, I'm down at the Embedded Systems Conference in San Jose, California. Today, Doug Gaff, Ian Skerret and I met a number of press people. There's one who's already published the results of our meeting with him. It's interesting the diversity of knowledge these guys have about Eclipse. But these are the guys you need to talk to to get the message out and I think we did a good job of getting them all one equal footing.

I spent a few hours at the Eclipse booth. As I mentioned in my last entry, I was pretty disappointed in the size and the location of the booth. But I was pleasantly surprised at the number of people who walked by and said they were starting to use Eclipse. I think by my account, most people who did walk by had something to say, which was pretty cool.

A person from a trade journal from India stopped by and I had a nice talk with her. She mentioned that all of Bangalore was using the CDT. That's a lot of software developers! I also had a couple of people who were wondering about extending the CDT for the specific needs of their projects. And I had a guy walk by and gave us a woo-hoo. I guess he likes what we're doing.

Talking with Ian, it is clear we need to find a good way to get publicize everything and start building a community around embedded. Doug Gaff and the people participating in DSDP have done a great job at getting the big players in the market involved. But I'm a grassroots kind of guy and I think there is also opportunity to get the word out to the average Joe.

My idea is to revive a tutorial someone had written about using the CDT to build a tiny app for an ARM7 chip that had no operating system and just ran on the bare metal. I'm doing work in CDT 4 to support gdb debugging with JTAG so it's perfect timing. And this was affirmed by a couple of conversations I had with guys looking for exactly that. So I'll work with Ian to get a webinar or something together, so stay tuned...

Hello from ESC Silicon Valley

This is my first ESC (Embedded Systems Conference) since my days at ObjecTime, umpteen years ago and that was in Boston and was a smaller show. I just did a walk around to get myself oriented with the San Jose Convention Center and quickly went through the exhibit hall. It's quite a site. This is really my first "ultra-hype" trade show and the booths here are amazing. Some are set up as theatres with lots of chairs, some have multi-storing temporary structures (you won't catch me going up there!). It's pretty cool and I can't wait to see the action when it opens later today.

Now, I need to thank Ian Skerret for setting up the Eclipse booth and to those that helped set up meetings with members of the trade press. And no disrespect to the Eclipse Foundation, but the Eclipse booth is pretty tiny. We talked about this at EclipseCon in the Embedded panel and I again put forth the challenge...

Eclipse is an important force in the embedded industry. Unless my Google Alerts are making up stuff, more and more embedded vendors are taking advantage of the power and flexibility of Eclipse as the foundation of their tools offerings. But Eclipse is successful because of the vendors and developers that contribute to it. Eclipse had a huge marketing impact in the Enterprise market because of the huge vendors that promoted it.

We need to see the same thing in the embedded space. We need to take that ground swell in the grassroots and promote Eclipse as the phenomenon that it is. This will go a long way towards encouraging those that are using Eclipse to contribute back and help convince more vendors to join the community. Those of us who got in early, did so because we saw the vision of what Eclipse can do in the embedded space. It's time to start shouting about it!

Get your swim suit on

A colleague pointed this out to me and it really made my day. With the stress of getting CDT 4.0 M6 out the door while getting ready for a week at ESC, you need to have a laugh to help keep you going.

Here's what you do. Go to Google maps and click get directions. Start from somewhere in North America and end somewhere in Europe.

The directions will guide you to Long Wharf in Boston and then instruct you to "Swim across the Atlantic Ocean", which apparently is 5572 kilometers. You then get out of the water in La Havre, France, and continue on your way.

I love the sense of humour software developers have. I can imagine the design discussions around putting this feature in. I'm sure they're still laughing about it, just like we are now. Now, what can we do with the CDT, hmm...

GPLv3, freedom at what price?

I've got my fresh copy of the GPLv3 Third Discussion Draft Rationale and started to try to dissect it. It's a pretty intense legal document as it tries to build up a GPL that has no holes. I'll have to read it on the plane down to San Jose for next week's ESC to help me fall asleep.

But one thing that has struck me as I dig into it and read others' comments on it, is that the view of freedom put forth by Richard Stallman and the FSF may actually turn detrimental to the community instead of helping it.

There is no doubt that open source is successful today because of the GPL. The freedom for users to be able to download, possibly modify, build, and run their code is really what got a lot of developers involved in open source to begin with. And that momentum has grown and the qualify of the open source code has grown to become a serious force in the industry. But that all started in the day when most open source users were developers.

The issue I have is that the FSF does not differentiate between user and developer. But when it comes to critical systems, especially in the embedded space, I'm not sure users even want that freedom. While having GPL code in my TV is cool and I'm glad the manufacturer was able to take advantage of it, and theoretically ended up with good code cheaper than if they would have licensed something. And hopefully the developers contributed back to those open source projects to make them even better. But the last thing I want to do as a user is change that code, even though I know I could.

So while the GPLv3 tries everything in its power to ensure that the user can modify the code, my fear is that it will handcuff the developers working on this code. It's not free to them to make sure the user gets everything they need to do the modification. The manufactures can't charge for the code, but you can be sure device prices will go up if they need to ship an SDK and hardware with it.

GPL in embedded devices has always been a tricky subject, and GPLv3 seems to make it even trickier. I'm not sure the FSF is willing to listen to the concerns of embedded manufacturers, but I sure hope someone is making noise.

CDT 4, now 5 MB better

I was just checking last night's CDT build to make sure our source feature got generated correctly (it did, thanks Andrew!). One thing I noticed was that our run time downloads are now around 17 MBs. CDT 3.1.2 was around 12 MB. That's 5 MB more CDT! We haven't added any new plug-ins so it's all enhanced feature content and improved internal data structures that's contributing to it.

Our M6 is coming along nicely and should be released late Friday or over the weekend. It'll be a part of Europa M6 which I think is still a week or so away. The biggest thing you'll notice is a new New Project wizard that merges standard and managed make into a single experience. Standard make projects are now "Makefile" project types, something VS users will be familiar with, and allows us to associate tool chains with standard make projects to make it easier to set them up for indexing.

For our Windows users, you'll see a new MinGW tool chain which uses various tricks to find your MinGW installation so there is no more need to add this to your path. Also, it uses the CDT's internal builder to call the build tools directly so there is no need to install MSYS or Cygwin to get make and the other command line tools. This will feed into the EasyEclipse MinGW distribution that I have promised to deliver.

Aside from that, the index is more complete and has more information in it to drive all our parser based features. This also feeds into better performance for content assist and open declaration since we no longer do a full parse of the file and all it's includes. It's not often you get 'oo's and 'ah's over a content assist, but I did at our demo at EclipseCon.

I'll provide more details and a pointer to the New and Noteworthy on the CDT wiki once we wrap this thing up and get it out to you.

"Conflicts With"

So you know in Bugzilla you can mark a bug as a duplicate of another bug. I have no problem with people raising new bugs even if they end up being duplicates of another bug. That way we make sure we get them all.

But I've often run into cases where I'd like to mark that a bug "conflicts with" another bug. If you mark one of them FIXED, the other one automatically gets marked INVALID.

The tough part, though, is picking the one to fix...

Ever hear of EFS?

We're at a critical juncture in the evolution of the CDT. Everything has been going so good lately, with tons of indexer and build improvements, that I guess we were due for a bit of a crisis. The issue we are grappling with is how to use the CDT with projects that have source files sitting on a remote server, possibly across the Internet somewhere. It's a tough environment that will definitely stretch Eclipse and the CDT if we are ever successful in making it work well.

The solution I've been hoping to push is to use EFS to access those files and, in theory, the CDT should just work. What's EFS? Well, it's the Eclipse File System, which provides a layer of abstraction at the same level as java.io.File, but allows different implementations. Using the right IResource APIs and URIs instead of IPath, and everything should work. It seems designed to meet our needs.

I managed to build a FileSystem using FTPClient from Apache's common.net. My test is to run it against the ftp server on the CDT build machine at Eclipse. Saving files is a bit slow, and I guess that's to be expected and we can probably introduce some caching and worker threads to at least get it off the UI thread. I imagine RSE does this already and will need to take a closer look at their EFS implementation to see if it would work better.

The bigger issue is the number of instances where plug-ins are assuming IPath works and IResource.getLocation() works. And I kind of knew that going into it, that we'd have some work in the CDT to solve this, and that proved to be true. So I tried a General project and text files worked fine. So I though I'd try something more involved. How about a Ant build.xml file. Well, there again, massive NPEs because the build.xml editor and the AntModel assumes IPath to work. Not only that, they use java.io.File to get at the file. Ouch.

It is pretty clear that little work has been done to try out a real EFS implementation as backing for IResources, or at least not enough work. My feeling is that this remote project idea must transcend the CDT, and we've even thought about starting a new Eclipse project to focus on it. It's clear there is a lot of work to do and we need to start organizing if we really want to make EFS remote projects work.

Now that's Embedded!

I ran across this really cool project this morning. A guy built a little device that recorded light patterns, i.e. bright/dark, and then replayed them on an LED. He used a tiny Atmel micro controller and programmed it in C using, you guessed it, the CDT, with the WinAVR cross compiler toolchain.

It's projects like this that make working on the CDT so cool. We have this little device with 1K flash and 64 bytes, yes bytes!, of RAM and people can use the CDT to program it. Then we have the big iron supercomputers that the Parallel Tools Project gang work with, and you can use the CDT to program it as well. And, of course everything in between, from mobile multi-media devices, to Linux server applications, to the PC desktop, to gaming consoles. But this little project really made my day!

Mozilla Desktop Environment?

Snooping around the Internet news sites, I ran across a discussion on the Mozilla planning list about the potential of a desktop environment built out of Mozilla and their GUI language XUL (O.K. I found it through the trusty Slashdot, I'm sure you did too :).

I always thought the idea of building a desktop around a browser was intriguing and to me this is the promise of mobile tablet-like devices. But simply using Mozilla and it's plug-in architecture might be overkill. I think simply using the browser function with JavaScript would do the trick. Imagine, you fire up your tablet, the operating system boots and the window system comes up, and it could be any operating and windowing system, and the only thing that gets launched is the Browser. Small and fast. Perfect for mobile.

You'd probably have some dynamic HTML pages locally on the device to do desktop-y things like managing files, writing documents, playing media files, etc. Maybe that does require some help from Mozilla and its set of plug-ins. And add in Internet connectivity and AJAX and you can get a whole suite of applications that can leverage the power of the servers out on the Internet or at your office as well.

Once I get some time, I really want to dig deeper into the RAP (Rich AJAX Platform) project at Eclipse and see how their tools can be used to build such applications. And Mozilla probably is the platform of choice on the device since it is open source and readily available and works on many platforms. This all leads to some really interesting combinations of open source pieces to make something really different and fun for people to use. And Microsoft says there is no innovation in open source...

CDT Forever!

It's funny what Google Alerts will give you some times. When I got one pointed at http://www.cdtforever.com, I was pretty excited. A fan site for the CDT? Or just a well planned plot by my wife to buy her a new ring...

Ego-less Development

I remember early on in my CDT career running into an old boss and mentor. He asked how things were going and I told him great and that we were starting to build a C++ parser for the CDT. He was skeptical that we could do that and make it work well with the time and resources we had. But I was so excited to be building a C++ parser, I chose to put aside his wisdom. "It's not that hard."

Well, for years after, looking back now, I probably should have listened closer to him. The CDT's parser performance was abysmal. I really feel for the guys at QNX and others who were building products based on those early CDT releases. And, of course, you can guess what my first task at QNX was when I got there. Fix it!

Luckily I pulled a rabbit out of my, uh, hat. CDT 3.1's indexer is incredibly faster than our previous tries. I totally changed the approach we had taken and threw away the requirement of being 100% accurate that we had be preaching in futility since we started. Now, each file gets parsed only once and again only after it is saved. A lot of good work has gone into CDT 4.0 as well to add more information to the index so that we can use it for all parser based features making them incredibly faster too. It's not 100% accurate, but for 99% of CDT users, that's just fine.

But I still remember those years of pain as we fought hard to make the parser faster. Everything we tried resulted in only minor improvements. And every time we analyzed the issue, nothing really jumped out as to the cause. Unfortunately, it was probably our egos that kept us from giving up and trying a whole new approach. And if we had taken that approach earlier maybe we could have saved so many in the community the grief of an indexer stealing all their CPU on them.

"Ego-less development" is a mantra of mine and we should have followed it. Always question your design, try to understand the big picture and the impact of your design on the entire system. And if someone points out a flaw, listen and make sure you really are doing the right thing. I think this is especially true in open source with so many eyes on your work and so many hands trying to pull it in different directions. It's a huge challenge but if you are successful, the rewards are great, for everyone.

OpenKODE at GDC

You know, all my dreams will have come true if I am ever standing at a conference center attending a GDC, the Game Developers Conference. Interesting enough, this year, it was held the same week as EclipseCon, and in the same area of the world, so I was close. But it is just a dream, and we can probably save that one for another life time. At any rate, I am always interested to see if there is any big news that comes from there, especially in the tools/SDK front that might be of interest to CDT users.

One thing I did notice was that Khronos, the open media API people gave a number of presentations there and have nicely posted them to their site. The one I was most interested in was on OpenKODE, a core API that allows you to build portable applications. What I found most interesting in that presentation was a "State of the Union" of handheld multimedia applications. It shows why the industry needs to be interested in standard APIs to allow for growth of applications for these devices. And I firmly believe that and am glad Khronos is taking this on. They also confirmed what I believe about Java, that, while it really helps with cross platform development, if you need performance, like most 3D applications do, you really need to code natively, and that means C.

In theory, with OpenKode, you can build for one platform, say Windows, and with a simple recompile, run your application on another, say a handheld device running some other operating system. This is similar in purpose to SDL, but SDL has a patchwork architecture, where OpenKODE seems to be much cleaner. Acrodea has a sample implementation of it for Windows, and I wouldn't mind seeing someone do an open source licensed version of it so that we can pass it along in our upcoming EasyEclipse CDT distribution.

We can debate whether C code that requires a recompile to run on another platform is portable. However, if you look closely, to get any performance out of Java, you need a JIT which recompiles your Java anyway. With C, you just need to do it manually and ahead of time. As we build up a collection of portability libraries, I don't see that there is a big need to jump on the Java bandwagon.

A Visit to Google

Chris Recoskie and I had a couple of free hours on Thursday last week at EclipseCon and decided to accept a request to go visit to Google campus. One of their developers is a fan of the CDT and has produced some of our best bug reports on the indexer, so I felt it would be a great opportunity to learn more about what they were doing with the CDT.

It was a great visit and we all wished it could have been longer. It turns out that a few of them are using the CDT to work on some of their Linux C++ server applications. A lot of my early work on the CDT was meant to support this area, but I hadn't actually met anyone who was doing C++ server apps. So it was great to know that there is still a need there and an area where we need to keep in focus.

After our meeting, we went to their famous cafeteria for lunch. It was very cool. They had everything from hamburgers and hot dogs to gourmet pizzas and ethnic food. And, of course, it was free. The only problem we had was finding a place to sit since Google is really growing these days.

They are doing some cool stuff at Google and I am really honored that they are using the CDT there. It'll be a great story for both sides if we can solve any issues they have so that the CDT can be the IDE of choice for C++ server apps there.

EclipseCon The Day After

Here's three happy guys:



That's me on the left with Dave Inglis, the team lead for our Momentics IDE holding our award for best commercial Eclipse-based development tool, and Fred Plante, our fearless leader and manager of the QNX Tools team.

On our way home from EclipseCon, we stopped off to visited a good customer of ours. They like what we've done with Eclipse and had some good constructive criticism on areas where we could improve. They love the tools that are our value add with Momentics, but they confirmed what I always thought, that those tools don't matter much if the edit, build, debug cycle doesn't work as well as it can, or worse, hurts productivity.

But it really hit home coming the day after EclipseCon. It really showed how important our work on the CDT is to our customers. And judging from the passion of the people we work with on the CDT, this is true for all vendors that ship the CDT. I was very pleased that a lot our our customer's concerns have been addressed 3.1.2 and will be addressed in 4.0. I think we'll all have happy customers as CDT 4.0 hits the streets.

I also have one final thing to say about EclipseCon. Someone commented to me that they felt that there wasn't much buzz this year. Others though felt otherwise. What I think happened was that we saw much less marketing push from vendors and much more excitement from the engineers that were there. That's a different type of buzz and from an open source project perspective, much more important. Eclipse is driven by those who contribute to it and as someone else said, this truly was a "Community Party", and in my view, that's what will give Eclipse the biggest boost.

Another EclipseCon in the Books

Well, it's all over. What a great week. For me and the CDT community, it was the best EclipseCon ever. We had plenty of talks about the CDT and really got our message out. We had a lot of time to talk to each other. The CDT demo and project meeting went really well. I also had a lot of time to talk to others in the greater Eclipse community about possible future integrations and maybe even some new projects down the line.

The morning of the last day started with the panel I moderated on multi-language support in Eclipse. A lot of interesting scenarios came up where users want to work with multiple languages at the same time which really stretches the bounds of what we're doing. But it really shows that we need to work together more as Tools projects and the JDT to make sure that our plug-ins work together well and make it easier to create new language IDEs, maybe with the help of Safari.

The day concluded with the ending panel where Mike drilled the PMC leads for all the top level projects. I got to represent the Tools PMC and it was fun. One of the best suggestions I heard from the audience was that we really need to market the benefits for companies that put people to work on Eclipse projects. There are a lot of examples of companies that really do benefit financially by putting developers on Eclipse to make sure it meets their needs. We really need to showcase those examples to the world.

Anyway, it's been a long week and I'm pretty tired. Time for bed...

Time for a SAFARI?

I was pretty impressed by the SAFARI presentation by Bob Fuhrer from IBM Research. They have a lot of tooling that generates most of the cool features you get with an Eclipse language IDE, for any language that you specify. There are lots of wizards that generate Java plug-in code and most of it finds information based on a grammar you provide. A lot of it is pretty simplistic, but enough to get you going. At the very least, the generated code can serve as a great head start.

The good news is that they are close to bringing it to Eclipse for others to use. Of course we're building a multi-language framework with the CDT. So we'll have to figure out where we can leverage some of the cool things they have and where they can leverage some of the cool things we have. But this will be a great tool to help those poor souls who ask us how to build an IDE like the CDT for their own language.

CDT Morning at EclipseCon

My phone is in my room charging so I didn't get any pictures this morning. But it was cool to sit through the two CDT focused long talks and see the interest. We had over 50 people in each and a lot of attentive faces.

The first talk was by Phillipe Ombredanne from nexB and one of the leaders of the EasyEclipse open source Eclipse distribution. He presented a huge collection of plugins that extend the CDT in various useful ways. I was surprised at how many they were and thanked him for all the hard work he did to find them all and try them out. I am looking forward to working with him on EasyEclipse to build up an open source Ecilpse C/C++ distro that will be hugely useful for new users.

The second talk was by Chris Recoskie and Beth Tibbitts from IBM on how to use the CDT's parser data structures and the index to build static analysis utilities. This is a topic close to my hear since this is the area that got me into the CDT and where I've spent most of my development effort. But it is a pretty dry topic and I was very please to see people following closely what they were saying and we even had a couple of really good questions on the details of the DOM at the end. Maybe we'll see some cool features that can help users of the CDT write better code faster with the help of all the parser work we've done over the years.