High tech isn't always the answer

I have two sons, one of whom, Alex, is Autistic. That presents its own challenges in our life but my wife and I are getting to be seasoned veterans in that battle now that he's 14, and our "typical" 13 year old son is getting that teenager attitude which is a whole ball of wax on it's own.

At any rate, Alex has an obsession with a few things like most Autistic kids do and I appreciate most of them. Two of them are video games and Star Wars. And, of course, that means we have pretty much every Star Wars video game that's out there. The latest addition was one he found on eBay Canada called Star Wars Dark Forces. I wasn't sure what it was, but given the $6.95 price, I was sure it was pretty old. But it was cheap enough, so I ordered it for him.

Well, when it arrived I looked at the CD and found it was copyright 1994. Will this thing even work on our XP box at home? Well after I found and installed a Sound Blaster emulator, it runs like a charm. And you know what? He loves it! It looks like DOOM II and probably uses the same engine from id Software. It's pretty well done for 1994 but cheesy as all hell.

But I felt a lesson in all this. We often work hard in the software industry to pile features into our boxes and provide the latest bleeding edge of technology. But we often lose site of what this hole industry is about. It's about the people who use this stuff. If they're not happy with the end product, they we just wasted a lot of our time and money. And probably a lot of theirs.

And I guess that's what's led me in the last little while with the CDT. I don't work on the indexer much any more. I'm glad that we have a few other guys looking at it. And that's given me more time to focus on cleaning up the user experience. Because, no matter how fancy our stuff is under the hood, if people feel that the CDT is too hard to use, they won't even get far enough to try out the fancy stuff. And sometimes, at least in my house, 1994 technology is plenty fancy enough :).

A new era for QNX

So how to you take a great technology and bring it to a wider audience? For those of us working on open source projects, we know what to do. You create a community for it. And that's what QNX is doing starting today with the introduction of the Foundry27 portal.

Now lets get something straight from the start. This isn't open source. We've taken great pains to make sure we didn't mislead anyone into thinking that it is (unfortunately some members of the press haven't). But it does give everyone access to the source code and to the developers via forums. And, as long as you are using it for non-commercial use, you have free access to the tools and the SDK to play with it. We are even working on ways for people to contribute any of their cool work through the portal to others that may be interested. But, commercial users still have to pay for development seats and run-time royalties.

The idea is really to lower the barriers that people feel when they approach commercial software. Cut through the sales people and lawyers and just let me play with your stuff to see if I like it. This is something that many companies, even Microsoft, are trying to do to woo developers. QNX is going pretty far though by giving a public face to its staff, which is a pretty major culture change that I remember going through when I started working on the CDT. But we're intent on making sure developers get what they need and feel comfortable in this new community.

As with all communities, this one will need to be nurtured and it'll take a bit of time to work out some of the kinks just a Eclipse did in the early days. But we have learned from that experience and there is a lot more software out there to support something like this. But the focus really is on people as it should be with communities and this is a much friendler face on some really good technology.

Hibachi, a lesson in diversity

I've always been interested in Ada. Unfortunately, I've never had the opportunity to work on a project that required me to use it. But from studying it over the years, I can immediately appreciate it's strong type system, inherent support for multi-tasking, and focus on helping the programmer write quality systems. I can think of many projects that would benefit from Ada, especially in the safety critical embedded systems arena.

As part of my vision for the CDT (or dream, I guess) has always been to support multiple programming languages and Ada was one I wanted to see. But with a lot of ideas I have for the CDT, I neither had the time or real community support for it. So when Tom Grosman from Aonix approached me proposing they contribute their ADT code to Eclipse, I jumped at the chance to help.

And I'm glad I did. The Hibachi project is the result of Tom's work and is nearing it's creation review. I have been trilled at how Tom had gone about talking to his competitors and other vendors and interested parties and built up a strong community, even before it got started. They have a firm plan that immediately focuses on meeting objectives that would enable all vendors to get value out of Hibachi. This will then promote additional contributions to make Hibachi the best Ada IDE in the business. I can't wait to see them in action.

There are a lot of things at play here that is a trend I've seen with successful Eclipse projects. In many ways the forces are there to make it happen. Customers are demanding high quality IDEs based on Eclipse, the free community would love to see Eclipse support for their favorite tools, vendors would love to provide it all but to do a good job of it, they really need to work together and share the burden. If you can harness all this energy and make your project a friendly place to participate, then I think you have the clear formula for success.

Eclipse vs VS on Slashdot

A pointer to an article comparing Visual Studio and Eclipse has been posted to Slashdot. The article isn't really strong, but I am finding a ton of value reading the comments on Slashdot. The people there make a lot of good points and there is even some discussion on the CDT.

It's interesting to see that the number of pro-Eclipse versus pro-VS comments is pretty even. One thing that I did pick up from it for Eclipse, other than a bunch of usability concerns, is that if we truely want to be compared to VS, Eclipse really needs to support .Net development. That and we really need a GUI builder. I wish I had more time to take a look at NAB and see if we can use it more generally as the NAB team suggests.

Wascana Back to School Special

Tuesday is the first day of school here in Ottawa. Other places have already started but it does make more sense to start right after the labour day weekend instead of a day or two before it.

This has co-incided with a marked increase in the downloads of my previous CDT for Windows release (now numbering over 800). So it hit me that if I want students to use the new rebranded version of Wascana I'd better get it out now. And interestingly enough one of the people on the forums is an instructor who has chosen to use Wascana over VS Express (thanks!).

So I took a couple of hours out of my weekend and put together Wascana, MinGW Edition 0.9.3. You can download it from the Wascana web site. It includes the 3.4.5 MinGW gcc and all the current MinGW and MSYS packages. It includes the SDL and now the wxWidget libraries so you can build multi-platform applications. Stay tuned for tutorials on using these libraries with Wascana, and hopefully some project templates that set things up for you.

Does "Eclipse Europa Needs Some Polishing"?

I ran across this review of Eclipse Europa on eWeek that came out yesterday. It confirms almost every concern and hope that I had for the Europa C/C++ package that people have been downloading in droves from eclipse.org. And it was really interesting that this guy is a former Windows C++ developer who is very familiar with Visual Studio who is now doing Mobile development, the exact scenario that we see a lot of and is the biggest growth area for Eclipse on the C/C++ side.

The first thing that hit me was his summary: "Eclipse Europa is a solid IDE, but it could use more refined packaging for the Windows platform". If you're a regular reader of my blog, what more can I say.

The sore points that he ran into really home for me. His biggest complaint was the install. He had expected the C/C++ IDE package to include the gcc compiler. There are still way too many steps to getting this package to be useful for C++ developers. "Nothing leaves a more sour taste in a Windows user's mouth than an application not working properly, or requiring additional manual configurations, after clicking finish on the installation wizard's final panel."

He also had trouble dealing with the Eclipse workspace paradigm. Visual Studio is much more flexible about what files are included/excluded from a project. This is an area we really need to deal with to make these guys comfortable.

He had some good things to say too, though, and he really showed why I think Eclipse will be attractive to Windows developers once we clean things up. The CVS integration is unbeatable. He loved the CDT editor and navigation features including CDT's new Call Hierarchy view. It's these features that really bring the CDT into the mainstream.

One thing to notice, though, is that the title of the article seems to address all of Europa, at least that's what readers will see first. That's why we really need to be careful when we present Eclipse as an IDE. It isn't an IDE accross the board (not to open that debate again). But users see the word IDE and have pretty high expectations. And when it falls short, it looks bad on everyone.

The Need for Diversity

I'm in shock. Amongst other emotions that I'm still trying to figure out.

I came into work this morning and checked my e-mail to find that Danny Smith from the MinGW project had sent an e-mail "Bye" to the mingw users and developers mailing list. Bye? What do you mean bye?! Just as I was getting excited about the future of MinGW with it's spanking new modern compiler, the only guy working on it has quit. I don't know what to think. Is it a bad joke? Has someone broke into his e-mail account and sent the message. The responses from the other MinGW developers leads me to believe not as they politely wished him well in his future endeavors as well as expressing their fear for the future of the project.

And fear we should. I was always concerned over the lack of progress with the MinGW compilers. The seemed stuck on 3.4.2 as the official release for a long time (and now, probably even longer). Danny had come to the rescue and offered hope that the wait was over and we'd soon be able to enjoy all the great improvements to gcc in recent years. But now it appears someone else will need to take on this challenge. And it appears to be a big challenge as there were a number of bug reports flowing in (one of which was mine) and I was getting worried that Danny would get overwhelmed.

The timing of this is interesting, especially after my blog entry yesterday. But I've also been in a number of discussions in Eclipse lately over the need for diversity for projects to succeed. If contributors to a project all come from one company, what happens to that project when the company needs those resources elsewhere. The CDT was able to survive such an incident because we had contributors from many organizations who stepped up to fill in the holes (and I still can't thank them enough :). But there are projects at Eclipse who haven't worked hard enough to make sure they diversify like this and it is something to worry about if you rely on such projects.

And that's the position I find myself in. I was relying on MinGW's 4.2 compiler to make Wascana a super appealing environment for Windows development, even for commercial use. Now, I'm not sure what I'll do. Maybe it's time to apply some focus again on the Windows SDK compiler and debugger integrations. Although, unless by some miracle Microsoft let's me redistribute their SDK, it violates Wascana's primary mission as a simple to install complete IDE. And I doubt I would have ample time to contribute to MinGW and I don't really have the expertise anyway. And I have QNX work piling up. And CDT stuff to prepare for. Like I said, I'm still trying to figure this whole thing out...

The True Meaning of Wascana

While the progress on Wascana has been slower than I may have liked, it is progressing. And I've been very pleased with the positive feedback I've received on it. Almost everyone I've heard from says it's the right solution at the right time. A complete CDT IDE is hard for people to set up themselves, especially for noobs, and that is Wascana's primary mission in life, to make this easier.

But there is another reason for Wascana, and one I use to justify spending some of my work time on it. I've often seen marketing staff from various vendors promote their Eclipse-based tools as, well, Eclipse-based tools. Now in the Java space, that definitely means something. But in the embedded world, it doesn't have the same punch. It's almost like customers are saying "yeah, so?".

This has been the main driver behind my work on improving the CDT for the "grassroots" segment of our industry. These are the guys just getting into programming, or are doing it as a hobby, or people working in a start-up. People who don't have a lot of money to spend on expensive tooling but who would benefit from a good free IDE. And while there are good IDEs out there for free, there is so much more upside to Eclipse.

But I had reached a road block in my pursuit of supporting the grassroots. We had reached the point that their biggest hurdle was setting up the CDT with a good compiler and debugger and set of run-time libraries. This is the stuff that Microsoft's Visual C++ has always been good at. And if you look around, thanks mainly to the growth of Linux, there is getting to be a pretty good set of open source tools and libraries.

And I guess that's why the time is right for Wascana. I think we can build a pretty good free open source IDE from all this, and the feedback I've received is that it will be very popular. And if that becomes true, then commercial products based on Eclipse will benefit from the extra visibility and the investment will have be worth it. So while I've had to pursue Wascana out on SourceForge due to licensing and IP requirements on Eclipse projects, I consider Wascana to be an important part of the CDT, both for the desktop developers who want a good open source IDE based on it, and for commercial vendors who want their CDT-based IDEs to be successful.

Too clever for me...

Brian Kernighan, of Kernighan and Richie fame, or K&R (if you did't know, Richie created the C language and Kernighan helped him right the book on it). At any rate, Brian has this famous quote which I can't seem to find the root source but I found it quoted many times:

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."

And that perfectly explains my frustration in the last couple of weeks trying to understand two very clever code bases. Not only is debugging twice as hard, but being a new guy trying to understand the code is at least twice as hard. Hmm, maybe here's a new quote for people:

"Learning code is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, the only one who'll be able to understand it."

Unless the person learning the code is twice as smart as you. Or something... At any rate, I'm not twice as smart as most people so I get to struggle trying to figure this stuff out.

BTW, as I was searching around for the source of the Brian's quote, I ran across this interesting interview with him that gives a glimpse of the human side of Unix and C as they were created at Bell Labs many years ago now.

Lunar Eclipse filling my Inbox

Hey gang, there's a lunar eclipse Tuesday beginning at 3:51 CDT. I personally can't wait until it's over so that my google alert on "eclipse cdt" can stop reporting this thing...

From Old to New

I was hunting and pecking around looking to see what is happening in the industry as I do probably too regularly (I really got to get some code done...). At any rate, I ran across some slide shows that ZDNet were showing on old computers. I still remember the buzz and excitement us young geeks had as computers hit our neighbourhood streets. I don't think I'll ever see something like it again.

Anyway, one of the pictures was of the first computer I ever typed a program into. It was an HP-85 (click here for the real site that ZDNet borrows the pictures from) that my best friend's dad used at his work for the Fisheries Department at Government of Manitoba office in town. And it was where it all started for me and it was cool to see the picture. Yeah the thing had a tiny screen and a proprietary CPU, but it did speak BASIC and I remember being excited trying to figure it out.

Of course that is contrast to the latest computer, or at least processor, that caught my eye, Tilera's TILE64, a monster 64-core machine organized as a System-on-Chip (SoC, peripheral interfaces included). It especially sparked my interest because of the market it's trying to address, embedded systems for video and advanced networking. Intel can go on about their server and desktop monster multi-core machines, but there is a real need in the embedded space for this technology too. I can imagine some pretty wicked things that embedded devices could do with automation and robotics and such with this kind of horse power.

But as with all monster multi-core machines coming out, I still think we need a better way to program them so that we don't get lost in the complexity of getting our programs to do multiple things at exactly the same time. Hell, I spent a good part of the last couple of days solving a deadlock issue in the CDT, and that was just two threads colliding...

A lesson in scalability

I just read the Skype blog where a Skypian describes (well glossed over, but we get the gist) what happened with their two day outage last week. I don't use Skype very much but I know a few people that use it for their work and were at least inconvenienced by it. I read the report with somewhat the same reasoning that one watches Nascar, to see the big wreck and find out how it happened. But reading these things helps you think about how you could avoid such wrecks in your day job, so it's useful reading.

The story goes that 30 million computers around the world running Skype all downloaded a Windows Update and did a restart, all at the same time. I always wondered how Microsoft's servers could keep up with that, but I guess they did very well. But when those 30 million Skype users all tried to log into Skype after their restart all at the same time, bad things started to happen and everyone got booted off the system.

Now, being a software professional, it's not clear to me how this outage could last two days. Normally, you get a timeout if the server is busy and after some amount of time you retry. You'd think there would be a variability in the timeout so that everyone doesn't retry all at once, but maybe that was the flaw they found.

But the lesson of the day is to always consider the "impossible" since sooner or later, it may not be impossible. We run into that with the CDT. We find users who take the CDT and import any old project they may have and expect the CDT's parsers to find everything. In a lot of cases, we're fine, but we definitely don't take into consideration all possible scenarios. And I think that will be the next phase of CDT's lifecycle, to reach that maturity where our feature set does work on more and more projects and we can have more and more happy users added to our community. Openning our minds to the impossible will help us get there.

Just when you needed a Boost

I've been aware of the Boost C++ library for quite a while, but in the context I had to deal with it, it was painful. The Boost library is a collection of C++ templates intended as a trial ground for additions to the Standard Template Library that is part of the C++ standard. Some of them according to Bjarne, have made it into the next standard C++0x. But they stretch C++ templates to the limits, and as such, stretched the CDT's C++ parser to it's limits and broke it. In the early days of the CDT, we eventually just skipped it.

But lately, Markus on the CDT team has been testing his indexer work with Boost, and I've had a number of requests from people to include it in Wascana. So I decided to take a fresh new look at it.

Now I was expecting some simple container templates and utilities and such. And there were things that are much needed like a threads package for multi-threading your app and a regular expression utility class.

But I was amazed at some of the big constructs they have there. The first thing I ran into is a complete lexer/parser subsystem including a preprocessor. With that, it wouldn't take to long to build parsers in C++, and maybe even a C++ parser.

As well, there is a Statechart engine. This is something I've dealt with a lot in my past and it was cool to see a solution that involved templates and States as objects and some of the neat tricks it used to implement action code. Whether it scales to real size state machines, I'd have to dig deeper to see.

I've always been amazed at how powerful C++ templates can be and how the compilers can take all this template code and specializations and such and optimize it down to some pretty efficient code that you probably would have written by hand. But with templates, you work at a higher level of abstraction, meaning higher productivity. Boost gives you some pretty powerful abstractions. We'll see how easy they are to use in practice.

Debugging the Debugger

I've been trying out MinGW's new 4.2.1 gcc compilers. As I mentioned previously, they're experimental. But I've gotten really good feedback from people that moving to 4.2.1 is a great move and will help make MinGW a serious choice for developers.

They actually have two variants of gcc that they're working on. One of them supports exception handling based on the debug information gathered using the DWARF standard. It's apparently much more efficient than the default one based on setjmp/longjmp. I'm not sure what that all means, but my take is that the dwarf version is better.

At any rate, I had a problem using the dwarf version that I didn't have using the default (sjlj) version. If I specify the path to a file using Windows traditional back slashes, e.g. ..\main.cpp, gdb got confused and I couldn't set a breakpoint on a line. And, unfortunately, CDT's builder builds files this way and I my breakpoints failed to get set.

So, I downloaded the source to MinGW's gdb, configure and built it and set up a debug session, all within the CDT (this worked since configure generates forward slashes). I was able to set breakpoints, look at the dwarf symbol data that gdb was trying to use and found where the line number info was missing. And with that information, I was able to generate a hopefully helpful bug report that the MinGW developers can take, or if I find the time, I can try out different solutions. The only trouble I had was making sure which gdb was which :).

At any rate, this brought home again why I love using IDEs for development (which gave me a great intro for an article I'm writing). The productivity of using a debug environment that provides point and click visualization of debug information has to be at least ten-fold over using command line debuggers, and maybe a hundred-fold over using printfs. Once you start using it, you'll never go back.

The Master Speaks, Bjarne's vision of the future

I'm not sure whether you'd call him a Jedi master, or a Dark Lord. I guess that depends on your opinion of C++. To me, I've always affectionately called him Barney (which I'm sure he'd hate), and, of course, I treasure my copy of "The C++ Programming Language", the "Barney Book".

Bjarne Stroustrup, the inventor of C++, recently gave a rare public talk at the University of Waterloo, Canada's top university for computer science. The topic of the talk is the new version of C++ called currently C++0x (he mentions that if it slides into 2010, they may just call it C++0xa, and yes he has a pretty good sense of humour). But he also talked a lot about the past and present of C++. You can download the talk here but be warned it's huge and you may want to use the bittorent.

I'm a long term fan of C++ since I used it for my grad studies work back in 1989. It was a no brainer to me that it became so popular. Bjarne was able to bring object-oriented constructs and generics to C programmers without compromising on performance. And C++0x has performance square in it sites as it works to clean up some of the complexities of the language and bring new concepts that desperately need standardization like threads.

He also had some great examples why performance is criticial, even in today's world of fast computers with lots of memory. Embedded systems have always had performance as high priority, and in the world of mobile, high performance also means using less power, which makes power consumption a performance issue. Also, if your application uses less memory and is faster, that leaves more resources available to add more functionality and making the system even more useful.

So while the world still seems to be jumping on the Java/C# or Ruby/PHP/Python bandwagons, C++ still and always will have it's place. 3 million C++ programs can't be all that wrong...

Can we help these guys be successful?

I received a Google alert pointing at from this developer who is making the transition from Windows to Linux. Obviously, to do so, he has to drop Visual Studio as his IDE of choice and has picked Eclipse and the CDT for his Linux work. Despite some of the difficulties in setting up the CDT, he picked it as "rivaling Linux's best IDEs".

This transition was one of the scenarios we thought about in the early days of the CDT that would make the CDT successful for desktop development. But it is really only recently that I am starting to see this become more and more common. I think Ubuntu was desktop Linux's tipping point. And as Linux becomes more popular there, as the momentum in the press seems to be showing, I think there will be more and more developers looking at Linux when building their applications. And the CDT, technically, is positioned well here.

But the one thing that made me disappointed with the gentleman's post was the frustration he had setting up the CDT to work properly for him. And, you know, this frustrates me as much as it does these people. We've made long strides at improving the CDT, especially in this area, but it appears the message isn't getting out, and maybe we missed something.

And to be honest, the people that are getting the CDT from Eclipse directly and trying to make it work like this aren't getting the support they need to be truly successful with the CDT. If you get your CDT from a commercial vendor, then you have a much better chance since the vendors generally make sure the environments work properly for their customers (we do our best at QNX to do so for our customers). But the committers working on the CDT are kept very busy by their employers, as they should, and that is making it very difficult to get a focused effort to make the CDT truly work well out of the "open source" box.

And this comes back to the Platform versus IDE debate. The CDT at Eclipse.org is a Platform. It's not an IDE. It misses several components that make it a true IDE. Support for people trying to use it as an IDE is probably the biggest one missing. I'm trying to do something about it in open source with the Wascana project, but sometimes I get the feeling that I'm fighting a losing battle. And without success on the desktop, I don't think the CDT can truly be Uber.

Or maybe I'm just feeling down since my holidays are over and I have a ton of work piled up...

MinGW gcc 4.2.1 now available

Danny Smith from the MinGW project has posted his work on porting gcc-4 to the MinGW platform. From the mingw-user mailing list, it looks like a lot of people were waiting patiently for it. One thing I've learned in my days working in open source, it really makes you mad when someone complains that a release or some feature isn't ready yet. But, I know I'm glad it's finally here, even if it is currently experimental (with the target of 4.3 being the official version).

So what's the big deal with gcc-4 versus gcc-3? The biggest thing is a new optimization framework which promises to be able to generate faster code even accross functions. This is an area where commercial compilers excel. And gcc-4 brings MinGW into the mainstream being releases only days after the official 4.2.1.

The other interesting thing is that the port also supports OpenMP, a standard API and language extensions to support parallel programming. I know the PTP people will be very interested in that. Also, there is support for Objective-C++, which we've had numerous requests to support with the CDT.

I've started updating the Wascana installer to include this new compiler and a couple of other updates it requires. Tune into the web site to find out when 0.9.3 will arrive (with all it's fancy new branding too :)

Man, I could use a paint shop plug-in

So I'm stealing away a couple of hours here and there on my second set of holidays to work on Wascana. One of the things I've done is created a logo. You can see it on the home page. It's pretty simple, an orange moon to represent a lunar eclipse, and a blue 'W' for Wascana. I made it using Paint.Net, a photo editor written in C# (for some reason...). But it was pretty handy since it supported layers and let me draw simple shapes like the circle and text.

The next phase of the exercise was to then use the logo to create images for the icons and bitmaps in Eclipse. I wasn't too happy with how Paint.Net scaled the images and I wanted more control over the properties of the circle and text. I tried a number of different programs, an old version of Paint Shop Pro (when it was still owned Jasc) and GIMP for Windows. Neither did what I wanted.

But that got me thinking. Why isn't there a plug-in for Eclipse which does this. I am making bitmaps for my .product file. Eclipse is for everything and nothing in particular, why not image editing? (And this plays into one of my dreams for Eclipse - and why not take to the next level and do 3D model editing too, both needed for game development - but I digress.) Does anyone know of any activity in that area?

Eclipse CDT, More than Just for Writing Code

I think I've gotten more Google Alerts about "Eclipse CDT" in the last couple of weeks than I ever have. It's a great sign that people want to talk about their experiences, good or bad.

The latest one I got was from something called the "Reality Factory 2 Development Weblog". The author had struggled with the performance of CDT's content assist in Callisto, and I understand why. In Callisto it did a full parse including all the header files included by the file in the editor, which for C++ with lots of templates and stuff, it took quite a while. In Europa, one of the features my intern worked on over the winter was to migrate this to use the index for the contents of the header files. It's way faster, and the author was very happy to see it.

It's also interesting to see the comparisons with Visual Studio. The author states that in many cases, Eclipse and the CDT are actually faster than VS and he's switched back to Eclipse. That's cool to hear. It's been a while since I used Visual Studio regularly but it's looking like we're reaching one of the bars we set for ourselves of being as good as VS (being as good as JDT is the other one, BTW). This makes me believe even more that Wascana is important as a path for VS users who want to switch to Eclipse with all the components they'd expect of an IDE. This is definitely one of the main growth areas for the CDT I see in the coming years.

And the author of the blog entry really brought home why that's true. It's not just Eclipse and CDT's support for writing code. It's all the other plug-ins available to developers to help them in all areas of software development, including managing bugs with Mylin and source code repositories with Subversive. Even the built-in web browser gets kudos for helping out. Taking a look at the big picture of the Eclipse ecosystem, you really do get a sense of the value proposition of Eclipse. Embedded and enterprise developers get it, and we just need a vehicle to get that to the desktop developer too.

Using Eclipse CDT for MySQL C Development

I got a pointer to this blog entry in my Google Alerts this morning. It's an article on how to write a C application that talks to a MySQL database. And it's done using the CDT. Very cool!

This is exactly the kind of article we need more of, showing the CDT in action solving real world problems. The CDT can be used for anything but a lot of people still don't know that yet.

Baby's first "Hello, World!"

(Warning! Major geekness in this one :)

My youngest son is 13 now and his report card looks a lot like mine did. Good in math and science, sucks at literature. He's pretty much a twin, although he's way more interested in sports than I was at that age. And looking back, I guess I was about that age when I started getting interested in programming. My best friend's dad had an HP "portable" computer he used at work and we used to fool around on it trying to make it do stuff.

It was a pretty exciting time back then in the late 70's as home computers were born. TRS-80's came into existence and were the first machines I wrote for, or at least typed my first programs into while hanging out at the local Radio Shack (and, yes, that was in Regina, too). There was no way my family could afford one. And taking a look around my house now with three laptops and two desktops for four people, my kids are way spoiled!

So, I was at the book store the other day, looking to see what people were writing books about (in case the urge to write one ever overcomes my wife's objections :), I saw a copy of "Game Programming for Teens". It looked simple enough for a 13 year old to pick up and it uses a version of BASIC, just like I did when I started. And it's about making games, which he has expressed interest in, so I picked it up. Yesterday, he typed in his first program, the same program everyone types in when learning a new language. The eternal "Hello, World!". I was a proud papa, in the geekiest way a dad could be I guess. And I can tell by his enthusiastic "This stuff is confusing (meaning complicated), but I get it!", he's started down that same path that I did.

But what makes me a little upset, though, is that there isn't such a book to get kids started using Eclipse. The BlitzBasic demo that came with the book is one ugly IDE. I showed my son the CDT and it looks way more impressive. This is one reason I'd like to see Wascana become the IDE of choice for desktop hobbyists, and why I focus so much on supporting the grassroots of our industry with the CDT. If we can get people using Eclipse early in their careers, then it should be a much easier sell when those people come shopping for our products that are built on Eclipse. At least that's the dream, but I fear we still have a long way to go before making that a reality.

Introducing ... Wascana Desktop Developer

As I reported earlier, I have to change the name of "CDT for Windows" to something else. While doing so, I want to try address a big sector that I think we've kept trying but kept missing, and that's the C/C++ desktop application developer.

So here it is, the Wascana Desktop Developer, or simply Wascana. What's a Wascana? Well, since I was in Regina and I had some people comment on my last blog entry to name it after Regina, I did. Wascana is the Cree word for "Pile O' Bones", and was the original name for Regina. The beautiful park and lake in the center of Regina is called Wascana Park. They just finished a cool redevelopment project of the lake, which was an impressive engineering achievement on a tight budget. And this distribution really is a pile of bones, a collection of open source components that we're bringing together to make a skeleton that desktop application developers can dress up. Or something like that :).

I originally planned on addressing the needs of the Windows developer, but, given the strenghts of CDT at cross platform development, I'd like to expand this to cover all desktop operating systems, including Linux and Mac OS X. And I'd like to provide a set of open source cross-platform libraries that support a variety of desktop applications. The wxWidgets library is a natural. I'd also like to support the hobbyist game developer, one of probably the most interesting area of desktop development. Something like Ogre and SDL would fit that bill.

I am also making my Microsoft C++ compiler and Windows debugger support a part of this project to try and get more interest (i.e. help) with it. I've been disappointed in the progress of MinGW, e.g., there's still no gcc 4 port. And gdb is still gdb and has a lot of issues on Windows. So I think MSVC support needs to be there to for Wascana to make any serious inroads on Windows.

Linux will be interesting. Ideally, we'd like to leverage the tools and libraries as they come with the distributions. We'll need to make sure we get the right versions lined up, maybe with some super packages or something. We'll have to see.

And as for Mac, well I'll need help with that but I know there are a lot of Mac developers using the CDT.

BTW, I've renamed the CDT for Windows project at SourceForge and unfortunately, the didn't forward the old name to the new name so I apologize for the confusion and hope you can find us at http://wascana.sourceforge.net.

What's in a Name?

Well, I've been in this business long enough to realize that naming a product is tricky business. I had fears that calling the distribution of the CDT, tools and libraries for Windows, "CDT for Windows", that I'd get a letter from Microsoft one day telling me not to use the word Windows. Well, what I got first was a letter from Eclipse about the word CDT. As per Eclipse's new Trademark Usage Guidelines, CDT is a trademark of the Eclipse Foundation, and using it in the product name "CDT for Windows" is a violation. (Even though it is open source, you can still consider it a product, I guess).

Now no one get upset. I have a great relationship with the Foundation staff, and I can clearly see the need to protect the CDT brand. It wouldn't take much to clone the CDT and make some tweaks and still call it MyCDT (which has been done, BTW), and if not done well, could do harm to the work we've done at building up that brand. So while, I'm disappointed about the prospect of losing the most natural name for the CDT for Windows distribution (which took me about 5 minutes to come up with), I will comply.

The name change also triggers something else that has crossed my mind as I received feedback and looked at the state of the CDT. We are sorely lacking in contributions from the desktop developer space. Despite this, there is a huge contingent of CDT users that are using it for that purpose. And as the need for cross platform development tools grows with the growth of Mac and desktop Linux, we really need to start addressing that shortfall.

So to help with both problems, I would like to shift the focus of the CDT for Windows distribution, more towards CDT for Desktop Development and open the door to building a complete IDE that targets all desktop operating systems. If I can grow a community around that, and maybe even attract commercial interest, then hopefully we can close the loop and build momentum into that potentially lucrative space.

But first, I need a name for such a project that keeps all the lawyers happy...

BTW, at the moment, I'm visiting my birth place of Regina, Saskatchewan in the heart of the Canadian prairies. I didn't realize how beautiful a place it can be, at least in the summer :). If you ever get a chance, it's an interesting place to visit.

SourceForge vs. Eclipse

As you may know, I've started up an open source Eclipse distribution called CDT for Windows out on SourceForge. (If you haven't, feel free to take a look ;). My top objective for this distribution is to help out people who have Windows machines and who want to try out using Eclipse for C/C++ development. We've received numerous complaints and bug reports from people who've tried to make something like this on their own. And so far the reception for this distribution has been warm and I'm pretty happy with it and excited about its future.

One thing that is bothering me, though, is that I can't release something like this from Eclipse itself. Mind you now, I didn't really try, but I think the feedback I got when I talked about it was that there was no way the Eclipse Board would allow the tools and libraries that are GPL and LGPL licensed and with uncertain pedigree to be released from an Eclipse server. And I am the first one to stand up and promote why these legal requirements are important to ensure the pedigree of products that would like to redistribute Eclipse. But, I wouldn't begin to expect vendors to redistribute CDT for Windows. So why does this still bother me?

I guess it comes back to the reason why I'm doing CDT for Windows in the first place. I'm trying to grow the CDT community into the Windows development space. If you've heard me speak, you know I've been trying to do this for years. Visual Studio obviously rules there, but I think Eclipse has a lot to offer Windows developers. And there are lots of them. If we can grow the CDT community there and make them happy, this should help the CDT. And having a good base of CDT users should also make developer products that target for other platforms, like the one from my employer ;), more attractive, making the Eclipse membership happy too.

So, I'm doing this for the benefit of Eclipse. But I can't do it at Eclipse. And the irony isn't lost on me. But at the same time, I'm really enjoying the freedom I'm having doing this at SourceForge. It may come around and bite me in the behind at some point. But for now, CDT for Windows users can benefit from it even if does come from SourceForge, which is where many of the packages I'm distributing are coming from anyway...

Eclipse, a Platform or an IDE?

This debate has popped up again recently as it seems to regularly at this time of year. I've added comments to the odd entry or two. So I figured I should make a statement in my own blog and to make sure I'm clear on where I stand on this issue.

Is Eclipse a platform for tools vendors, or an kick ass IDE for end users? From my vantage point being a project lead on the CDT and working for an Eclipse member company, Eclipse is clearly a platform, not an IDE. A great product manager who I used to work with asked me a great question one day early in my career on the CDT. I mistakenly used the term IDE and he asked a great question. "How can you call it an IDE if there's no compiler?" In other words, how can it be an "Integrated" Development Environment, if it doesn't come with everything you need to actually use it?

But Eclipse is a great platform for making IDEs and they are available from a number of sources, just not from Eclipse itself. And a lot of those sources are from the board and add-in provider members who help fund the Eclipse Foundation and who fund the vast majority of the contributions to Eclipse. This is a business, and I have no problem with that, and this group is focused on making sure their commercial products get the most benefit from Eclipse as a platform as they can. Because of that, though, none of them are really working on, or even, why there are road blocks to making Eclipse from Eclipse a true IDE.

So it feels like there are two issues hanging in the air. First, if we are not really distributing a true IDE from Eclipse, then I think we really need to stop calling it one. The user community expectations have been set high and we often see them disappointed by what they get from Eclipse. I believe it is really limiting the success of Eclipse. Yes it's great now, but I am sensing a slip and everyone is all of a sudden worried about NetBeans for some reason.

That leads to the second issue. Eclipse needs to be a great IDE. And I mean the one they download from Eclipse.org. This is an area where I hear Netbeans is beating Eclipse. Sun is investing in Netbeans to make it a great free IDE out of the box. They don't seem to have a commercial interest in making money at it (which makes me wonder why Sun does any of the Java stuff that they do).

But at Eclipse, that isn't the mindset of the decision makers. I get the sense that they are actually afraid of the free Eclipse competing with their products. I guess if all you are doing is packaging up the free stuff and charging for it, you have a point. But the products that I see Eclipse members building aren't that, so why the fear?

I've invested a lot of personal effort into making the CDT a great IDE. I've seen the feedback we get when the average Joe developer tries to build his own IDE with it. It's not an easy task and they often fail. But I want these people to be successful. Because some of them, one day, will have a need to use my commercial product, and if their negative experience continues to linger, I don't want that impacting their purchasing decision...

CDT for Windows, an exercise in product management

Putting the "CDT for Windows" distribution together has been a lot of fun. I've been able to learn more about Windows installers, or at least Inno Setup which really dumbs down the process so that you can quickly put together a pretty sharp looking installer. Even their Pascal scripting engine brought back a lot of memories from my first year computing class (how do you create a comment in Pascal anyway? {}.)

I've also been able to learn more about the infrastructure that SourceForge offers to let me set up bug tracking, help forum, announcements mailing list and most importantly, the downloads section for the distro. That and it gives pretty good statistics information so I can see how many downloads I'm getting (~250 so far), and how easy it is to get into the 99.78 percentile in SourceForge project activity (number 415 out of 152,347 today!)

But what SourceForge doesn't give me, and something that I've never been good at, is a nice Web site and marketing material. Last night I finally put together the CDT for Windows web page and it's pretty stark at best. In fact, I had a hell of a time just picking the background color since I just couldn't stay with the default white which was blinding me.

It really makes me appreciate the work that the product marketing guys and their teams do. I've always had a great relationship with these guys in my career since they represent the customer as we're putting together the product, and they make sure the lines of code I create, get turned into a hot looking product that customers will find appealing.

Well, with CDT for Windows, being an open source project not really sponsored commercially or by Eclipse, I'm finding myself flying without a net. As much as I like hanging out with marketing people, their talents haven't rubbed off on me yet and the marketing material for CDT for Windows looks just plain ugly. Even the icons look eerily similar to the Eclipse Platform ;)

But, I still think CDT for Windows will be useful for people who have struggled getting the CDT working on their Windows machines. Hopefully, I'll attract more artistic people to help build it's own look and feel. This is the beginning of a journey that I thought would be simple, but having to do it on my own, it ain't pretty...

Re: Don't try this at home

As I mentioned recently, I had trouble with CDT for Windows with performance on my slower machine at home. Well, I've been able to trace it to the Harmony VM. Not only that, during testing, I was able to hang the VM when I went to do a CDT index rebuild. So for CDT for Windows 0.9, I've dropped the Harmony VM for now. The promise is there, but it's clearly still a pre-release and they have some work to do to get it fast enough and to fix some other bugs that we raised against them as well.

So after switching to the Sun JRE, things are much better, even on my Core Duo laptop. And I was able to run it successfully on my 666MHz beast of a machine, albeit, it was still pretty slow, but no worse than everything else that runs there...

Introducing - "CDT for Windows"

As I've mentioned a few times on this blog and elsewhere, we often get requests for a simpler way to get new users up and running on their Windows machines. This is especially true for students and hobbyist who have a need to learn and play with C++ and don't want to dive into Linux. As well, we get numerous bug reports from people who are trying to go through all the manual steps to set this all up and miss something.

Well, I am pleased to announce "CDT for Windows". This is a distribution of the Eclipse Platform and the CDT with the MinGW GNU tool chain for Windows and a Java Runtime wrapped up in a simple Windows installer. With a few clicks you'll be up and running debugging a C++ application.

The distribution is on SourceForge at http://cdt-windows.sourceforge.net/. It's currently at version 0.9.1 and contains the Europa Platform and CDT 4. My objective is to get it at a 1.0 level by the end of September with CDT 4.0.1. I have also included the SDL portable multi-media library. My hope is to get the wxWidgets portable application library into it as well.

As with the CDT, I really want to build a community around "CDT for Windows", mainly since I'm doing most of the work in my spare time (I've got a lot of work to do at QNX and the CDT too, you know :) and need to share the work, which really hasn't been too much so far. I've only done light testing so feel free to give it a spin, raise any issues you may find in the SourceForge tracker.

I think it looks pretty slick for something that only took me a couple of days to put together. I hope you find it useful as well.

CDT 4, Now Available!

It's been quite a year for the CDT. The growth of our community has been very rewarding. And CDT 4 is the fruit of our efforts. There are lots of new features and I counted 1152 bugs marked fixed. We have a What's New section in our user docs that doubles as our traditional Eclipse New & Noteworthy. You can see it on the Eclipse help machine.

Download instructions are available on our web site as well. There are lots of other ways to get it too. The Eclipse download site has the Eclipse Platform and the CDT in a single zip file. If you are on Linux, I'm sure you'll see it soon in your favorite distribution. If you are on Windows and want to build Windows application with GNU tools, check my other announcement ;).

Don't try this at home

I am putting the finishing touches on "CDT for Windows", my self-contained CDT with MinGW distribution that I'll be officially announcing later this week. To test it out, I thought I'd try it on my machines at home. One of them is this pretty old Pentium III 667 MHz (or as I like to call it in a throw back to my Iron Maiden days, 666 MHz, the processor of the beast, and I apologize if you've never heard of Iron Maiden :) with 256 MB of RAM but with a healthy 80GB hard drive. We've been having real performance problems running anything other than a web browser on it, and even then, I can't do things like TSN's flash video clips. But, heck, I thought I'd give it a try anyway.

Whatever the minimal required machine for Eclipse is, this ain't it. What a mess. It took forever to do anything. And all I did was create our Hello World project, build, and fire up the debugger. Now, I am using Apache's Harmony JRE which is still pre-release, so maybe that's not helping. But it's amazing to see what memory starvation does on big software.

One thing I've heard off and on over the years is people complaining about the sluggishess of Eclipse and the CDT, especially on older Unix machines that had multiple users sharing resources. Now I see why. Luckily, developers using Eclipse usually have plenty of horse power and memory anyway and this isn't as big an issue as it was a few years ago. But we still need to watch out. Even running the CDT on my 512MB, 1.8 GHz AMD Linux test machine at work has issues.

CDT 4, and so it begins

For all intents and purposes, barring emergency rebuilds, CDT 4.0 is done. If there are moments in my career that I hold as highlights, this is one of them. I am proud of the work that the committers and contributors have done and of the great feedback and bug reports from our community. Funny enough, I ended up having the final severe bug to solve, and it was a toughie, but I was glad to work my butt off and to put up my share to match all the great work everyone has done this year.

I have a webinar scheduled for July 12 and I invite you all to attend. I'll be walking through all of the features of the CDT with focus on what's new. I hope you'll be as impressed with the new CDT as I am.

But for now, I need some sleep. It won't be too long before I need to get back to work bringing CDT 4 to my QNX customers and start planning for next year and CDT Ganymede (the next Jupiter moon after Europa you know, at least that's what they tell me...).

Fun with ANTLR v3

A colleague of mine puts it best, "I have a habit of chasing shiny objects". I'm interested in so many things in this industry that I love to tinker with that I never actually get to finish any of them, other than a C++ parser I once dreamed of :).

The latest shiny object is the new ANTLR version 3, a fancy new parser generator. I've followed ANTLR with interest for a few years and we almost used it for CDT's parser back in the 1.x days. But at the time we felt that hand coding was the only way we could successfully deal with C++ ambiguous statements (e.g. x * y, is that expression, or a declaration of a pointer to x).

The new version comes with an LL(*) parsing mechanism that uses infinite lookahead (the *) to decide what rules a sentence matches. If you aren't familiar with what all that means, Terence Parr, the author of ANTLR, has written a great book that explains it all for you. The book itself is interesting since right now there is very little documentation other than the book. But it's only $24 dollars for the "non-dead tree" PDF version and is an interesting way to help fund the work.

But, this is essentially how Johnny C and I wrote the CDT parsers. We start at the high level concepts and break them down into finer grained detail until you get to the individual characters, creating an Abstract Syntax Tree (AST) along the way. You can base interpretation choices on where you are in the analysis and by looking ahead into the source stream as much as you need. It's a very powerful technique.

With ANTLR, however, you can specify the grammar at a higher level and have it generate a lot of boilerplate code for you. It may be the one parser generator tool that can convince me that it's better than hand coding. But to figure this out, I decided to try building a parser. Mike and the CDT guys at IBM are already working on a C parser using LPG, an LR parser generator (LR is bottom up, which, while faster, doesn't allow you to easily use context information when resolving rules, I prefer LL) and extending it for UPC, Unified Parallel C. And since I need to resolve some extensibility issues for GNU versus MSVC, and having a lot of experience in the past, I decided to try a C++ parser.

Now if ANTLR can handle that, I'm sold. It'll be an interesting journey and should allow me to try out some ideas on improving how we did some of the things in CDT's parser. Also this is just a prototype and I don't really plan on replacing CDT's parser with it. But it will help me learn ANTLR more and help me help others in the Eclipse community who want to use it. And who knows, another shiny object may fly by and take me on a totally different tangent anyway...

More word on massive multicore

I was just reading this article on Intel's work on trying to bring massive multi-core processors to market. The had showed a demo back in February that I blogged about back then. However that chip didn't really do anything other than run 80 threads at a time. It didn't have I/O or memory from what I could tell.

So now the press is trying to figure out how Intel will productize that. I think the article asks some pretty irrelevant questions, like "Will they be x86 cores? Will they run today's applications?" And the response is that they are working on it to make it easier for programmers to deal with.

But I think that's a mistake. Why would you run your favorite e-mail program on an 80 core machine? Eclipse, now that I can see, especially if you've ever debugged a run-time workbench and saw 30+ worker threads going at it. But really all we're doing there is taking the same old paradigm and stretching it beyond it's limits. The hardest bugs I've had to solve were when multiple threads were contending for the same resource and someone forgot to synchronize them. Happens time and time again.

No, 80 cores are great. Mix that with 128 stream processors like those on the latest GPUs, and there's some massive horsepower that we can take advantage of. You can hide all the complexity behind good compilers and run-times, but I don't think you'll get all the benefits of the hardware. I continue to believe that we need some new programming paradigm where we stop thinking of programs as a sequence of operations and more like a network of data flows. Turning to objects was the first step, but I think there's further we can go.

A new way to get your CDT

Not only am I the CDT project lead, I'm also CDT's release engineer. That means I get to write all the cool ant scripts and stuff that pumps out the CDT builds on a semi-regular basis. Actually, I'm more the release engineer team lead and cron is my underling that does almost all the work. Actually cron does such a good job I hardly ever have to do any release engineering at all unless I have to change something.

Well, the past couple of weeks I've had to change some things. The first one is to meet Europa's guideline of having our jars digitally signed for security. That way, when you download the CDT, you know you're getting the officially blessed version from Eclipse. I think the risks are pretty low, but it is possible for someone to make a set of zips that look like the CDT but aren't and do something nasty things. The infrastructure created by the Eclipse team makes it easy to do the official signing so I figured why not.

The other issue we've been running into lately is the sheer size of the CDT builds. They passed the 400 MB mark with all 9 platforms and the two main features, runtime and sdk, that we build. With only 10 GB (now 15 GB) of disk quota at eclipse.org, we kept bumping into the limit and constantly needed to clean up builds. When you take a close look, the 9 platforms are almost identical except for the small shared libraries that we have in the cdt.core fragments. The update site, as a result, is much smaller, around 40 MB, since you don't get that duplication.

So while I was wearing my release engineer hat anyway, I figured I'd fix all this. The recommended way to get the CDT is using our update site. We will have the runtime feature, the main one that you download, on the Europa update site. That feature along all of the others will also be on the CDT Europa update site. All this will be explained on the CDT's website when we release. The jar files will also be compressed using pack200 to make downloads a lot faster and save bandwidth on the Eclipse servers and mirrors.

The other big change is that we will no longer be releasing feature and platform specific zip and tar.gz files. Instead, we will have a single zip file called the CDT Master that is an Archived Update Site. So you can install normally via our update site on Eclipse.org, or you can download this zip and install it, still using the update manager, but from the zip file instead. We are currently doing our nightly builds this way, and it works well.

This will hopefully make the CDT easier to get up and running while saving a lot of space and bandwidth for Eclipse. It's good all around, but it is a change and, as always, I'd like to hear your feedback on it.

Happy 5th Anniversary CDT!

From: "John Duimovich"
Newsgroups: eclipse.tools.cdt
Subject: [ANN] - CDT project changes
Date: Fri, 7 Jun 2002 17:20:34 -0400

The Eclipse Tools PMC is pleased to announce some exciting developments to the CDT project hosted on eclipse.org. First, QNX will be contributing some C/C++ core technology from their recently announced product to eclipse.org. We believe that this will enhance the value of the technology hosted on eclipse.org and bring some industrial strength technology into the CDT project.

And so it began. Five years ago yesterday, the new CDT project was born. The QNX team had just gone through six months or so of compressed development in a remote location affectionately called the "Toolshed" (we're known as the Tools team). They had basically gone from nothing other than the source to Eclipse and JDT to come up with a fully functional C/C++ IDE. But QNX is an RTOS and run-time company. Tools enable that business but take a huge investment to do really well, and it was becoming clear that a lot of good can come if we would share the work and benefits of that work with others in the open. A handful of other vendors liked what QNX had done and worked together on starting this new direction for the CDT.

A meeting was held at QNX headquarters on July 17'th of that year in what turned out to be the first ever CDT Summit. In attendance were people from QNX, Rational, Red Hat, and MontaVista. From day one, we enjoyed the spirit of "co-opetition", co-operating competators, that still lives strong today. Here's a fuzzy picture of Sebastien Marineau from QNX on the right, the first CDT project lead, who really got this whole thing off the ground. He's sitting with John Prokopenko, a senior architect/manager from Rational and a mentor of mine.



I was at Rational at the time and this was my first exposure to the new CDT. And as we know now, it became a career changing moment for me. Who knew then that I'd spend the next five years living out a dream of mine, to make a software development tool that would help thousands of software developers write better code faster. I wouldn't have passed that up for anything.



There are many people to thank for getting us here, including Sebastien and the team here at QNX for having the vision to share their great work with the world. To my former co-workers at Rational/IBM (who coincidentally I had lunch with today) for helping get this crazy DOM thing in place. To the gang at Intel (especially Leo!), TimeSys, Chris at TI now at IBM, and everyone else who formed that initial spurt of a community outside the original two, to all the product vendors and users and contributors that now form what I estimate to be our 400,000 member CDT community.

Thank you all! And happy 5th anniversary!

CDT 4, What a difference a community makes

If you've ever been to a CDT meeting, chances are you would have heard me state one of my mantras, "Don't take a .0 version of an open source project and put it in a product". Certainly, in the past with the CDT this has been very true. And it certainly hasn't been the fault of the committers and contributors to the CDT who work on it with a passion and really want it to be the best C/C++ IDE it can be. A lot of it had to do with the lack of test pressure we'd get on these releases. And a lot had to do with the lack of feedback from the community until after the release. A lot of it had to do with a young community that was just starting to grow.

Over the last year, this has all changed. With the great bug reports we've been getting from the community and the volume of patches we've been getting from contributors, and the great work of our army of committers, I am happy to say that CDT 4.0.0, is the first .0 release of the CDT that I feel really good about. I'm sure everyone will be pleased with it from our open source community to the vendors that adopt it.

In the spirit of openness, I have to admit I have been very dissatisfied by our previous .0 releases. And I think I need to share what happened so that we can learn from it. Here are the "lowlights" of previous CDT .0 releases (BTW, the CDT 1.x.0 series actually went pretty well, so we'll start with 2):

• CDT 2.0.0 - first introduction of our new parser-based indexer and search. This was a dramatic change to the way we did indexing and the start of our performance woes. In fact I rewrote the CDT scanner (tokenizer) in 2.0.1 since it was brutally slow. We also introduced a new build model that wasn't really ready for prime time and really needed to be exercised with more tool chains.
• CDT 2.1.0 - This release was to meet one of the contributing vendor's product needs and was done while the other vendors were focused on the next release. It was a bit difficult to watch the extent of work they were doing mainly on their own. This really showed the need to standardize our releases (thanks Callisto, Europa, etc!) and how the model we had been following can't scale as the number of contributing, and consuming, vendors increase.
• CDT 3.0.0 - This was the big DOM release. Again, massive changes to the indexing and search framework done by one vendor. Even worse, that vendor later had to withdraw its resources on that massive bulk of code due to business reasons. This really showed us the need to have a diverse set of committers working on each component. Contributors will come and go, and you need to build a robust community to survive it.
• CDT 3.1.0 - Yet another new indexing framework. With the contributors to the original index gone, I took it upon myself to finally throw out our failing assumptions, 100% correctness as a requirement being the big one, and focus on a new strategy to address performance. At the end of the day, this new strategy is the right one and correctness was still pretty good. The problem was that I did it myself and bit off more than I can chew. It was an absolute necessity to get it done, though, but I did do a lot more work for 3.1.1 to make it product quality.

It's been a while ride over the last 5 years on the CDT (BTW, it was 5 years yesterday, I need to blog about that too!). I think we've learn a lot from it, probably enough to write a book about it. Being in the open, I hope others can learn from it too. At the end of it all, I think the most important aspect of any software platform project, commercial or open source, is that of community. And the most important job of a project is to build that community. And that there is probably nothing harder to do than build a community. As our industry goes through this shift, we really need more examples of what to do and what not to do. Then, hopefully, we won't have to always go through pain like this.

Some CDT Stats and Memories

In the rush to get numbers for Ian's Europa stats report, I got wind of a web site that gathers these stats for us. The CDT's numbers are here. There are a couple of items that caught my attention. First was that CDTs line of code count is only 646,651. I believe at one point we were actually more that 700,000 so we've really cleaned up a lot of dead features over the last year or so. It also shows the power of the Eclipse Platform that we can take it and offer a complete IDE for C/C++ in such little code.

The other number that caught my eye is the estimated 176 person years of effort to build all this. I'm not really sure how they got that number but looking at the breakdown by developer, it seems pretty reasonable. At a loaded labour rate of $150K US, that's about a $26M investment, which points back to the main reason I think the CDT is successful commercially. Not many companies could afford to do that on their own.

Taking a look at the contributors list, also reminded me of all the people that have worked on the CDT over the years, 32 in total. And although some have moved on, their contributions were huge and still pay a important role in the CDT today (thanks, John and Alain!). And we have new blood to take over and they are quickly catching up, especially the gang from Intel and Wind River, and the new team at IBM and Symbian are starting to role, and of course, our debug keepers from ARM, Nokia and Freescale.

On the eve of our biggest release, CDT 4.0, it was nice to sit back for a moment and think of all the great contributions we've received and the great people we've worked with over the last 5 years that play such a big part in getting us where we'll be tomorrow.

Linus on source control systems

I ran across this video on the weekend. It's Linus Torvalds giving a talk at Google headquarters about his thoughts on source control systems and why he likes the one he made the best. I thought I was overly opinionated, but wait until you see Linus. This talk was especially relevant to me as I am currently weighing the benefits and costs of moving from CVS to Subversion.

Despite his crass style, Linus does make a lot of points I have to agree with:

• CVS sucks. SVN tries to be a better CVS, but the problems are so fundamental and SVN doesn't make enough strides to make it much better than CVS.
• Merge is the key feature of any source control system. We've gotten used to CVS's weakness at merge and Eclipse's CVS support helps a lot. I actually think the SVN Eclipse plug-ins are a step backwards since I haven't seen a successful merge with them yet.
• Source control systems should allow everyone write access, at least on branches. I would love it if everyone who customized the CDT could do it in our main repository so we can see what they did and make it easier to incorporate their improvements. I'm not sure that is even possible in CVS and it requires central administration at least in SVN. I guess with Linus's GIT, allowing this is it's number one advantage.

I'm actually a big fan of Rational's ClearCase because of it's ability to manage branches and merges. It was a behemoth to manage, and a lot of people found it hard to work with, but I really appreciated the ability to work in a private branch and make occasional merges back to the main line. I'll definitely want to look at GIT and see how it compares. Of course, we would need an Eclipse plug-in to work with it...

BTW, when I visited Google during EclipseCon, I saw this stage. It's in their cafeteria and you can see people getting drinks in the background. I wonder how many other cool visitors they get like this.

Now there's a screen I can live with

I've just been reading a few notes on the web about Palm's new Foleo "Mobile Companion". If you haven't seen it yet, it's a mini-laptop type thing that's intended to work with Palm's Treo smartphone. But under the hood it's really a mini-laptop that has wi-fi and Bluetooth connectivity and a USB port and SD slot for expansion. And it runs Linux with an Opera browser. The price is also pretty reasonable at $600 US and that's with a 10" screen and full size keyboard, although there's no function keys.

So, I'm not promoting the product. Yes, I know people from Palm who are contributing to Eclipse and the CDT, which I'm sure they're using in conjunction with this product. But I think it could the be the start of a trend. Everyone loves smartphones and getting their mail in Blackberry's and such, but the size of the screen and keyboard on these devices really limits their usefulness beyond their mobility. People still need laptops to do their real work.

But I think there's room in the mobility market for devices like this one. The embedded system-on-a-chips are there now to do it. And I think you'll even see games on these things with the 3d capabilities of these chips. With solid state memory like SD cards getting bigger and cheaper, these could be really useful little machines. Palm was first and it fits their niche, but I wonder if anyone else will take the plunge and make a more generally useful "mini-laptop."

UML Action Semantics, Naturally Parallel

Earlier in my career, I had the honor of reviewing a part of the UML spec which involved a very surreal phone meeting with one of my heroes in this industry, Jim Rumbaugh. The area was on the UML's Action Semantics. At the time it was a separate spec but it is now intertwined in the Superstructure document as UML's Action behavior.

The idea according to Jim was to provide a sort of assembly language that all software behavior could map to. But I thought it provided a more powerful concept, that of the Action itself. An Action is a unit of behavior that has inputs, does some processing, and produces outputs. The outputs of one action feeds into the inputs of other actions. The "Ah-ha" is that all actions that have their inputs satisfied theoretically run in parallel.

This concept isn't new. Hardware designers have been thinking this way forever. I believe Petri nets present a similar idea in mathematical terms. But what struck me was this was a really powerful paradigm that can make it easier for programmers to write highly parallel programs. What was needed, though, was a good, 2-dimensional programming language that allowed programmers to create actions and hook up the inputs and outputs quickly and, of course, with minimal typing. But something like that really wasn't an objective for UML.

It's probably one of the reasons I'm keenly watching Eclipse's Modeling project. Aside from a great framework for creating domain specific languages, it has the capabilities that would be needed to build this "Action" language. And with a good back end that produced code for today's multi-core clusters, I really think this could be a good way to help programmers meet Intel's challenge that "Software has to double the amount of parallelism that it can support every two years" to catch up to the what the hardware guys are doing.

How Different Are Linux Distro's anyway?

In my years in software development since Linux has taken off in popularity enough to warrant commercial software vendors porting their wares to it, one thing I've seen vendors having trouble with is dealing with the massive number of different Linux flavours out there. Back when we were just Windows and Unix (commercial *nix's if you will), life was so much easier. The operating system vendors ensured that the releases were well defined so that we could easily put together a reasonable list of supported versions for our products.

With Linux, it really is next to impossible to do that. Novell and Red Hat do fill in that role as commercial Linux vendors that provide a stamp of approval over their versions of all the packages that go into a Linux distribution. But, really, none of the developers I know that are using Linux are using any of those commercial Linux'es. They're using Fedora, OpenSuSE, and more lately Ubuntu. It really is impossible to validate your products against all the possible combinations of Linux that your customers may want to use.

But, I then ask the question, so what? How different are these distributions anyway that makes it so hard to support Linux. Yes, you may have version differences in the packages, and things like the major versions of GTK can break under GUI applications like Eclipse. Also, it's pretty confusing the number of different ways to set up user's environment variables, but then applications shouldn't be relying on that anyway. I really wonder if there's much else that can affect most software products.

It bugs me every time someone tries to explain away a bug with, sorry, that version of Linux isn't a reference platform so we can't look at your problem, especially when the person is using a recent distro like Ubuntu. But it really does speak to the challenges that software vendors face with the fragmentation of the Linux market. But I guess it's part of the price we pay for "freedom".

cdt-dev is my office

Today is RC2 day for CDT 4. As we get closer to milestone build dates, I send out friendly reminders to the cdt-dev mailing list on what bugs are still open against that milestone. It's just a prod to get the developers to do something about them so that we have no open bugs on a milestone when we do the build. It's worked every time as we get the friendly but odd 'Zarro bugs found' message from my bugzilla query when we're ready.

RC2 was no different. This morning we had two left. Ken from Austin, Texas gave an update on his asking for feedback. I, from Ottawa, Canada gave some feedback for him to go ahead and fix it. Bala from London, England mentioned he had a patch ready for his and Mikhail from Russia replied saying he was looking at it. I'm confident we'll be ready in a couple of hours to fire off the build and get the RC out by the end of the day.

This happens regularly on the CDT and once in a while I stand back and think of what just happened (and I think I've probably blogged about this before too). We have a very effective development team working on the CDT, and the cdt-dev mailing list is the backbone of that collaboration. A lot of groups use different technologies such as instant messenger or IRC channels, but for us the cdt-dev mailing list works great. Bugzilla comes in at a close second. But then, we treat bugzillas as mini mailing lists anyway.

I think the biggest benefit of the cdt-dev list is that it's open to anyone. If you want to see what's happening with the CDT at a high level, that's the place to go. If you want more detail, then you really got to watch the bug reports and signing up to receive notifications on the cdt-*-inbox accounts are the best way to catch the train.

From my experience on the CDT, the most important tool you have to build a community is open communication, like mailing lists, forums, IRC. As your community grows, the only way to really talk to them all is via open communication, so it really forces you down that path and you end up doing it anyway. But in the early days, it was a hard habit to get into, especially when QNX was by itself, or even when I was at Rational and we started working with the QNX gang who were only a 5 minute drive from the Rational office. But open communication has really paid off in the end for the CDT and the reach of our cdt-dev mailing list impresses me time and time again.

Open Source Ripped?

Bjorn made me read this article by Howard Anderson. Well, he didn't make me, but it is a topic I'm very interested in as well: How does open source make sense in a commercial world?

It's actually a very interesting article. When I finished it, I had to remind myself of the title. In the end, I wasn't sure if he was for or against open source. His general thesis seems to be that open source is a tool used by small companies to gain market share against big companies. Yes, he's right. I've seen that. There are a lot of smaller companies shipping a world class IDE with their products making them more attractive. They leverage open source (i.e. Eclipse) to do it to lower costs since building a world class IDE is prohibitively expensive for most. I think it's a great business model. I guess he was just looking at it from the big proprietary company side.

There are a couple of areas where I have to disagree with Howard, though. He mentions open source is a "religion". Well in some circles, I guess open source participants see it that way. Certainly from the outside it looks like Richard Stallman is playing the part of a religious leader, and FSF is his church.

Howard also seems to believe that the people writing open source are doing so at night when they come home from their real jobs of working on proprietary software. But that's not what I do as an open source developer. Open source is my day job. The company I work for is one of those companies that is reaping the benefits of the open source business model, and is willing to invest in open source to help build a community where we can share the work with each other. And there are lots of developers like me from many companies. Open source is not a religion to us, but a business means to a business end.

So while it'll probably be impossible to shake the stigma of the open source "religion" from what we do, open source in the spirit of "co-opetition" (co-operating competitors) is a vital tool available to the commercial world. Some communities are set up for this to work well, like Eclipse, while others, not as much (and I won't name them unless over a beer :). But the ones that are, seem to be the ones that the big proprietary companies fear the most. Which means we must be on to something...

Bye-bye 32-bit Windows

I just read that Microsoft was putting an end to 32-bit support with their operating systems. I guess that shouldn't really be a surprise. It is a real struggle for device driver writers to support both and I think Windows 64-bit has really been hurt by that. The same is probably true for Linux as we don't see much demand for the 64-bit Linux version of the CDT either at 2.5% versus the 32-bit Linux at 20%.

Maybe this will finally trigger people to focus more on 64-bit and start writing programs with that in mind. The biggest change for C/C++ programmers is the size of pointers changes. I've seen a lot of code that assumes you can merrily cast a pointer into an int and back and everything is happy. One example of this is with Java native code where we like to stow away pointers in Java fields for later calls back into native-land. Well with 64-bit, while the size of pointers change, the size of int does not. I'm also hearing that there are different interpretations of the sizeof(long) where on some platforms it's 64-bits, but on others it stays as 32. Then there's long long (gcc) and int64 (msvc) which in the 32-bit world also means 64-bits.

Suffice it to say that the 64-bit world gets a little messy. We'll think back to the simple days of 32-bit with a sigh. But then, I think things will still be better than the now ancient 16-bit world was (now who's old enough to remember that?). Now they say that this will start after 2008, but given the length of time it took to get Vista out, people with 32-bit only machines shouldn't worry too much. They'll be ready for the dumpster by then anyway. And do we really need another operating system version beyond Vista? Microsoft hopes so, and I'm sure they'll use the 64-bit push as a marketing ploy to help you think so too.

Eclipse Wants You!

A few days ago I posted a blog entry worrying about the openness of the Eclipse Platform team. I thought it would generate a flood of comments saying I was wrong. And to a certain extent, I probably am wrong. But I think its a serious issue that people need to think about.

I guess the point I was really trying to make is that we as the Eclipse community outside of the Platform have depended too much on IBM/OTI's great contributions, to the point where we expect them to fix all of our problems. My experience with open source projects is that it just doesn't work that way.

Open source developers usually work on open source projects for a reason. They are trying to get something done for themselves, and really as a side affect they hope that others will find it useful as well and maybe come help out. Because open source software is free, I think people start to think it's more like a charity, but it isn't. And I think this is even a bigger factor with Eclipse since the vast majority of developers are employed to work on Eclipse projects. They respond to the community as much as they can, but at the end of the day, if their empoyer asks them to work on something else, that has to have priority.

So if you have a bug that isn't getting the attention you think it deserves, please think of the people at the other end. There's a good chance it's not because they think you're problem isn't important, but that they have probably been assigned work elsewhere and really just don't have the time. Do as much leg work as you can. Create a really good bug report that has a patch and a really good justification that shows you thought about the fix as much as the committer would have. Make it as easy for the committer to fix your problem as you can.

And if you find you really depend on certain functionality that isn't being provided or bugs that you really need fixed, and you do enough great patches, you can become a committer too. The more committers we get from different employers, the better off we'll all be. That kind of redundancy is important in open source and is something we've really learned to appreciate on the CDT project.

I love bugs from playstation.sony.com

I have no idea what this group is doing with the CDT. They've been on the outskirts of our community for quite a while and this bug was just marked VERIFIED to show they are still around. In fact, one of my first experiences as a CDT "dude" was at the first EclipseCon where someone from the Sony Playstation group stopped Sebastien, the first CDT project lead, and I and asked us for information about CDT extensibility. We thought it was pretty cool then, and it's still cool now.

Forget OO, C++ is a better C

I was working on bug 176353 trying to get interrupt signals sent to cygwin-based gdbs. Part of the magic for that in the CDT is something called the Spawner. Spawner subclasses Java's Process and adds the ability to send Unix type signals to them. It also implements some fancy I/O but that isn't supported on Windows. This class is a little Java and a lot JNI native code that interacts with the operating system to implement the signals as well as the other Process type things like starting up and waiting for completion of the process.

So, I guess this code was written originally with Visual Studio 6 many moons ago. However, continuing with my theme of using MinGW for Windows development, I've created makefiles to build the spawner DLL. What made this situation a little weird was that the original creators of the spawner were guys who didn't really know C++, so they did it in C. I always find it weird doing C in VS, but that's what these guys did. So when I wrote the makefile, I used make's default of running gcc on these files. Makes sense.

However, I was having trouble when I added a calls to a couple of Windows routines. I was getting undefined references at link time to the two calls I added. Weird, I didn't get any compile errors. When that happens it usually means I forgot to add the library. So I added it and still got the errors. What's going on here? Is it something broken in the MinGW port of these libraries? Was my code just getting tired and cranky?

Well for some reason, maybe I was getting tired and cranky, I wondered if it was because I was using C instead of C++. Part of my debugging technique is to start assuming the least likely cause and testing it out. This was really damn unlikely. But I tried it out. I changed the compiler to be g++ and ran it over the .c files. I thought it would have treated them as C files and nothing would really be different.

But to my surprise, g++ compiled the .c files as C++. And I got a ton of errors. And almost all the errors were for passing the wrong types to functions, especially since I was using UNICODE and not everything was really using wchar_t. Well, no wonder things weren't working. Of course the one thing it found were compile errors with the two functions I was calling. I had forgot to add the include to the header that defined them which specified the correct calling convention, WINAPI, which is why I was getting the link errors in C. But then C was happy to play along without having to see the declarations of the functions and made some bad assumptions. I wasted a good couple of hours trying to figure out what was wrong.

So, forget the object-oriented, templates, namespaces, operator overloading, and all the other cool features of C++, C++ at its core is just a much better C. It has proper type checking that helps you find those errors before link time, or worse, run time. It all feeds into helping you build better software faster, which is what this whole tools industry is all about. And if you've programmed in C++ for years and have to go back to C, don't forget to make that paradigm shift back to the 80's...

A lesson in release management

One of the first things I remember learning about managing projects came well before I even considered doing so. It came from the lore at the big telecomm company I worked at. They had an old telecomm switch that was doing quite well sales wise, but they had started working on their latest and greatest architecture that would pave the way to the future (which it did in the end). However, they were so excited, they started announcing it to their customers well before the release date.

Well, guess what happened. The customers got excited, too. They didn't want to buy the old switches anymore, they wanted the cool new one. Unfortunately, the dates ended up getting delayed and that spelled trouble since sales of the old switches were drying up. Lesson learned, though, and you notice a lot of companies holding back release information for that very reason.

Well, I think the same thing is happening to the CDT. For the first three months of this year, we've been hovering around the 65,000 downloads mark. It's not our biggest. That happened last October and November when we hit 85,000. But it was steady.

Well, I just did the numbers for April and found them at a disappointing 55,000. Maybe it's just a glitch. Maybe people are happy with getting the CDT from other places, like Linux distributions.

But it makes me wonder if this is a side-affect of CDT 4. We've been making a lot of noise about it, and we're finding that a lot of people are using the CDT 4 milestone builds, especially starting at M6 which just happened to be at the beginning of April. I haven't been counting the milestone builds in our figures.

We'll see how May's numbers are, but it would be interesting if we're seeing the pre-announcement affect in open source projects too. And I guess, why not?

gdb is a-calling

I've been working over the past week trying to get process interrupts working on Windows so that we can get suspend working again with gdb. The suspend call gets translated to a SIGINT signal raised on the process, which on Windows gets converted into a Control C event. For some reason it stopped working lately, mind you I don't remember ever actually trying it. It was fun, though. It got me back to my roots as a Windows developer and renewed my dislike for Hungarian notation.

Now that I got it working, though. I am finding that when you suspend gdb, both the cygwin version and the mingw version, that you end up in a pretty useless state. The stack trace has gdb totally confused. The addresses don't even look like real addresses. I thought for a moment that it was because of something I did in the CDT. But when I tried it from the command-line I got the same result. I did a Google search and looked at the cygwin and mingw forums and couldn't find anything useful. Maybe no one cares. Maybe it's my machine. Who knows (do you?).

At any rate, it has led me down a path to take a deeper look at how gdb is implemented and try and debug it myself. This is something that I've wanted to do for a while. We use gdb as our debugger in QNX Momentics so I may be able to help out our gdb guys too. It'll also give me something else to test the CDT with. I'm not sure we've tested Makefile projects enough in CDT 4 so this will give me a chance to do that. My focus will be on MinGW gdb and it would be cool to be able to contribute any fixes I come up with, or even a port to the latest gdb back to these guys.

AMD does big parallel, too.

Just read this at my favorite hardware site, Tom's Hardware (although I'm not so happy with their new look and feel). AMD is showing off their upcoming uber enthusiast platform which includes dual Agena FX quad-core processors and their (or ATI's, I'm still finding the new world order there weird) R600 multi-thread (320?) stream processor GPU based graphics card.

The demo was doing real-time face recognition on passers-by. It really shows that what used to be done as batch jobs can now be done in real-time with consumer massively parallel systems. We always wonder why we need more horse power - my web browser is working fine on my 3 year old system at home. But I think we're hitting a tipping point for a number of really cool applications that will change the way Joe Consumer looks at computers.

Which makes me even more excited to get into the raytracing hobby time project I've started to show off the CDT as an IDE for "really cool stuff". Now, to figure out how to get one of these machines :)