Sunday, May 31, 2009

Catch the Wave

It's a cold and rainy day today. We just had the freakiest hale storm I've seen in quite a while. It's a good day to be inside. While watching the Jays get pounded by the Red Sox (man, they Jays need better pitching), I watched the Google Wave keynote from last week's Google IO. BTW, I finally found a good use for twitter as I followed a couple of guys I know who were there who kept us up to date on what was going on. That was good because the video didn't show the standing O that the presentors got (it was good but standing O good?)

Anyway, I am quite impressed by the workflows and concepts behind Wave. If you haven't seen it yet, Wave is really just a good realization of collaboration tooling that we kind of see in IBM's Jazz, but more general. You could also see it as a redesign of e-mail systems to take into account and interface with or replace all the other social networking tools out there. The coolest feature is the real-time collaboration you can do, and of course, the API that allows you to play in this world too.

We've talked about IDE in the cloud a bit in Eclipse and, while not for everyone, I think it has potential. And Google Wave seems like a good framework to make this happen. Add an extension to interact with a server side Eclipse, and you get instant integration with the rest of your collaboration tools. Interesting potential.

But there was something else that struck me. Google is open sourcing pretty much all of this. They want to build Wave as an ecosystem to get as many people working through their browser as possible. I still haven't figure out how that makes Google money, but I'm sure there's a master plan at work there. But one thing you'll notice, is that Google worked for two years behind closed doors before pushing it out. This is the same way IBM put together Eclipse, how QNX pushed out CDT. It happens a lot.

And that got me thinking. Is this the only way innovation happens, i.e., in closed environments. How much innovation really happens in open source projects or at least how efficient is it? Realistically, it's a lot less than you'd think. Innovation happens when get get a crack team together in a highly collaborative environment where you don't need to spend time working with a community of diverse interests. So while open source helps make a technology popular, I don't think it's possible to create it there. But maybe that's stating the obvious.

Friday, May 29, 2009

Bjorn says "I had more to say but..."

And he turned off comments on his blog basically forcing me to put the comment here. The CDT committers know I refer to Bjorn and his use of Jedi mind tricks. And I'm falling into it again, making my thoughts visible on my own blog instead of hiding it in his comments page. You're Jedi mind tricks don't work on me, well, actually they do...

There are a lot of good things happening at Eclipse, and that you can't argue with. This week's demo of e4 self-hosting is definitely something very cool. Using the CDT in my hobby time this week doing something intense has helped me realize what a good thing we have and the great work that the contributors have put together. So as much nay saying and navel gazing we've done in recent weeks, things aren't all that bad.

But for those of us who have been around a while, the excitement at Eclipse is certainly well below the levels of those early days. I'll never forget those early EclipseCons and looking around at all the vendor positioning going on, trying to grab a piece of the spotlight. It was pretty wacky, and really fun for this lowly developer from Saskatchewan to be a part of.

But Eclipse has grown up. As Bjorn says, it is what it is. It ain't all that bad. Is it as good as we'd individually like it to be? No, it's not. That's the frustration you see. Especially as us old timers start to deal with that reality. But I'm proud to be associate with Eclipse and it has rewarded me well. And I think that's true for many.

So we keep plugging away. People will come and go, Bjorn included, and Eclipse lives on because it still meets that need that sparked the excitement in the early days. And we should all be proud of that.

Java is making me dumb

Sorry everyone who loves Java. But if you're a regular reader, you know about my love/hate relationship with Java. I hate it because I can't do the things I can do in C++ with it. But I love it because I'm making a pretty good living using it on Eclipse-based projects. So I deal.

Now, in my playing with Android and doing JNI development with it and OpenGL ES, I found myself immersed again in C++ for the first time in a very long time and loving it. One of the things you do with graphic programming is matrix multiplication. So I thought it would be cool if I created a Matrix class and an operator overload for the multiplication operator. So I could write code like this:

M4 myTransform = baseTransform * currTransform;

with all of these things as 4x4 matrices. Much less typing, no? :). So I coded it up and for the life of me I couldn't figure out why it wasn't working. Well, after mucking with it and trying different combinations I remembered something Bjorn Stoustrup mentioned in the C++ "bible". If you aren't doing anything special in your constructors, don't provide them. The compiler will take care of it for you and will do a better job of it than you could. So I removed them, and bingo, it worked!

Man I felt dumb, and at bit of a loss as to how I forgot how to do intense C++ programming. Clearly doing Java almost full time like I do, I'm out of practice. Now, you could argue that that's why people should stay away from C++, because it's too complicated. And, sure, it's not for the meek.

But I was trying to show the incredible benefits the gcc compiler gives you with performance over Java, especially in mobile where the JIT's aren't up to snuff (and Dalvik in particular doesn't even have one) and gcc can optimize using extended hardware capabilities like SIMD instructions. So there is a method to my madness and you put up with the pain to reap the rewards. Anyway, more on that after the Ottawa demo camp. Don't want to spoil the show :).

Tuesday, May 26, 2009

Tim Sweeney Interview, old story new

Gamasutra, a gaming news site, published an interview they did with the legendary Tim Sweeney of Epic Games, the Unreal and Gears of War gang. You can read it here:

http://www.gamasutra.com/view/feature/4035/from_the_past_to_the_future_tim_.php

I've read interviews from him before and have always appreciated the honest look he gives of the industry. I was even more impressed when he showed up on some of the "making of" material of my boys' Gears of War II game. He is quite the geek, but man he knows his stuff technically and he understands the business, which is a hard to find in a single person.

In this interview, the thing that really struck me was how he started Epic out of his parents house and dorm room in between jobs mowing lawns and his school work. Back in the 80's and early 90's the software industry really was driven by independent developers like this. The big guys at the time just weren't nimble enough to work with all the new technology coming along. It was an exiting time.

Zooming ahead 15-20 years, it seemed that this environment, the ability of independent developers to build software by themselves or in small teams and produce applications that sell enough to provide them with a decent living were long gone. But I'm starting to see that change. The are now a few stories of guys building iPhone apps in the same way Tim did back then and making it "rich". And as the number of mobile platforms grow and with app stores becoming an easy way for these independent developers to sell their wares, I'm sure there's a next generation Tim Sweeney coming alive out there.

Monday, May 25, 2009

Building Communities

There's been a couple of times now that presentations I've made have been recorded for posterity. I don't usually listen to them, mainly because I'm worried it really sucked and I felt embarrassed. But the main stage presentation I gave at EclipseCon was different on many levels. And having just finished listening to it, I'm glad it's been captured. It's available at Eclipse Live here:

http://live.eclipse.org/node/739

The title of the presentation is "The Rise and Fall and Rise of the CDT: Lessons on Building Communities". It is a very personal look at the history of the CDT and the lessons I learned about building communities through the roller-coaster ride I've been on for the last 7 years working on the CDT.

This little presentation is my proudest moment of my career. I put a lot on the line with it and you can tell at the beginning I was a bit nervous about that. But listening to it today, I am very happy with how it went.

If you are curious about my philosophies on working in open source communities, or just curious why I'm such a crazy man, here's an open window into my soul. And there is some humour to keep things from getting too serious.

Sunday, May 24, 2009

Common Navigator misses the mark

What's wrong with this picture:



This is a Android project that includes both JDT and CDT natures to build an Android app with a native library using the upcoming Android Native Development Kit. As you can see both JDT and CDT contribute to the list of things in the project. The Resource system would put it's own twist on the project if I hadn't turned it off.

The Common Navigator was a great idea when it was proposed. Have a single navigator framework that projects could contribute to. I have the sense it wasn't meant to handle the use case where you'd want to see a project that that has multiple natures leading to multiple contributors. The result is confusing at best, and embarrassing at the least.

But it's a good example of where the Eclipse dream of interoperability is a bit of a failure. Yes, there is a bug on this (didn't have time to look up the number but one of the CDT committers warned that this was going to happen). And my understanding is that it won't make the cut for 3.5 meaning this behaviour will be seen in commercial products as well, unless the vendors fork and fix it locally. And it's just not the JDT's fault, CDT is helping here too, and not knowing, I wonder if it's something missing from the framework to help co-ordinate this.

It's pretty disappointing and, yes, embarrassing, as I try to get the Android community to use the CDT for this style of application. There's no reason to create separate projects for Java and C++. Everything else is working fine in the one project. They'll just have to make sure the expand the right version of the src folder to find their Java source, at least there's icons that differentiate it, unlike the assets and res folders.

Friday, May 22, 2009

Mobile, the new TRS-80

I'm embarrassed to tell this story, but it gives a sense of where I come from. I was in high school in the early 80's. This is when the personal computer world had mystique. You had the Commodore PET's, VIC 20's, and 64's. You had Radio Shack's TRS-80. Before I made enough money to buy my own Coleco Adam, I used to head down to the local Radio Shack in Regina, not far from Wascana Lake, BTW, and the sales guys there would let me type away at the TRS-80 they had there. This was where my passion for computer programming got it's first taste of fuel. It was great.

The Coleco Adam which ran Apple Basic was where I got my first full time experience and wrote my first program, a simulation of the old Stock Ticker board game. For university, I convinced my parents to lend me money for a Commodore Amiga 500. That was amazing. It was also my first experience with the mantra, the best technology doesn't always win and the IBM PC blew Amiga and Apple II and the TRS-80 et al away. It wasn't until I had kids that I finally broked down and bought my first PC.

Anyway, I'm telling this story because I see that same mystique in the new mobile platforms that we have today. You have that same diversity in operating systems and user interfaces and hardware shapes and sizes. You have the fight in processors, this time ARM versus Intel Atom, that we had back then with Intel's 8086 versus the 6502 and Motorola 6800. Who's going to win? I hope they all tie.

It's an exciting time. Programmers have a whole new set of platforms to build applications for. We've been getting pretty bored building the same old apps for desktops. And now we finally have good development tools that we can use to target any of these platforms (and yes, we need to get Eclipse working for iPhone app developers). For the Ottawa Demo Camp, I plan on showing what I'm doing with Android using their Java environment along with the CDT for native libraries. The productivity I'm getting is exactly what we hoped.

I'm having a blast. Programming for Android reminds me a lot of those days I spent down at the Radio Shack with that TRS-80. This time, though I'll probably hedge my bets and take a look at the other platforms out there as well, like iPhone, Blackberry, Nokia's Qt on Symbian, and even at Moblin, which I plan on getting up and running on my Via Artigo box.

Can you tell? I'm glad that mystique is back...

Wednesday, May 20, 2009

Wascana is over

Yes, I'm giving up on the dream. For most of my time working on the CDT, I've tried to help make it the C/C++ IDE for anything and everything. A lot of people flocked to it naturally mainly in areas where it had no real competition, especially in the embedded space. And lately I'm seeing a lot of uptake in the Linux desktop space as well. And looking at the vendors contributing to the CDT, it matches what they were investing in. And that's great. We've really met, and at times exceeded, the objectives we set out for ourselves.

But I always yearned for one more. The Windows desktop space. It was Microsoft Visual Studio 6 that got me into using IDEs and built my passion to work on one. And the CDT has always had Visual Studio as one of the bars we were working towards reaching. And as Microsoft reduced their investment on Visual C++, we were able to meet that bar and scream past it. I remember reading an article where they were struggling with indexing, an area where the CDT is now really strong. That was a proud moment for me.

Yet to really show we've met the bar, I felt we needed to have a real IDE distribution that had equivalent functionality to Visual Studio. That meant including a compiler and a strong SDK for building Windows applications. Wascana was my attempt at building that IDE. And I had a lot of great feedback on it. People loved the simplicity of a single install that gave them everything to do their work.

But a few things have happened since I started Wascana. Mainly, I switched jobs and am no longer working 100% on the CDT. In fact it is much less than that, probably just enough to fulfil my duties as project lead. So any work on Wascana would have to be on my own time.

And I need to be honest with myself and you, the passion I once had for CDT for Windows (which should have been the name of this project) is waning. As I blogged about last, my attention is turning to mobile. There are lot of really cool mobile platforms out or about to be released and there is a lot of opportunity in that space to make a difference. CDT for Windows, not so much. It's too much of a fight. A fight I don't have the time or the passion to fight any more.

I know a number of people will be disappointed by that. And to those who still want to see something like Wascana, I encourage you to carry the torch. I'm not going anywhere and would be happy to provide guidance. And who knows, maybe one of the vendors that I know sell a Wascana type thing will make their work more readily available.

Anyway, I'm very excited about what's happening in the mobile space. CDT has a huge roll there so my passion for that does not wane. In fact, there is one area that I think we really need to fix, i.e. CDT's build system, that can really help mobile app developers. And if I stop pushing for an internal builder and a fancy build model that I felt we needed to match Visual Studio and instead just provide a good Makefile editor and template engine, maybe we can dig out of that hole too.

Sunday, May 17, 2009

It's Android Time

Well, I've had a lot of fun trying different things with embedded Linux and creating simple platforms. The idea was to help hobbyists and students start using CDT for building embedded applications. We have the remote download and launcher (using RSE) and gnu cross compiler support in the upcoming CDT 6.0 to help make it a real force in this area.

That work was pretty much all on hobby time. And it was fun to see it working. But I was describing that the hobby was just like work just on my own time. And my wife make a point that really hit home, "yeah, it's like work but you don't get paid". Uh, yeah.

That and there are already a lot of platforms out there for hobbyists to get starting using Eclipse for embedded development. And one of those platforms has a Market place that allows you to sell the little apps that you can make. Hobby, that pays. Hmmm.

Of course, the platform I'm talking about is Android. It has a great set of Eclipse tooling for writing apps. Yes, it's Java, and yes I bash Java regularly. But it gets the job done. And I think I have something to offer Android. Myself and a few others out on the webisphere have figured out how to build JNI libraries for Android. That gives you the best of both worlds. JDT for Java, CDT for the native libraries. You probably only need native libraries for compute intensive tasks that you can use the underlying hardware to help accelerate. But at times that is a need.

And building native libraries is pretty easy. Do a google search for Android Build System and you'll see a general description of it and the special Android.mk file you need to provide. You need to build your library in a subdirectory under the Android source tree. But once it's there you can create a CDT project that points to it. "make mymodule showcommands" and you're off to the races. And CDT's scanner discovery will even pick out the include path from the gcc/g++ commands.

Now I'm not sure how the native libraries work on real devices. It may be prohibited due to security concerns. So I'd like to try it on one before I push the idea too far. Rogers up here in Canada is finally getting Android phones in June, the same ones as T-mobile. I'll have to invest in one and see. But going through the SDK docs, I am getting the feeling this will be pretty rewarding, in more ways than one ;)

So what does this mean for my work on Wascana, which also competes for my hobby time? I'm still keen on it, and my new strategy of "stealing" the RPM contents from Fedora will allow that to take less time. But it comes back to what I was saying in my previous blog entry. It's a big burden trying to promote CDT for Windows development. Sometimes I feel alone and I wonder if anyone really cares. It's a burden I'm getting tired of carrying. Hopefully someone will step up and help, or some vendor will come in and fund some of it. We'll see.

Thursday, May 14, 2009

Too few cooks

On Planet Eclipse we've had some harsh but healthy discussion on how Eclipse could be better organized for success. There are probably little things that need to be done, but at the end, I think the conclusion is that the Foundation staff are doing a pretty good job and Eclipse is organized to accomplish what it was intended to, to provide an open ecosystem for companies to work together on Eclipse platform technologies.

But I am still very worried about Eclipse. And the main problem I am seeing is that there are too few people making the wheels turn, especially as we push Galileo out the door. I figure if we removed probably 3 or 4 people from the equation, Galileo would stop dead in it's tracks and the release wouldn't happen. That is scary in my books. Even on the CDT, if I didn't do the countdown to releases, as I've done for the past 5 years or so, the CDT wouldn't release either. Or at the very least, someone else would have to jump in. And I won't even start with the e4 flexible resource project which currently has one part time guy actually doing code.

That's a pretty big burden for those few. And give the number of commercial vendors that rely on Eclipse technologies, I wonder if it's fair. The good news is that these few people have a lot of passion and are very focused on making Eclipse releases happen. So it will be a success. But understaffed projects is a chronic problem at Eclipse and that's where I fear it's future.

Maybe this fear comes from my own situation where I've been reduced to pretty much working on CDT in my hobby time plus a few hours here and there in my day job. But I know a lot of people in the same position and really wonder how we'll accomplish our goals in the upcoming year for Eclipse. That's the reality we need to figure out how to deal with and improve. Eclipse governance problems is incredibly minor compared to that.

Friday, May 08, 2009

MinGW Cross for Linux

I mentioned in my last entry that building stuff on my laptop leaves scorch marks on my legs (if you haven't read it, that won't make sense). This is especially true when building in my Windows VM. But I figure it wasn't so bad when building from within Fedora itself. And since I knew from my experience on the MinGW mailing lists that the developers there mainly use Linux as their build hosts, I figured that maybe a better path towards building what I need for Wascana would be do it that way to. Build a cross-compiler that ran on Linux and built Windows libraries.

Then I remembered that I ran into Andrew Overholt from Red Hat and he was excited to let me know that they were working hard on mingw packages for Fedora. That was before my Fedora days so I kinda blew it off, as "sounds good, I guess" (sorry Andrew!). So when I googled to find out how to build a cross compiler for MinGW, of course, I ran into the work at Fedora that he was talking about. And it blew me away. Not only are they packaging up the cross compiler, they're also packaging up a pretty thorough list of libraries to help you build Windows apps, many more than I was thinking of for Wascana.

So that's led me to wonder, since I'm now a Fedora fanboy :P, why not sync Wascana up to these packages. That way, whether your building your apps from Linux or from Windows using Wascana, you have the same library set available to you. I should also make sure the versions line up for the toolchain so that there are no ABI issues on the C++ side (which probably isn't a real problem anyway).

And interestingly enough, it brings into play the Cross GCC compiler integration that I build for the tutorial I gave at EclipseCon which I have just contributed to CDT 6.0. I should make sure you can use it easily in this environment too.

The team building the cross mingw stuff for Fedora have made all the decisions and I can extract what I need from their RPMs to install into Wascana. In fact a number of the packages already have zip files ready to grab with p2. Thank you Fedora team!

Thursday, May 07, 2009

At home in my Fedora

Well, it's been about a month now and I'm starting to really feel at home with Fedora on my main machine, the Dell D830 laptop. The longer you go, the more packages you install that you find you need and eventually you end up with something pretty decent. My biggest issue was dealing with 32-bit apps and missing the 32-bit libs they need to run which you don't get when your in 64-bit mode (I have 4GB RAM and want to use it all). But that's pretty solvable if not time consuming. And then there's the never ending issue of docking and undocking the laptop, going from single screen to my second 24" LCD at work to my alternate second 22" at home. I still have to reboot to get the thing to recognize the second screens, and I even get the occasional hang when coming out of standby. But I'm dealing with that.

There are some things that just rock. I've been playing with the VIA Artigo I got at ESC a couple of years ago and building embedded images and toolchains and kernels and libraries is a dream with the tools available on Linux. There's no beating "-o loop" to peek at and create a file system image (just remember to type the right /dev name when doing a mkdosfs, doh!). Not to mention having all the utils and libs necessary to build cross compilers and such. And I've played with both buildroot and OpenEmbedded which are things you only want to do on Linux.

One thing I am learning though is that anything compute intensive probably shouldn't be done on a laptop. I think I'm melting my video chip as the external screen goes into fits as the disk and CPU get busy. Not to mention the scorch marks on my legs trying to watch the NHL playoffs and build a cross gcc at the same time. So I'm starting to use my Dell desktop at work more where I've learned the slickness of "ssh -Y". You can't do that on Windows.

But I still have a Windows XP image running in VirtualBox for those times I do need Windows. I still use Outlook for work. There's no beating it for the high volumes I get and the Calendar is top notch, and despite Evolution's attempt at Exchange integration, it's not quite ready for prime time. And of course, I need Windows to finish up Wascana 1.0 which almost ready for beta trials with Galileo. And I have a partition open to try it on Windows 7 (especially now that I mkdosfs'ed over my old Windows install, same doh!).

But I can't remember how many times I've gone between the Windows VM and Linux and forgot which one I was in. That is telling me that the Linux desktop isn't all that bad. I still hate the fonts (been complaining about that for years) and the multi-screen situation, but the rest is making it worth it. Highly recommended, at least if you're someone like me.

Monday, May 04, 2009

Navel Gazing

Man, we're sure doing a lot of navel gazing on Planet Eclipse about the future of Eclipse, thanks a lot to Bjorn's own navel gazing. I'm not sure why we are doing this since I am sure that Mike and the gang do that quite a lot themselves already and I'm not sure we're helping.

Anyway, I think it is a natural process to go through at this stage of Eclipse's life. Since the original consortium was created back in November 2001 (according to wikipedia, but I think that's pretty accurate), the software industry has changed a lot as you'd expect in 7+ years. Open source was pretty new back then and it was a bold move by IBM to start this thing. And, at the time, having an organization to promote and manage the growth Eclipse was critical to it's success. It made open source friendly to companies that feared it and it really pushed them into a new way of thinking.

Fast forward to today, there's no doubting that open source is now, not only accepted, but for many companies, it's the preferred way of working, especially on commodity platforms. Companies that used to play commercially in that space have moved on to higher ground or found niches to sell their wares, or disappeared all together, or are Microsoft. Open source is a commercial force, no matter what the Free Software people would like to think.

So what does this mean for Eclipse? I'm not sure. And that's why I am appreciating the navel gazing going on. What should it mean?

That made me wonder about Linux. Yes, it has a Foundation, but in the timeline of Linux, that's actually a fairly new development. Linux survived for years as a free software project. But if you look at the latest stats that I found (Apr 2008), you'll see that more that 3/4's of Linux development is done by developers that work for commercial vendors that have a vested interest in it. Surprisingly 9% of that development is by IBM.

I think it's pretty easy to imagine that if horrible things came to pass and the Foundation ceased to exist, Eclipse would live on. And for the same reasons Linux has. Too many companies have a vested interest in it to leave it for dead. And yes, the larger companies would want some sort of insurance against bad things, so you'll always need a Foundation, just like Linux has.

But how influential should that Foundation and the members that support it be in the day to day operation of Eclipse developers? Right now, it's a lot. The freedom of developers is very restricted relative to other open source projects. For some, it's even more restrictive than what their employers allow. And, sure, it's less restrictive than many commercially sponsored open source projects (OpenOffice comes to mind). But where are we on the open source project health scale? Or does that matter at all? Are things really all that bad right now?

No, no solution to the navel gazing here, and I guess more questions than answers. But it is a good time to think about it. Or maybe, I'm just doing my own navel gazing...