Well we got off to a great start at the Summit today. The day went by real fast and we were all pretty burned out by the end of it so it must have been good :). One highlight for me was when I asked for hands on who was a committer. I got the 7 or so I was expecting. I then asked who had contributed patches. To my happy surprise, I got over 20. That explains why we have so many patches outstanding in bugzilla for the CDT. It is certainly one sign the CDT contributor community is healthy, but its also a sign that we have a lot of work ahead to keep up and to start nominating more committers.
We spend the day introducing eachother and then dug deep into the CDT DOM. I have to admit that one was really dry, and I was the one giving it. We then got an update from the Intel team on what they'd like to do with the build information in the new project wizard and the project properties. Looks like a big change that should hopefully smooth out some workflow issues that we have there. Another big day tomorrow as we review some of the new source navigation and indexing features and dig into debug.
Another thing I'm trying is Skypecast to broadcast the proceedings. You can check it out by following the link. It is definitely a technology preview and we had a hard time getting remote people hooked up to the sound system we have running without horrible echo and feedback. But the broadcast out sounds O.K. (as long as I mute the mic on my laptop, sorry Norbert!) I'm sure it would work better if everyone was working through headsets, instead of trying what we're doing with capturing the audio through a sound board. But it is an interesting way of communicating when working on open source projects.
Wednesday, September 20, 2006
Monday, September 18, 2006
CDT Summit Eve
The CDT Fall summit for this year starts tomorrow here at QNX headquarters and I'm getting both excited and nervous, as I guess I should. We've had a number of summits in the past starting all the way back with the first one in July 2002 when QNX brought the CDT as we know it to the world. That summit and pretty much everyone since then has had kind-a the same feel. Lots of people have been interested in what was going on, but few have had the resources to commit to helping out.
But you know, there's nothing wrong with that. It's a pretty difficult decision for corporations to commit resources to work on open source projects. It's difficult to track the return on their investment and there are a slough of legal issues that need to be addressed and tracked to make sure the IP walls are set up correctly, and I'm not just talking networking. I've learned to be patient, not get too down when hopes fail to materialize. In the end, simply using the CDT and distributing it in their products means that the CDT is getting good test coverage, which is just as important these days.
My feel for this one, though, is different. Maybe it's because I'm starting to be overly optimistic. We have around the same number of attendees registered as we did last year, and a lot of them are the same faces. However, this year most of the attendees have been contributors to the CDT. Some have become committers over the year and some will become committers shortly after the summit. We'll use this as an opportunity make sure we are talking to eachother and co-ordinating our work. We'll also use it as a team building exercise as I'm sure we'll find a few battles along the way. It sure helps when you know the person at the other end of the bugzilla entry when smoothing over issues.
Finally this year, it looks like our contributors summit will focus much less on recruiting contributors and much more on co-ordinating actual contributions. It's a much funner summit to run and hopefully a much funner summit to attend.
But you know, there's nothing wrong with that. It's a pretty difficult decision for corporations to commit resources to work on open source projects. It's difficult to track the return on their investment and there are a slough of legal issues that need to be addressed and tracked to make sure the IP walls are set up correctly, and I'm not just talking networking. I've learned to be patient, not get too down when hopes fail to materialize. In the end, simply using the CDT and distributing it in their products means that the CDT is getting good test coverage, which is just as important these days.
My feel for this one, though, is different. Maybe it's because I'm starting to be overly optimistic. We have around the same number of attendees registered as we did last year, and a lot of them are the same faces. However, this year most of the attendees have been contributors to the CDT. Some have become committers over the year and some will become committers shortly after the summit. We'll use this as an opportunity make sure we are talking to eachother and co-ordinating our work. We'll also use it as a team building exercise as I'm sure we'll find a few battles along the way. It sure helps when you know the person at the other end of the bugzilla entry when smoothing over issues.
Finally this year, it looks like our contributors summit will focus much less on recruiting contributors and much more on co-ordinating actual contributions. It's a much funner summit to run and hopefully a much funner summit to attend.
Friday, September 08, 2006
Open Source Hardware
I'm sure I've told this story before somewhere, but I used to sit across from an ASIC designer many years ago. This is when I first started using ObjecTime Developer for software modeling back in the early 90's, a couple of years before I joined the company. He was marveling at how we software designers had started using graphical tools, just as ASIC designers were abandoning similar tools for textual description languages. I'm not sure why they made that transition, but given the complexity of the chips they were designing, my bet is that the graphics didn't scale well for them and the tools back in the early 90's weren't very good - no Eclipse back then!
This guy was coding in a hardware description language called Verilog. I peaked over his shoulder one day and saw that it looked a lot like C code. I found that very interesting but it took many years before I sat down and took the time to learn a bit more about the language and what it could do (there wasn't much of an Internet back then either). It indeed was C-like and was structured a lot like C, and I'm sure suffers the same scalability issues that programming in C can sometimes cause. Thankfully, there is an Eclipse plug-in to help you write your own Verilog code.
Fast forward to the recent future and my interest in MultiCore processing, I found it quite interesting when Sun announced that they were open sourcing their Niagara line of processors. Diving deeper, I was able to find the Verilog code for their T1 chip published on www.opensparc.org. Other than being cool to look at and maybe interesting for students to learn CPU design with, I didn't really see the benefits of open sourcing a CPU design.
Then yesterday, I ran across an announcement from Simply RISC that their engineers had taken the open source T1 code and made a simple SPARC embedded processor out of it. Of course with the T1 source being GPLed, they have released the source for their CPU as well. Is this the start of something? I'm still a bit doubtful. Chip companies make most of their money on the designs they come up with, not necessarily the chips themselves. But it is an interesting phenomenum to watch out for.
This guy was coding in a hardware description language called Verilog. I peaked over his shoulder one day and saw that it looked a lot like C code. I found that very interesting but it took many years before I sat down and took the time to learn a bit more about the language and what it could do (there wasn't much of an Internet back then either). It indeed was C-like and was structured a lot like C, and I'm sure suffers the same scalability issues that programming in C can sometimes cause. Thankfully, there is an Eclipse plug-in to help you write your own Verilog code.
Fast forward to the recent future and my interest in MultiCore processing, I found it quite interesting when Sun announced that they were open sourcing their Niagara line of processors. Diving deeper, I was able to find the Verilog code for their T1 chip published on www.opensparc.org. Other than being cool to look at and maybe interesting for students to learn CPU design with, I didn't really see the benefits of open sourcing a CPU design.
Then yesterday, I ran across an announcement from Simply RISC that their engineers had taken the open source T1 code and made a simple SPARC embedded processor out of it. Of course with the T1 source being GPLed, they have released the source for their CPU as well. Is this the start of something? I'm still a bit doubtful. Chip companies make most of their money on the designs they come up with, not necessarily the chips themselves. But it is an interesting phenomenum to watch out for.
MultiCore: The True Promise of Eclipse
So, I needed a board to help try out some JTAG things (for those readers not involved with embedded development, a board is a little computer kind-a thing). We had just received an OMAP board which uses a TI chip that contains both an ARM general purpose processor as well as a TI DSP (digital signal processor). Of course, my focus was on the ARM processor that runs our operating system and it was pretty cool to getting it up and running with little effort.
But after a while, I started wondering what people use this board for. I've been away from embedded development for a few years and man have things changed while I was away. I soon discovered that the main use of this thing is for audio processing. There are some audio jacks as well as a connector to plug in an LCD screen. By programming some audio processing algorithms into the DSP, you could make a pretty cool multimedia device with this thing.
My curiosity then wondered over to how one would program the DSP. If I had a compiler with an integration with the CDT and a debugger that understood how to debug the DSP and that was also integrated with the CDT, then I'd then have a complete multi-core development solution where I could have regular software projects and DSP projects and work on them all at the same time.
It's a very interesting time in the embedded industry with the multi-core phonmenun. I think we'll see a lot of new processors come out that have specialized parts. What I hope to see, and I'm pretty sure it will happen, is different vendors working together integrating their Eclipse-based technologies and unify their development activities into a single workflow for the developer who sees these boards as a single target. That is the true promise of Eclipse!
But after a while, I started wondering what people use this board for. I've been away from embedded development for a few years and man have things changed while I was away. I soon discovered that the main use of this thing is for audio processing. There are some audio jacks as well as a connector to plug in an LCD screen. By programming some audio processing algorithms into the DSP, you could make a pretty cool multimedia device with this thing.
My curiosity then wondered over to how one would program the DSP. If I had a compiler with an integration with the CDT and a debugger that understood how to debug the DSP and that was also integrated with the CDT, then I'd then have a complete multi-core development solution where I could have regular software projects and DSP projects and work on them all at the same time.
It's a very interesting time in the embedded industry with the multi-core phonmenun. I think we'll see a lot of new processors come out that have specialized parts. What I hope to see, and I'm pretty sure it will happen, is different vendors working together integrating their Eclipse-based technologies and unify their development activities into a single workflow for the developer who sees these boards as a single target. That is the true promise of Eclipse!
Sunday, September 03, 2006
Did someone say doughnuts?
I just read Bjorn's note comparing doughnut stores to open source businesses. From what I hear that note probably made more an impact on Canadians that it did others. For some reason, we've rallied around the doughnut as our national past time and our waste-lines are paying the price!
At any rate, I totally agree with his assessment. My spin on it, you can make money by packaging up open source and selling priority support for it, and you can make money by taking open source and customizing it for a small vertical market. Certainly we at QNX are doing the second, taking Eclipse and customizing it to work well for developers writing applications for our operating system.
Another analogy I thought of also has to do with Tim Hortons. After Wendy's (the burger Wendy's) merged with Tim Hortons you started seeing a lot of Wendys and Timmy's co-located in the same restaurant. So the analogy could go that people love doughnuts. So when the come to Tim's and get their fix, they see the Wendy's there and decide to stay for lunch.
So what I've also seen vendors do is package Eclipse as a sort of loss leader to get people interested in their higher margin products. My recent blog on the JTAG vendor Ronetix is an example of that. And I think we'll see a lot more as well as Eclipse becomes ubiquitous (2.27 million users!). Vendors will find they have to play the Eclipse game just to keep up with the Jones.
At any rate, I totally agree with his assessment. My spin on it, you can make money by packaging up open source and selling priority support for it, and you can make money by taking open source and customizing it for a small vertical market. Certainly we at QNX are doing the second, taking Eclipse and customizing it to work well for developers writing applications for our operating system.
Another analogy I thought of also has to do with Tim Hortons. After Wendy's (the burger Wendy's) merged with Tim Hortons you started seeing a lot of Wendys and Timmy's co-located in the same restaurant. So the analogy could go that people love doughnuts. So when the come to Tim's and get their fix, they see the Wendy's there and decide to stay for lunch.
So what I've also seen vendors do is package Eclipse as a sort of loss leader to get people interested in their higher margin products. My recent blog on the JTAG vendor Ronetix is an example of that. And I think we'll see a lot more as well as Eclipse becomes ubiquitous (2.27 million users!). Vendors will find they have to play the Eclipse game just to keep up with the Jones.
Saturday, September 02, 2006
Windows SDK RC1
I've stated a few times now that although the CDT is highly used in the embedded and Linux/Unix markets, I don't think we've conquered the world until the CDT is seen as a valid alternative to Visual Studio for Windows development. Looking at the whole Eclipse ecosystem and all the components available for it, I just think it has a higher value proposition than Visual Studio. At the very least you get a cross platform development environment that you can theoretically build any application with, once all the pieces are there that is.
So I've been working a little on adapting the CDT for Windows development starting with support for Microsoft's C++ compiler. Over the last couple of years they've been shipping it for free, first as a separate toolkit, and now as part of the .Net 2.0 SDK. But, in order to get it working, you had to download a few pieces, including the Platform SDK, and if you wanted to do debugging outside of Visual Studio, the Debugging Tools for Windows. I felt it was pretty complicated to set up, especially for newbies, and of course these pieces aren't redistributable so we couldn't shrink wrap it for you.
But someone pointed me at the new Windows SDK which is part of the Vista program (which is why I was confused since I thought it was a Vista thing only, but it is not). This SDK has recently reached Release Candidate 1. As described in this MSDN TV program (these programs are pretty useful and something we should consider for Eclipse), this new SDK is really a combination of all the pieces you need to build Windows applications, both managed (i.e. .Net) and unmanaged (i.e. native).
What I found interesting was their focus on providing command line tool support for "people who like to work that way". Now, I don't know anyone developing Windows applications that like to work that way. So I read into it that they are really talking about 3rd party IDEs such as the CDT. With the tools provided by this SDK, it should be a pretty simple matter of integrating them as a tool chain just as we do with the gnu tools. Download the SDK, download the CDT and the Windows integration, and you are off and running.
At least that's my hope, which of course will only be successful if it receives community attention. But it sure would be a boost for Eclipse to be seen as the development environment for everyone, without prejudice.
So I've been working a little on adapting the CDT for Windows development starting with support for Microsoft's C++ compiler. Over the last couple of years they've been shipping it for free, first as a separate toolkit, and now as part of the .Net 2.0 SDK. But, in order to get it working, you had to download a few pieces, including the Platform SDK, and if you wanted to do debugging outside of Visual Studio, the Debugging Tools for Windows. I felt it was pretty complicated to set up, especially for newbies, and of course these pieces aren't redistributable so we couldn't shrink wrap it for you.
But someone pointed me at the new Windows SDK which is part of the Vista program (which is why I was confused since I thought it was a Vista thing only, but it is not). This SDK has recently reached Release Candidate 1. As described in this MSDN TV program (these programs are pretty useful and something we should consider for Eclipse), this new SDK is really a combination of all the pieces you need to build Windows applications, both managed (i.e. .Net) and unmanaged (i.e. native).
What I found interesting was their focus on providing command line tool support for "people who like to work that way". Now, I don't know anyone developing Windows applications that like to work that way. So I read into it that they are really talking about 3rd party IDEs such as the CDT. With the tools provided by this SDK, it should be a pretty simple matter of integrating them as a tool chain just as we do with the gnu tools. Download the SDK, download the CDT and the Windows integration, and you are off and running.
At least that's my hope, which of course will only be successful if it receives community attention. But it sure would be a boost for Eclipse to be seen as the development environment for everyone, without prejudice.
Friday, September 01, 2006
CDT everywhere
I continue to be surprised by how many vendors are redistributing the CDT with their products. Lately I've become interested in JTAG hardware debuggers and how to best hook them up to Eclipse for some real low level bit hacking debug workflows. This is something that probably deserves it's own blog entry and it's not really the theme of this story.
At any rate, I ran across a JTAG vendor called Ronetix who appears to build a pretty full featured device similar to the Abatron device I've been playing with lately. So quickly browsing Ronetix web site, I see that they have a Starter Kit that they sell. Low and behold it "Includes Eclipse IDE". Going to the product page for the starter kit I see they have a screenshot of Eclipse in action, and, yes, it is the CDT.
At some point I need to sit down and figure out what is driving the success of the CDT. It certainly fills a need that maybe isn't getting addressed by others, i.e., an IDE for non-Windows development that is extensible and ubiquitous (mind you I'm still keen on CDT for Windows development too). I'll have to ask the 34 developers that are currently registered for the upcoming CDT Contributors Summit why they find the CDT important enough to invest in. No matter the reason, it's been a fun ride and we're looking forward to a great year of collaboration toward CDT 4.0.
At any rate, I ran across a JTAG vendor called Ronetix who appears to build a pretty full featured device similar to the Abatron device I've been playing with lately. So quickly browsing Ronetix web site, I see that they have a Starter Kit that they sell. Low and behold it "Includes Eclipse IDE". Going to the product page for the starter kit I see they have a screenshot of Eclipse in action, and, yes, it is the CDT.
At some point I need to sit down and figure out what is driving the success of the CDT. It certainly fills a need that maybe isn't getting addressed by others, i.e., an IDE for non-Windows development that is extensible and ubiquitous (mind you I'm still keen on CDT for Windows development too). I'll have to ask the 34 developers that are currently registered for the upcoming CDT Contributors Summit why they find the CDT important enough to invest in. No matter the reason, it's been a fun ride and we're looking forward to a great year of collaboration toward CDT 4.0.
Tuesday, August 15, 2006
Greenphone, the open source phone
Everytime I look at the net these last few days, something pretty cool pops up. This time was an announcement from Trolltech, the Qt people, about a new product they plan on releasing in September called Greenphone. This is a GSM/GPRS phone built on an embedded Linux kernel with Trolltech's embedded version of Qt called Qtopia. It is really only sold as a development platform and comes with the necessary SDK.
Now, I think this is a bit different than the open source gaming device I talked about earlier. I don't think Trolltech wants to get into the phone business. In some ways, I think they are just curious about the kind of applications people will build for such a device. And, of course, in the end their goal is to sell more Qtopia licenses to commercial developers.
But I've always wondered what kind of applications make sense on such a small platform. Web browsing when the screen is only 240 pixels wide makes even less sence than browsing the web on a TV. I'll be watching along with Trolltech to see what people will come up with. And, as always, it'll be interesting to see how many people use the CDT to develop for this platform.
Now, I think this is a bit different than the open source gaming device I talked about earlier. I don't think Trolltech wants to get into the phone business. In some ways, I think they are just curious about the kind of applications people will build for such a device. And, of course, in the end their goal is to sell more Qtopia licenses to commercial developers.
But I've always wondered what kind of applications make sense on such a small platform. Web browsing when the screen is only 240 pixels wide makes even less sence than browsing the web on a TV. I'll be watching along with Trolltech to see what people will come up with. And, as always, it'll be interesting to see how many people use the CDT to develop for this platform.
Microsoft XNA Express, maybe they do get it...
Continuing on my vacation game development theme (and yes, I am spending a lot of time with my family and doing things around the house, it's not all geek time ;), Microsoft has just announced that they will be releasing an Express version of their XNA Game Studio for free for Windows development and only $99 for Xbox 360 development. This offering will build on top of their free Visual C# Express IDE and will include some tools for integrating content as well as their XNA Framework game-engine-type-thing. They are really pushing for game development in C# and the CLR, even for the Xbox 360.
As the guy in their XNA Overview video mentioned, the game developer market is pretty small relative to others and selling tools to this market isn't going to be a money maker. What's important to Microsoft is that they help developers as much as they can to get them building content for Microsoft's platforms. It doesn't really matter how much they charge for the tooling and frameworks since they will make their money on the platforms. And with good free offerings, they'll get the kids hooked making games for Microsoft platforms and that will carry that into their careers as professionals.
I am still of the opinion that Eclipse can be an even greater game development environment since it is truly multi-platform. There's no reason why we couldn't build a set of plug-ins that allow developers to target all of the consoles and all of the desktop platforms, including Microsoft's.
Actually there may be one reason, who's going to pay for it? Microsoft is busy devoting itself to Visual Studio, and I haven't seen much interest from the other vendors in contributing to such an open source project (although I know from bug reports and one quick discussion years ago that Sony Playstation group is or at least has used the CDT). It would take some sort of consortium to organize and pay for the project and get involvement from the various players. It could be done and it would be cool for Eclipse but I'm not sure that industry is ready for such co-opetition as much as the embedded industry is.
As the guy in their XNA Overview video mentioned, the game developer market is pretty small relative to others and selling tools to this market isn't going to be a money maker. What's important to Microsoft is that they help developers as much as they can to get them building content for Microsoft's platforms. It doesn't really matter how much they charge for the tooling and frameworks since they will make their money on the platforms. And with good free offerings, they'll get the kids hooked making games for Microsoft platforms and that will carry that into their careers as professionals.
I am still of the opinion that Eclipse can be an even greater game development environment since it is truly multi-platform. There's no reason why we couldn't build a set of plug-ins that allow developers to target all of the consoles and all of the desktop platforms, including Microsoft's.
Actually there may be one reason, who's going to pay for it? Microsoft is busy devoting itself to Visual Studio, and I haven't seen much interest from the other vendors in contributing to such an open source project (although I know from bug reports and one quick discussion years ago that Sony Playstation group is or at least has used the CDT). It would take some sort of consortium to organize and pay for the project and get involvement from the various players. It could be done and it would be cool for Eclipse but I'm not sure that industry is ready for such co-opetition as much as the embedded industry is.
Sunday, August 13, 2006
GP2X - The open source handheld gaming system
Well, I'm on holidays right now but I still like to keep in touch with what' s happening in the industry and still monitor a few Internet rag sites regularly, including my favorite, The Inquirer. Today, I saw in one of their Hardware Roundup postings a link to a review of the GP2X Personal Entertainment System which uses an ARM dual core processor that runs Linux. I've always been interested in game development, so finding a handheld gaming machine that ran Linux sent me off on a trail to find out more.
Well it turns out it's made in Korea by Gamepark Holdings as a follow up to a previous edition handheld which was actually made by another company called Gamepark. Apparently the engineers didn't like what the original company wanted to as a follow up so spun out and made an almost identical company to do it the way they wanted. Interesting inside story there, I'm sure.
Anyway, they advertise this machine as the "Open Source Gaming Device", which I find pretty cool and again fits into the model I've seen over and over again with open source development. The company sells the device (and it's pretty cheap at only about $200), and then fosters an open source community around writing software for it and manages an SDK of open source libraries to support them. They also use a number of the open source Linux apps to build up a suite of multi-media functions for video and audio for users to get started. I haven't seen any analysis about how successful they've been but the community forums seem to be pretty active.
I was a bit disappointed, of course, when I saw that the SDK didn't ship with Eclipse/CDT components, but I was happy to see someone in their community blogging about using the CDT in this environment. Of course, it's a natural fit with CDT's built-in support for gnu development, including cross-development for embedded operating systems such as Linux (and QNX Neutrino ;). I would be quite interested in helping anyone who would like to push to make the CDT a more formally "supported" development environment for this cool little box.
Well it turns out it's made in Korea by Gamepark Holdings as a follow up to a previous edition handheld which was actually made by another company called Gamepark. Apparently the engineers didn't like what the original company wanted to as a follow up so spun out and made an almost identical company to do it the way they wanted. Interesting inside story there, I'm sure.
Anyway, they advertise this machine as the "Open Source Gaming Device", which I find pretty cool and again fits into the model I've seen over and over again with open source development. The company sells the device (and it's pretty cheap at only about $200), and then fosters an open source community around writing software for it and manages an SDK of open source libraries to support them. They also use a number of the open source Linux apps to build up a suite of multi-media functions for video and audio for users to get started. I haven't seen any analysis about how successful they've been but the community forums seem to be pretty active.
I was a bit disappointed, of course, when I saw that the SDK didn't ship with Eclipse/CDT components, but I was happy to see someone in their community blogging about using the CDT in this environment. Of course, it's a natural fit with CDT's built-in support for gnu development, including cross-development for embedded operating systems such as Linux (and QNX Neutrino ;). I would be quite interested in helping anyone who would like to push to make the CDT a more formally "supported" development environment for this cool little box.
Thursday, July 27, 2006
Ballmer: Software is becoming a service
Remember my blog entry on "Software as a Service Industry". Well, I had a chuckle when I read today's ZDNet top story: "Ballmer: Software is becoming a service". See, it's not just me, lol.
I think Microsoft will have a very hard time turning into a services and solutions company. They've spent decades now focusing on building and selling great products. The paradigm shift will certainly confuse their customers for the first little while, if not their employees.
But I see it everyday. Every time a customer comes in with a specific requirement that really only applies to their environment, the stronger I feel that selling software out of a box just won't cut it any more, at least for complex software we tools builders end up making. With the ongoing costs of development and maintenance of that software, it makes more sense spreading out the revenue to match. And it places an even higher importance on the extensibility of that software, just as we see in Eclipse projects today.
So, we'll see how this all pans out, but if Mr. Ballmer says its true, it must be true, :)
I think Microsoft will have a very hard time turning into a services and solutions company. They've spent decades now focusing on building and selling great products. The paradigm shift will certainly confuse their customers for the first little while, if not their employees.
But I see it everyday. Every time a customer comes in with a specific requirement that really only applies to their environment, the stronger I feel that selling software out of a box just won't cut it any more, at least for complex software we tools builders end up making. With the ongoing costs of development and maintenance of that software, it makes more sense spreading out the revenue to match. And it places an even higher importance on the extensibility of that software, just as we see in Eclipse projects today.
So, we'll see how this all pans out, but if Mr. Ballmer says its true, it must be true, :)
Tuesday, July 25, 2006
"AMD to buy ATI"
Now, have you not seen that headline enough yet?
Any time there's a bit of a shakedown in our industry I'm always intrigued. It's not what the industry analysts have to say about it, and certainly not what you read in the press release from the parties involved. It's the story behind the story that piques my interest.
So, I blast through all the reports and try to piece together what is really happening and what it means to our future. For the AMD/ATI thing, the Inquirer yet again puts forth a interesting view on the insider story. Whether what they say is true or not we may never know, but I have seen a lot of rumors posted there that eventually became fact, including the AMD/ATI deal.
I do think that they present a good argument for what is happening, and it seems to be driven by the end of the MHz race (thanks, I can cook a roast in my PC case now, enough already!) and the push for many-multi-core a la Sun's Niagara architecture. AMD also has some pretty cool ideas on how to integrate co-processors that do cool things into their cache-coherent architecture and I'm sure the ATI acquisition will help speed some of these along. And the Inq is pretty sure Intel is working on similar architectures.
So what does that mean for us tools developers? Well, these events really give me more confidence in my prediction that a programming model change is a-coming. Applications will more and more need to take advantage of a multi-threaded environment to get performance gains. We can no longer rely on ever increasing MHz to save us. For C and C++, it means building more multi-threading constructs into the language. Something the Parallel Tools (PTP) people are working on building tooling for APIs like OpenMP.
As I'm sure everyone who's built a multi-threading application (such as Eclipse plug-ins) know, working in this environment is difficult and somewhat unpredictable. The door is wide open for a new set of analysis tools that we can use to scope out when things are going wrong. And I'm sure our experience with such tools in the embedded industry, where we have had to deal with unpredictability of environments for a very long time now, will become of value to everyone.
It's an interesting time again in our industry and we'll all need to keep our eyes on it and be ready to hold on tight as yet another paradigm begins to shift.
Any time there's a bit of a shakedown in our industry I'm always intrigued. It's not what the industry analysts have to say about it, and certainly not what you read in the press release from the parties involved. It's the story behind the story that piques my interest.
So, I blast through all the reports and try to piece together what is really happening and what it means to our future. For the AMD/ATI thing, the Inquirer yet again puts forth a interesting view on the insider story. Whether what they say is true or not we may never know, but I have seen a lot of rumors posted there that eventually became fact, including the AMD/ATI deal.
I do think that they present a good argument for what is happening, and it seems to be driven by the end of the MHz race (thanks, I can cook a roast in my PC case now, enough already!) and the push for many-multi-core a la Sun's Niagara architecture. AMD also has some pretty cool ideas on how to integrate co-processors that do cool things into their cache-coherent architecture and I'm sure the ATI acquisition will help speed some of these along. And the Inq is pretty sure Intel is working on similar architectures.
So what does that mean for us tools developers? Well, these events really give me more confidence in my prediction that a programming model change is a-coming. Applications will more and more need to take advantage of a multi-threaded environment to get performance gains. We can no longer rely on ever increasing MHz to save us. For C and C++, it means building more multi-threading constructs into the language. Something the Parallel Tools (PTP) people are working on building tooling for APIs like OpenMP.
As I'm sure everyone who's built a multi-threading application (such as Eclipse plug-ins) know, working in this environment is difficult and somewhat unpredictable. The door is wide open for a new set of analysis tools that we can use to scope out when things are going wrong. And I'm sure our experience with such tools in the embedded industry, where we have had to deal with unpredictability of environments for a very long time now, will become of value to everyone.
It's an interesting time again in our industry and we'll all need to keep our eyes on it and be ready to hold on tight as yet another paradigm begins to shift.
Tuesday, July 18, 2006
Sustaining Open Source Projects Through Turnover
When you have an open source project such as the CDT that has been around for a while, you end up having to deal with turnover in the people that are working on that project. There are usually a couple of reasons I've seen as to why this happens. Either they have been revectored or promoted to work on something else, or they've left the company that was contributing the resource to a company that doesn't want to invest their resources that way. (As an interesting side note, we have quite a few examples now of people who have switched companies but are still working on the CDT, including yours truely, but that's a topic all on its own...).
In dealing with turnover, I find myself going through a paradigm shift from young project to mature project. In a young project, you are struggling to get people and organizations to contribute to your project. So you find yourself accepting contributions that may not perfectly fit the mould and architecture you are trying to set out, but getting those contributions mean getting people involved and showing the world that your project has momentum and is "the exciting place to be".
But with turnover, without proper documentation, automated tests, and good architectural fit, you start finding that that code that helped get your project going now becomes extra baggage. You start struggling to add new features and you find you need to either replace or simply remove the functionality it provided. Without someone to keep the code alive, it quickly gathers "rust" which starts to spread to places where you are trying to do new work.
So the lesson of the day for me is too keep the long term vision, including a well laid out architecture, for the project front and center from day one. Try to influence new contributors to follow that vision and to manage the churn in that vision so that you can sustain the code as long as you can. This is all basic software engineering school stuff, but it applies to open source projects as much as it does to commercial. And I think I am now of the opinion that having a strong vision like this can serve as much of a draw for contributors as a wide open door does. Or maybe the growth in the CDT lately has given me a bit more confidence. Or maybe its my new rose colored glasses...
In dealing with turnover, I find myself going through a paradigm shift from young project to mature project. In a young project, you are struggling to get people and organizations to contribute to your project. So you find yourself accepting contributions that may not perfectly fit the mould and architecture you are trying to set out, but getting those contributions mean getting people involved and showing the world that your project has momentum and is "the exciting place to be".
But with turnover, without proper documentation, automated tests, and good architectural fit, you start finding that that code that helped get your project going now becomes extra baggage. You start struggling to add new features and you find you need to either replace or simply remove the functionality it provided. Without someone to keep the code alive, it quickly gathers "rust" which starts to spread to places where you are trying to do new work.
So the lesson of the day for me is too keep the long term vision, including a well laid out architecture, for the project front and center from day one. Try to influence new contributors to follow that vision and to manage the churn in that vision so that you can sustain the code as long as you can. This is all basic software engineering school stuff, but it applies to open source projects as much as it does to commercial. And I think I am now of the opinion that having a strong vision like this can serve as much of a draw for contributors as a wide open door does. Or maybe the growth in the CDT lately has given me a bit more confidence. Or maybe its my new rose colored glasses...
Thursday, July 13, 2006
JUnits are my friend
Now I'm sure everyone who writes code in Eclipse is well aware of the power of the JUnit, but I just felt like expressing my appreciation for them right now.
I am in the middle of adding a few constructs to CDT's new index that didn't make it into 3.1.0 and was worried about whether the code I had just written was correct or not. Of course, the CDT is chalk full of JUnit tests for the DOM and other features, but in the mad rush to get the new indexing framework in I cut corners and didn't write any JUnits for it. Instead, I had my new Index View that I used to browse the index and visually verify things. (Now that view was supposed to be hidden since it's not quite complete but thanks to those who found it and have raised bugs against it :).
Well, now that I have a bit more time, I figured I had better make the plunge and start writing some. To my surprise, with the new indexer architecture it was actually pretty easy to programatically create a project, import some files from my test plugin into the project and run the indexer over them. I was then able to easily write some code to search the index and make sure everything was there that was supposed to be there.
Alas, of course, it showed me that it didn't and I have to now go and find out why that reference to my enum didn't get added. In the end, writing JUnits will have saved me more time than it took to write them. No more excuses. And thanks to Mr. Joe Unit for saving the day yet again!
I am in the middle of adding a few constructs to CDT's new index that didn't make it into 3.1.0 and was worried about whether the code I had just written was correct or not. Of course, the CDT is chalk full of JUnit tests for the DOM and other features, but in the mad rush to get the new indexing framework in I cut corners and didn't write any JUnits for it. Instead, I had my new Index View that I used to browse the index and visually verify things. (Now that view was supposed to be hidden since it's not quite complete but thanks to those who found it and have raised bugs against it :).
Well, now that I have a bit more time, I figured I had better make the plunge and start writing some. To my surprise, with the new indexer architecture it was actually pretty easy to programatically create a project, import some files from my test plugin into the project and run the indexer over them. I was then able to easily write some code to search the index and make sure everything was there that was supposed to be there.
Alas, of course, it showed me that it didn't and I have to now go and find out why that reference to my enum didn't get added. In the end, writing JUnits will have saved me more time than it took to write them. No more excuses. And thanks to Mr. Joe Unit for saving the day yet again!
Friday, July 07, 2006
How many engineers does it take to turn a CDT?
We had our regular monthly CDT contributors call yesterday. These are usually low key things where we quickly touch base, talk about release planning and the occasional technical issue. We've had calls that have lasted only 20 minutes. Sometimes they'll stretch to the whole hour if someone brings up a technical issue and we talk slow enough about it.
This months meeting struck me a little differently though. First of all, I was able to get a full head count and we had 21 people on the call. Of those people, I'd say 16 of them were people that have contributed code or are planning on contributing code. I also know that there were 3 or 4 such people that weren't on the call. I found that I had to cut off discussions and table them for future meetings because we were going to run past the hour we have allocated.
When I joined QNX last year and was handed leadership of the CDT, I remember mentioning to Mike M. that we had a hard time attracting contributors. At the time we really only had 5 or so people actively contributing. We knew the interest in the CDT was high and just needed to find a way to turn at least some of that interest into contributions so that we could continue to grow the CDT.
I'd have to say now we are finally getting the attention that the CDT needs. With contributors counting around 20 and a lot of people out in the community testing and raising bugs, I'm starting to feel like we can actually reach the goals I had personally for the CDT and go way beyond. We have a bright collection of talent now and they are all doing great things. Even over the last week as we opened up CDT 4.0 development, there have been some cool enhancements going in (like common navigator support) and I can't wait to try our first weekly build on Monday.
But the thing that really struck after the meeting was that I am going to be a busy man. With this many people contributing to the CDT, it's going to be a great challenge to make sure we don't run over each other. Communication is going to be key and I will take on the responsibility to make sure this communication happens and to facilitate the resolution of any conflicts that may arise. It's going to be a great run, though, and I can't wait to see what we accomplish as a team.
This months meeting struck me a little differently though. First of all, I was able to get a full head count and we had 21 people on the call. Of those people, I'd say 16 of them were people that have contributed code or are planning on contributing code. I also know that there were 3 or 4 such people that weren't on the call. I found that I had to cut off discussions and table them for future meetings because we were going to run past the hour we have allocated.
When I joined QNX last year and was handed leadership of the CDT, I remember mentioning to Mike M. that we had a hard time attracting contributors. At the time we really only had 5 or so people actively contributing. We knew the interest in the CDT was high and just needed to find a way to turn at least some of that interest into contributions so that we could continue to grow the CDT.
I'd have to say now we are finally getting the attention that the CDT needs. With contributors counting around 20 and a lot of people out in the community testing and raising bugs, I'm starting to feel like we can actually reach the goals I had personally for the CDT and go way beyond. We have a bright collection of talent now and they are all doing great things. Even over the last week as we opened up CDT 4.0 development, there have been some cool enhancements going in (like common navigator support) and I can't wait to try our first weekly build on Monday.
But the thing that really struck after the meeting was that I am going to be a busy man. With this many people contributing to the CDT, it's going to be a great challenge to make sure we don't run over each other. Communication is going to be key and I will take on the responsibility to make sure this communication happens and to facilitate the resolution of any conflicts that may arise. It's going to be a great run, though, and I can't wait to see what we accomplish as a team.
Saturday, July 01, 2006
How many engineers does it take to push a button?
One of the benefits of being located in Ottawa is that I get to rub shoulders with the who's who of Eclipse at interesting times. One of those times happened again yesterday as the button for releasing Callisto was pushed. Now, it wasn't really a button and it took about a half an hour from the time Denis started when the mirrors were ready until all the web sites were updated and we could download Callisto. But it was a moment.
It was particularly underwelming for the newspaper guy who was there, but I did get a chance to interview with him and hopefully sent him off with something interesting to write other than a bunch of computer geeks hitting refresh until we could see the magic "3.2" appear. But such is our life.
I came away very impressed with the work that Denis and his team do. Sometimes we forget how complex an operation that a site such as eclipse.org is. But it takes a team of dedicated professionals to pull it of and my hats off to Denis, Matt, and Nathan for pulling off one of the most challenging releases you'll see in this industry. And it was pretty cool to be in the nerve center as it was happening. Not to mention, they were all using Eclipse to managed the site which was also cool.
It was particularly underwelming for the newspaper guy who was there, but I did get a chance to interview with him and hopefully sent him off with something interesting to write other than a bunch of computer geeks hitting refresh until we could see the magic "3.2" appear. But such is our life.
I came away very impressed with the work that Denis and his team do. Sometimes we forget how complex an operation that a site such as eclipse.org is. But it takes a team of dedicated professionals to pull it of and my hats off to Denis, Matt, and Nathan for pulling off one of the most challenging releases you'll see in this industry. And it was pretty cool to be in the nerve center as it was happening. Not to mention, they were all using Eclipse to managed the site which was also cool.
Tuesday, June 27, 2006
What does Callisto mean to the CDT?
I've been lucky enough to be involved with the CDT since the day QNX proposed it to world back in 2002. It's been a very interesting journey. In the early days, the CDT was almost a side project at Eclipse where a few vendors had a dream of building a great C/C++ IDE and tried desparately with the few resources we had to reach the bar that the JDT guys continuously raised and continue to raise on us. But in those days the people working on the CDT didn't have a whole lot to do with the other projects at Eclipse.
Callisto has changed that in a lot of ways. First of all, just delivering at the same time as the other 9 projects opens up opportunities for working with them to bring their features to the C/C++ world. I've had discussions with TPTP with thier static analysis features built on top of the CDT. It's still small but a start. And others will arise in the future I'm sure. But the biggest benefit was our tighter schedule with the platform where we became early adopters and were able to get bugs fixed before having to wait for a maintenance release. And the platform team was very eager to help us out.
For the CDT, even the fact that we knew about 8 months in advance when our delivery date was going to be was a huge benefit. Until then, the release dates for the CDT were at the whim of the vendors providing committers to the CDT as we tried to match vendor release plans with CDT release plans. It made feature planning very difficult (we even had a 4 month cycle once!). And we look forward to the next release in a years time which will give us the opportunity to put forward a great program and make the major version jump to CDT 4.0.
For me personally, though, it was just the opportunity to work together with the 9 other project leads and Bjorn, Ward and Ian from the EMO. These are great people and it was a pleasure to work with them towards this great common goal that even Mike said wasn't possible. We proved them all wrong and have started a new era at Eclipse. And I hope you all enjoy the fruits of our labour, Callisto!
Callisto has changed that in a lot of ways. First of all, just delivering at the same time as the other 9 projects opens up opportunities for working with them to bring their features to the C/C++ world. I've had discussions with TPTP with thier static analysis features built on top of the CDT. It's still small but a start. And others will arise in the future I'm sure. But the biggest benefit was our tighter schedule with the platform where we became early adopters and were able to get bugs fixed before having to wait for a maintenance release. And the platform team was very eager to help us out.
For the CDT, even the fact that we knew about 8 months in advance when our delivery date was going to be was a huge benefit. Until then, the release dates for the CDT were at the whim of the vendors providing committers to the CDT as we tried to match vendor release plans with CDT release plans. It made feature planning very difficult (we even had a 4 month cycle once!). And we look forward to the next release in a years time which will give us the opportunity to put forward a great program and make the major version jump to CDT 4.0.
For me personally, though, it was just the opportunity to work together with the 9 other project leads and Bjorn, Ward and Ian from the EMO. These are great people and it was a pleasure to work with them towards this great common goal that even Mike said wasn't possible. We proved them all wrong and have started a new era at Eclipse. And I hope you all enjoy the fruits of our labour, Callisto!
Wednesday, June 21, 2006
Can't talk now, coding...
I've been pretty quiet lately with the blogging. The main reason is that I've been working certain parts of my body off as I try to implement a new indexing architecture for the CDT. There is a lot of good news and a little bad news with this project. The good news is that I can now index Mozilla in 14 minutes on my laptop! In CDT 3.0, that took around 50 minutes, and improvement of around 75%. As well, as you change files, you hardly notice the indexer running were as it could take up to 12 seconds to deal with the change in 3.0. I almost fell over when I got the first timing at 14. Followed shortly by a dance of joy.
How did I do it? Well I took a hint from the precompiled header feature that most compilers are starting to support. As I'm indexing, and potentially other parse activities as well, I skip over header files that I have already parsed previously and get the symbol information from the index. This required building a more structured database for the index as opposed to the string based flat table in 3.0. It turns out to be much faster since parsing C and especially C++ is a lot slower than the database lookup. This is why incremental times are so fast. I just didn't realize the whole reindex operation would be so fast as well (my target was 20 minutes for Mozilla).
The bad news, is that while it is incredibly faster, it does suffer from being young. There is less captured in the index than there was in 3.0, for Mozilla about 20% less symbols. So searching for certain things aren't going to get you everything you were looking for. But I have been able to capture the high runners. More bad news, is that we are getting spurious StackOverflow errors because not all information is in the index and some of the algorithms we have for symbol resolution weren't prepared for that. So as a result, the new index is only used for Search actions where we can recover gracefully and not for content assist and open declaration.
But back to the good news, as we work more on improving the contents of the index I'll be able to direct all parser operations to it and make the CDT much more responsive for all operations (including my baby - content assist). And even as it is today, there is enough information there for the majority of workflows. Even the field engineers at QNX are extremely happy with it and these are the front line guys who need to make sure their customers are happy. More good news is that I'm getting more help with the indexer, both testing and coding. It's tough to do this as a one man show and I am appreciating all the help I'm getting from the community.
With the new indexing framework in place in CDT 3.1, the opportunities for exciting new features is wide open. And one of the major objections to using the CDT on large complex projects has been eased greatly. It's time to get the message out, now that I can lift my head away from the code!
How did I do it? Well I took a hint from the precompiled header feature that most compilers are starting to support. As I'm indexing, and potentially other parse activities as well, I skip over header files that I have already parsed previously and get the symbol information from the index. This required building a more structured database for the index as opposed to the string based flat table in 3.0. It turns out to be much faster since parsing C and especially C++ is a lot slower than the database lookup. This is why incremental times are so fast. I just didn't realize the whole reindex operation would be so fast as well (my target was 20 minutes for Mozilla).
The bad news, is that while it is incredibly faster, it does suffer from being young. There is less captured in the index than there was in 3.0, for Mozilla about 20% less symbols. So searching for certain things aren't going to get you everything you were looking for. But I have been able to capture the high runners. More bad news, is that we are getting spurious StackOverflow errors because not all information is in the index and some of the algorithms we have for symbol resolution weren't prepared for that. So as a result, the new index is only used for Search actions where we can recover gracefully and not for content assist and open declaration.
But back to the good news, as we work more on improving the contents of the index I'll be able to direct all parser operations to it and make the CDT much more responsive for all operations (including my baby - content assist). And even as it is today, there is enough information there for the majority of workflows. Even the field engineers at QNX are extremely happy with it and these are the front line guys who need to make sure their customers are happy. More good news is that I'm getting more help with the indexer, both testing and coding. It's tough to do this as a one man show and I am appreciating all the help I'm getting from the community.
With the new indexing framework in place in CDT 3.1, the opportunities for exciting new features is wide open. And one of the major objections to using the CDT on large complex projects has been eased greatly. It's time to get the message out, now that I can lift my head away from the code!
Monday, June 05, 2006
Software as a Service Industry
Curt Schacker, apparently a veteran of the embedded software industry (well, his resume looks good anyway), has an interesting article on LinuxDevices.com on how he sees the state of the embedded software industry. His contention is that we've been been trying to shove a giant square peg in a giant round hole (his words, not mine), and that the embedded software industry is really a service industry and isn't well served by off the shelf software.
Now mind you Curt is a co-founder of, you guessed it, an embedded services company. But I have definitely seen the trend, especially in the tools area. It is really hard to sell software development tools in a box. Every customer seems to have different processes, different configuration management systems, build systems, coding standards, you name it. It is very difficult to build a suite of tools to satisfy them all.
The biggest success stories I've been a part of in this industry is when we sell the customer a box, but then follow it up with intensive support or custom development to make the software in the box work best for them. There's nothing worse, for me anyway, to have a customer who bought my box, but then let it sit on the shelf because it didn't really meet his needs. It's not so good for the reputation and future sales.
This is where programs like Eclipse really play into the business needs of software vendors. First, by sharing the development costs with other companies, our boxes are cheaper to produce. However, with Eclipse's extensibility and customizability, it is easier to take those products and customize them for individual customer's needs. Selling services may be more difficult and, as Curt mentions, doesn't provide the multiples that products do, but it might be the right approach that customers have always wanted and the best road to profitibility for software vendors.
Now mind you Curt is a co-founder of, you guessed it, an embedded services company. But I have definitely seen the trend, especially in the tools area. It is really hard to sell software development tools in a box. Every customer seems to have different processes, different configuration management systems, build systems, coding standards, you name it. It is very difficult to build a suite of tools to satisfy them all.
The biggest success stories I've been a part of in this industry is when we sell the customer a box, but then follow it up with intensive support or custom development to make the software in the box work best for them. There's nothing worse, for me anyway, to have a customer who bought my box, but then let it sit on the shelf because it didn't really meet his needs. It's not so good for the reputation and future sales.
This is where programs like Eclipse really play into the business needs of software vendors. First, by sharing the development costs with other companies, our boxes are cheaper to produce. However, with Eclipse's extensibility and customizability, it is easier to take those products and customize them for individual customer's needs. Selling services may be more difficult and, as Curt mentions, doesn't provide the multiples that products do, but it might be the right approach that customers have always wanted and the best road to profitibility for software vendors.
Sunday, June 04, 2006
Web server on your phone?
One of my "too many" interests in the computing industry is how to best serve up web content from embedded devices. The main use I see for such a capability is to allow maintenance personnel an convenient and standard way at getting at state and configuration information from the devices under their care. You see it very commonly used for configuring home routers such as my Linksys.
If you were at the CDT BOF at EclipseCon 2005, you would have seen a demo I gave of using gsoap to do this kind of thing. Since then, I've come to the conclusion that SOAP and related protocols are oversolving the problem. You can do what I was trying to do with simple http GETs. And with the coming out of AJAX to provide more interactive content with web pages using simple http requests, this really starts to look like the right architecture.
The problem I had was how to you integrate an http server with your embedded application. There are a few httpd library packages around but none of them appear to have enough momentum behind them to take the industry by storm. I had considered making my own but going through the http spec I quickly came to the conclusion that it would take a little more work than I wanted to soak into it at this point.
Then I ran across Nokia's Raccoon project where they've ported Apache to the Symbian OS that they use in their cell phones. My head almost fell off. I thought Apache was this big monolithic web server that is driving the bulk of the web servers on the internet, big iron types. Could Apache be made small enough to fit into embedded devices. Nokia seems to have been able to do it. And looking at Apache's modular architecture, it looks like you could write some cool modules that can interact with the software on the device without having to resort to the slow and clunky CGI interface. Very cool, and something I need to look into more.
If you were at the CDT BOF at EclipseCon 2005, you would have seen a demo I gave of using gsoap to do this kind of thing. Since then, I've come to the conclusion that SOAP and related protocols are oversolving the problem. You can do what I was trying to do with simple http GETs. And with the coming out of AJAX to provide more interactive content with web pages using simple http requests, this really starts to look like the right architecture.
The problem I had was how to you integrate an http server with your embedded application. There are a few httpd library packages around but none of them appear to have enough momentum behind them to take the industry by storm. I had considered making my own but going through the http spec I quickly came to the conclusion that it would take a little more work than I wanted to soak into it at this point.
Then I ran across Nokia's Raccoon project where they've ported Apache to the Symbian OS that they use in their cell phones. My head almost fell off. I thought Apache was this big monolithic web server that is driving the bulk of the web servers on the internet, big iron types. Could Apache be made small enough to fit into embedded devices. Nokia seems to have been able to do it. And looking at Apache's modular architecture, it looks like you could write some cool modules that can interact with the software on the device without having to resort to the slow and clunky CGI interface. Very cool, and something I need to look into more.
Subscribe to:
Posts (Atom)