It's Friday. It's been a pretty busy and long week. Got lots done. Working on lots of Eclipse related projects both internal and external. The Red Wings won the Stanley Cup (although I'm very proud of the Pittsburgh fans for their behavior after their team lost at home, they're real hockey fans). I bought a new car (a little Mazda 3 Sport, love it! but hate car shopping). So I think I'm allowed to think some crazy thoughts once in a while.
This one sprouts from a couple of different triggers. First, the latest Emacs discussion and how it would be nicer if we could integrate Eclipse better in a command-line environment. Second, comes from my interest in GUIs for embedded systems. Flash is one idea and trying to figure out how you'd hook a C/C++ app on a device to a Flash-based UI. Third comes from a bit of OSGi envy in that it would be a huge help to C/C++ developers if we had a component system like it.
So the crazy idea is this. Rewrite Eclipse in C++. Maybe rewrite isn't the right word since I hear there are people out there who actually like Java ;). But start producing C++ components that look a lot like Eclipse. I'd start with SWT which is a layer over top of some C and C++ code anyway. Shouldn't be that hard. You could then look at a C++ implementation of OSGi. Bundles would be easy to make, just use DLL/so's instead of jars (of course, I'm missing the point on other componentization issues that OSGi deals with but I can't be that far off). Then continue to add things as they make sense or people have a need.
I did a quick search, since I'm pretty sure the C++ SWT idea had been thought of before. I found a company, PureNative Software, that has actually done it. Mind you, they are using their proprietary Java to C++ compiler to do it, and I would think that leaves in a lot of the glue that SWT uses to bridge the Java/C++ world. But they do have a compelling story.
So I'm going to try and contact myself in a parallel universe to start working on it. Or maybe it'll just remain a dream. But I'll throw it out there to see whether you think it's such a crazy idea or not.
Hey Doug, I would start by doing OSGi for C++ first. If you do that well, then the UI will come along.
ReplyDeleteWith OSGi support you could already do so many things, like printing, ripping a CD, all the things that Java doesn't support today.
I agree with Antoine. I believe that there is interest at OSGi in seeing their specs move to C++. That would be a real contribution.
ReplyDeleteWell, there is already a "OSGi" for embedded as part of DSDP: the RTSC project proposal! It is not exactly OSGi, but it is a very light weight component model. I think the C/C++ world needs a component model. I don't care which one it is, as long as it provides some of the OSGi capabilities.....
ReplyDeleteBe careful or Loud Marvin will try to sell you something again...
ReplyDeleteAn interesting idea, indeed. I am _far_ from an OSGi expert, but have done a system with C++ that was essentially a home grown component system. In the end it was fairly simple to use (DLLs were autoloaded by the classes you wished to use), but it never felt totally abstracted, unlike the way OSGi components for Java feel.
ReplyDeleteSo a couple questions... I'm sure I could find others, but I just have a couple minutes:
1) Is this basically a cross platform, COM-like solution? What would it do the same or different?
2) Are there any previous technologies or concepts you would envision this project being based on? You mentioned dynamic libraries for one. Any others? (i.e. DBUS or ICE for the communication layer, etc)
2) How much would you abstract the underpinnings of the component system? With our solution, we needed to be careful with details, like who owned the memory we were passing around. This was especially tricky in windows where the process and the loaded DLL have separate heaps. Everything that crossed the boundary had to be passed by intrinsic types. Quite a few times, some new developer would come along without reading the documentation and say, "hmmm, why do they use char* everywhere?? Let me get with the times and change this to std::string" And then they'd wonder why it crashed. For all the arguments against Java, it sure is nice to have the JVM own all the memory (among other things).
SWT in C++: GCJ is your friend here.
ReplyDeleteGCJ is one of those mysteries of open source Java. It's got little love from the Java community and little love from the C/C++ community.
I used it (along with SWT) a few years back to build a native installer for my Java app. It was perfect: No need to grovel for what JVM is installed on the user's machine because the app *is* native.
I sure wish Java people would get why an ahead-of-time compiler can be cool.
gcj gets no love from C++ developers either. It's still Java.
ReplyDeleteHey Doug,
ReplyDeletehow do you try to solve dll-handling for dependant libraries and versioning. For example,
a.dll from bundle a-bundle-1.0 depends on log.dll from bundle log-bundle-1.4. If I now upgrade from log-bundle-1.4 to log-bundle-1.5, I don't want to change a.dll from a-bundle-1.0.
I would love to hear from such possibilities.
I used Wascana (great work!) to start intercepting dll-loading, but had only little success.
The idea of driving OSGi to the native world isn't new, it's called Universal OSGi, you can find more info here: http://www.osgi.org/blog/2007/10/universal-osgi.html
ReplyDeleteI had seen the Universal OSGi thing before (I didn't claim it to be a new idea, actually few of my ideas are really new). But it's hard to find any information on it other than that one blog.
ReplyDeleteI'm not sure this will solve DLL hell, but a bigger issue will be dealing with different C++ ABIs. How long until this starts looking a lot like COM?
In the near past, I have seen 3 different teams in different mobile device companies that were planning/studying to create eSWT as a native UI toolkit. One of the teams changed direction, the other canceled the project and I am still waiting to hear back from the third.
ReplyDeleteAll these teams had different problems they were trying to resolve by the port, which indicates that there is an interest. I am not sure if it indicates something but they have all decided a different solution.
On the Flash integration, have you checked SonyEriccson's capuchin? It provides Flash, java integration but the idea is simple enough to be applicable for C++.
No, please. C++ is more part of the
ReplyDeleteproblem than part of the solution. Let's not go there.
I'd be much more interested in making it possible to use JRuby to write Eclipse-plugins in Ruby.
Aside from the great benefits this would have to large C++ projects, also consider the great potential for integrating dynamic and functional programming languages as well.
ReplyDeleteIf something like this existed, I'd wrap it with SWIG and use SWT from Python! Or mix together modules written in Haskell, C++, and Python.
The C/C++ level is the lowest common denominator for enabling many, many other frameworks and languages. By excluding everything that's not Java, you really limit yourself.
I think its an excellent idea.
I dug a little into Universal OSGi and it was more about native bundles and still required jumping through Java to the core.
ReplyDeleteMulti-language is definitely very interesting and maybe the right approach to this. And, yes, using C as the core (C++ has ABI issues), and since almost all languages have C bindings, it would really enable this.
Maybe starting a new community driven technology project would be a way to get this started. Seems to be real interest in it.
Doug, have you looked at the DSDP/NAB eclipse project yet? They have a lib called MWT which seems to be something like SWT written in C++.
ReplyDeleteBut I have to admit, I don't think writing a new eclipse in C++ would be worth the effort.
When it comes to standards and/or platform independence, C++ is lagging far behind Java in so many areas (UI, database access, Unicdoe support, multithreading, etc.). Different solutions exist, but don't interoperate pretty well. Think of all the different string classes out there despite the fact that standard C++ provides one.
This all should have been addressed ten years ago. With todays JVMs running nearly as fast as native compiled code, I'd just use Java from the start.
Yes, C++ also has features that Java does not have, like the powerful template system. But have you ever had to explain code that uses template expressions to a novice programmer?
Excellent, someone who thinks rewriting Eclipse in C++ is crazy, of course it is. But some of it might be interesting.
ReplyDeleteMy original thought was for SWT since they are working on a Flash integration and we could reuse that architecture in C++.
My thoughts have now turned towards OSGi, which could be a good platform independent COM. That's the next blog entry...
OSGi in native C++ is really a great idea. I have done a C++ implementation of OSGi last year. It's based on the POCO C++ Libraries and has some quite interesting features. It uses the C++ compiler's ABI (I did not go down the COM route). To achieve portability, a bundle can contain multiple shared libs, one for each supported platform. So you don't have to mess around with COM, while still having the ability deploy the same bundle to different platforms. You can find more information here.
ReplyDeleteA few sideways libraries of interest:
ReplyDeleteGUI: Wt http://www.webtoolkit.eu (if you dig into it, you'll see a paint-on-canvas capability, may be not flash like, but is a web gui none-the-less, if that is the direction you were taking).
ICE http://www.zeroc.com/ice.html may not be as high-fallutin as OSGI, but it might be a modular alternative to it.
The DLL hell under Win32 could be solved by using the VERSION_INFO that you can embed into each Portable Executable in its resource area. It can be easily read out without loading the dll. Otherwise you could use something similar to the OSGi Manifest file in native OSGi zip/rar/cab files. The code for each bundle would be a platform native compiled and which one this is, is defined in the manifest as well. I have already successfully loaded DLLs into process memory from a memory location (directly from uncompressed zip stream).
ReplyDelete@snapp_action: Actually in Win32 Dlls use the *same* heap as the hots process. They use different *stacks*. Thats why std::string will fail to deallocate its memory when you pass it around. But the easy solution would be to allocate Strings from the framework. Just like COM does with SysAllocateString.
Otherwise I do not see why different compiler ABI would be a problem? Its all binary code after call.
I have already a C++ JNI wrapper based on templates in place which could easily bridge C++ bundles to the Java world. Its really easy to use as I will demonstrate in a future blog posting. Basically you could hand in all Java classes (OSGi classes) into a C++ bundle. You could then write platform specific code and provide its functionality to Java just by creating a class in the VM from the C++ side. You could register OSGi services from C++ as well of course.
A generic wrapper could maybe even expose functionality of COM libraries to OSGi as services.
Minor clarification: SWT C++ itself does not require any special tools in your build process, just the necessary library files and your C++ compiler. That's it. However, to produce SWT C++, today you need SWT Camp and the translator to get from SWT Java to SWT C++. The present approach teaches the developer how to fish the ocean of open source instead of giving them just one catch. That being said, a pre-translated SWT C++ source code download is in the offing. Here are further reasons for the present approach:
ReplyDelete1. Technology Demonstration. The translated source code could have been distributed instead of the translator, but the present approach really shows developers what's possible with NewJ. SWT C++ is not a one-off. Practically any Eclipse or Java project, including OSGi itself, could be automatically translated to C++.
2. Single Source Base. Most importantly, this resists forking. It also means there is still only one implementation of SWT. SWT C++ benefits from all the improvements in SWT Java. Further, for testing and deploying SWT C++ apps, it means that the apps are feature-for-feature and bug-for-bug compatible with Java SWT.
3. License Agnostic. SWT C++ does not assume that the original source code being translated or the generated source code is EPL. The developer using the translator is responsible for determining how the developer can license and re-license the generated source code and compiled binary.
Thanks for the clarification. I still think it would better to do this by hand and skip all the Java-isms.
ReplyDeleteYou may be resisting forking but you do realize SWT itself is a pretty big fork for each window system it supports.
Yes, SWT Java provides an implicitly-shared common API that is entirely re-implemented (forked, per se) for each target platform. Even the actual class and interface declarations can slightly differ from platform to platform.
ReplyDeleteOn the other hand, Java AWT and certain other platform-independent libraries use a multiple-platform design pattern for their implementations. The class and interface declarations are explicitly shared and exactly the same for all platform implementations. Further, the implementation of platform-independent methods may also be explicitly shared and be the same for all platforms. Only the platform-specific implementation (native) methods of the library are forked, per se.
Each approach has its pluses and minuses.
With the SWT approach, we just need to implement SWT C++ PI (mostly just wrapper classes) for each target platform. Everything else can be automatically translated, and that’s a huge advantage over writing from scratch in C++.
Regarding Java-isms in SWT C++, there’s a Java-thing that many C++ programmers want (from the novice to the inventor): And that is a standard library as rich as Java. The “isms” in SWT C++ are primarily due to leveraging an existing extra-linguistic rich standard library, the Java API. Despite this fact, it is realistic to expect C++ purist APIs and libraries on top of SWT C++, be they OOP C++ purists, generic programming C++ purists, or another paradigm. SWT C++ is a much easier starting point than the native C platform API. The C++ purist’s library may even hide SWT C++ implementation altogether the way your C++ vendor hides the Standard C++ Library implementation from you and any C implementation modules it might contain.
Lastly, new SWT C++ apps are only a piece of the pie, maybe a small piece. For the immediate future, it is expected that SWT C++ will more popular for translating existing SWT Java apps to C++.
"For the immediate future, it is expected that SWT C++ will more popular for translating existing SWT Java apps to C++."
ReplyDeleteNow there's the crazy thought of the day...
Crazy cool! Immensely useful too to be able to run popular SWT apps where they have never run before (small devices w/o Java or SWT) or in a way they have never run before (deterministic memory management, instant startup, super-low resource usage).
ReplyDeleteEvery next-great C++ GUI toolkit awakes on 0-day-0-developers. Instead, SWT C++ is born to a supportive family of developers and a heritage of existing apps. It’s a strategic pattern: C-to-C++, Win32-to-WinCE, Mac OS X to Mac OS X Mobile, and so on. Over time, the distribution between the new developers/apps and existing developers/apps levels out. Viability depends not on precise distribution but on actual developers to distribute.
I work on a portable open source framework implemented in C++ which provides a OSGI-like infrastructure (including loading bundles from DLLs).
ReplyDeleteIf you're are interested in please have a look at http://sof.tiddlyspot.com
> Rewrite Eclipse in C++.
ReplyDeleteIsn't it what Qt Creator guys have done?
An IDE written completely in C++.