Tuesday, June 21, 2011

A UI Revolution is Coming. Are we Ready?

This topic has been brewing in my mind, and in this blog, an in my tweets for months now. And one final event made me certain it is going to happen. And that event was the introduction of Windows 8. Not that Windows 8 is really that big a deal. But it confirms for me a trend that's going to change the way we interact with the desktop applications we use daily, including Eclipse. Yes, a UI Revolution is coming. And we need to make sure we're ready, or Eclipse is going to look old very quickly.

What am I talking about? Well, let's start at the beginning. There is no question that Apple started the revolution with the iPhone and iPod Touch. Yes, there's an app for that. There's an app for everything. Apps are small, easy to create with a simple UI that was intuitive even for "regular" people (and, no, computer people aren't very regular). The rest of the world outside the walled garden got their fix thanks to Android and Windows Phone and a few others. Smartphones have revolutionized how we think of applications.

Of course, once we have smartphones figured out you want to see if the same ideas scale upwards. The next big hole to fill was tablets. Tablets have been around for a long time albeit with the old UI borrowed from the desktop. It didn't work. But could the smartphone UI scale up to tablet sizes. Indeed. iPad is gigantically successful and Android and Playbook are following behind. We've finally figured out how to make tablets commercially successful by using UI that again was intuitive for everyone.

Now, as I've been wondering here and there, what about the desktop? Does it make sense to bring these same principles to machines with larger and multiple screens with keyboards and mice and fast CPUs and GPUs? Well, I think we hit a tipping point. And that point happened when someone got the bright idea to slap a keyboard and trackpad onto a tablet, i.e., the Asus Eee Pad Transformer. Watching the videos people have posted onto YouTube of the Transformer in action, I am convinced that a tablet UI would work just fine on a desktop platform.

And apparently, so is Microsoft, which brings us back to Windows 8. Early videos of it show off exactly that. It's the Windows Phone 7 UI, scaled up for both tablets and the desktop. And I think it looks pretty sharp. And I don't see any reason why Android couldn't do the same. And with the work Nokia/Trolltech are doing with Qt version 5, I can see how we can do the same for Linux. We all love the usability of our mobile devices, I want to love my desktop just as much.

But there's problem. Watch the Windows 8 demo when they bring up Excel. It's revolting, but in the wrong direction. You finally see how bad the old time UI that we've had for decades and that has been replicated on every desktop OS we know really is. We really need to invest in upgrading the UI and potentially reinventing those applications if we want to keep them around.

And, yeah, that includes Eclipse. Unfortunately, the old technique of putting a common API over the native UI frameworks isn't going to work. The new frameworks are just too different. And making them common looses the gains they bring individually. We got away with it before because they used to be so similar. Not any more.

So what are we going to do? There's many ways, I suppose. The itemis guys gave a great presentation at EclipseCon on one of the strategies they used. They use a DSL that implements the logic and general layout and generate code for each of the UI frameworks they support. It was cool but I'm not sure how complex of an app you can build that way.

But I do think they are on to something. You need to program the UI in the language and technique that the framework supports. But the core logic of the application should be able to stay the same. You know, Model-View-Controller, and you only need to swap out the View part? And we should be set up for that, no? Isn't that why we broke things up into ui and core plug-ins? Or do we have so much logic in the ui plug-ins that this won't work?

So this is an area I'm going to try and prototype. And I'll start with Qt5 with their QML Javascript-ish UI language and see whether I can hook up Eclipse services through some sort of QObject C++ to Java bridge and make them dance. I have a feeling it's going to be a dismal failure. But you have to start somewhere. Doing nothing doesn't leave me feeling very comfortable about Eclipse's future after the revolution.

10 comments:

  1. Hi Doug,

    Fascinating topic. I think this is the whole point of model-driven UI.
    The really revolutionary step is going to be abstracting the functionality at a higher level than the "widget" level. As I you're pointing out, that's a very different kind of thing. So rather than say, ok, we want the photo here, and over here we've got our toolbar, which has a tools for sharpening, retouching, etc.., and over here we have a button that takes you to a histogram.. We want to be able to say something more like "the user is interested in images and they can be presented as lists, tiles, panels, but they all share this ordering" "here are some things that can be used to transform pictures".

    As always, there is a mapping between API approaches and Model-Driven approaches, IMO the model just makes everything much more straightforward and explicit, and most importantly means that you don't have to build an adapter layer between your high-level abstraction API and the implementation -- something that is very hard to do and very typically sloooww.

    In MVC, everyone always forgets the "C" part. A really good example of abstracting out the higher-level aspects is actually GEF, I think. If you look at that, they have actually created a setup where you can define how UI objects are related to each other without having to worry about the widget context. So you can take exactly the same providers and use them with lists, tables, diagrams, graphs..

    In AMP -- although our focus has changed so that our primary target is now really the Eclipse API -- we actually write an entire application to multiple frameworks that have completely different UIs and APIs. In my experience by far the most interesting and challenging aspect of this is not the code generation -- that's trivial -- but in coming up with the rich abstractions above. An older article on that subject.. http://milesparker.blogspot.com/2009/04/meta-models-for-arbitrary-targets-iii.html

    cheers,

    Miles

    ReplyDelete
  2. Doug,

    This is part of the design goal for e4/Eclipse 4.x and the modeled workbench + CSS Styling. You can change the way your UI is rendered through dependency injection and CSS Styling.

    ReplyDelete
  3. Please let us know about your work, Doug!

    And I think starting from Qt (whether normal Qt GUI widgets or with QML) is a great choice!

    ReplyDelete
  4. My experience with the Transformer is that the UI works great (=easier and more fun to use) by reducing the complexity. I think this is both driven by the necessity of a touch friendly UI and a design focus on a better UI (rather than on feature bloat). Thus while I love my transformer I'm not sure how applicable the design is for Eclipse (here I mean only the IDE not RCP). While we certainly have some complexity we could and should reduce, I don't see how an IDE can be made significantly less complex. And I haven't seen any examples of applications as complex as e.g. an IDE or an office suite working well in the Ipad/Android/Win8 UI. I would be surprised if Microsoft is moving VisualStudio or Office to the Win8 UI.

    ReplyDelete
  5. Thanks for the responses gang, and keep them coming. This is an important topic for the future of Eclipse.

    A couple of responses:

    @Ian and Miles: e4 and modelled UI doesn't go far enough. You actually want a different UI that's native to the new platforms, Android, iOS, Windows 8. I'll dive deeper into that in an upcoming article.

    @Roland: That's the big challenge and why I'm excited about the revolution. What would an IDE look like if it was an Android application? We might be surprised at how good it can be.

    @Hendy: Will do. If anything, Qt5 with Lighthouse and QML has the capability to be that common framework we need for Eclipse.

    ReplyDelete
  6. The "Eclipse 4.x Application Platform" provides you with a runtime layer providing you things you'd have to write your own. It needs to have Java and OSGi so running native on the constrainted devices might give you some headaches.

    ReplyDelete
  7. Hit the reply button too early. Beside that you can code the UI in any UI-Technology you want to. I've shown that already for Swing and JavaFX - for Qt all missing is the Java integration

    ReplyDelete
  8. I really think it could indeed be achieved with Qt, given the multi-platform nature of Qt. Qt runs on nearly every desktop platform out there and on quite a few mobile ones. Even on Android, just not on iOS and Windows Phone 7 right now. But one doesn't really know what Nokia is up to with WP7, so maybe we'll see something there as well.

    ReplyDelete
  9. @Joachim that's a great point about WP7. The Qt team seems to have renewed vigour. Maybe Nokia has something up their sleeve. Or maybe the Trolls are just trying to get re-acquired to keep their great tech alive.

    ReplyDelete
  10. Qt has Java integration, it is called "Jambi" and it is decent (though I 'm not sure if it is maintained to latest Qt versions).

    I don't think that the problem to make an IDE like Eclipse/CDT "phone-like" easy is so much on the "view" part. Of course that can change.

    But what will need most work is the "backend" code so that it becomes smarter, more "context sensitive", more "able to read the programmer's mind" etc. Only then you can remove all the junk^wuseful buttons and menus that are now all over the place.

    For me the "phone-style" IDE will look somewhat like this: http://textroom.sourceforge.net/

    ReplyDelete