Wednesday, August 11, 2010

Why Vaadin - line of argument

While I told in my former post how it came that we left the GWT/ExtGWT couple behind us and finally deciding for Vaadin, I want to elaborate why we think Vaadin is the best option.

First I will explain how Vaadin helps us to solve some of the crucial problems and why it is so effective to build the new AdminCentral with it. Afterward I will response to some valid concerns raised by the Magnolia CMS community.

Benefits

Server side execution
Executing UI events on the server is controversial - more on that later - but it has tremendous benefits.
  • Reflection can be used. While GWT code can't use reflection server side code can. This is for instance very helpful when we transform dialog definitions into real dialogs.
  • Security respected. At any time we can use ACLs to define what should be possible in the UI.
  • Less web services. The UI is build on the server, but rendered on the client. So there is no need to define additional services for receiving dialog and tree descriptors.
    We just have to expose the content related stateless REST services.
  • Agility. With GWT it is difficult to support agility as you cannot change the final code without compiling it. With Vaadin we can for instance use our powerful Groovy support.
  • Magnolia API. Full Magnolia API can be used and no DTOs are needed.
  • Testing. All code can be tested with normal junit test cases. Even the UI code. The server side widget is linked to the client side widget by annotation.
Efficiency
We are a small team and efficiency matters, as time-to-market is crucial.
  • Collaboration. The two communities and the backing companies are looking forward to support this collaboration and to help each other to make this successful.
  • Documentation. Vaadin's documentation is just great. The Book of Vaadin, which is freely available, makes it easy to get started.
  • Compilation. No GWT compilation is needed unless a new client side widget has to be written which should be rarely the case once AdminCentral is rolled out. Composite widgets are not requiring any GWT coding.
  • Theming. As many other things theming is a service provided by Vaadin. Once we decided for a design it will be easy to implement a Vaadin version.
Third party and custom modules
It will be much easier to adapt the AdminCentral and plugin module specific UI than if we had chosen for GWT. Firstly one hasn't to perform GWT compilations and secondly reflection/injection will be used to plugin into the AdminCentral.
It is the ultimate goal to make the AdminCentral extendable and usable for more generic content management related use-cases.

Performance Concerns
After the decision was made the community raised concerns about the performance of such a solution. Originally I shared this concerns but want to explain why I think they are not justified or at least don't neutralize the benefits.

Factor in the scenario
We are going to develop the AdminCentral and not the next generation gmail application. The AdminCentral is the backend and is typically used intranets.

Number of Requests
The server side event handling will lead to more requests and will add the network latency to most of the user interactions. This sounds dramatic in the beginning but is it really true? Let's have a closer look.

Scenario A) Ask for data (GWT) = send event (Vaadin)
Most of the user interactions will result in a request to the server. The only difference is the reason for sending it. While GWT will ask for the data to show or send some data, Vaadin will send the event information.
Examples:
  • creating a page
  • moving a page in the tree
  • open dialog
  • click save button
Scenario B) One request to rule them all
Note that any client side UI will also request data from the server. So it is actually not unlikely that a client which uses stateless REST services will send more requests to assemble the data its needing.

A typical use-case is opening a dialog for paragraph editing which needs additional information to fill a table or read information from a page we are linking to.

GWT (2 - 3 requests):
  1. event -> execute open-dialog action
  2. ask the server for the dialog to open and its descriptor (might be cacheable, but not necessary due to some complex rules like paragaph to dialog mappings, ..)
  3. populate the dialog -> request to fetch the data details
  4. we need some sub-nodes to fill in a table -> request to fetch the data
Vaadin (1 request):
  1. event -> immediate request to server
  2. event handler builds the dialog (this is fast as no rendering is involved)
  3. some data is read. Reading additional information as meta data and sub-nodes are at hand
  4. bind the data to the dialog
  5. Vaading sends back a JSON representation of the UI changes
  6. client side rendering engine applies the changes
Here we actually save requests by Vaadin's roundtrip approach.

Scenario C) Visibility changes - Vaadin says not No to client side events
In other use-cases as clicking on a tab or navigating in menus none of the frameworks sends events to the server. In both cases it is client side code which toggles the visibility.

Conclusion
While we surely expect to have more requests, we don't think that we talk about doubling the numbers of request. In fact I believe that we will not see an increase bigger than about 10-20 %.

Responsiveness of the Demos
Some stated rightly that the ExtGWT examples are snappier. In my opinion this is due to the fact that the examples just don't use server side services to read the data. But is this a realistic scenario? How often will you perform user interactions without having to read or send data?

Example: Sorting in the table
Client side sorting is a nice thing to do, but normally you will request the data on demand (paging). How will you sort the data which you have not received yet?

Optimize where you expect the biggest performance gain
While we can discuss for a very long time about the client performance I think that an end user will judge the overall performance including search, filtering of data, ...

And here we can do much on the server side. I prefer to invest time on that end than to optimize the very last bit and byte sent over the wire.

Content API
  • return iterators not collections (same as JCR)
  • don't apply the security checks, delegate that to JCR (possible since JCR 2.0) 
JCR / Content Structure
  • mixin for metadata instead of using a sub-nodes
  • versioning friendly structure: extra node to separate paragraphs and subpages
  • use more and proper node types: searching, ..
I am very sure that the resulting responsiveness will benefit much more from this kind of improvements.

Proof of concept
As for now we have decided for Vaadin but still we are just entering a proof of concept phase in which we want to proof that Vaadin is really the better choice. Here are the scenarios we explicitly test.


Page Editing
Here we expect some GWT coding. It is sufficient if the dialogs and toolbox can be Vaadin components.
  • show that we can inject behavior into existing HTML: add edit button which opens edit dialog
  • mouse over: show paragraph information on mouse over
  • inline editing: proof that inline editing will work
Tree Grid
Vaadin is working on the tree grid. If we find some shortcomings they will most likely help to improve it.
  • move: drag and drop
  • context menu: per node type
  • editable row
Paragraph
If we use a technology to build the new AdminCentral we want to ensure that it can be used to write more complex paragraphs as well. As an example I take the page commenting. Each page has a commenting paragraph. The scenario is different as it must work for high traffic websites too.
  • initial form should not create a session (the page should be served by the cache)
  • coexistence: two independent Vaadin paragraphs

Conclusion
We really think Vaadin has great potential and deserves its chance. Still we are new to that technology and will have to verify that our assumptions are right.

We won't rush into anything

18 comments:

breun said...

Nice posts. Good luck!

Jose María Arranz said...

I invite you to test a different framework: ItsNat

In some way ItsNat is similar to Vaadin (server centric) with some key differences:

* Absolute freedom of web design
* Absolute control of layout in a Single Page Interface (SPI) environment.
* In spite of your web site can be SPI, ItsNat is SEO compatible (not important for Admin, very important for the public version)
* New components are very easy to build because ItsNat is basically DHTML in the server.
* Client layout is clean of framework artifacts so client JS frameworks (jQuery, Dojo...) can easily add special effects (movement, opacity changes...) and custom JS code can be sent easily to the client.
* Current layout and behavior of Magnolia could be ported easily to ItsNat

http://www.itsnat.org

runger said...

Hi!

Thanks for the detailed and informative posts!

I'm not really a "GUI guy", so I can't add much to the GWT vs Vaadin discussion.

Anything which improves the following aspects of magnolia is certainly welcome though:
* useability in backend
* seperation of UI and logic in backend
* extensibility of backend UI

So I'm excited to see what Vaadin and Magnolia 5 will bring us!

One specific comment on your article - you state that

"... I think that an end user will judge the overall performance including search, filtering of data, ..."

I have to say, I totally disagree. The responsiveness of the UI is a primary 'quality' criteria for users - understandably, since they have to 'click' hundereds of times per day to get their work done. If each 'click' is a few seconds faster or slower, that makes a big difference.

No one likes waiting, and that's true for searches and other operations too, of course. But in the end it will be the UI responsiveness for the common operations (navigating in adminCentral and the edit-mode, opening and closing edit dialogs) that will determine the user's impression of performance, not the search.

Regards from Vienna,

Richard Unger

Casper Bang said...

Why haven't you guys looked at SmartGWT? A more liberal license and IMHO better designed solution than GXT.

"How will you sort the data which you have not received yet?"

You capture the intent of the sort and/or filter, and pass this to the associated client datasource which talks to the server backend. SmartGWT can even do client-side sort/filter if it determines it already has the required data.

Philipp Bärfuss said...

Richard Unger said ...
   "No one likes waiting, and that's true for searches and other operations too, of course. But in the end it will be the UI responsiveness for the common operations (navigating in adminCentral and the edit-mode, opening and closing edit dialogs) that will determine the user's impression of performance, not the search."

Perhaps I was not clear enough about the impact of this changes. I was not mainly talking about searching in the front-end/back-end.
This influences how fast the list and tree views work, as well how long it takes to active or save content, ... especially if you have a lot of content the changes proposed to improve the performance on JCR level will have an effect in many places. Some of the features like the inbox are mainly based on JCR queries. For instance we will have a lists to show the recently modified pages, ... I still belief that we can gain a lot in that field which contributes to the responsiveness of the UI.

Richard Unger said ...
   navigating in adminCentral and the edit-mode, opening and closing edit dialogs

Most of this operations will need extra data and will cause a request no matter if you have server or client side event handling: tree needs data to show, dialog too, closing a dialog means saving, ...

Anyway we are now building a mini AdminCentral and will be soon able to judge on a real world example ;-) So lets see how balanced the final impression is and how many of the events can be handle with client side evens.

Norman said...

You choise of Vaadin over GWT caused a lot of discussion even within our company. Checking the very slow Online-Demo of Vaading does not provide good arguments for server side event handling.

With your explanation, the decision seems more comprehensible. So I would like to thank you Philipp for making Magnolias decision that transparent to the community.

I'm eagerly waiting for the result of the mini-AdminCentral test drive.

Philipp Bärfuss said...

Some short comments on the other frameworks mentioned here.

SmartGWT
It was probably not obvious but we went through a first evaluation phase a year ago. We decided for ExtGWT/GWT and were also willing to pay the OEM license. At that time we evaluated SmartGWT but I din't like that this is mainly an adapter to the JavaScript library and I struggled a lot when I wanted to use my own models.

We have put down some of this findings in our wiki.

Anyway, the problems we faced with GWT/ExtGWT are the same we had with GWT/SmartGWT.

ItsNa
This was not on my monitor for now but the samples and the website are not convincing. The license is less liberal (ItsNa is GPL, Vaadin is Apache licensed) and we actually like the fact that Vaadin is based on GWT, which we still like as we can write client code in Java. Vaadin just solves some of the crucial problems we had with the plain GWT approach.

Jose María Arranz said...

I don't want to fill your blog with a discussion about ItsNat vs Vaadin. Anyway some comments:

* ItsNat does not impose a look and feel, ItsNat is look-and-feel agnostic you can use/clone the cool HTML/CSS layout you like more. For instance see

this "false" ExtJS example

and this clone of a real world web site.

* Client code is also written in Java using W3C DOM Java APIs (custom JS and external JS frameworks are optional).

* Commercial licensing cost is a bargain.

Alamoos said...

GWT/ExtGWT are quite different from Vaadin.

GWT and ExtGWT are client-side approach, while Vaadin is server-side approach. They have their own pros and cons and the choice depends on your requirement.

GWT/ExtGWT is strong in client programming: full control, faster responsiveness, and, arguably, seperation of UI and data tier. ExtGWT is also strong in widget's completeness/functionality. Vaadin is strong in easy accessing the backend services and no exposing logic at client.

When discussing Ajax in pure Java, one framework you must check: ZK. IMHO, it got the most innovative approach: server+client fusion. Like Vaadin, it allows developers to access backend easily. Like GWT, it allows developers to control the browser completely. Like ExtGWT, it has a complete set of widgets. Since it is avaiable quite a long time, it also has many unique features, such as markup language, grail... However, the weakness is the client-side control, though optional, must be done in JavaScript (jQuery). It will be the best choice to me if ZK allows developers to write the client-side functionality in Java.

Philipp Bärfuss said...

True, ZK has a very similar approach and in my opinion it is a matter of taste if you prefer one or the other. We liked the face of Vaadin, the license, the services and the community and we like that the client side part of our application can be written in GWT. We like GWT so we like Vaadin ;-)

Marc Logemann said...

Really interessting. We are at the same phase http://www.logemann.org/2010/09/ext-js-from-java-programmer-perspective.html. We evaluated Ext GWT and someone point me to Vaddin which i have not known so far. We will do a comparision wrt widgets and overall programming style in the next weeks. Pretty exciting.

Charles Kendrick said...

Actually, GWT and SmartGWT will typically send far fewer requests than the corresponding UI
implemented in Vaadin (or other server-centric) technology. This is in fact one of the prime
strengths of client-centric technology. See this SmartGWT sample:

http://www.smartclient.com/smartgwt/showcase/#grid_adaptive_filter_featured_category

Vaadin and other server-centric applications will hit the server for every sort and filter.
SmartGWT will do most operations client-side. These are the critically important requests
since they are requests for data.

As far as your dialog example, in SmartGWT this would also be one request, even if parts of the
dialog are dynamic - you would use queuing (see SmartGWT QuickStart Guide, Chapter 8,
"Queuing") and in many cases, queuing is automatic.

http://www.smartclient.com/releases/SmartGWT_Quick_Start_Guide.pdf

Finally, the lack of client-side event handling means it is not possible to handle certain
common RIA use cases in a server-centric model, specifically customized drag and drop, as well
as rapid data entry with interdependent fields - too many server trips.

And you can never have an offline mode.

These may not be requirements in your application today, but requirements change.

While we're on the topic, the other very serious problem with server-centric architectures is
clustering. Vaadin requires a chunk of server-side session state which is totally unnecessary
in SmartGWT and other client-centric architectures, and in a clustered deployment, this state
has to be replicated amongst all servers in the cluster.

Between far more server trips and poor clustering performance, large deployments of
server-centric architectures will never come within 10x the performance of client-centric
systems like SmartGWT. More often, it's 50x or 100x or more.

So in conclusion, be very careful adopting any server-centric architecture. If (more likely
when) tougher requirements come up, you're just going to end up abandoning the technology and
moving on to a client-centric system.

Mark Waschkowski said...

"Vaadin and other server-centric applications will hit the server for every sort and filter.
SmartGWT will do most operations client-side. These are the critically important requests
since they are requests for data."
This often requires going server side because the client will not have the full data set to sort/filter on anyway.

"While we're on the topic, the other very serious problem with server-centric architectures is
clustering. Vaadin requires a chunk of server-side session state which is totally unnecessary
in SmartGWT and other client-centric architectures, and in a clustered deployment, this state
has to be replicated amongst all servers in the cluster."
Nonsense, that is completely dependent on the type of clustering implemented - sticky sessions commonly used for this.

The 10x and 100x arguments are off base - many different things contribute to performance characteristics. Just making statements like that make me realize that you have an agenda to push. The I saw you were from SmartGWT - surprise, surprise.

Sigh

Charles Kendrick said...

I linked to the example so you could see a further explanation and verify the claim yourself. The Adaptive Filtering & Sorting behavior means that local filtering and sorting kick in automatically when the dataset has been reduced to the point that it's all cached. User very very rarely choose to scroll through a dataset that's too large to cache, so this feature tends to eliminate server contact for the vast majority of sort and filter operations.

This is just *one category* of server request that SmartGWT eliminates, that's why the aggregate benefit is 10-100x.

As far as clustering and sticky sessions:

1. GAE does not support them, and you can expect this to be the case for other global-scale cloud services. It's difficult to support sticky sessions when you have dynamic DNS and sophisticated load balancing scenarios involved.

2. sticky sessions do not avoid replication overhead in many scenarios. Even with stick sessions enabled, clustering solutions often replicate anyway, to at least one other cluster member. And they should, because it's not acceptable in many applications (banking, portfolio management, telecom ops, defense) that a batch of user sessions are lost if a clustered server crashes.

And this is just the scalability aspect. Big picture:

1. offline is impossible

2. the application will always feel sluggish relative to a client-centric solution, especially when latency is high or the server is loaded

3. advanced drag and drop scenarios in which logic must run on every mouse move basically cannot be achieved with acceptable user experience & server load. Likewise anything where you want to run logic per-keystroke.

4. scalability and clustering issues as discussed above

And yes, I'm the CTO of Isomorphic (not exactly a secret), regardless, what I've said is both accurate & verifiable.

Mark Waschkowski said...

1. offline is impossible
I write web apps. I've never written an offline app. If you need offline, then you will want to consider this.

2. the application will always feel sluggish relative to a client-centric solution, especially when latency is high or the server is loaded

'always', 'especially when' - I call FUD and plain misinformation! You simply can't say 'always' when talking performance claims, you are never right! The applications I've written with Vaadin are very responsive, an exceed our latency requirements by a significant margin. Sigh...

3. advanced drag and drop scenarios in which logic must run on every mouse move...
I don't care, and most developers won't in this fringe scenario...you make every potential concern sound like a 5 alarm fire.

4. scalability and clustering issues as discussed above
I've already referred to those above.

And yes, I'm the CTO of Isomorphic (not exactly a secret), regardless, what I've said is both accurate & verifiable.

Mark says:
Actually, its clear that you are making statements like 'always' and 'has to' when its verifiably not the case.

Please stop with the hyperbole so a better conversation might be had.

Charles Kendrick said...

Mark, it sounds like we've each worked on very different sets of applications.

To explain my perspective, I've either directly implemented, been the managing architect, or been deeply involved in about 80-100 large scale enterprise web applications across the global 2000, many enterprise ISVs, and lots of SMBs and startups.

I can tell you that:
1. custom drag and drop is not at all "fringe" in these types of applications. Companies build custom software because existing off-the-shelf solutions aren't good enough, so it's very common that the UI is advanced and has requirements for client-side interactivity on at least one screen. And it only takes one screen for the overall requirements to be unable to be met by a server-centric technology.

2. yes, it's accurate that client-centric solutions are "always" faster across all of these applications. Please realize that you're referring to latency requirements while I'm talking about the *complete absence* of a trip to the server. Client-centric solutions can provide instant response in many, very common use cases (grid filtering, live input checking, etc) - when you watch expert users ripping through large datasets in a SmartGWT interface, then you'll understand my unqualified statement that a server-centric solution is going to feel sluggish.

Sergey said...

I've used SmartGWT about one year in our company, but finally we decided to migrate on Vaadin, and eventually on ZK because during a proof of concepts phase it shows a more acceptable performance. This was a collective decision (whereas I personally would prefer a Vaadin due to its 'all in Java' approach - feel disgust to JS & markup chaos). As to SmartGWT... it looks & feel very well but work in real word... not so good... I was forced to spend many time in FireBug to determine its bugs or right call semantics/usage.

Taguato said...
This comment has been removed by the author.