Monday, December 15, 2014

Re: efficiently push data

Thanks again for your answer Peter!

ok, I think I'll do it by myself over websockets.
Andy



Am Donnerstag, 11. Dezember 2014 20:51:08 UTC+1 schrieb Peter Donald:

Hi,

... 
Names I read when searching for gwt and server push or synchronization between server and client are for example:
  • gwteventservice
  • RequestFactory
  • gwt-comet
  • Atmosphere
  • Jboss Errai
From that list I would suggest Errai as being the best candidate but I have not actually used it while the others I have investigated and found that they didn't work for one reason or another..


In short:
A framework where I can register a list of Pojos on server side and have the corresponding list on client side to work with including the possibility to listen for changes on client side. Is there something coming close? Of course it would be great if the framework is doing something fancy to minimize the necessary traffic too. But the transparent event and data exchange is more important than that.

However to be honest I think if I was in your scenario I would simply create simple json messages on the server and explicitly send them to the clients. I would typically do this over websockets if supported by the browser, otherwise use long polling. 

I have attached a response I made to a similar question that may help you.


On Fri, Nov 7, 2014 at 7:08 AM, Peter Donald <pe...@realityforge.org> wrote:
Hi,

On Thu, Nov 6, 2014 at 12:07 AM, Marcel K <marcel...@gmail.com> wrote:
Which framework for Server Push do you recommend, since Atmosphere doesn't seem to work with GWT 2.7.0?

It sort of depends on your environment and what your underlying platform supports. Below is a response I made to another similar question on the mailing list.

In short if you can use latest JEE or jersey then I would recommend that you use 

We use all three in our server push apps with GWT clients.

HTH 


On Fri, Oct 31, 2014 at 5:22 AM, Peter Donald <pe...@realityforge.org> wrote:
Hi,

We have built several systems like this where we want to push data to clients from the server. The client is responsible for subscribing to a channel possibly with filters in place. Rather than use GWT-RPC we have tended to move to using raw messages that are typically json blobs. We have used 4 basic strategies depending on the clients supported and the application requirements. The strategies are;

1. WebSockets
2. Server-Sent Events
3. Long polling
4. Periodic Polling

I had a poke around and found a good stack overflow answer to this (See [1])

When we faced the same issues, we ended up putting together a couple of libraries to help with in GWT.

* websockets: https://github.com/realityforge/gwt-websockets
* Server-sent Events: https://github.com/realityforge/gwt-eventsource
* Polling: https://github.com/realityforge/gwt-webpoller (Warning this
API is not yet at 1.0 status)

The tricky part is actually managing the channels and messages sequencing in each channel but we never got arounf to releasing our code for that part of the problem.

-- 
Cheers,

Peter Donald


On Fri, Dec 12, 2014 at 2:12 AM, andymel <sound...@gmail.com> wrote:
HI Peter,

thank you very much for taking the time to write this very interesting post! Especially the 'delta-hierarchy'-graphs and the TransactionRegistry will inspire me if I have to implement it by myself. But my main hope was that someone could point me to something that handles the synchronization between server and client for me. 

More about my context:
My server doesn't use any persistence framework. Different clients connect and stream byte data to my server directly to disk. During the stream I extract meta data. When the client ends the stream the metadata is saved as file to the streamed data on disk. 
The browser gui is used to browse and work with this data in different ways and should be updated automatically when new streams arrive.

During my research I found a list of libraries that seem to be meant to solve the synchronization between server and gwt-client side objects. 

Names I read when searching for gwt and server push or synchronization between server and client are for example:
  • gwteventservice
  • RequestFactory
  • gwt-comet
  • Atmosphere
  • Jboss Errai
As there is no time to try everything and find all pro and cons before I decide my best case scenario would be if someone could point me in the right direction after I write down what is needed.

In short:
A framework where I can register a list of Pojos on server side and have the corresponding list on client side to work with including the possibility to listen for changes on client side. Is there something coming close? Of course it would be great if the framework is doing something fancy to minimize the necessary traffic too. But the transparent event and data exchange is more important than that.

Only if nothing fits I will implement it by myself. Thanks Peter once again for adding a great answer for that exit scenario.

Andy


Am Donnerstag, 11. Dezember 2014 01:49:00 UTC+1 schrieb Peter Donald:
Hi,

So we have several applications where we follow a similar pattern. We use java EE but any framework that allows you to intercept your business services and supports JPA EntityListener's should be capable of directly implementing this. 

Our EntityListener implementation has a @PostPersist, @PostUpdate and @PreRemove that captures the delta of the jpa entity. These deltas are stored in TransactionSynchronizationRegistry. We need to use the registry rather than thread locals to deal with EJBs that are run in separate threads. 

Then we add an interceptor on our business services and the boundary/very first service that is invoked by the client is responsible for collecting all the changes out of the TransactionSynchronizationRegistry and routing them to "interested" clients. The clients then receive the deltas and apply them to their local model. The changes are packaged up into one message per transaction, which ensures the client-side state is always consistent.

The content based routing is the hard part. Clients subscribe to "graphs". Typically we define graphs that have a single root entity. When an entity delta is routed we allocate it to 0 or more graphs. Then the graphs determine which clients the delta is routed to. Then the deltas are aggregated into a single message for each client.

Imagine a domain model as simple as the following.

class Company { int id: String name; }
class Person { int id: String name; Company company; Person reportsTo; }

And there is two possible subscriptions. You can either subscribe to a Company and receive all changes for a company or you can subscribe to a person and all the direct reports.

So if we imagine data like

Company {1, "Bobs Bobcats"}
Company {2, "Marys Mowers"}

Person {1, "Bob", 1, null}
Person {2, "Bill", 2, 1}
Person {3, "Mary", 2, null}

So imagine a change happens to Person 2. The change would be associated with the "Supervisor Graph" rooted at Person 1 and it would be associated with the "Company Graph" rooted at Company 1. If a change happens to Person 3. The change would be associated with the "Company Graph" rooted at Company 2.
 
The changes are pushed out to the clients based on a combination of long polling, server-sent events and websockets.

Of course this is quite complicated so we have a DSL that generates the majority of this with a couple of extensions points where developers can write routing code etc. 

Mostly this means have fairly complex apps with up to 80 entities that are replicated to the client with most developers being completely unaware of how it works and being able to write simple code paths. The UI developers only care about receiving changes and reflect them on the UI, the server side developers just write simple state transform code, the network layer is extremely efficient and it is trivial to build multi-client interactive applications.

HTH

On Wed, Dec 10, 2014 at 4:27 AM, andymel <sound...@gmail.com> wrote:
Hi,

I think about how to implement a browser GUI that primarily will show a list of a lot of data objects. When something is added to the list on server side or one of those objects is changed, the client side GUI should be updated. So I need a server push. I already have a websocket connection and used the AutoBean functionality for something else. In theory I could immediately implement the whole thing by either pushing the whole list each time something is changed or even do any suitable caching. But I'm relatively new to GWT and I'm sure there are a lot of things that I don't know yet. I'm curious how people with more experience would implement it.

During my research today I discovered the RequestFactory where caching seems to be included. At least I found the sentence "Furthermore, the EntityProxy interface enables RequestFactory to compute and send only changes ("deltas") to the server." Can this be used to have a List/Result entity on the server and client that is updated automatically with just a little bit of traffic in between without thinking about what was changed by myself?

Do you recommend anything else for this use case?
Thanks in advance!
Andy

--
Cheers,

Peter Donald

--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit+unsubscribe@googlegroups.com.
To post to this group, send email to google-we...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit.
For more options, visit https://groups.google.com/d/optout.



--
Cheers,

Peter Donald

--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit+unsubscribe@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at http://groups.google.com/group/google-web-toolkit.
For more options, visit https://groups.google.com/d/optout.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)