A live polling system in Ember.js

In the Hstry application, we use a live-update system similar to the one in Facebook. When a student or a teacher is looking at a timeline, he automatically sees the comments and messages that the other users are posting without having to refresh the page. I spent some time figuring out how to get this to work in Ember.js but eventually I managed to find a solution.

Update user comments in “real-time”

On our page, we have a list of posts. Each post has a list of user comments, which we want to live-update. Any comments that are created by a user must be propagated to the clients within a reasonable time frame.

Backend

In the back-end, we have a REST endpoint /updates. Given a parameter last_request_timestamp, it returns a JSON object containing the comments that have been created since that timestamp.

Polling object

On the front-end side, we define a Pollster object that has methods start() and stop() (see this blog post) . When we call start(), it starts executing onPoll() at a regular interval. Conversely, it stops polling when we call stop().

App.Pollster = Ember.Object.extend({
  interval: function() {
    return 5000; // Time between polls (in ms)
  }.property().readOnly(),

  // Schedules the function `f` to be executed every `interval` time.
  schedule: function(f) {
    return Ember.run.later(this, function() {
      f.apply(this);
      this.set('timer', this.schedule(f));
    }, this.get('interval'));
  },

  // Stops the pollster
  stop: function() {
    Ember.run.cancel(this.get('timer'));
  },

  // Starts the pollster, i.e. executes the `onPoll` function every interval.
  start: function() {
    this.set('timer', this.schedule(this.get('onPoll')));
  },

  onPoll: function(){
    // Issue JSON request and add data to the store
  }
});

Note: I encounter a problem when testing this route. When running a test, the ember-testing framework waits until all promises are resolved before moving on to the next test. Therefore, a test on this route never finishes because of the infinite pollster. To get around this problem, I wrote the tests for this route in the back-end, in our case through Capybara (we use Ruby on Rails). If anyone has a solution to this problem, please let me know!

Update (5th May 2014, credits to Mark): Instead of the setInterval function, it’s advised to use the Ember.run.later method (see http://emberjs.com/api/classes/Ember.run.html#method_later). The callback is then executed within the Ember RunLoop, allowing it to coalesce data bindings and view updates.

In the Route for which we want to do live updates, we create and start a Pollster in setupController and stop it in the deactivate hook.

App.Route = Ember.Route.extend({
  setupController: function(controller, model) {
    if (Ember.isNone(this.get('pollster')) {
      this.set('pollster', App.Polster.create({
        onPoll: function() {
          /* Defined below */
        }
      }));
    }
    this.get('pollster').start();
  },
  // This is called upon exiting the Route
  deactivate: function() {
    this.get('pollster').stop();
  }
});

Pushing data to the store

Ok, so now we have a polling object that calls its onPoll function at regular intervals. How do we push data into the Ember store?

In our onPoll function, we need to issue a JSON GET request to the /updates endpoint. This call will return with a a list of Comments in JSON format and we need to push these into the store. For that, we use the push() method (doc). This method roughly takes a JSON object as input, transforms it into an Ember model and adds it to the store. We also use recordIsLoaded to check whether an object is already present in the store.

// This code is executed in the 'setupController' hook of the Route
var route = this;

onPoll: function() {
  Ember.$.getJSON('/updates', 'GET').then(function(json_obj) {
    // The JSON structure is as follows:
    // {
    //   comments: [
    //     { ... },
    //     { ... },
    //   ]
    // }

    // Iterate through the comments
    json_obj.comments.forEach(function(comment) {
      // Make sure that the comment is not already in the store
      if (! route.get('store').recordIsLoaded(App.Comment, comment.id)) {
        route.get('store').push('comment', comment);
      }
    });
  });
}

Automatically refresh the template

We are not there yet because changes in the store are not automatically reflected in the template. This is explained in the Model FAQ in the Ember.js guide. I cite:

Because the server decides which records match the query, not the store, queries do not live update.

It also says that if you want to bind the data in your templates to the store, you should use filters.

Filters, on the other hand, perform a live search of all of the records in the store’s cache.

Our (simplified) template for displaying a Post is as follows.

<article class="post">
  <h2>{{title}}</h2>
  <p>{{content}}</p>
  <aside>{{render "comments" comments}}</aside>
</article>

We need to override the comments property in the controller so that it returns a filtered list of comments. This way, this list will automatically update when Comments are added to the store.

App.PostController = Ember.ObjectController.extend({
  comments: function() {
    var postId = this.get('id');
    return this.get('store').filter('comment', function(comment) {
      return comment.get('post.id') == postId;
    });
  }.property()
});

And it works!

Conclusion

Using jQuery’s getJSON and Ember’s push and filter, we can create a powerful live-updating system where the data shown in the view is synchronized with the models in the store. Ideally, you would like to use a push system such as WebSockets instead of a polling system like the one described here. However, they bring issues with them (cross-browser compatibility for one) and require a different server setup too. As long as your use case does not require hard real-time updates (e.g. user comments), a polling system with a sufficiently long refresh interval can go a long way.

Write us your thoughts about this post. Be kind & Play nice.
  1. Joe Fiorini says:

    Great post, thanks for the reference to my blog post. My onPoll now defaults to `this.store.find(“model”)`. Is there any reason you can’t do that too? That would take care of updating existing records and adding new ones at the same time.

    What would you think if the onPoll callback was a route action? It could still have a default behavior to reload the model, but it could be overridden from the route. And by making it an action it would get called all the way up the route hierarchy.

    Reply
    • yoran says:

      Hi Joe,

      I’m glad you liked the post, and thank you for writing your initial post! To answer your first question, your suggestion is indeed simpler when you need to update a single model. After all, you then make use of Ember-Data instead of re-inventing the wheel by manually making the request. However, in my case, I actually have multiple models that I need to update (I left this detail out in my story for the sake of simplicity). By calling “/updates”, my back-end takes care of including all the models that I’m interested in updating. Using ‘this.store.find(“model”)’ would result in a HTTP request per model, whereas now I have them all with a single HTTP request.

      Your second suggestion is very interesting. By default, polling would be disabled in the Route but you could set a property ‘doPolling’ to ‘true’, which then enables polling. However, I am trying to think of a use-case where you would want to define ‘onPoll’ on another route than the leaf route, so that making ‘onPoll’ an action would be useful. I can’t find any though. It’s definitely a cool idea, did you discuss this already on Ember Discuss?

      Reply
  2. Why not use `Array.forEach` instead of `Ember.$.each`?

    Reply
  3. Marc says:

    I believe it’s recommended to use Ember.run.later instead of setTimeout. http://emberjs.com/api/classes/Ember.run.html#method_later

    Reply
  4. Ray Tiley says:

    Am I missing something or is the pollster defined in setupController a local variable. Its not going to be available in the deactivate hook. The way it reads the polling is never going to end and your going to get an error in your deactivate hook when you try to calls top on an undefined variable.

    The idea in this post is great. Thanks.

    Reply
    • Yoran Brondsema says:

      Hey Ray, you’re absolutely right, thanks for noticing it! I updated the code example. When the Route is first instantiated it sets the property `pollster` on the Route. This is probably slightly better than creating a new Pollster object every time we enter the Route, although garbage collection should take care of cleaning up.

      Reply
  5. Joe says:

    Any reason for using the for each? As far as I can see, the push method in ember seems to not add the messages are already in the store (i.e. it doesn’t duplicate ids).

    Reply
    • Yoran Brondsema says:

      Hi Joe, when I wrote the post it wasn’t clear to me that `DS.Store.post` updated existing models. You’re right, it now says it in the documentation. Thanks for the comment, I added a comment in the code snippet.

      Reply
  6. Moises says:

    Great post, I was looking for this use case exactly..

    Reply
  7. Peter says:

    Thanks, I needed this! Small question though. My server respond with snake_cases and Ember dutifully converts these to camelCases when needed but not in this example. The view updates and loads the new data (`post.id` for example is defined) but I have a `created_at` attribute as well in the JSON and the template doesn’t find that value. It does work if I change my server response to send camelCases. Any idea?

    Reply
  8. Jaydev says:

    I was able to get around the problem of the infinite run loop in my async tests by scheduling a task to kill all timers before the never-ending poll is triggered:

    test(‘my awesome test’, function() {
    Ember.run.next(this, function() {
    Ember.run.cancelTimers();
    });
    // now call your infinite poll
    });

    Reply
    • Yoran Brondsema says:

      Hi Jaydev, thanks for your trick. Unfortunately, the thing happens during integration tests so wherever I call the timer cancellation snippet, it’s either too late or too early. Also, I’m a bit weary of globally turning off all timers. There are some timers that I might want to keep, as they don’t create an infinite loop for instance.

      Reply
  9. victor says:

    Hi,
    very good article.
    How can I do to show a loading spinner while onLoad is processed?

    Reply
  10. Benoît says:

    Thank you Yoran, very useful post, it worked instantly !

    Reply
  11. Brommersman says:

    You may have already solved this since this post is quite old now but in terms of getting your tests to pass I have found that you need to tell Ember to run the later function instantly and not to reschedule it e.g.

    allow interval to be set (or read it from application config – import config from ‘config/environment’)

    then in testing set the interval to be 0 (this will cause Ember to run the later function straight away)

    schedule: function(f) {
    return Ember.run.later(this, function() {
    f.apply(this);
    if(this.get(‘interval’) > 0){
    this.set(‘timer’, this.schedule(f));
    }
    }, this.get(‘interval’));
    }

    the check if( this.get(‘interval’) > 0 ) will prevent the function from rescheduling itself.

    This obviously doesn’t allow you to test if it will run every X seconds – but then you are testing the Ember.run.later function rather than what your function does.

    Reply
  12. Gaurav Sharma says:

    Hi Yoran,
    Very clear and simple post.
    I have implemented polling in my project and it is working fine. But I am facing issue regarding automatically updating of template.
    Scenario: I want to perform certain event with an instance by clicking on its checkbox. For that I have used itemController. But when I check the checkbox it automatically uncheck it by rendering the new template by polling which is creating issues. Please help me in this issue.

    Reply
  13. Ujjval says:

    Hi, Thanx for this document. Very well explained and It helped me a lot in my implementation. I have further queries about polling in ember. I have model ‘A’ which contains one to many relationship with model ‘B’ & model ‘B’ also contains one to many relationship with ‘C. I want updated data of all of these models and for this I am using your documentation. But, I have observed that when I bring data for ‘A’ model from server, It goes to update DOM of all of its child property (model ‘B’ and ‘C). But only few properties of those objects is changes. I want to know how can we restrict ember to update DOM by matching each property and updating DOM only for the property which is updated.

    Reply
    • Yoran Brondsema says:

      Hi Ujjval, sorry about the (very) late reply. As far as I know, since HTMLBars was introduced (Ember 1.10 I believe?), Ember does “smart-updating” of the DOM. This means that at every run-loop, it only updates the parts of the DOM whose dynamic parts (computed properties) have been updated. I think they use the same ideas as the concept of virtual-diff introduced by React.js. Also, with the introduction of Glimmer 2 in version 2.10, the rendering performance has only increased. So I think this issue has been taken care of and it’s not something you need to worry about.

      Reply
  14. Martin says:

    I wonder if there’s a way to stop the pollster from the implemented onPoll function itself?
    Imagine that the gotten payload may include a flag like ‘completed’ which remains false as long as the server might send some new data… And once the flag gets true the pollster is expected to stop the polling. I noticed that the pollster stops while leaving the route but never managed to get ’em down while staying on the route.

    Reply
    • Yoran Brondsema says:

      Hi Martin, the Pollster object has a `stop` function. What I imagine would work, would be to call this function when you get a payload with the ‘completed’ flag set to true. Does that answer your question?

      Reply
      • Martin says:

        Hi Yoran, I followed your code almost straight forward. The thing is that I can get a reference to the pollster fairly easy with ‘_this.get(‘pollster’)’ within the closure and I can even call the method with ‘_this.get(‘pollster’).stop();’
        The thing is that it does not destroy the timer at all – I can see the onPoll handler still ticking. I am on Ember 2.10 using Chrome on a Mac.

        Reply
        • Yoran Brondsema says:

          OK I see. I think you’ll need to add a `isRunning` property on the Pollster object. This will be set to ‘true’ in `Pollster.start`, to ‘false’ in `Pollster.stop`. Also, you’ll need to wrap the body of `Pollster.schedule` with `if (this.get(‘isRunning’)`.

          PS: a lot has changed in the Ember world since this blog post came out. The `ember-lifeline` addon has a `pollTask` functionality which is a bit more robust than the implementation in this blog post. See https://github.com/rwjblue/ember-lifeline#polltask for more info.

          Reply