• A live polling system in Ember.js

    In the Hstry application, we use a live-update system similar to the one in Facebook. When a student or a teacher is looking at a timeline, he automatically sees the comments and messages that the other users are posting without having to refresh the page. I spent some time figuring out how to get this to work in Ember.js but eventually I managed to find a solution.

    Read more

  • Uniqueness validations in Rails

    Validations on models in Rails are very powerful. They allow you to express certain properties that attributes of your model must possess, like uniqueness, numericality or presence, and Rails automatically takes care of enforcing them when a model is persisted. Moreover, Rails also provides customizable messages that tell you what went wrong when certain validations didn’t pass.

    The uniqueness validator ensures that rows in your table are unique for certain columns. For instance, you can use it to ensure that your users’ emails are unique by setting a uniqueness validator on the attribute email of your Users table. However, because validations are not enforced at the database level, things can go wrong when your application consists of multiple processes. The following diagram makes this obvious.

    Consistency failure for uniqueness validations in Rails

    From http://robots.thoughtbot.com/post/55689359336/the-perils-of-uniqueness-validations

    The uniqueness validation is not atomic, therefore we can a typical synchronization issue that arises when you have concurrent processes.

    Unique indexes on database

    The way to solve this problem is to declare unique indexes on the unique column. This way, the second creation in the diagram would fail because of the constraint on the database. This new situation is shown in the diagram below.

    Uniqueness validations are consistent with database indexes.

    From http://robots.thoughtbot.com/post/55689359336/the-perils-of-uniqueness-validations

    Using Rails migrations, you can declare an index as follows.

    class AddEmailIndexToUser
      def change
        # If you already have non-unique index on email, you will need
        # to remove it before you're able to add the unique index.
        add_index :users, :email, unique: true
      end
    end

    The consistency_fail gem and Git

    The consistency_fail gem analyses your models and identifies situations where you declared a uniqueness validator on an attribute without a corresponding database index. In order to identify such cases early on, I run the gem before each commit using a pre-commit hook in Git. This forces me to fix the problem because I am confronted with it each time that I make a commit and the commit fails if consistency_fail identifies a problem.

    My .git/hooks/pre-commit file looks like this:

    #!/bin/sh
    
    # Invoke consistency_fail for uniqueness index failures
    consistency_fail

    Make sure it is executable.

    chmod +x .git/hooks/pre-commit
  • First Ember.js Meetup in Brussels

    On Thursday 31st of October, I’m organising the first Ember.js Meetup in Brussels. The purpose of the group is to have a local community where Ember.js users and fans (appropriately, we call ourselves “Embeerists”) will share knowledge and experience, have discussions about the framework, learn from each other and just have an interesting evening among peers. The plan is to organise a meetup about once a month.

    So if you live in or around Brussels, you are interested in Ember.js or you are already using it, feel free to come to the first meetup.

  • A round of optimizations on Hstry.org

    At hstry.org, we re-enact historical events through social media in the form of timelines. Our timelines tend to be quite big, with a lot of DOM elements, images, video embeds and audio embeds. We noticed pages were starting to load slower and slower so it was time that we looked at the performance of our website in detail. The result is a series of optimization steps, both at the front-end and the back-end. After all, a bad performing website with long loading times is bad for user experience and also negatively impacts SEO.

    Performance is hurting UX

    One example of our timelines is the Everest timeline, where we re-enacted the first successful ascent of Mount Everest by Edmund Hillary and Tenzing Norgay. We pretended that Edmund Hillary had a smartphone back then and was tweeting about his ascent. Since the ascent took 47 days, a total of 400 tweets were posted. Before our round of optimisations, they were all loaded during the initial load. With this many elements, including about 50 images, 3 YouTube embeds and a Spotify embed, you can guess that the performance was suffering. When the page was fully loaded, a total of 5.6MB had been downloaded. This is already far from ideal on desktop, let alone on mobile. The poor performance has a very negative impact on the user experience but also on SEO so it was time that we did something about it.

    Front-end optimisations

    The biggest bottleneck was the front-end so there was a lot to gain here. We mainly used two techniques to cut down the initial loading time.

    • infinite scrolling. We divided each event into chapters. This improved both the user experience, as it became easier to navigate the timeline, and we could use it to implement an infinite scroll. When a chapter is shown, it sends a request to fetch the next chapter from the server. As the user scrolls down and almost reaches the end of the chapter, the next chapter is shown. This goes on as the user scrolls down. With this technique, only the first chapter is loaded at the initial load and the other chapters are loaded if necessary. For robots to be able to crawl the page, we added static links to the different chapters in <noscript> tags.
    • lazy loading of resources. We only load images, video embeds and audio embeds if the user sees them. As a user scrolls up or down, a resource is loaded a bit before it actually gets into the user’s viewport. This technique had a drastic impact on the loading time because only a small portion of the resources are loaded initially.

    In both cases, I used a customized version of the Waypoints jQuery plugin. With this plugin, you can trigger events when the user scrolls at a certain position. I used it extensively to trigger the appropriate events for showing a chapter or loading a resource.

    The effect on the loading time was big: we went from 5.6 MB to 550 KiB of initial load.

    Back-end

    Although the greatest gains were to be made on the front-end side, we also put some effort in performing some optimizations on the back-end side.

    • Caching. As the database grows, queries take and it also takes some time to generate the view. Rails makes it very convenient to cache parts of a view using fragment caching. You tell which block of the view you want to cache. At a first load, it generates a static HTML file for that block and saves it either on disk or in memory, depending on the configuration. At the next request for this block, the static version is returned instead of being regenerated. A cache digest and a cache key is added to the filename so that it is easy to control of the expiration of a cache item, for instance if the block is updated or a specific row in the database is updated.
    • Cloudfront CDN for assets. To store our images, stylesheets and Javascript assets, we use the Amazon S3 storage service. It’s cheap, reliable but not very fast as this blog post shows. Instead, we switched to the Cloudfront CDN for delivery of the assets. Since browsers serialize requests to the same domain, we use 4 different domains: assets0.hstry.org, assets1.hstry.org, assets2.hstry.org and assets3.hstry.org. They all redirect to the same Cloudfront bucket but it enables the browser to download assets in parallel.
    • Server-side compression of HTML pages: To reduce the size of GET requests to HTML and JSON assets, we enable compression with the heroku-deflater gem.

    Conclusion

    Performance is a very important factor in the user experience. If a user has to wait too long for a page load, he will very likely leave before the load has finished. Therefore, this series of optimization steps were important for a better user experience overall.

  • Lessons learned from Hstry’s pilot: thick clients are good

    Last Friday we had Hstry‘s first pilot at a school in London and it was the first time we tested our educational product in a real class with real pupils. I was busy beforehand working on the product and at one point during the development I had an architectural decision to make: I could implement the view logic of one specific feature either on the server, at the controller level in the MVC paradigm, or as Javascript running on the client. Back then, I thought it would be good practice to keep the view as dumb as possible and therefore I implemented it in the controller. Only later did I realize that, after we had had server-overload issues during the pilot, I should have gone the thick-client way.

    Read more

  • Estimate the complexity of your software with planning poker

    At the startup that I joined recently, Hstry.org, we are currently building a product for the education market. Our goal with this product is to make the process of learning history more interesting and fun for students. Ultimately, we hope that our method will have a positive result on the outcome of learning. To test our product, we are running a pilot at a summer school in July and so we want to have a prototype ready by then. A few weeks ago me and my CEO attended a workshop about agile development and we thought that building this prototype would be a good opportunity to apply some agile techniques in practice. To estimate the development complexity of the features of our product, we used the “planning poker” technique and I have to say that it was a very valuable exercise.

    Read more

  • The “Learning Ruby” Project

    Two weeks ago, I started learning Ruby. To get a first acquaintance with the language, I followed the Ruby course on Code Academy. I had never followed any tutorial there and it’s alright. It’s quite simplistic because they target beginner programmers. For me, it was good to get to know the syntax of the language and get in touch with a few basic Ruby constructs like blocks, procs and modules. So far I am really enjoying the language; I like the fact that it is so concise.

    The next steps for me are to practice the language by solving some Euler problems while at the same time reading Programming Ruby, diving into the Ruby on Rails documentation, follow the Ruby tutorials on Code School and writing tests for an application that I took over. Needless to say it’s going to be busy times!

  • Write maintainable CSS with SASS

    I recently started using the SASS framework for any front-end development work that I do. The trigger came from a ever-going frustration about trying to write maintainable CSS code. Now I wish that I had discovered SASS earlier. I’m even so positive about it that I wrote this article to share my experiences.

    Read more