How we deploy the Hstry application

As an application grows in size and in users, having a good deployment process becomes increasingly important. It’s not only important in order to avoid breaking existing functionality when pushing changes but also to make sure that everyone on the team (developer and non-developer) is happy with the implementation of the feature that is going live. In this blog post I will describe how we structured our deployment process at Hstry.

The overview

A picture says more than a thousand words so let’s start with a pretty diagram.

An overview of the deployment process for the development of the Hstry application.

An overview of the deployment process for the development of the Hstry application.

Rationales

I’m explaining the different steps in the process down below, but here are the underlying reasons for why we came to this process:

  • Early feedback. Being able to get feedback on the implementation of a feature from everyone on the team early is important to us. As developers, we want to show the work-in-progress of a feature as soon as possible to the rest of the team so that we can quickly adjust. After all, we are still an early-stage startup and we can’t waste time on throwing away large amounts of work. Using the incredibly useful fork functionality of Heroku, we wrote a custom Rake task that can deploy a feature branch to a throwaway application with a single command.
  • Confidence through testing. The continuous integration server, which runs our entire test suite for every build (both unit and full-stack integration tests), is a crucial step in giving us the confidence that the changes that we push won’t break anything major. Of course, small bugs may arise (unless you want to run several hours of integration tests at every build) but it’s important that nothing big breaks.
  • Opportunity to do manual testing with production data. Our database is still small enough that we can copy the production database every night to our staging environment. This allows us to do a final check of the changes with the actual production data (of at most 24 hours old) before we do the big push.
  • Automation. We automated every step with custom Rake tasks so that we can deploy to any environment with a single command.

The different stages of the process

As you can see on the overview diagram, we have five remote environments, each corresponding to a different stage in the process:

  • remote development
  • fork
  • continuous integration
  • staging
  • production

Remote development

When I was the sole developer on Hstry, having a single remote development server was enough to show ongoing implementations of features to the rest of the team. However, when we expanded the development team and we started working on multiple features simultaneously, a single server wasn’t enough because of conflicts. So we introduced the fork servers.

Fork

Forks are like throwaway applications. We only use them to show ongoing work of a feature branch to the rest of the team. Once a feature branch has been integrated into the master branch, we destroy the fork application corresponding to that branch. Since moving to this model, we only use the development application as a template to generate forks.

Continuous integration

We use Codeship as our hosted integration server. Every time we push changes to a branch on our BitBucket repository, a build of that branch is run on Codeship. It runs our entire test suite (back-end, front-end and integration tests) and therefore alerts us if our changes are breaking some of the existing functionality. If it is the master branch that is being built and all the tests succeed, it is deployed to both the remote development and staging server.

The continuous integration server is crucial for us to be confident that one of our changes doesn’t break something major. We have a relatively big suite of unit tests (and are constantly aiming for a higher coverage) and we have a few integration tests that test typical scenarios for a user on the application.

Staging

The staging server is the final step before production. We use it to do a manual QA with actual production data before going live. Currently, our database is small enough that we can do a copy of the production database to the staging server every night. When that is no longer feasible (we hope that day will come soon!) we will have to resort to follower databases.

Production

Production is well, production. It’s what you see when you go to https://edu.hstry.co!

Conclusion

For now, this process has been working well for us. However, we are still a small development team and therefore we expect the process to evolve as the team grows.

If you have any questions or comments, please don’t hesitate to reach out to me at yoran@hstry.co.

Trackbacks for this post

  1. Agile development at HSTRY | Yoran Brondsema