Putting Code to the Test With Tapad’s Custom SBT Plugin
by Kurt Kopchik
Before shipping new code to production, it’s important to be confident it will work as expected. The last thing you want to do is deploy an untested or poorly tested change and end up like Knight Capital Group, where they lost $460 million within minutes. At Tapad, we have a set of quality gates that code must pass through prior to deploying to production. The final step in this process is deployment to a staging environment that matches the production environment as closely as possible. This is a great sanity check to perform a final validation on your changes; however, there are a number of issues with using this approach as a team grows. Here are a few we’ve found:
- Staging is a shared resource, which will result in contention. For example, Developer A and Developer B both have changes to deploy that touch the same feature areas.
- If a staging server goes down or maintenance needs to be performed, everyone (including releasing to production) is on hold.
- Consistency issues of what bits are running in staging crop up over time. Once a developer’s code is deployed in staging and there was an issue did they cleanly roll it back? Did they clean any bad database records that were created?
- By the time a change gets to staging, there should be high confidence that it will work correctly. Staging should not be used as a place to test out code that was “too difficult” to test locally.
Some of the above issues can be managed by being very diligent about the use of staging, but as the team grows this approach does not scale well. We knew we needed to develop a way for developers to have their own isolated and disposable staging environments.
###pDocker with Docker-Compose(formally known as Fig) to build these environments. Docker leverages Containers instead of Virtual Machines. As a result, it is much more lightweight because the underlying OS and hardware stack can be shared between containers. Docker-Compose is a layer on top of Docker that provides an easy-to-use Yaml-based file format to define different services and how they communicate with each other. At Tapad we call this a ‘topology definition’.
Once you have enabled and built Docker images for the repositories you are interested in and have defined how they interrelate, you can start up an instance by simply running:
docker-compose up –f <topology.yaml>Although this approach worked, there was too much context switching that needed to occur to use docker-compose out of the box. Tapad writes the majority of its code in Scala and developers live in “sbt” which is the build tool for Scala. As a result, developers would have to compile their code in sbt and then switch over to the shell to kick off a docker-compose instance. There also wasn’t a good way for developers to start an instance with their latest changes without first having to manually compile code and build a new Docker image.
Our solution: a custom-built plug-in sbt-docker-compose that integrates Docker Compose functionality directly into sbt. This plugin takes care of compiling your code, building a new Docker image, and starting up a new uniquely named instance with dynamically assigned ports. Now a developer just needs to enlist in a repository and run one command “dockerComposeUp” from sbt and they are up and running.
We didn’t stop there. The plugin also contains the ability to pass the docker-compose instance information into ScalaTest test cases for execution against the running instance. This allows you to now run more advanced Integration and End-to-End tests inline with your Unit Test cases. The plugin is also designed so that it can be extended to add additional features that are custom to your internal systems. For example, at Tapad, not only can launch instances locally we can also kick them off in our Apache Mesos system. In the future, the same could be done as an extension for AWS. There are many more advanced features and examples detailed on the sbt-docker-compose GitHub repository.
After great success with using this plugin at Tapad, we wanted to share it with the world. So if you are a Scala developer and want to raise the bar on how you test your code, we strongly suggest you give it a try!
