Matt Soldo

Upgraded Autobuild System on Docker Hub

Docker Hub‘s automated build system is one of it’s most popular features. One quarter of all repositories on Docker Hub are created through the automated build system, and the system is now processing over 60,000 builds per week – a 300% increase since the beginning of this year. Today, we are releasing a major upgrade to our Autobuild system which features dynamic matching of git branches and tags, a faster and more secure build infrastructure, and user control over build parallelism.

Dynamic Matching

With this new release, the Docker Hub build system can now be configured to dynamically trigger builds as your team creates new git branches and tags. So if for example you create a new git feature branch named “great-new-feature”, the build system will automatically build a docker image tagged as “great-new-feature”. This allows Docker Hub autobuilds to fit seamlessly into your development and deployment workflow. It is configurable to support full dynamic matches, regex filters, or simple static matches.

By default, newly created autobuilds will build match and build all branches of the linked git repository. You can customize it by creating any number of build rules. Each rule can match either a git branch or tag. The rules can be static (e.g. only the git “master” branch) or dynamic. Dynamic rules are created by entering a regular expression, so you can filter branches or tags however you wish. If for example your team uses git tags to mark version numbers, you could crete a rule to build all tags matching /^[0-9.]+$/.


Faster, More Secure Builds

In addition to these dynamic builds, we have also overhauled the backend of our build system so that builds our faster and more secure.

The previous hub builds infrastructure was designed with a global build queue and a fixed fleet of builds machines. Builds have a “rush hour”, typically from 2-5pm US pacific and this frequently caused delays (as long as 2 hours) for a a build to get through the queue.

With our new infrastructure, each user has their own private build queue This way your builds will start quickly and in a predictable manner.

In addition, builds are now more secure. The build system spins up a new build node for each build request. This guarantees that private build artifacts that may be left over from one build cannot be retrieved by a malicious user.


Control over Parallelism

Finally, we are providing control over executing multiple builds in parallel. The Automated Build system will execute as many builds in parallel as you have private repositories. So if for example you are subscribed to our 10 repo “small” plan, you could execute 10 builds simultaneously. Users of our free plan (which includes 1 free private repo), can execute one build at a time.


Using the new Autobuilds

With dynamic build rules, faster builds, and control over build parallelism, now is a great time to try the autobuild system if you haven’t already. If your team already is already using autobuilds, you do not need to take any action to use the new system and you can add dynamic rules to them by visiting the build settings menu on Docker Hub.



 Learn More about Docker

• New to Docker? Try our 10 min online tutorial
• Sign up for a free 30 day trial of Docker
• Share images, automate builds, and more with a free Docker Hub account
• Read the Docker 1.9 Release Notes
• Subscribe to Docker Weekly
• Register for upcoming Docker Online Meetups
• Attend upcoming Docker Meetups
• Register for DockerCon Europe 2015
• Start contributing to Docker


, , ,

Matt Soldo

Upgraded Autobuild System on Docker Hub

3 Responses to “Upgraded Autobuild System on Docker Hub”

  1. Alexey Bobyrev

    That’s cool, but now there is an issue with automatic builds with submodules:

  2. Kevin

    Please please, we need a feature to support security updates properly. git often does not have all the changes. Often our containers are made up of only ‘yum install x’ and when a security update in a repo the containers use happens, the container should be updated as well. Having a set of hooks for kicking off builds when ‘yum upgrade’ or ‘apt-get upgrade’ has updates to do inside the container is a really important, but lacking feature.

  3. DockerCon Europe 2015: Day 2 « David Currie

    […] websphere-liberty image for which I am a maintainer. There was also a demo of the enhancements to auto-builds in Docker Hub and the use of Tutum, Docker’s recent Docker hosting […]


Leave a Reply

Get the Latest Docker News by Email

Docker Weekly is a newsletter with the latest content on Docker and the agenda for the upcoming weeks.