On February 17th, the Docker project had its second Docker Governance Advisory Board ( DGAB ) meeting to discuss the operations of the project. As this was our second meeting, it was great to see everyone again.
As a bit of background, the DGAB was established June 2014 with the intent of providing greater openness, while serving as an essential check to ensure that the Docker Project is serving the needs of its users. The goal of this advisory board is “to advise the Docker Project leadership on matters related to supporting the long term governance, structure, and roadmap of the Docker project.” The DGAB consists of representatives of the community, including contributors, corporations and our users.
Docker community’s astounding growth
Our first DGAB meeting was in November, during which we highlighted the phenomenal growth Docker was experiencing. The growth of Docker has been unprecedented for a open source project in its first two years. It’s clear Docker’s community is growing at a mind boggling speed: the number of contributors almost doubled in just six months and right now sits a hundred shy of a thousand (gasp!). The number of open source projects mentioning Docker in their description has tripled in the same time span to over 30K. The contribution process even at this scale appears quite healthy: pull requests stay open for a median of just one day and the merge rate is around 74% which is not too shabby. This means that nearly three fourths of the pull requests sent to the Docker Project make their way into the project.
Interesting project stats
- 900 contributors, 96% 6 month growth
- 6200 pull requests, 98% processed
- 30K+ open source projects (over 200% growth in 6 months)
- 240+ million container pulls
- 69% PR approved from community, 85% from Docker Inc.
- Mean Days a PR stays open: 10.1 for community, 5.2 for Docker Inc.
- Median days a PR stays open: 1
- About equal number of PRs by community vs Docker Inc. employees (3.1k vs 2.8k)
- Overall merge rate is 74%
We started the meeting by discussing the largest update to the project since our last meeting, namely the new project leadership structure. We discussed what it meant to the project and introduced the new Chief Maintainer, Chief Architect and Chief Operator.
Core team focus on extensibility and pluggability
One of the challenges that comes with growth is it is increasingly hard to focus on the right things. The top priority of the Docker Project is to make Docker more modular in design. This has been our users’ top request and will serve to facilitate a better user experience and enable our community to take Docker to new places by providing their own components through standard APIs. Consequently, the primary focus of the core Docker team is redesigning Docker to have a fully pluggable architecture. The team is focusing its effort by starting with the network stack. During the meeting, Solomon Hykes was asked to list the project priorities in order of importance and these areas were mentioned: security, extensibility, networking, signing and distribution, storage.
The idea is to produce a new component, a pluggable network stack for Docker because the maintainers feel the implementation lacks the focus that clear APIs would provide.
Docker seeks to address every area of functionality that may have extension points. Every default functionality should be a removable with a swappable extension from another source. For areas like networking and storage, the goal is to have a extension point for each place where multiple approaches could be taken. For example, on the networking side today, Docker creates a bridge and sets up IP tables in a specific way; however;this is an area where pluggable extensions could really help as there are many different ways to accomplish this.
There were two major areas of work on storage: storage drivers and movable storage were emphasized. This second topic area was related to image storage: how to transfer them, how to hash them, name them and where can they be stored e.g. on an S3 account, etc. The maintainers will also look at read-only volumes. This work will come after security and distribution of images.
The Docker team is aware of the complex problem of distributing logs from containers. Exposing volume control is the ultimate goal for volumes: there should be a powerful way to customise how volumes are implemented with the best storage system and a much better API to deal with directory and hosts. The goal is to cleanup and refactor everything into a nice auditable architecture.
Cascading updates is a critical topic too
A DGAB member brought up a complex issue: Docker has a problem with cascading updates of base packages. There is no easy way to manipulate the layers.
Solomon acknowledges that it’s a big topic. How much should Docker decide about this flow? It’s a matter of deciding what goes into the container image. The maintainers will go through some iterations of possible workflows but to some degree there will always be different ways to do it.
He said that Docker is not at the point where standard tools have emerged. However, the model for containers is not to think in terms of “this is a machine”, but instead “this is a binary”, which illustrates one of the challenges to standardizing too early. Docker is a new solution and people naturally are still trying to define it via terminology and concepts of earlier solutions.
Do you consider a Docker container a machine or a binary? Is it compiled statically? If you see the Docker container as a binary then it’s clear you want to rebuild them all. For example, lots of developers use rpm or deb and those must be rebuilt every time too.
But they are aware that users want squashing of images, garbage collection, rebasing layers, merge layers. It’s a contentious topic already. At some point the Docker team will have to make some calls.
One of the best ways the DGAB can serve the Docker Project is to provide critical and honest feedback as noted above. The DGAB discussed the largest issues facing the Docker Project and what we can do to help them. We also discussed how the DGAB can best serve the needs of the project.
Delicate balance and standards
What happens when people are both competitors and partners at the same time?
In his own words, Nicola Paolucci from Atlassian describes his perspective:
“When the conversation moved to the delicate question if the Docker project would consider and adopt the App Container Specification, I realized the delicate balance needed by an open organization where both enthusiasts and commercial entities collaborate to create a flourishing ecosystem. I loved the friendliness and the pragmatism of all the actors – differences notwithstanding”.
Docker and CoreOS are competing companies but Solomon wished the following: parties can compete outside of the DGAB meetings but inside, we have to work from the position of making the Docker project successful. If we can accomplish this goal, then the organization can stay healthy and productive.” In the spirit of collaboration, Brandon, CTO of CoreOS, expressed that his primary concern is that the next version of the Docker Image specifically address some of the shortcomings that the ACI team has identified. Solomon identified the efforts which have been underway around security, signing, making Docker more modular, and enabling the API’s with batteries included but swappable model and expressed interest in help from the community with seeing those efforts succeed.
The DGAB meeting concluded with Steve Francia, the new Docker Project Chief Operator, leading a discussion around which improvements the DGAB felt needed to be made to ensure that the Docker contributor experience is optimized.
The complete transcription and notes are being compiled by Van from Rackspace who serves as DGAB chairman and provided the agenda for this meeting. We will be pushing those notes as soon as they are ready.
Learn More about Docker
- New to Docker? Try our 10 min online tutorial
- Share images, automate builds, and more with a free Docker Hub account
- Read the Docker Engine 1.5 Release Notes
- Subscribe to Docker Weekly
- Attend upcoming Docker Meetups
- Celebrate the Docker Project’s 2nd Birthday
- Register for DockerCon 2015
- Start contributing to Docker