Category: Author Archives: Solomon Hykes

Welcoming the Orchard and Fig team

Today I am extremely proud to announce that the creators of Orchard and Fig – two of the most polished and exciting projects to come out of the Docker ecosystem – are joining the Docker team. Fig is by far the easiest way to orchestrate the deployment of multi-container applications, and has been called “the perfect Docker companion for developers”. As it turns out, these are currently the two most important questions for the tens of thousands of people building applications on the Docker platform:

  1. How to orchestrate Docker containers in a standard way?
  2. How to make Docker awesome for developers?

With Fig, Ben and Aanand got closer to an answer than anybody else in the ecosystem. They have a natural instinct for building awesome developer tools, with just the right blend of simplicity and flexibility. They understand the value of a clean, minimal design, but they know from hard-earned experience that every real project needs its share of duct tape and temporary hacks – and you don’t want to be standing between an engineer and their duct tape. By incorporating that experience upstream, we have an opportunity to deliver an awesome solution to these problems, in a standardized and interoperable way, for every Docker user.

First, in parallel to maintaining Fig, Ben and Aanand will help incorporate into Docker the orchestration interfaces that they wished were available when building their own tools on top of it. “There are a thousand ways to implement orchestration. But those should be a thousand plugins, exposed to the developer through a unified, elegant interface”. We agree, and can’t wait to build this interface together.

Second, they will lead a new Developer Experience group – or DX for short. The goal of DX is to make Docker awesome to use for developers. This means anything from fixing UI details, improving Mac and Windows support, providing more tutorials, integrating with other popular developer tools, or simply using Docker a lot and reporting problems.

As usual, all development on Docker happens in the open, and we’re always looking for volunteer contributors and maintainers! If you want to join the Orchestration or DX groups, come say hi on IRC – #docker-dev / Freenode is where all the design discussions happen.

If you’re an Orchard user, there is a detailed post on what this means for you, and what to do next.

Lastly, since Orchard is proudly based in the UK, we are happy to announce that Docker is opening its first European office in London. If you’ve been considering joining Docker but don’t want to move to California – get in touch! We offer both on-site and remote positions.

Welcome Ben and Aanand – let’s go build it!

Additional Resources

Read more about this news in the press

vb  iw  gigaom  forbes   logo_infoq   zdnet siliconANGLEtwittervg

Docker 0.10: quality and ops tooling

Today we are happy to introduce Docker 0.10. We hope you will like it!

We’d like to thank all the awesome community folks who contributed to this release: Tianon Gravi, Alexander Larsson, Vincent Batts, Dan Walsh, Andy Kipp, Ken Ichikawa, Alexandr Morozov, Kato Kazuyoshi, Timothy Hobbs, Brian Goff, Daniel Norberg, Brandon Philips, Scott Collier, Fabio Falci Rodrigues, Liang-Chi Hsieh, Sridhar Ratnakumar, Johan Euphrosine, Paul Nasrat and all the awesome folks at Docker.

This release is the next step on the road to Docker 1.0. The changelog is particularly large, with a dominant focus on quality and improving ops tooling.

Quality

Firstly, we’ve continued our focus on quality as we near 1.0. This release includes the results of a week-long sprint where we fixed bugs, improved testing and documentation, cleaned up UI glitches, and so on. In that week alone we closed 48 tickets and merged 68 pull requests. Here is a small sample:

  • A new integration test harness will help us limit any regressions on the command line interface.

  • Output issues during ‘docker build’ have been fixed

  • Various performance and stability issues when running thousands of containers on the same machine have been fixed.

  • Symlink handling during ‘docker build’ has been fixed

  • The ‘docker build’ command can use client credentials when pulling private Git repositories

  • Multiple reliability and performance improvements to devicemapper storage

  • Caching issues during ‘docker build’ have been fixed

  • `df`, `mount` and similar tools can now be used inside a container

  • ‘docker build’ can now call commands which require MKNOD capabilities

  • Dozens of minor documentation improvements

  • Better test coverage across the board

  • Shell completion has been fixed

  • tmux and other console tools can now be used inside a container

  • Content detection in ‘docker cp’ has been fixed

  • The lxc execution driver works with lxc 1.0

  • Issues with high-throughput allocation of network ports have been fixed

  • ‘docker push’ now supports pushing a single image to the Docker index instead of all tags

  • The content and volume of error messages has been made more readable

  • Issues with ‘docker run –volumes-from’ have been fixed

  • Apparmor issues on certain versions of Ubuntu have been fixed

  • Race conditions, slow memory leaks and thread leaks have been fixed

  • The output of some commands is sorted to be more predictable

As you can see, some of these issues are individually quite small. But in aggregate, they make a big difference! We plan on continuing to fix issues, large and small, over the next releases. If there’s an issue you would like us to address faster, please bring it to our attention! You can always open an issue or comment on an existing issue to express your interest. And of course you are looking to contribute, we will be happy to point you to issues which need attention, and help you get started. Come say hi on the #docker IRC channel on Freenode!

Ops tooling

With this release we are starting a new phase in our march to 1.0: ops-readiness. To be used in production, it’s not enough for Docker to not crash. It needs to integrate well with the tools sysadmins use today: logging, system initialization, monitoring, remote administration, backups, etc.

Obviously we won’t reach this sysadmin-friendly nirvana in just one release, but with 0.10 we are taking several important steps in that direction:

  • Stop behavior: The default behavior of ‘docker stop’ has been changed to err on the side of “application safety”. Specifically, if an application fails to respond to the SIGTERM signal, docker will return an error instead of force-killing it with SIGKILL. This means ‘docker stop’ can safely be used on the most critical or brittle applications without the risk of data corruption or other side effects. (Note that you should still design your application to be resilient to abrupt termination – Docker cannot prevent power cords from being pulled!)

  • Signal handling: The Docker daemon itself now handles signals in the same way. When receiving SIGTERM, Docker will forward it to all running containers, wait for them to gracefully exit, then exit. If containers fail to exit gracefully, Docker will transparently expose that behavior and wait forever. It is then up to the external tool to choose between 1) waiting further, 2) giving up, or 3) force termination with SIGKILL. Note that because of how SIGKILL works, Docker cannot forward it to the application: instead it detects “orphaned” containers the next time it starts, and sends the SIGKILL now. In short: Docker never, ever sends SIGKILL to a container unless it receives SIGKILL itself.

  • TLS auth: One feature which has been requested many times is the ability to expose the Docker remote API over the network in a secure way. That is now possible with Docker 0.10, with built-in support for TLS/SSL certificate security. You can now use SSL certificates to retrict access to the API to only those hosts or users with the appopriate certificate. This is only the first step in securing the Remote API and we have plans in the future to provide more granular role-based access control as well as other forms of authentication.

  • Systemd slices: Docker now ships with a systemd plugin, which automatically detects when the underlying host has been initialized by systemd. If systemd is detected, Docker will automatically use the low-level systemd APIs to manage control groups, instead of the default behavior of accessing /proc directly. For sysadmins currently using the systemd tools to manage resource allocation, this means that individual Docker containers will show up automatically in those tools.

  • Release hashes: Every release of Docker now includes SHA256 and MD5 hashes of all build artifacts. These will be published on the documentation site and download page, allowing you to verify that your installation has not been tampered with. For example you can verify the SHA256 of the official Linux and Darwin binary builds with the following command:

 

curl https://get.docker.io/builds/Linux/x86_64/docker-0.10.0.sha256
curl https://get.docker.io/builds/Darwin/x86_64/docker-0.10.0.sha256

Finally, with the release of Docker 1.0 in the near future, we would ask that you aggressively test Docker 0.10. Please log tickets and let us know your feedback!

Thank you everyone for your support and help, and happy hacking!

The Docker maintainers

Docker 0.9: introducing execution drivers and libcontainer

Ship it!

Fellow Dockers,

Today we are happy to introduce Docker 0.9. With this release we are continuing our focus on quality over features, shrinking and stabilizing the core, and providing first-class support for all major operating systems.

In addition to dozen of bugfixes, Docker 0.9 includes 2 major improvements: execution drivers and libcontainer.

As usual, for a complete list of improvements, you can check out the Changelog.

 

Execution drivers

First, we are introducing an execution driver API which can be used to customize the execution environment surrounding each container. This allows Docker to take advantage of the numerous isolation tools available, each with their particular tradeoffs and install base: OpenVZ, systemd-nspawn, libvirt-lxc, libvirt-sandbox, qemu/kvm, BSD Jails, Solaris Zones, and even good old chroot. This is in addition to LXC, which will continue to be available as a driver of its own.

There are already several projects underway to develop more drivers. Want to join the fun? Come say hi on #docker-dev on Freenode, and we’ll help you get started.

 

New default driver: libcontainer

docker-execdriver-diagram

Second, we are introducing a new built-in execution driver which is shipping alongside the LXC driver. This driver is based on libcontainer, a pure Go library which we developed to access the kernel’s container APIs directly, without any other dependencies.

Thanks to libcontainer, Docker out of the box can now manipulate namespaces, control groups, capabilities, apparmor profiles, network interfaces and firewalling rules – all in a consistent and predictable way, and without depending on LXC or any other userland package. This drastically reduces the number of moving parts, and insulates Docker from the side-effects introduced across versions and distributions of LXC. In fact, libcontainer delivered such a boost to stability that we decided to make it the default. In other words, as of Docker 0.9, LXC is now optional. To switch back to the LXC driver, simply restart the Docker daemon with docker -d -e lxc. Of course we will continue to support the LXC driver going forward.

 

Using libcontainer for your Go projects

We have developed libcontainer in the hope that other projects will reuse it. If you’re interested in playing with the native container features of Linux – namespaces, cgroups, capabilities etc – then we encourage you to start hacking! To get started go get the Go package and check out the API docs:

 

go get github.com/dotcloud/docker/pkg/libcontainer
godoc github.com/dotcloud/docker/pkg/libcontainer

Objective 1.0

This release is a major step towards a stable, production-ready 1.0 release. We plan on making our next release, 0.10, the first release candidate for 1.0.

 As discussed previously, the goals for Docker 1.0 are:

  • production quality

  • first class support of all major operating systems

  • a shrunken core and a stable plug in architecture

  • well documented

  • able to be commercially supported by Docker and our partners

  • Docker able to offer long term support

We are already hard at work preparing 0.10, with several exciting improvements that we think you will like. If you would like a sneak peek, or if you feel like contributing – come say hi! We are on #docker on Freenode. We welcome enthusiasts of all levels and can help you get started with your first contribution. As always, thanks go out to our community of contributors, now 352 strong!

Thanks and happy hacking!

 

The Docker team

 

 

Docker 0.8: Quality, new builder features, btrfs, OSX support

Fellow Dockers,

Today we are happy to introduce Docker 0.8, with a focus on Quality and 3 notable features: new builder instructions, a new BTRFS storage driver, and official support for Mac OSX. You can see the full Changelog on the repository, and read below for details on each feature.

This release is special in several ways:

First, this is the first Docker release where features take the backseat to quality: dozens and dozens of bugfixes, performance boosts, stability improvements, code cleanups, extra documentation and improved code coverage – that’s the primary feature in Docker 0.8. We still have ways to go, and there are still many open bugs! But we are making progress and will continue to focus on Quality until it becomes a defining characteristic of Docker.

Second, this release marks the beginning of a new release cadence, which we hope you will find simpler and clearer. It’s a very simple cycle:

  • One release per month. Every first week of the month, we release a new version of Docker. For example, in the first week of March we will release 0.9.

  • Master always works. We only merge patches which we consider ready to be released. And we only release what is in master. This makes it very easy to test upcoming features or distribute a “bleeding edge” version of docker. Simply build from master and you will get the current release candidate.

  • Not feature-based. Release dates are not linked to any specific features. If a feature is merged before the release date, it gets released. Otherwise, the next merge window is only a month away.

  • Simple numbering. We follow the Linux convention for numbering versions. The first digit indicates a major change in the project’s lifecycle. For example, 1.0 indicates that the project is considered ready for production use. The second digit indicates regularly scheduled releases. The third digit is reserved for hotfixes, stability backports etc.

Lastly, this release marks the beginning of our support for platforms other than Linux on 64bit x86. With Docker 0.8 we are focusing on Mac support – but expect us to start supporting more and more platforms over the next few releases. As many of you have pointed out, “run anywhere” is only useful if you can actually, you know… run anywhere :)

Thank you to the entire Docker community, and happy hacking!

The Docker team

(more…)

Docker 0.7 runs on all Linux distributions – and 6 other major features

So, Docker 0.7 is finally here! We hope you’ll like it. On top of countless bug fixes and small usability improvements, it introduces 7 major features since 0.6.0:

  • Feature 1: Standard Linux support
  • Feature 2: Storage drivers
  • Feature 3: Offline transfer
  • Feature 4: Links
  • Feature 5: Container naming
  • Feature 6: Advanced port redirects
  • Feature 7: Quality

You can see the full Changelog on the repository, and read below for details on each feature.

(more…)

Announcing Docker 0.6

Events API, build and registry improvements, expert mode, security updates, and more.

Notice:

If you are currently using the Ubuntu PPA to install docker you will need to modify your APT sources in order to upgrade to docker 0.6.0.  Please visit http://docs.docker.io/en/latest/installation/ubuntulinux/  for the new repository information.

Dear Dockers,

Wow, we have quite a release for you today. Since the last release just one month ago, the project has received 378 commits by 40 different contributors! In addition to boatloads of usability improvements and bugfixes, 0.6 introduces a websockets events api, upgrades to the Dockerfile syntax, a major improvement of the registry download and upload protocol, tighter security and access control, an “expert mode” for advanced usage scenarios, and much more. You can see the full changelog here.

We hope you like it.

Solomon & the Docker team

(more…)

dotCloud and Docker join the Linux Foundation

Linux Foundation

We are very excited to announce that dotCloud has joined the Linux Foundation.

The Linux Foundation is a nonprofit consortium dedicated to fostering the growth of Linux and collaborative software development.

Docker obviously depends on a number of technologies developed by the members of the Linux community—including LinuX Containers (LXC), cgroups, and the kernel itself.

Ultimately, our goal is to build the “button” that enables any code to instantly and consistently run on any server anywhere. Achieving that goal is dependent not only on having a thriving open source community for Docker, but also by the continued success and spread of both Linux and collaborative software development in general.

In less than four months since Docker launched, we’ve been thrilled to see Docker integrations into open source projects such as OpenStack, Chef, Puppet, Vagrant, and mcollective, as well as the “dockerization” by our community of hundreds of open source projects, such as Redis, Memcached, PostgreSQL, Ruby, and more.

By joining the foundation, we are excited to support the amazing open source contributions that have come before us, and support those yet to come!

> For more details, you can read the full official announcement on the Linux Foundation website.

Docker 0.4: Remote API, Build, Openstack Integration

Introducing Docker 0.4.0
==================

Dear Dockers,

Today we are happy to introduce a new release of Docker. In addition to numerous stability and usability fixes, this release introduces 2 highly anticipated features: Remote API and Build, as well as a very cool Openstack integration.

Contents
========

* What is Docker?
* 0.4.0 summary
* Remote API
* Build
* Openstack integration
* What’s next?
* Broader kernel support
* Cross architecture support
* Even more integrations
* Plugin API
* Externally mounted volumes
* Better documentation
* Production-ready
* Community news
* Contributor of the month: Backjlack
* New maintainer: Victor Vieux
* How you can help
* Hack day on June 11

(more…)

Docker 0.3 released: public index, data volumes and open-source registry

Hi everyone,

Today we’re excited to announce a new version of Docker. This version brings three highly demanded features: data volumes, a searchable index and the first open-source release of the docker registry!

Let us know what you think! We’re always available on irc.freenode.net #docker. Feel free to open an issue on Docker’s repository.

# CONTENTS

* What is Docker?
* 0.3.0 summary
* Data volumes
* Searchable index
* Open-source registry
* What’s next?
* Remote API
* Runtime API
* Build!
* How you can help
* Hack day on June 11

(more…)