Ying Li

Introducing Docker Secrets Management

Containers are changing how we view apps and infrastructure. Whether the code inside containers is big or small, container architecture introduces a change to how that code behaves with hardware – it fundamentally abstracts it from the infrastructure. Docker believes that there are three key components to container security and together they result in inherently safer apps.

Docker Security

A critical element of building safer apps is having a secure way of communicating with other apps and systems, something that often requires credentials, tokens, passwords and other types of confidential information—usually referred to as application secrets. We are excited to introduce Docker Secrets, a container native solution that strengthens the Trusted Delivery component of container security by integrating secret distribution directly into the container platform.

With containers, applications are now dynamic and portable across multiple environments. This  made existing secrets distribution solutions inadequate because they were largely designed for static environments. Unfortunately, this led to an increase in mismanagement of application secrets, making it common to find insecure, home-grown solutions, such as embedding secrets into version control systems like GitHub, or other equally bad—bolted on point solutions as an afterthought.

Introducing Docker Secrets Management

We fundamentally believe that apps are safer if there is a standardized interface for accessing secrets. Any good solution will also have to follow security best practices, such as encrypting secrets while in transit; encrypting secrets at rest; preventing secrets from unintentionally leaking when consumed by the final application; and strictly adhere to the principle of least-privilege, where an application only has access to the secrets that it needs—no more, no less.

By integrating secrets into Docker orchestration, we are able to deliver a solution for the secrets management problem that follows these exact principles.

The following diagram provides a high-level view of how the Docker swarm mode architecture is applied to securely deliver a new type of object to our containers: a secret object.

Docker Secrets Management

In Docker, a secret is any blob of data, such as a password, SSH private key, TLS Certificate, or any other piece of data that is sensitive in nature. When you add a secret to the swarm (by running docker secret create), Docker sends the secret over to the swarm manager over a mutually authenticated TLS connection, making use of the built-in Certificate Authority that gets automatically created when bootstrapping a new swarm.

 

$ echo "This is a secret" | docker secret create my_secret_data -

 

Once the secret reaches a manager node, it gets saved to the internal Raft store, which uses NACL’s Salsa20Poly1305 with a 256-bit key to ensure no data is ever written to disk unencrypted. Writing to the internal store gives secrets the same high availability guarantees that the the rest of the swarm management data gets.

When a swarm manager starts up, the encrypted Raft logs containing the secrets is decrypted using a data encryption key that is unique per-node. This key, and the node’s TLS credentials used to communicate with the rest of the cluster, can be encrypted with a cluster-wide key encryption key, called the unlock key, which is also propagated using Raft and will be required on manager start.

When you grant a newly-created or running service access to a secret, one of the manager nodes (only managers have access to all the stored secrets stored) will send it over the already established TLS connection exclusively to the nodes that will be running that specific service. This means that nodes cannot request the secrets themselves, and will only gain access to the secrets when provided to them by a manager – strictly for the services that require them.

 

$ docker service  create --name="redis" --secret="my_secret_data" redis:alpine

 

The  unencrypted secret is mounted into the container in an in-memory filesystem at /run/secrets/<secret_name>.

$ docker exec $(docker ps --filter name=redis -q) ls -l /run/secrets
total 4
-r--r--r--    1 root     root            17 Dec 13 22:48 my_secret_data

 

If a service gets deleted, or rescheduled somewhere else, the manager will immediately notify all the nodes that no longer require access to that secret to erase it from memory, and the node will no longer have any access to that application secret.

$ docker service update --secret-rm="my_secret_data" redis

$ docker exec -it $(docker ps --filter name=redis -q) cat /run/secrets/my_secret_data

cat: can't open '/run/secrets/my_secret_data': No such file or directory

 

Check out the Docker secrets docs for more information and examples on how to create and manage your secrets. And a special shout out to Laurens Van Houtven (https://www.lvh.io/) in collaboration with the Docker security and core engineering team to help make this feature a reality.

Docker Security

Safer Apps with Docker

Docker secrets is designed to be easily usable by developers and IT ops teams to build and run safer apps. Docker secrets is a container first architecture designed to keep secrets safe and used only when needed by the exact container that needs that secret to operate. From defining apps and secrets with Docker Compose through an IT admin deploying that Compose file directly in Docker Datacenter, the services, secrets, networks and volumes will travel securely, safely with the application.

Resources to learn more:

, , ,

Ying Li

Introducing Docker Secrets Management


13 Responses to “Introducing Docker Secrets Management”

  1. jens

    broken link for https://lvh.io

    Reply
  2. Kai Poitschke

    Great feature, but why is it only available in swarm mode?

    Reply
    • Ying

      Ying

      We wanted to have a secure place to store the encrypted secrets at rest, which was the raft store in swarm. We would like to eventually provide parity for `docker run`, but we are not there yet.

      Reply
      • Dario Nascimento

        Any document or draft about how you plan to encrypt data at rest?

        Reply
  3. Swapnil Kulkarni

    Thank you very much for the introductory article. The steps are mentioned to view the contents of secrets in container will not work when the redis container is created on a worker node.

    Reply
    • Ying

      Ying

      Good point thank you – these instructions assume a single-node swarm. We should probably mention that.

      Reply
  4. Leslie-Alexandre DENIS

    If I understand it correctly, Swarm is a requirement for the Docker secret management ?

    Isn't that possible to deploy, like an etcd, the distributed store to a standalone instance and tell the Docker engine to access it ?

    My 2 cents.

    Reply
    • Ying

      Ying

      Hi there! Yes swarm mode is a requirement. We wanted to use the swarm raft store, because swarm by default communicates with all nodes using mutual TLS without additional setup, and our swarm logs in 1.13 is also encrypted at rest.

      In the future, we would love to be able to to provide other secrets backends, such as Vault (which encrypts secrets at rest, as opposed to etcd), but there is still a lot of work to get to that point and other features we would like to provide first.

      Reply
  5. Henrik

    First of all, great addition to docker, but it has a few downsides (especially since you are not able to update content of a secret):

    This is just a simple example with a password, but you could easily imagine use cases with SSL certificates, SERVICE configuration etc. where this will become an issue.

    echo "p45sw0Rd" | docker secret create my_secret_pass –
    docker service create –secret my_secret_pass IMAGE

    Great, everything is fine, but some time later on when I want (or need) to update the password I will have to remove the secret (since it cant be updated):

    docker service update –secret-rm my_secret_pass SERVICE

    (This will restart the service, and might possibly fail due to the missing secret)

    Now I will have to delete the secret (since a CONTAINER might depend on the name)

    docker secret rm my_secret_pass

    Now recreate the secret with the new content:

    echo "n3wP45sw0Rd" | docker secret create my_secret_pass –

    And now I will have to add the secret to my SERVICE

    docker service update –secret-add my_secret_pass SERVICE

    (which once again will restart the SERVICE)

    Therefore, PLEASE add an ability to update the content of a secret.

    Reply
    • Ying

      Ying

      https://docs.docker.com/engine/swarm/secrets/ (the final example using mysql and wordpress) demonstrates updating the secret. It does require a restart of the service, because secrets are immutable, but it should only require one.

      When you want to update the secret, you can create a new secret with a different name, and update the service to require the new secret but present it as the same file name.

      Reply
  6. Jon Harrison

    Are Docker Swarm secrets supported with Windows Containers?

    Reply
  7. Sudha

    Thanks for this article. In the picture that you have added to explain the architecture, can you mention, what key is denoted in RED color, and what key is denoted in BLUE color. Are you referring to data encryption key that is unique per-node (OR) the cluster-wide key encryption key, called the unlock key, in this picture. Also why are the users denoted next to WebUI, are in RED and BLUE color, can you explain that part as well.

    Reply

Leave a Reply to Ying

Click here to cancel reply.

Get the Latest Docker News by Email

Docker Weekly is a newsletter with the latest content on Docker and the agenda for the upcoming weeks.