Jim Armstrong

The Journey to 150,000 Containers at PayPal

Updated Dec 6, 2017 to clarify that the 50% boost is an enhancement for developers’ build / test workflow and complements their CI environment. PayPal developer use Docker for Mac and Docker for Windows on their local machines as part of this workflow.

PayPal is committed to democratizing financial services and empowering people and businesses to join and thrive in the global economy. Their open digital payments platform gives 218 million active account holders the confidence to connect and transact in new and powerful ways. To achieve this, PayPal has built a global presence that must be highly available to all its users: if PayPal is down, the effects ripple down to many of their small business customers, who rely on PayPal as their sole payment processing solution.

PayPal turned to Docker Enterprise Edition to help them achieve new operational efficiencies, including a 50% increase in the speed of their developers’ build-test-deploy cycles when developing and testing locally. At the same time, they increased application availability through Docker’s dynamic placement capabilities and infrastructure independence; and they improved security by using Docker to automate and granularly control access to resources. On top of the operational benefits, PayPal’s use of Docker empowered developers to innovate and try new tools and frameworks that previously were difficult to introduce due to PayPal’s application and operational complexity.

Meghdoot Bhattacharya, Director of Cloud Engineering at PayPal, shared the journey his team has helped PayPal undertake over the course of the past two years to introduce Docker in to the PayPal environment and grow its usage into the standard platform for application delivery. You can view the full DockerCon EU 2017 session here:

PayPal’s Past, Present, and Future with Docker

While PayPal may not be an “old” company, over the course of their 17+ years they have built a global payment processing platform consisting of hundreds, if not thousands, of individual applications and services. As is usual with this level of rapid development and growth, there are a plethora of different tools and processes for building, testing, deploying and running each of these applications. For both the developer teams and for operations, the lack of standardization created issues. Developers were locked into a limited set of tools and frameworks and it was difficult to add anything new to the mix because of the burden it placed on test and operations teams to figure out how to deploy and run each new stack. For operations, the strain of deploying apps to multiple architectures and multiple clouds, each with different sets of instrumentation and procedures, led to slow turnaround times for deployment and maintenance.

PayPal’s initial use of Docker was simply as a means to create a single, consistent application packaging format that could be used with as many of their existing applications as possible. They focused on the operational side of the house, creating a unified application delivery and operational model, to address their issues around application silos. PayPal undertook this phase prior to the launch of Docker’s official Modernize Traditional Applications program, yet they shared many of the same strategies and benefits: start with existing applications where you already have in-house expertise around the day-to-day operations, and focus on migrating the applications to containers without changing the code. At this stage, PayPal recognized several important benefits, even though the apps themselves remained the same:

  • Decoupled deployment from framework stacks: PayPal now had a platform that could deploy almost any application using the same procedures, regardless of programming languages, libraries, and stacks
  • Modernized OS and kernels: Since Docker packages all the application dependencies into the container, they were able to decouple them from the underlying OS and thus modernize the infrastructure without changing the app. Some applications got 10-20% boosts in efficiency for free, simply by running the containers on top of a modern OS and kernel.

PayPal Docker

As the Docker container format and operations became well understood, PayPal opened the door to developers to learn and use containers. Because Docker provided a standard methodology for shipping and running the applications, developers were free to experiment with new programming methodologies and frameworks, while operations and support teams did not have to worry about those complexities. With developers joining the Docker Supply Chain, PayPal recognized a greater than 50% boost in developer productivity by developing, building, and testing locally with Docker for Desktop. The developers’ local development environment also complements their cloud SDLC environment. With Docker, every developer, operator, tester and automation tool was able to run the same application, with the same configuration. For system administrators, there were additional benefits as well. OS upgrades, which used to take a full year to roll out, were now done in one shot by simply updating the Dockerfile for a container image. As word of these benefits spread within PayPal, other organizations not originally part of the pilot started to adopt Docker for their own use cases and the Docker pilot team at PayPal helped them get going as well.

Results and What’s Next

In just under two years since the inception of phase I, PayPal has moved over 700 apps to Docker, running in 150,000 containers. Along the way, PayPal has seen a 50% increase in developer productivity in building and testing their Dockerized applications. Projects like OS upgrades that used to take a year can now be done across dev, test, and production with a single change. And developers are free to explore emerging technologies for their applications, while operators and administrators enjoy a single set of operating procedures no matter the application.

In their final phase, PayPal has begun to roll out a full Container-as-a-Service (CaaS) platform across all their clouds. In this phase, they are focusing on efficiency gains. So far, the applications moved to the new Docker CaaS platform have reduced resource consumption, increased security and availability, and enabled PayPal to provide a single platform for tooling, monitoring and operations across all environments.

Later stage Docker benefits for PayPal

PayPal’s journey to implement Docker is a great blueprint for other organizations to follow. Focusing on adoption in manageable steps, building a team of in-house experts that can consult and assist other teams, and setting clear goals and measuring results are key to implementing a transformation that can drastically improve operations and development.

Learn more about PayPal by viewing their case study.

To learn more about Docker Enterprise Edition:

, , ,

Jim Armstrong

The Journey to 150,000 Containers at PayPal


Leave a Reply

Get the Latest Docker News by Email

Docker Weekly is a newsletter with the latest content on Docker and the agenda for the upcoming weeks.