Kubernetes and containerization trends (according to the reports of 2021)
In October’21, Datadog unveiled its updated report — “10 Real-World Container Usage Trends“. The authors examined over 1.5 billion containers run by tens of thousands of Datadog clients. We compared the results with the other relevant studies of the last year to complement Datadog statistics and see whether their results reflect the general market trend for Kubernetes.
Some exciting stats from the Datadog report
- Nowadays, almost 90% of Kubernetes users use managed services from leading cloud providers: Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS). Last year, there were only 71% of such users.
- The average number of Pods per organization has doubled. As of July 2021, each company runs about 1000 Pods. That is twice as much as in 2019. The number of hosts is growing at the same rate. This is a significant indicator of the increased usage of containers for different workloads.
- Pod auto-scaling is growing in popularity. Horizontal Pod Autoscaling (HPA) is used by 40% of organizations, while Vertical Pod Autoscaling (VPA) is much less popular (1%). Datadog attributes this to the fact that managing the number of Pods is more accessible than managing the resources of individual Pods. (Some technical details and our guidance on this topic you can find here.)
- Adoption rates of the OpenShift platform are obviously skyrocketing as the number of platform users has increased by 28% over the past year, and, for the most part, those are big businesses.
- Companies tend to use containerd instead of Docker as their container runtime in K8s. The popularity of Docker has decreased insignificantly (only 6%). There are also data available that show even faster growth in demand for containerd (see below).
The alternative view of the trends
The current state and tendencies of the container market are regularly analyzed by commercial and non-profit organizations such as Cloud Native Computing Foundation (CNCF), Red Hat, VMware, Canonical, and Sysdig. Let’s see how their recent observations correlate with the findings of the Datadog report — as well as some other trends we can easily notice.
The rise of the managed services
The Datadog report says that 90% of Kubernetes users use EKS, GKE, AKS, and other managed services. Such a high percentage is partly due to the specifics of the Datadog respondents: being Datadog customers, they willingly use third-party services (SaaS) to cover various needs. However, the world is a tricky place: for instance, according to CNCF End User Technology Radar, most companies won’t choose Datadog over Open Source solutions such as Prometheus, Grafana, and Elastic for their observability needs.
Still, other researchers confirm that the percentage of managed services among K8s users is pretty high. For example, the Flexera 2021 State of the Cloud Report states the following:
- 51% of respondents use AWS Managed Services (EKS and ECS),
- 48% run Kubernetes themselves,
- 43% use AKS,
- 31% use GKE.
It is worth noting that the majority of Flexera respondents are Enterprises (637 out of 750 respondents).
There are also surveys on the use of Kubernetes in local development, and even there, we can see a noticeable share of managed services.
Managed Kubernetes in local development
According to Cloud Native Survey 2020*, 26% of the CNCF community members (1,324 respondents) use managed Kubernetes in development; this is 3% more than a year before. It is worth noting that: a) 19% of those surveyed do not use Kubernetes at all in local development, and b) the number of on-premises Kubernetes users for local development has also increased to 31% (compared to 23% in 2019).
The Canonical Kubernetes and cloud native operations report 2021 surveyed about 1,200 specialists. 24% of respondents use on-prem Kubernetes installation in local development, while 22% prefer managed Kubernetes. When asked, “Who manages your organization’s cloud infrastructure?” the following responses were given:
- 37.5% — DevOps engineer of the company;
- 20.6% — platform engineer of the company;
- 14.8% — system administrator of the company;
- 11.5% — SRE;
- 7.9% — intern of the company;
- 7.7% — we use managed service.
Here’s how Alexis Richardson, founder and CEO of Weaveworks, comments on that:
“I don’t believe 37.5% of the DevOps folks are all managing their own on-premise infrastructure. I think some of them are using tools like EKS. But maybe they are answering as if they are managing them because they own the relationship with the clouds.”
Pod count growth and the rising container popularity
The “number-of-pods-has-doubled” trend, in the view of Datadog analysts, points to how organizations are investing heavily in Kubernetes — and shifting more of their infrastructure and workloads to containers.
Let’s have a look at the containerization trends identified in other studies:
- The CNCF Annual Survey 2021 shows that more than half of companies (69% in Europe, 55% in North America, and 54% in Asia) use Kubernetes in production. CNCF’s previous (2020) report states the percentage of companies running more than 5,000 containers was 23% in 2020 (vs. 11% in 2016).
- The report The State of Enterprise Open Source 2021 by Red Hat (1,250 respondents) states that just under 50% of respondents worldwide use containers in production. 37% of respondents use containers for development only.
- The Sysdig report, which is based on an analysis of millions of clients’ containers, says that 45% of their customers run 250 or more containers, and 57% of them have at least 50 Pods deployed in one cluster.
- The Portworx 2021 survey (500 respondents) notes that 75% of organizations are running more than 40% of new applications in containers.
- The annual Flexera report highlights that 53% of companies use Docker containers.
- Gartner’s analysts predict that by 2022, more than 75% of global organizations will be running containerized applications in production, up from less than 30% in mid-2020.
OpenShift expansion & growth
Kubernetes grows in popularity, and so do different platforms that make it possible to run Kubernetes on different infrastructures (bare metal, private and public clouds) and easier to operate. They are an excellent alternative to managed services for users seeking to focus on business tasks. Examples of such platforms include OpenShift and VMware Tanzu.
So, Datadog’s stats help confirm the popularity trend of K8s platforms using OpenShift as an example (the most popular K8s platform according to Forrester). But what do other reports have to say about it?
- According to Red Hat’s State of Kubernetes security report (500+ respondents), 37% of responders use OpenShift for hybrid cloud deployments. 32% of respondents use AWS Outposts, and 25% — Microsoft Azure Arc for the same purpose.
- 15% of Sysdig responders seem to be relying on OpenShift for container orchestration. The platform took a big jump from 9% in 2020, and it is noticeably ahead of the other K8s platforms involved in the survey: Docker Compose (4.8%), Swarm (2.5%), and Mesos (1.3%). Sysdig notes that its customers value OpenShift’s ability to run in multiple cloud environments.
- Flexera reports that 20% of respondents use OpenShift (or a bit higher percentage, 23%, for the enterprise segment).
- In the Portworx report, 8% of those surveyed primarily use OpenShift.
Docker gets less popular as the container runtime
Every year, Sysdig conducts a study on the security of containers run by customers (which include companies of all sizes). The latest report says that there has been significant growth for containerd and CRI-O over the past year, from 18% to 33% and from 4% to 18%, respectively. (The chart below from Sysdig, most likely, contains an error since the CRI-O share quadrupled rather than doubled.) At the same time, the Docker share has dropped by 37%. Nevertheless, Docker is still the most popular container runtime:
The trend was spurred by a warning from Kubernetes developers that the dockershim component will be deprecated in v1.24. K8s developers are still concerned about the confusion this change may cause, so they surveyed users to see if they are willing to ditch dockershim.
Container security trends
Red Hat and Sysdig also touched upon the subject of container security in their reports.
We can draw the following conclusions based on Red Hat’s 2021 Kubernetes adoption, security, and market trends report 2021:
- 55% of respondents cite insufficient out-of-the-box security as the main reason preventing the deployment of Kubernetes applications into production.
- Almost all of the respondents experienced at least one security incident in their Kubernetes environments in 2020.
- 74% of organizations are actively embracing DevSecOps methods to secure the app development life cycle.
Sysdig notes an increase in the number of companies (74% of those surveyed) scanning images for vulnerabilities and other potential security risks during the build (that is, before deploying).
At the same time, Sysdig analysis shows that more than half of DevOps teams (58%) run privileged containers, which poses the risk of compromising containers, nodes, and even clusters under certain conditions.
Note that most of the trends mentioned above do not contradict each other. The discrepancy in the stats is due to each researcher covering a narrow market segment. Overall, however, the reports show noticeable growth in the container market in at least three areas:
- application containerization;
- Kubernetes popularity in production;
- managed services (KaaS) and container orchestration platforms (OpenShift, etc.).