Fairwinds | Blog

How You Can Scale Kubernetes Without Leaving Security and Operational Excellence Behind

Written by Joe Pelletier | Feb 27, 2023 6:10:29 PM

Techstrong Research polled its community in 2022, asking DevOps, cloud-native cybersecurity, and digital transformation readers and viewers about their Kubernetes environments. We recently joined Mike Rothman, Chief Strategy Officer of Techstrong Group and General Manager of Techstrong Research, to talk through the results. One thing is crystal clear: the use of Kubernetes is exploding. Just 25% of respondents don’t have any production applications running in a Kubernetes environment, but even more (31%) have six or more apps running in production. 

While the adoption of Kubernetes is undeniable, company size plays a significant role in the data we’re seeing. Small companies, for example, may not even have six applications total (deployed in Kubernetes or not), while larger organizations may have many more. As K8s adoption increases over the next year, reports like this may become more granular, so we can track adoption by size of company as well as showing more detailed information about how many apps or services are running in production. 

Kubernetes adoption is hard to do right

We’ve seen a steady increase in K8s adoption over the past eight or so years (the initial release was September 9, 2014) and many organizations have moved slowly, starting by testing out Kubernetes and learning how to operationalize it to bring it into their environment. At the same time, other companies have already fully embraced it. And during this time, the community has grown, Kubernetes has grown, and there have been a lot of advancements in open source tooling. All of these things are contributing to the maturity of Kubernetes and the Kube ecosystem, which makes it easier to get more teams and applications in production K8s environments and help development teams ship faster. But moving from two or three applications to two hundred applications migrating to the cloud presents new challenges. 

Operationalizing Kubernetes

While many teams have been able to get a single container to run reliably and consistently in Kube, it becomes a lot more difficult to manage when organizations are trying to move hundreds of apps and services to K8s in the next year. One of the biggest challenges is achieving a baseline of consistency. Policy enforcement has become one way organizations are taking a more prescriptive approach to consistency and best practices, rather than making one-off environments for each team and tuning every setting manually.   

Kubernetes offers a lot of flexibility and many options, and the default settings aren’t necessarily the most secure, reliable, or cost efficient. To accelerate adoption, it helps to remove some of the complexity and automatically enforce best practices, while also making sure that it integrates with your CI/CD to help developers get the information they need where they need it. 

Techstrong Research’s PulseMeter report shows the biggest challenges teams are encountering as they seek to operationalize Kubernetes. Unsurprisingly, the biggest challenge was related to security issues and concerns at 24%. It’s understandable that in this complex environment, dev and ops teams are worried about misconfigurations and security vulnerabilities. 

Troubleshooting can also be a serious challenge in Kubernetes — developers generally aren’t Kubernetes experts, and so they turn to platform engineers to be a Kubernetes help desk. Troubleshooting is typically time-consuming and it causes a lot of interruptions to other critical work, slowing everything down. The other top two challenges are tied: delays in application deployment and addressing cloud costs. This is interesting, because security concerns and troubleshooting both contribute to deployment delays. 

Cloud cost, on the other hand, is something that I think organizations will focus on even more as an issue in the year ahead. There are a few reasons I believe that: 

  1. The economy is rather hard to predict this year, and many organizations are looking for more ways to control costs

  2. Insufficient or nonexistent Kubernetes cost monitoring is causing organizations to overspend, according to the CNCF’s FinOps for Kubernetes report

  3. Rapidly increasing adoption could easily lead to out-of-control cloud spend 

According to the CNCF report, 68% of respondents saw increased Kubernetes costs in 2022. And the survey showed that most could reduce their expenses with a Kubernetes cost-monitoring strategy that was more active and granular. A FinOps approach, which is the practice of identifying unit costs related to cloud spend across silos within an organization, can help organizations understand their spend and implement Kubernetes cost avoidance strategies. 

Automation facilitates operational scale 

Configuration presents a serious challenge in Kubernetes because it’s easy to look at a single YAML file and find configuration issues. When you’re in a complex distributed environment like Kubernetes, which is layered on top of a cloud provider and their configurations, there’s a lot more than just one file to consider. There are plenty of challenges to be mindful of: misconfigurations, known vulnerabilities, out-of-date Docker images, anomalous activities, role-based access controls, regulatory compliance, and reliable apps and services.

Doing all of this manually simply isn’t possible at scale. You need to create a shared framework between development, security, and operations to bring that visibility and drive ownership of the different types of findings. Creating policies for your organization and automating policy enforcement can help you create feedback loops with individual teams, so they can quickly gain visibility into where problems exist, understand how to fix them, and know which issues present the greatest risk. This will help you move faster and deploy more applications, while still maintaining a high level of visibility across the Kubernetes environment.

Respondents to the TechPulse survey were almost equally focused on these five issues, and worried about how much manual effort or time they required:

  • Configuration complexity when deploying software (20%)

  • Tracking down costs and overruns (19%)

  • Checking and remediating vulnerabilities (18%)

  • Staying in compliance (16%)

  • Gathering and reporting compliance evidence (14%) 

Accelerating Kubernetes adoption with security and operational excellence

While there’s still a lot of hype around adoption, Kubernetes is still an emerging technology. Many organizations are increasing adoption and putting more and more workloads into production in the year ahead. As this adoption ramps, organizations serious about implementing Kubernetes and other cloud-native technologies need to investigate and implement cloud native solutions that provide guardrails to help developers deploy apps and services fast without worrying about unintentionally releasing vulnerable code, not meeting regulatory compliance requirements, or increasing cloud spend unnecessarily. 

Watch the DevOps.com webinar: Scaling Kubernetes without Compromising Security and Operational Excellence