Cloud native technology is revolutionizing the development and delivery of applications. It’s helping companies modernize and drive their digital transformation. In particular, Kubernetes is a powerful platform for running containerized applications. It has many attributes – auto-scaling, auto-recovery, etc. – which are critical for running applications in the on-demand, 24x7 world in which we live today.
As CTO of Black Duck Software, I saw first-hand the benefits of leveraging cloud native technology, agile development techniques, and DevOps practices, as we launched a new SaaS platform employing containerized micro-services running on Kubernetes. While the benefits were tremendous once we finally had everything deployed in production, there were certainly a number of challenges. It made clear to me the need for tools, processes and an organizational mindset and commitment to be successful.
While DevOps implies a tight coupling between the Development and Operations organizations, the reality is far different. For many organizations, there is a large gap between Dev and Ops due to:
A lack of tools for enforcing standards and best practices across multiple development teams and production environments
Knowledge gaps in the organization
Philosophical differences and competing priorities – Development teams want to iterate and push out new features quickly. Operations teams want to ensure everything is properly tested and keep the production environment as stable as possible.
At Fairwinds, we’re in a unique position to address these challenges. We have a team with extensive Kubernetes and DevOps expertise, and experience gained through managing hundreds of clusters at dozens of companies. Simply put, it’s the marriage of people, process and technology to drive positive outcomes.
The first challenge that most companies have is around security. As Kubernetes and containers represent a new approach to deploying applications, one which is foreign to most operations and infosec teams, the first question often asked is: Will my applications and data be secure with this new way of developing and deploying applications? Many of the traditional security tools and approaches no longer apply. Likewise, developers are being forced to take on some of these new security challenges - a role to which they’re both unaccustomed and reluctant to embrace.
Security considerations around application and platform vulnerabilities, appropriate permissions, ingress/egress controls and cert management make managing security complex (see my blog on the addressing Addressing Kubernetes Security Vulnerabilities with Policy Enforcement). Unfortunately, this list goes on as in today’s environment, security is a never ending task. In the world of DevOps, it has now spawned the term “DevSecOps” because of its importance and the need to implement security across the complete development and operations chain.
Reliability
After an approach and tooling for finding and fixing security issues has been implemented, DevOps teams will move on to other challenges in managing the environment. Keeping applications available 24x7 is very important. Kubernetes has a number of built-in features to ensure the reliability of applications. However, developers need to be aware of these features and implement code to take advantage of them. For example, each micro service or application service running in Kubernetes should have liveness and readiness probes set. This enables Kubernetes to ensure a service is running correctly and automatically kill and restart it if it’s not.
Polaris and Goldilocks that we use daily across the Kubernetes clusters we manage. We set out to use the open source tools as a base to build upon and provide a deeper configuration validation platform that assesses deployments (application containers) and the underlying Kubernetes infrastructure for security, reliability, performance and cost issues and provides real time feedback to both Dev and Ops teams.
Some of the key objectives we had in mind:
Provide developer enablement while ensuring an Ops perspective, i.e., have applications run securely, reliably and cost effectively.
Allow flexibility and enable Ops to move quickly and provide rapid feedback to identify problems early so that developers still have the context – e.g., perform the checks and analysis during pull requests to give developers realtime feedback on potential issues.
Live where the developers and SREs live, i.e., integrate with the existing tools that these teams use for their daily jobs.
Monitor, find, prioritize and remediate issues. Do this in a way that seamlessly ties into the existing software development lifecycle.
Drive a common language and communication process between Dev and Ops to forge a tighter bond and integration.
Enforce best practices, make policy ingrained in the process.
Fairwinds Insights provides a common framework for organizing, prioritizing, communicating, reporting and remediating issues.
We’ve built Fairwinds Insights as an extensible platform that integrates multiple open source tools (OSS by Fairwinds and best-of-breed 3rd-party open source), as well as the ability to integrate other commercial security and auditing tools. The SaaS platform includes built-in checks for security, reliability and cost. The various integrations allow users to surface issues in their cluster in the tools that developers and DevOps engineers use on a daily basis, e.g., Slack, Git, JIRA, etc. Along these lines, Fairwinds Insights was featured as part of Datadog’s marketplace launch in August. Through our Datadog integration, users can surface all of these issues into a dashboard and tie into Datadog’s monitoring and alerting capabilities.
Interested in using Fairwinds Insights? It’s available for free! Learn more here.
I’m even more excited with our latest release of Fairwinds Insights as it starts to fill out our vision of baking the checks and balances into the process and bridging the gap between Dev and Ops in an automated fashion. With our new release we are delivering:
CI/CD integration – Continuous Integration/Continuous Deployment (CI/CD) integration enables Insights to plug into the software development pipeline at a critical junction point. As developers check in new versions of software, CI tools such as CircleCI, Jenkins, etc. kick off automated building and testing of the software. Fairwinds Insights can run validation checks against these new versions and provide immediate feedback to developers
OPA Support – Fairwinds Insights now includes a built in policy engine based on the Open Policy Agent (OPA) open source project. This enables teams to implement policies for security, reliability, performance, cost, etc. and to enforce these practices throughout the SDLC from development to deployment. It provides policy checks, guardrails and enforcement of best practices to ensure consistency across development teams (applications) and operations (infrastructure).
Admission Controller – the Admission Controller provides additional safeguards to prevent problematic containers from being deployed into production clusters. It runs policy checks against the container before the container is deployed.
These new capabilities support “shifting left” in the development process, giving developers earlier visibility around issues so that they can be addressed much earlier, and deliver the premise of DevSecOps where security is tightly integrated throughout the process.
Fairwinds Insights is powerful platform for addressing many of the challenges organizations face in deploying cloud native technology. As the use of Kubernetes continues to increase, it provides a framework in which to enforce standards across the organization where there are multiple development teams and clusters, ensuring that applications are deployed securely, reliably and cost effectively.
I’m very pleased with the tremendous progress our team has made over the past year in executing on this vision and set of goals.