Kubernetes is increasingly the go-to container orchestration platform for businesses seeking to manage their containerized applications efficiently. One of the critical aspects of building a Kubernetes platform is deploying applications using Continuous Integration and Continuous Deployment (CI/CD) pipelines. When you think about platforms — internal developer platforms (IDPs) for Kubernetes — you’re talking about something that abstracts away infrastructure decisions, enables self-service environment builds, integrates with continuous CI/CD processes, and assigns and manages role-based access control. Essentially, the IDP is a self-service layer for developers, but a good one provides the guardrails and security features you need to make sure nothing breaks in the apps and services you’ve deployed.
A platform in the Kubernetes world includes four main components. You’ll need some add-ons — the default tools that your platform engineers or DevOps folks want installed in your clusters. This includes things such as how you want to handle DNS, how you want to handle certificates, and how you want to handle exposing your workloads to external clients or sources. Another component of a Kubernetes platform is governance, which is how you secure your platform, secure your environment, and try to prevent security events from occurring. The last component is feedback — identifying issues and providing suggested remediation to developers during code review.
That governance component could include how you enforce policies or what you want enforced, such as setting resource requests and limits on workloads that get submitted to your clusters. It can also include Linux capabilities that you want to either enforce or prevent on workloads, use of labels, creating name spaces that people can deploy to. A lot of that is focused on the platform from the DevOps perspective, but what does it look like to deploy into an environment that has these things in place in the IDP?
CI/CD enables development teams to automate the process of building, testing, and deploying applications to multiple different environments. The automation ensures that the code is always in a deployable state, which can help reduce the risk of deploying unstable builds. It can also accelerate the development process and minimize the chances of human error.
Kubernetes and CI/CD go together because both focus on automating application management processes. Kubernetes enables the deployment, scaling, and management of containerized applications, while CI/CD pipelines automate the process of integrating new code changes and deploying them to production environments.
As you build your platform, you know some of your developers will know more about Kubernetes than others. So, you want to make it easy for all of them to deploy an application, but not give them too much access. You’ll need to make sure that your developers don’t have more access than they need, so you’ll need some role-based access controls (RBAC).
RBAC Manager is a Fairwinds open source tool that makes building RBAC bindings easier. You can associate an RBAC binding to a specific group and ensure that the group can only view some things in the cluster. RBAC Manager can also do dynamic role bindings, which means that administrators can create name spaces that get RBAC bindings associated with them automatically as soon as they are created and labeled. That rule applies to any new namespaces you create as well, even if you label a namespace with a label that matches an admin label — which means there is no available privilege escalation. When giving developers access, you need to give them the right amount of access so that they can do what they need to do, but not allow them to do anything or everything because that introduces unnecessary security risk.
Kubernetes gives you the base layer API, but you also need to do things on top of it, such as:
Get traffic into the cluster
Expose apps to the world
Add certificates
Configure DNS
Improve autoscaling
Provision new nodes as you add workloads into this cluster
That’s why you need to install add-ons into the cluster — they provide a lot of things out of the box for your developers as part of your platform. There are a lot of open source tools available that you can use to build the platform to manage all of these needs. You can use Argo CD to manage your add-ons as well.
Argo CD, an open source GitOps tool, monitors your clusters and declarative infrastructure (stored in Git), resolving the difference and essentially automating application deployment. Argo CD allows you to set it so that anyone can view the infrastructure but can’t delete or modify it. This is helpful, particularly for debugging issues. If you’re using an ingress nginx object and can't tell whether a request is making it through the ingress controller to your pod, you can make sure developers can view the logs for the ingress controller to see what’s happening. It can also allow developers to look at the live manifest and see what’s going on in the cluster while obfuscating the secrets, so you can maintain security while empowering developers to troubleshoot issues.
This approach allows you to empower and enable your developers rather than restricting them. As the platform team, you can set up your GitHub repository and the credentials that allow Argo CD to talk to the repositories that you need, making it easy to provide access to your code repos automatically and deploy new applications. You can see where the infrastructure repo is added in the Argo CD configuration. Argo does this via secrets that have specific annotations on them using specific labels. And you can prepopulate the credentials and split them between two different objects. You can also create a repo template in Argo CD that's been deployed with the credentials. It’s a good idea to configure that ahead of time so you don't have developers trying to create access tokens that have more access than needed.
You can set it up as a specific type of object in Argo CD using the user interface, but you can also set it up so your developers can create applications using YAML. You can even let teams build their own app of apps for Argo CD, which enables a self-service model. It allows your developers to deploy the way they want and maintain visibility into what’s happening, so they can troubleshoot more effectively.
Using role-based access controls and continuous delivery, you’ve made it so your platform enables developers to create anything they choose in their name space without allowing them to delete things outside their namespace or view secrets. However, there are still a lot of variables to deploying Kubernetes. By default, Kubernetes allows a lot of capabilities that aren’t necessarily secure, cost efficient, or reliable. Policies can help you ensure that your platform enables developers to deploy apps and services that align with the best practices you have defined for your organization.
In your platform, you can put policies in place that check whether the memory limit is too high or the CPU limit is too high and automatically block that early in the development process. It’s helpful to provide guidelines and templates up front that outline how to set up limits and permissions.
A critical part of your platform is providing feedback to your developers. You need to make sure that you’re using the policies and guardrails to prevent issues in deployment, but also showing devs how to remediate problems when they pop up. It’s important to have a flexible platform that makes it easy for devs to get work done, uses the tools they are already familiar with, and still lets them do the complex things they want to do. The ideal platform makes it easy to deploy and adjust so that it runs as smoothly as possible but doesn’t require devs to become Kubernetes experts to deploy on it.
Fairwinds Insights helps your developers by putting the information needed into a dashboard, so it’s easy to see cost and efficiency issues and how to improve them, or view security issues, so your developers can quickly decide what to remediate first. If you’re not currently using Insights, there are a few options that get you started quickly: a free tier up to 20 nodes, two clusters, and one repo, a team level for 20+ nodes, unlimited clusters and repos, and an enterprise level for teams managing multiple clusters and critical applications.
CI/CD is a critical aspect of building a successful Kubernetes platform, as it automates the process of integrating, testing, and deploying applications, ensuring a reliable and efficient development process. By following best practices and using the right tools, you can build a robust and scalable Kubernetes platform that enables your development team to deliver cost-efficient, secure, and reliable applications quickly and consistently.