It’s wild to think that 2022 is wrapping up and that many of us are prepping for what’s next in 2023. As the new year fast approaches, it’s time to consider what Kubernetes open source tools you need in 2023.
By 2027, Gartner predicts that more than 50% of enterprises will use industry cloud platforms to accelerate their business initiatives. As we head into 2023, we’ll experience some of that acceleration with the adoption of cloud native technologies, containers and Kubernetes. For those teams adopting Kubernetes (or accelerating its usage), there are four open source tools to help you create and manage governance policies that make your clusters more stable, more secure, and more efficient.
Polaris is an open source policy engine for Kubernetes with a straightforward Helm installation, which you can run as a dashboard, as a command line interface (CLI) tool, or as an admission controller. Polaris can show you a score related to the number of passing versus failing checks. You can quickly see errors, things that may be dangerous to your cluster health, warnings, and passing checks. Polaris gives you an overall score and the corresponding grade based on Kubernetes best practices.
You can dive into the different checks. For example, Polaris indicates that you should have a pod disruption budget for your deployment. If you are not sure what that means, you can click on the question mark and go to the documentation page for more information. When you click through, you can see the pod that’s missing the description for the distribution budget.
We built a lot of these checks into Polaris, which are documented under reliability, efficiency, and security. These checks are all Kubernetes best practices that the Fairwinds team has learned, followed, and now suggest to all our customers. You can also add custom checks to Polaris to look for specific things that you want to enforce related to your environment or other organizational needs.
In the config section you can see the list of checks and their severity level.
There are three different levels available:
Ignore: these do not even show up in the dashboard
Warning: show in the dashboard as warnings
Danger: these also show in the dashboard but can also be blocked by our validating admission web hook
Once you have gone through your checks in Polaris and have fixed all the warnings and put in exemptions, you can start blocking things from getting into your cluster that violate some of these rules. You can mark them as “danger” and block them at the admission request level. Check out the full tutorial.
If you want to require that all images come from a specific list of registries, you can write a policy in Polaris to enforce that. Polaris policies are written in JSON Schema. If you go into the Polaris repository, you can see that all the default checks are in the checks folder in YAML files. If you look at the image property of the container, you can indicate that it must match any of these string pattern matches. If any one of these does not match it will pop up as a warning in the Polaris dashboard.
The pod disruption budget for cert manager is a good example. Cert manager is a single pod controller in this cluster. There is no need for it to run more than once because it runs as a reconciliation loop. If it goes down or disappears for a little bit, it comes back and takes care of its business. You can add an exemption for cert manager by going to controller names and set the rule, so it is exempt from the missing pod disruption budget rule. Once you save it, rerun the Helm install that references this values file, and update Polaris in that name space. Then the cert manager will no longer trigger a warning in Polaris.
Polaris sends reports about setting resource requests and limits. Everyone using Kubernetes is aware of the need to follow best practices around setting resource requests and limits for the CPU, memory, and workloads. Doing that can be a challenge, though, because to do it right you need a lot of information, including metrics (collected over a reasonable span of time), traffic information at your high lows and your low lows, and many more data points. That is a lot to do manually, which is why Fairwinds created Goldilocks to do that for you.
The open source tool, Goldilocks, helps users with Kubernetes resource optimization by setting proper CPU/Memory. It requires you to install a metric server and Vertical Pod AutoScaler, which are both standard. (Most people already have them installed in their cluster.) The Vertical Pod AutoScaler has three components to it: the recommender, the updater, and the admission controller. You can install the metric server using their Helm chart for installing the Vertical Pod AutoScaler; we also have a Helm chart for the VPA because there is not an official one. By default, our Helm chart just installs the recommender and the updater.
The recommender gets metrics from the metrics server and then uses those metrics to make recommendations to the VPA objects that are created for the deployments that you have attached to VPA. Goldilocks uses those values to make recommendations or to surface them for you based on the VPA. You can read the full tutorial for rightsizing Kubernetes workloads here.
If you want Goldilocks to create VPA objects for your workloads, you need to label namespaces with a particular label. If you are starting with a pure vanilla installation, there is a little block of text that tells you exactly which command you need to run to manually label your namespace. Once you label a namespace, you need to refresh the dashboard. That is how easy it is to add workloads to Goldilocks. Everything that is now deployed in that demo namespace will be picked up and will have a VPA associated with it.
With the Vertical Pod Autoscaler, if you have the updater installed, the updater can vertically scale your resources on your pods for you based on the information from the recommender. If you have these set to off, it does not automatically scale for you. You can pass another label to your namespace to enable the update mode to auto.
The recommendations that come from the recommender play a part in how we recommend Kubernetes resource limits and requests in Goldilocks. Similar to what you saw in Polaris, Goldilocks senses that there is a namespace. Based on that, it shows you:
The workload
The container
Goldilocks recommendations for your resources and requests
You can see the current setting and what is recommended for guaranteed quality of service and for burstable quality of service. And for each of these prints out some YAML that you can then copy and paste yourself into your Helm chart.
Goldilocks includes a glossary that explains the difference between the guaranteed quality of service, the burstable quality of service, and how Kubernetes treats your workloads when it is under resource pressure. Goldilocks recommendations are starting points for setting resource requests and limits that you can test out and tweak based on how they work in terms of your traffic patterns.
Nova helps you find outdated or deprecated Helm charts. It scans your cluster for updates to Helm charts — and container images if you are not using Helm. It is useful for keeping track of add-ons, cert managers, and external DNS that you want to keep as up to date as possible. There are many security patches and stability patches that you need to keep updated to maintain the security and stability of your cluster. Without a tool like Nova, it can be quite difficult to keep track of this.
Nova is simple to use. By default, Nova will output in JSON. If you pass the format table flag, it will give you a little table that shows you the release name chart, name, namespace, and the Helm version.
If you look at the screenshot above, you can see the information Nova gives you:
Install is the chart version in your cluster
Latest is the latest chart version that Nova knows of
Old is a Boolean that indicates whether your version is old or not
Deprecated is a flag that can be set as deprecated to indicate that the Helm chart should not be used in the future
Nova gets this information about the different versions of charts and whether they are deprecated by pulling artifact hub. One thing to remember is that you can fork charts and sometimes those forked charts wind up in artifact hub. Nova does not have a way to see which upstream chart is the one that your release came from. It is important to keep that in mind. Fairwinds does some matching and scoring to try to mitigate that, but it is possible that Nova is referencing a different fork than the one you are using.
You can read a full Nova tutorial here.
Nova can also find and scan the containers running in your cluster. The scan will show you the current (installed) version, whether it is old, the latest major version, the latest minor version, and the latest patch version. This gives you more information on what to update.
For example, you might not have any concerns about updating your add-ons to the latest patch version or even the latest minor version. Using Nova, you can make linear or granular decisions about how you want to handle patching your add-ons and do those upgrades. Nova also offers more configuration options for you to try out.
The team at Fairwinds wrote Pluto when we ran into challenges with Kubernetes 1.16 upgrades. Pluto helps you find deprecated and removed API versions in your Kubernetes clusters. All the old deployment extensions V1, Beta1, API versions were removed in 1.16, so if you had a lot of old YAML around or you were deploying old Helm charts that had deprecated API versions in them, it was a struggle to update those. We needed a way to tell our customers what they were using that had been deprecated or removed.
The Kubernetes API server automatically translates API versions from one to the next. That makes it easy to upgrade in place, but it breaks your ability to deploy to the cluster after you have done that upgrade. We wanted our customers to understand that instead of simply blocking them from updating their clusters,
Pluto is another CLI tool, so when we run Pluto it will give us a table output, which includes:
A list of objects
What kind they are
The version
The API version
The API version that replaces it
Whether it has been removed
Whether it has been deprecated
Watch a video overview of Pluto (with transcript).
Thanks to a community contributor, we now have the detect API resources command, which looks for an annotation Kubernetes adds to some resources that contains a string copy of the last applied YAML. If you keep CTL applied to your YAMLs, then that annotation gets set. Pluto can detect your deprecated API versions and you can use it in your CI to block builds or fail builds if people are trying to deploy things that are deprecated or removed,
If you are running all four of these Kubernetes open source tools, fixing everything they find, and following the recommendations they provide, you should be in a much better place in your Kubernetes journey. If you don't want to install all these tools, run them yourself, write your own CI/CD code, and deploy them into all your clusters, Fairwinds Insights is our commercial SaaS platform. It combines these four open source tools along with more tools and adds more functionality, reporting action items into a dashboard.
You can use Fairwinds Insights for free, foreverLearn more here.
Insights can help you put Kubernetes security, cost, compliance, and guardrails in place to make sure Kubernetes is working properly for your organization.
Watch the webinar on demand to get a demo of how these four Kubernetes open source tools work.