Whether you are looking to separate security boundaries, isolate the impact of a scary configuration change, scale applications closer to customers, achieve data sovereignty, or going for the multi-cloud dream, you likely have multiple Kubernetes clusters in your life. In this post I will describe some characteristics and tools that increase your success managing multiple clusters.
As Kubernetes clusters multiply, following standards and having repeatable processes becomes doubly important to produce consistent behavior and maintain order.
An example of such a standard is all clusters having the same version of Ingress or DNS controllers installed. An accompanying process documents or codifies multiple perspectives from well-rested engineers about how to safely upgrade those controllers.
Standards and processes will adjust as your infrastructure expands and you support differences between clouds, services that are not available in all regions, or new architectures. It can sometimes feel like the extra burden of maintaining standards and processes gets in the way of progress, but these habits help your infrastructure grow faster and farther, once you reach escape velocity. Part of maintaining standards does include adjusting to change.
Policy helps keep non-standard changes from being made to your Kubernetes clusters and workloads. Enforcement of policy should provide guardrails where necessary, early in development or engineering workflows. For example, if a Kubernetes Deployment should be disallowed due to an insecure configuration, ideally it should fail during Continuous Integration instead of when the application gets deployed to Kubernetes.
More specific policies can be accomplished using Open Policy Agent or similar tools. For example, a labeling standard for Kubernetes resources helps you track how your platform is being consumed, and eases deployment and cleanup of feature branches. Labels can be used to instruct CI/CD about where an app should be deployed, RBACBindings necessary for developer access to development environments, network access, or when an ephemeral development environment should be cleaned up. Labels are only reliable if they exist everywhere. A policy can enforce that particular labels are set to a range of values by writing a more expressive policy in Rego — the language used by Open Policy Agent.
Reactive or short term adjustments can get lost in a sea of clusters. Here are some tools that help spot unintended deviations in your fleet.
To help wrangle multiple dashboards and provide a unified place to manage results from great tools such as these ... there's an app for that. :)
Insights aggregates the results of great open source tools, including the ones described above, into a single view, which allows you to assess and enforce your standards across multiple clusters. Insights uses Polaris and Open Policy Agent to provide central policy management for some or all of your clusters. Whether you would like Slack alerts or ticket creation for issues surfaced by Insights, your engineers can track their fixes without constantly returning to the Insights dashboard.
Fairwinds Insights is available to use for free. You can sign up here.
Managing multiple Kubernetes clusters across teams and clouds inevitably introduces inconsistencies that can cause errors and cost time and productivity. Many of these challenges are addressed by our open source tools, but for a greater scale and improved management, check out Fairwinds Insights.
