Policy and governance are terms that often come up in the Kubernetes space, but how do we actually do this? This is where the Open Policy Agent (OPA) comes into the picture. OPA is a tool that allows us to put guardrails in place by defining policy as code using the Rego language.
With the Open Policy Agent a few examples of the types of policies we can create include:
Blocking deployments to particular namespaces
Restricting which container repositories we can pull images from
Setting a minimum number of replicas on deployments to enforce reliability
Block ingresses from using duplicate hostnames or when hostnames that are too long
Enforcing best practices around security, reliability and more
Oftentimes organizations will try to set up a loose policy and governance system by using a template that someone created that follows Kubernetes best practices. They will then share it with the rest of the team, and assuming no one ventures away from that template then everything should work. Unfortunately the reality is things do get deployed that deviate from that template, and if an attacker manages to get into a cluster due to compromised credentials you can be sure they’re not going to honor your template as they deploy additional images or security configurations that allow them access to more information.
OPA, by itself, can’t actually block resources from entering Kubernetes clusters. This is where the Admission Controller fits in. The Admission Controller is the last check in the Kubernetes API to decide whether or not a resource should be allowed into the cluster. OPA works with the admission controller to let it know if the resource that is trying to enter the cluster conforms to the policy that we have defined. If it does, then the resource gets applied to the cluster. If it doesn’t, the resource gets blocked.
notInNamespace[actionItem] {
# List Kubernetes namespaces which are forbidden.
blockedNamespaces := ["default"] namespace := blockedNamespaces[_]
input.kind == "Pod"
input.metadata.namespace == namespace actionItem := {
"title": "Creating resources in this namespace is forbidden",
"description": “This pod is forbidden from the specified namespace”,
"severity": 0.7,
"remediation": "Move this resource to a different namespace",
"category": "Reliability"
}
}For the above OPA policy we’re looking at one rule named notInNamespace. The notInNamespace rule has a number of conditions, and all of those conditions are listed within the curly brackets. In order for a resource to be blocked from entering the cluster, all of those conditions must evaluate to true. Let’s take a look at them:
The first and second conditions, blockedNamespaces := [“default”], and namespace := blockedNamespaces[_], define the list of namespaces that we wish to block. In this case it is simply the default namespace.
The third condition, input.kind == “Pod”, checks to see if the resource that we’re trying to apply to the cluster is a pod or not. If it is, we are going to continue to evaluate the conditions, and if it’s not, then it passes this policy and is free to enter the cluster.
The next condition, input.metadata.namespace == namespace, checks the namespace of the pod against the list of namespaces that are forbidden. If the pod namespace isn’t in the forbidden list, then it passes this policy and can enter the cluster. If it’s not, it continues evaluating the rule.
The last piece of this rule only gets evaluated if all of the previous conditions evaluated to be true. A pod is trying to enter the cluster, and it wants to be deployed to a namespace that we have forbidden. At this point it’s going to create this action item with the appropriate description, title, severity, remediation and category since this rule caught the pod trying to be deployed to a forbidden namespace.
From here, the Admission Controller needs to be configured in a way to know what to block. This is where Fairwinds Insights and OPA work well together. In the Action Item example listed above, we’re using this concept of action items and severity to tell the Admission Controller what to block in our specific implementation of OPA. Anything above 0.7 in severity gets blocked, and anything below that simply creates an action item within the Insights platform.
So what is Insights and how does it connect the Admission Controller and the Open Policy Agent?
Insights is a platform that surfaces information around security, reliability, efficiency, and cost and savings information in Kubernetes. The platform uses a variety of plugins, mostly open source, to gather this information either in clusters or in the CI pipeline. These action items can be automated to deliver information to Slack, create tickets in Jira, or send alerts to PagerDuty. The UI also offers a way to login and gain visibility into the cluster and cost information.
Fairwinds Insights is available to use for free. You can sign up here.
The way the Admission Controller is setup within Insights causes Action Items with a severity of 0.7 or greater (high or critical) to be blocked. In the above rule we created an action item with a severity of 0.7 if the pod was trying to be deployed to a forbidden namespace, so in that situation the pod would be denied entry into the cluster.
