In the world of containerized everything, Kubernetes has emerged as a platform to automate the process of deployment, scaling, and management of these applications and services. However, as your Kubernetes environments expand, maintaining control and ensuring security can become increasingly complicated. This is where Kubernetes governance becomes critical, and the Admission Controller can help you meet some of those governance goals.
Admission Controllers function as gatekeepers for your Kubernetes clusters, reviewing requests made to the Kubernetes API server and determining whether to admit those requests based on the policies you've set. By enforcing your policies at the entry point, an Admission Controller can improve your Kubernetes governance in three key areas: security, consistency, and resource management.
Fairwinds Insights can run as an Admission Controller. This means it will reject any Kubernetes resources from entering your cluster if they don't conform to your organization's policies.
Insights runs the following report types in Admission Controller:
Polaris for validating configuration against best practices
Open Policy Agent (OPA) for running custom policies
Pluto to discover resources that are using deprecated APIs
The default installation requires cert-manager v1.0 or greater, so make sure you install that first. If you don't have cert-manager or you want to use your own certificate for the webhook, you can use the caBundle and secretName parameters to pass a CA Bundle and the location of a TLS certificate stored in your cluster (you just need to have both of those ready as well). By default, the Admission Controller listens on port 8443, so you may need to configure a firewall rule or security group to allow the master nodes to send traffic to the worker nodes on that port.
To use the Admission Controller in Insights, you need to first install it on your cluster by following these steps:
Visit your organization's Clusters page
After selecting a cluster, go to the Install Hub page
Hover over the Admission Controller report and click the Quick Add button
Once you’ve added the report, re-install the Insights Agent using the Helm chart in your cluster. By default, the Admission Controller is installed in Passive Mode. That means that the Admission Controller is monitoring all activities, but it will not deny any deployments. When you’re ready, you can disable Passive Mode.
At Fairwinds, we encourage a phased approach to rolling out policy enforcement when using the Insights Admission Controller. While the idea of policy enforcement is appealing, many struggle with how to implement it effectively. Using the stages below, you can gradually increase policy enforcement over time, which will help your platform engineering team balance shipping code quickly while still driving security and reliability improvements.
When you’re just getting started, we recommend that you set up the Admission Controller to warn users when their YAML code or Admission requests contain misconfigurations identified in your Cluster-wide Policies or Scoped Policies. The easiest way to do this is using Passive Mode. You can also install Insights in your Continuous Integration (CI) pipeline to discover misconfigurations earlier in the development and deployment process.By default, the CI integration in Insights is configured NOT to fail CI pipelines. Make sure that your fairwinds-insights.yaml file at the root of your repo has the options.setExitCode set to false. You can gate pull requests with the Insights CI integration if you discover specific issues in a scan.
This stage enables you to enforce your Cluster-wide Policies at time of Admission, notifying you of Scoped Policy violations. Using the same settings in the last stage, you are now going to disable Passive Mode in the Admission Controller, which is done on a per-cluster basis:
Go to the Install Hub and click on the Admission Controller report
Select the Configure tab and toggle the Passive Mode option to disabled
Now test it out! Create a deployment with a high level severity Action Item by allowing privilege escalation. Here’s a sample you can try yourself:
bad-config.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-deployment
namespace: testing
spec:
replicas: 1
selector:
matchLabels:
app: busybox
template:
metadata:
labels:
app: busybox
spec:
containers:
- name: busybox
image: busybox:1.32
securityContext:
allowPrivilegeEscalation: trueNow type:
kubectl create ns testing
kubectl apply -f bad-config.yamlYou’ll see a message that says:
Error from server (Privilege escalation should not be allowed: Failure: true): error when creating "STDIN": admission webhook "insights.fairwinds.com" denied the request: Privilege escalation should not be allowed: Failure: true
Now you’re ready to enforce your Cluster-wide Policies and Scoped policies. You can also enable the Insights CI script to fail pipelines where Cluster-wide Policy issues are present. At this stage, the Admission Controller should be installed and Passive Mode disabled. You also need the Insights CI integration installed on at least one pipeline and you need to make sure the options.setExitCode is set to false.
Disable the 0002-enforce-cluster-wide-policies-only Automation Rule. To do that:
Ensure you have Owner permissions in Insights
Navigate to the Automation page
Click on the 0002-enforce-cluster-wide-policies-only Automation Rule
Toggle off the 'Enable' switch in the upper right corner
Click 'Update Rule' in the lower left
You can also create an Automation Rule that enforces Scoped Policies for specific Namespaces. Check out our docs to walk through an example. You can also configure a workload to bypass the Admission Controller by adding a specific annotation to that workload, and use an Automation Rule to bypass Admission Controller or creating an "allow list" within an Insights Automation Rule that forces the Severity of that Action Item to Low.
Admission controllers can be validating, mutating, or both. Validating admission controllers can’t modify objects related to the requests they admit, while mutating controllers are able to modify related objects. You can use a mutating admission controller to apply best practices automatically to every deployment, which can help your dev teams move faster. The potential drawback to this is that you may experience configuration drift, which means that the infrastructure-as-code in your git repo is not matching the runtime state.
Polaris is an open source policy engine for Kubernetes that includes over 30 built in configuration policies (and the ability to build custom policies). It validates and remediates resource configuration, and can be run as a dashboard, an admission controller, and a command line tool. When run as a mutating webhook or on the command line, it can automatically remediate issues based on policy criteria. Polaris is built into Fairwinds Insights, making it easy for you to continuously scan workloads against your mission critical policies, identify issues, and automate remediation.
Walk through what the Admission Controller is for and how (and why) to use it in this video.
If you’re already using Insights, install the Admission Controller to help you keep your Kubernetes environment functioning in line with your policies. If you’re not an Insights user yet, try the free tier (available for environments up to 20 nodes, two clusters, and one repo). If you have any questions about the Admission Controller or other ways Insights helps you apply standards across all clusters and all IaC repos, please join our Slack Community.
