Google’s managed Kubernetes service has always been one of my favorite ways to run a Kubernetes cluster. Google Kubernetes Engine (GKE) provides some incredible functionality that makes Kubernetes even more enjoyable to work with. With that said, one of the best features of GKE is commonly misunderstood, the relationship between Google Cloud IAM and Kubernetes RBAC. When configured correctly, they can provide a simple and powerful approach to Kubernetes authorization.
With most services in Google Cloud, configuring IAM roles is all that’s needed to set up authorization. Although that’s also an option with GKE, you’ll likely need to add some RBAC configuration as part of an effective Kubernetes authorization strategy.
Essentially, all IAM configuration applies to your entire Google Cloud Project, and all clusters within that project. Kubernetes RBAC configuration applies to each cluster individually, and enables fine grained authorization at a namespace level. With GKE, these approaches to authorization work in parallel, with a user’s capabilities effectively representing a union of IAM and RBAC roles assigned to them.
Both Google Cloud IAM and Kubernetes RBAC provide default roles that act as a great starting point to any authorization strategy.
Based on the above graphic, you might think that the Google Cloud IAM roles on the left match up with the corresponding Kubernetes RBAC roles on the right. Although that’s mostly true, there is a significant variation when it comes to the cluster admin roles.
In Kubernetes RBAC, the cluster-admin role grants full administrative access to create, edit, or delete just about any resource in the cluster. With Google Cloud IAM, the Kubernetes Engine Cluster Admin role actually only grants the ability to change the cluster infrastructure, but not actually access any of the resources within a cluster. Of course both of these role names make sense in isolation, but given the overlap between the effects of IAM and RBAC, this naming can get confusing. Here’s an approximation of how the IAM roles actually relate to RBAC roles:
These IAM roles don’t match the RBAC roles perfectly, but the capabilities in the roles these arrows connect generally line up pretty closely. It’s also worth noting that the project wide Owner, Editor, and Viewer IAM roles include the GKE Admin, Developer, and Viewer capabilities respectively (with a few interesting exceptions).
With the Principle of Least Privilege as our guiding light, we can use a straightforward approach with both IAM and RBAC to achieve well rounded authorization. Since IAM roles grant access across all clusters in a GCP project, they should be used sparingly. As an example, granting the “Kubernetes Engine Developer” role will grant edit access to all namespaces, including kube-system allowing core system components to be modified by anyone with this IAM role. Of course we still need to grant some form of basic access with IAM to enable a user to even get credentials to access a cluster. On GKE, RBAC is powerless without at least some IAM to grant that initial cluster access.
With this in mind, we generally recommend using the “Kubernetes Engine Viewer” IAM role to grant read only access. That access can can then be extended with granular RBAC configuration that grants write and/or admin access to specific namespaces. With GKE, IAM is the foundation for authorization and RBAC is the structure built on top of that.
In practice, that means granting edit or admin roles within your cluster to specific users at a namespace level to build on the read only access you’ve granted them with IAM. A simple role binding to grant Jane edit access within the web namespace would look something like this:
kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: jane-web namespace: web subjects: - kind: User name: jane@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: edit apiGroup: rbac.authorization.k8s.io
Defining RBAC configuration like this works relatively well at a small scale, but can quickly become difficult to manage across a variety of namespaces or users. It can also be difficult to get a full picture of authorization in your cluster when half of the configuration is in Google Cloud IAM while the other half lives in Kubernetes RBAC. After running into these problems consistently, we created some open source tools to help.
At Fairwinds, we manage clusters for a wide variety of organizations which means we’ve seen firsthand just how complex RBAC configuration can become at scale. To make matters worse, it can be quite difficult to automate changes to RBAC configuration in any meaningful way. Like many organizations, we try to use CI/CD and infrastructure as code wherever possible, that’s been really difficult with RBAC. Role bindings can’t be changed and instead need to be deleted and recreated when assigning a user a different role. Additionally, it becomes quite complex in any CI/CD workflow to notice the absence of a resource and make changes based on that. With RBAC, that’s a very important detail. Removing a role binding is required to revoke access, and that can get difficult to keep track of.
We built rbac-manager with all of this in mind. Built as a custom operator, rbac-manager can easily be deployed to your cluster and dramatically simplify your RBAC workflows. With rbac-manager’s custom resources (RBAC Definitions), we often see the amount of yaml cut to 1/3rd of the original configuration. Importantly, changing and revoking roles with rbac-manager all just works as you’d expect it to. If a role bound to a user is changed in a RBAC Definition, rbac-manager takes care of deleting and recreating the associated role binding with the new role. Additionally, if a role bound to a user is removed from a RBAC Definition, the associated role binding is automatically removed.
All of these improvements are relatively small, but when combined into a single tool, they can make RBAC much simpler to work with.
With the unique combination of RBAC and IAM that GKE offers, it can get difficult to get a full picture of who has access to a cluster. We created rbac-lookup to help with that. With a simple command like rbac-lookup rob, it will output all users, service accounts, or groups matching that name and show the RBAC roles they have been given in the cluster.
In our most recent release, we’ve introduced GKE IAM integration, that will also include the container IAM roles associated with these users in the output. So if you run rbac-lookup rob --gke it will include all the relevant RBAC roles, along with all the relevant IAM roles associated with GKE access for the matching users.
At Fairwinds, we believe that it should be easy to configure and understand authorization on your Kubernetes clusters. We hope that both rbac-managerand rbac-lookup help make RBAC more approachable for everyone.
