When selecting a managed Kubernetes service like AKS, EKS or GKE, it’s good to know the strengths and weaknesses of each. All of these services have solved how to easily deploy Kubernetes, but are different for how you configure clusters correctly or effectively leverage the service. By understanding these strengths and weaknesses, you can select a service right for your application and also save time having to learn multiple services and each of its specific nuances.
As a disclaimer, if I’m asked what to use, I’d counter by asking what your application is doing and what services it depends on. Kubernetes isn’t the only thing your application needs so you’ll need to factor in requirements - security, additional cloud services, and node customization for example. Without digging into that for this blog, here is a high level summary. If you do want to discuss your specific application, feel free to get in touch.
TL;DR: AKS is ideal if you're a Windows or .NET shop or already using Azure; EKS is good if you require a high level of customization; GKE is easy to get started and use.
Strengths for all managed Kubernetes
- Selecting AKS, EKS or GKE allows you to not worry about maintaining Kubernetes state or its API. Each managed service abstracts the control plane from view. Operators only need to worry about selecting the right server types for running their workloads.
- Each managed Kubernetes service fully supports its underlying cloud provider. This empowers Kubernetes to create cloud resources on your behalf - such as load balancers or persistent storage. While you can run Kubernetes from scratch, it can be a challenge to run on Azure/AWS/GCP while seamlessly interacting with cloud services.
- All the providers support private nodes and private APIs. From a Kubernetes security perspective the offerings are pretty similar. However, there are times when a container workload will need permission to interact with the cloud provider API. Each provider has its own service account and IAM hierarchy. Whatever provider selected, you’ll need to get up to speed with its security terminology and best practices.
Weaknesses for all Kubernetes services
- All providers are dependent on upstream Kubernetes development. Kubernetes is being developed faster and faster. Each service provider must keep up with the pace of development, especially as Kubernetes is only patching the latest three cycles. If you are still on version 1.15, don’t expect any patch support.
- With each service you are unable to configure the control plane. This limits an operator's ability to turn on/off Kubernetes API features. For example, if there were an alpha feature or configuration flag your version of Kubernetes supports - it cannot be enabled on a managed service provider.
- Autoscaling servers into and out of the cluster can be challenging on all providers. Each service leverages a tool called the Cluster Autoscaler. While the cluster autoscaler does a great job, it does have limits on what it can do. It is important to know these limitations and be able to account for those on your selected cloud provider. For example, the cluster autoscaler isn’t zone aware. This can lead to node imbalance if a node pool spans multiple availability zones. Because of this limitation, you may want to run a single node pool or autoscaling group per availability zone.
- Finally, while all three providers have been certified as CNCF Kubernetes conformant, none are running directly from source - they are pre-packaged and introduce additional features. This isn’t really a weakness, but you may run into some inconsistencies when comparing a managed service with the Kubernetes code base.
Strengths and Weaknesses of AKS, EKS, GKE
In alphabetical order, here is a quick summary of AKS, EKS and GKE. It is not an exhaustive list and could be debated as some strengths or weaknesses you might view as a strength for your individual application.
AKS Strengths
- If you are a .NET or Microsoft shop, AKS has first class support for Windows.
- Configuring the virtual network and subnet is simple.
- Robust command line support using the az cli.
- Integrated logging and monitoring solution.
- Azure Active Directory integration for cluster authentication.
AKS Weaknesses
- AKS is relatively new compared to GKE and EKS. As a result many features are still in alpha or beta.
- At Fairwinds, we are proponents of infrastructure as code, making use of Terraform a lot. The Azure Terraform provider doesn’t fully support all of the Azure APIs, so there are some gotchas. The az command line tool can be used to supplement Terraform.
- You are limited on your selection of underlying operating systems you can run. The only choices are Linux (Ubuntu) and Windows Server. The virtual machines do not support customization directly and there is no ability to provide a cloud init or user data script.
- You have to run a default node pool when deploying a cluster, it always has to be there and you can’t change server types once deployed.
- Support for multiple node pools is a preview feature.
- Node updates are not automatic, compared to GKE auto-updates.
- Nodes do not automatically recover from kubelet failures, compared to GKE auto-recovery.
EKS Strengths
- AWS is very mature and tools like Terraform integrate nicely. Amazon has published and maintains a very robust set of EKS Terraform modules with many features. If you only need to interact with EKS, there is an official command line tool, eksctl.
- EKS nodes are more customizable. You can use your own machine images (AMIs). This allows you to customize your operating system and configure servers for your exact needs. You still have the option for using a pre-set AMI, but sometimes these AMIs will not be viable due to security compliance regulations.
- AWS is the most widely used and offers many additional cloud services. Most Kubernetes tooling (DNS management, certificate management, etc.) fully support AWS integration. Kubernetes on AWS is widely documented and offers a large community of users.
EKS Weakness
- While it is simple to create an EKS cluster, adding and customizing node pools can be complex.
- Node updates are not automatic, compared to GKE auto-updates.
- Nodes do not automatically recover from kubelet failures, compared to GKE auto-recovery.
- Pod density and CNI limitations based on instance type and subnet sizes.
GKE Strengths
- GKE makes it really easy to deploy a Kubernetes cluster. The command line tool, and web console are both very friendly.
- Updating the cluster version and node pools is a simple one click process. The version can also be set to automatically update when possible.
- Node pools can be configured to self heal, preventing manual intervention when the underlying kubelet has issues.
- GKE cherry picks bug and CVE fixes into the version of Kubernetes they ship. The downside to this is if a user is running the GKE version of 1.15.9 and they wanted to look at the Kubernetes code, they couldn’t be completely sure that code wasn’t altered.
GKE Weaknesses
- You cannot customize your server configuration. You have to use one of the two server types they offer: Container OS or Ubuntu. You don’t get to pick the versions or kernel versions. If you want a deeper level of customization on your underlying hardware, you cannot do that with GKE.
- GKE runs managed cluster-addons (kube-dns, ip-masq-agent, etc…) that are not overly configurable to the end user. They cannot be modified to use node selectors or tolerations. Any changes made to these addons are reverted.
- While not a huge weakness, GKE has the concept of Zonal vs Regional clusters. By default, a cluster's control plane (primary) and nodes all run in a single compute zone that you specify when you create the cluster. This cannot be changed after the cluster is created. If you need a production ready GKE cluster, be sure to create a Regional cluster.
Fairwinds + EKS, GKE or AKS
Leveraging a managed Kubernetes service is not the same as using Kubernetes effectively. Fairwinds helps companies using AKS, EKS or GKE configure clusters correctly and effectively leverage the service.
Get Kubernetes security, cost allocation and avoidance, compliance and guardrails in one platform for free with Fairwinds Insights.
Learn more about what Fairwinds adds to AKS, EKS or GKE.
