When building an application, often the most important technical decisions come down to what to build yourself, and where to rely on third parties. Thanks to a vibrant open source ecosystem, modern teams can focus on delivering features that delight users and differentiate their product from the competition, relying on third parties for the boring, boilerplate parts of their application.
But once you’ve decided to adopt third-party software for some mission critical part of your app - maybe the database, authentication mechanism, security auditing, or compute infrastructure - you still have a major decision to make:
Do you want to adopt something free, and rely on your wits and a community of volunteers to support you? Or are you willing to pay a partner to help ensure your success?
You may be tempted to go with “free”, and in some cases it does make sense, especially for smaller, well-constrained problems. But in many cases, “free” comes with a big asterisk - you’ll tend to pay for free software with something much more dear than dollars: your time. And you’ll lose a lot of the safety and security that a commercial partner can provide.
I've been building commercial software for over a decade, at everything from a FAANG enterprise to a two-person startup. I've noticed that early architectural decisions - often made hastily in order to ship an MVP - often have huge ramifications down the line. My bias has typically been towards free software with a quick time-to-value, and in certain cases that bias has led to real pain for me, my team, and my company's bottom line.
Let’s dig into some of the hidden costs of free software.
Solving a complex problem with open source can be difficult, tedious, and error-prone. There will be difficult decisions to make: you’ll need to decide which packages to use, how to integrate them into your stack, and how to best configure them for your needs.
You’ll need to start by developing a deep understanding of the software, but just copy/pasting the quickstart instructions is asking for problems down the line (see Cost #3). If the software is complex enough, you’ll likely want to designate one or more team members to become your resident experts.
You’ll also need to figure out how to train the larger organization on the software. This can be tricky, especially if the software affects many different personas (e.g. SREs, developers, and managers). New hires may not be familiar with the software or your particular instantiation of it, so you’ll need a process for onboarding them in the future.
You may also need to provision some additional infrastructure to run the software. A compute platform like Kubernetes, when combined with a package manager, can greatly simplify this process; but regardless, be prepared to introduce a little more complexity in your environment.
Finally, you’ll likely need to spend time tuning your setup to suit your organization’s environment and needs. It can be hard to tell when you’re “done” here - just because the software is working doesn’t mean it’s stable or secure.
With a commercial partner, a lot of the setup costs go away. Your partner will have a whole team of experts who can advise you on the best setup for your environment. They’ll have a product team focused on creating a fast “time to value,” and will provide a mixture of software and services to get you up and running quickly. And if it’s SaaS, you won’t have to worry about the infrastructure complexity at all.
Once you’ve deployed a new piece of software, the maintenance costs are difficult to estimate or predict, but they tend to fall into two buckets: upkeep and expansion.
On the upkeep side, at minimum, you’ll need to ensure that updates are being regularly downloaded and applied. This is not just to get the latest features - it also ensures that bugs and security holes are getting fixed soon after being discovered.
You’ll also need to be prepared for spontaneous issues that arise as you use the software. Your environment is constantly evolving, and engineers may begin to use the software in new and unexpected ways. If the software starts to slow down, crash, or hog compute resources, you’ll need to shift priorities and dedicate some immediate engineering time to deal with the problem.
But the bulk of maintenance costs will come from expansion. Typically off-the-shelf open source works well in solving small, simple problems in single environments. But for bigger problems that span multiple applications, clusters, or cloud environments, off-the-shelf simply won’t work.
In those cases, you’ll need to put a lot of effort into scaling and maintaining an open source solution - as your team grows and your application evolves, so will your needs. You may need to integrate with other software, or enable additional security features (e.g. to meet a compliance standard like SOC2 or HIPAA). And if there’s a new feature your team or customers have been asking for, you’ll need to find the time to build it; if the software affects a large number of different personas or stakeholders, this can become a massive burden.
With a commercial partner, 90% of the maintenance costs disappear. Commercial partners are particularly incentivized to help large customers with complex environments, ensuring the software is highly scalable, both in terms of infrastructure complexity and organization size.
With a SaaS platform, updates will be applied automatically; but even in self-hosted cases, your partner will provide you with a clear upgrade path, and will let you know which upgrades are critical and which can be put off. And if any spontaneous issue arises, a whole team of people will spring into action to get it fixed as quickly as possible.
Furthermore, your partner will be routinely working on new features based on customer feedback. Their offering will be well-suited to a wide variety of environments. And as a partner, you’ll get to exercise some influence over the roadmap.
In addition to the labor burden involved in operationalizing free software, you have to worry about the potential for catastrophe. There are several questions you should be asking to insure against major failures:
What will the fallout look like if it fails?
How well will it scale with our environment?
Is everything configured securely?
Are we backing up any associated data?
How do we ensure that everything continues working as intended?
Do we have monitoring and alerting in place?
Depending on what specifically you’re deploying, there are likely several more “unknown unknowns” you’ll be exposed to. But in general, there are three major failure modes you should worry about:
Security - if the service were misconfigured, what could an attacker accomplish?
Data loss - if the underlying data were wiped out, how easily could you recover?
Downtime - if the service stops working, how will it affect your customers or your team?
The key word here is risk. Open source software is generally distributed as-is, meaning it’s up to you to understand the ins and outs of the software. Even if your team has scoured the docs, there’s a strong chance for undocumented behavior or an unintentional bug to bite you down the line.
Again, with a commercial partner, these concerns all but vanish. Your partner is incentivized to spend every day ensuring their product is secure and reliable. A commercial partner takes full responsibility for their software, absorbing the risk you would otherwise take on yourself. Any issues are more likely to affect their bottom line than yours.
At Fairwinds, we’ve built a commercial platform for auditing Kubernetes clusters - Fairwinds Insights. Many of our plugins are open source, such as OPA, Aqua’s Trivy or our own Polaris, so one question we often get in the sales process is:
“Why can’t I just deploy all those tools on my own for free?”
The answer, of course, is that you could deploy the open source on your own, but it certainly wouldn’t be free. We’ve spent over 10,000 engineer-hours operationalizing these tools - designing a common report schema and translation layer, building out dashboards, integrating tools like GitHub and Slack, and crafting engineer-friendly workflows. Thanks to this work, Fairwinds Insights users get valuable intel on their Kubernetes clusters on day 1, and can continue to expand their usage without pain.
Looking for a complete Kubernetes governance platform? Fairwinds Insights is available for free. Get started today.
Assuming you chose to do all this yourself, let’s look at where the hidden (and not-so-hidden) costs would lie.
The hardest part of auditing your Kubernetes infrastructure is simply figuring out which tools to use. We’ve identified dozens of open source auditing tools in the wild, and spent months looking at each individually in order to identify the best-in-breed for each type of audit. Anywhere we haven’t found a great solution, we’ve built our own. With Fairwinds Insights, we’ve crafted a strong, well-researched, growing portfolio of auditing tools that can cover any organization’s needs.
We’ve also built an installer that packages all these auditing plugins together in a single Helm chart, and runs them in a uniform way. That might sound simple, but take Trivy as an example: by default, Trivy runs on the command line and can check a single container image for CVEs. To fully operationalize Trivy, we built a CronJob that:
runs the CLI on a configurable schedule (typically every hour)
uses the Kubernetes API to gather a list of images running in your cluster
aggregates the results, deduplicating vulnerabilities shared between different images
maintains a list of which images have been scanned most recently
utilizes that list to prioritize new scans
Each audit tool in our portfolio has a similar wrapper to fully operationalize it. We’ve put hundreds of engineer-hours into building and maintaining a highly configurable, one-line installer that runs up to a dozen different tools in your Kubernetes cluster in a consistent way.
Finally, we’ve spent weeks researching each of these tools, and have developed strong opinions on how they should be rolled out. For example, there are a number of different ways to deploy OPA to audit Kubernetes. But after studying best practices and real-world implementations, we settled on a solution that:
Stores policies in a policy-as-code repository
Syncs those policies with a remote server
Federates the policies out to every cluster in your fleet
Allows you run the same policies in CI, Admission Control, and in-cluster audits
Allows you to turn particular policies on and off depending on context
A homegrown solution would likely miss some or most of these best practices.
Fairwinds Insights community edition is free to use forever. Try the full edition for 30 days by signing up here. Test in GKE, AKS or EKS or run on a test cluster.
Each of our auditing tools is reasonably simple to run on an ad hoc basis, to get a point-in-time snapshot of your cluster. For example, you can run polaris dashboard to see a list of each of your workloads and how they could be improved.
But if you care about security and compliance, glancing over ad hoc reports isn’t sufficient. You’ll need to run the audits on a regular schedule, store the results somewhere, and track changes over time. This will involve provisioning storage (to stash the results), some scripting (to diff one audit against another), and maybe some third-party integrations (to send alerts when new problems arise). You’ll want to be sure to regularly update the auditing tools you’re using. And, when an update to one of these tools is available, you’ll need to rewrite all that additional infrastructure accordingly.
So just to get a regular audit running with persistent results, you’ll have to maintain a bunch of extra code and infrastructure. And that’s before your security, development, and ops teams start to ask for extra features - what happens when they want to assign items to particular people? Get alerts in Slack? See historical trends? Create issues in GitHub?
The Fairwinds Insights team is constantly talking to customers and building out the most requested features. We’ve built user interfaces to help:
ops teams understand which resources don’t adhere to their policies
developers dig in when their CI pipelines catch a configuration issue
engineering managers understand how secure, efficient, and reliable their clusters are in a single glance
If you’re just running the free open source tools, it’s up to you to make sure they’re actually running. If one of the audits starts to fail, or silently skips some part of your infrastructure, you may end up with gaps in coverage. An unintended lapse could take you out of compliance, or worse, lead to a security breach.
If you’re storing the audit data persistently (and you should be!), what are the ramifications of losing that data? You’ll need to ensure it’s backed up appropriately.
Furthermore, while the open source maintainers behind each of these tools might be willing to help you with problems that come up, they have no obligation or incentive to do so. They’re volunteers, and are often stretched to capacity. If you run into trouble, your team might be left digging into other people’s source code to try and discover the problem.
The Insights team has 24/7 monitoring and alerting built around the application, and any downtime will get an immediate response. When customers have problems, we provide support via email, zoom, and Slack. We’re strongly incentivized to make sure our users are getting value out of the product, and to keep their data safe and secure.
To wrap up, there are major costs to using off-the-shelf open source to solve a mission-critical problem. Your team will:
spend a non-trivial amount of time learning and deploying the software
get sucked into fixing, upgrading, and enhancing your deployment
always run the risk of outages, data loss, and security breaches
If I had to pick a single word to summarize the reasons to choose a commercial partner when deploying mission-critical open source, it’d be: trust.
Open source software comes as-is, and is maintained by volunteers; while some will do their best to be helpful, they have no obligation or incentive to educate you, answer your questions, or support your particular needs.
Paying for software neatly aligns incentives. Don’t think of a license as paying for “bonus features” - think of it as a partnership. A commercial license gives you a fixed, predictable, measurable price for using the software, replacing the hidden, variable costs of self-service. A commercial partnership also comes with additional benefits, like influence over the roadmap, and rapid support when you have problems or questions.
Best of all, your engineers will be free to focus on what matters - delivering valuable features to your users.
