Innovation in computing has been a sequence of battles against three major bottlenecks.
The first, the cost of raw compute, was solved by decreasing the cost and size of semiconductor chips leading to personal computers for the masses.
This then opened a second major hurdle: information transfer cost as more powerful computers increased demand for more data, with ugly transitional solutions like the floppy disk or cd-rom.
This was solved by the internet, allowing computers to stream data between each other over TCP, but has in turn created a third major bottleneck: distributed systems operational cost, as the need to network large configurations of computers created intrinsic operational complexity.
At sufficient scale, the likelihood of a large, multi-node system failing in some way approaches 1, and the skillset needed to debug a failure in such a complex system efficiently requires a rare mix of generalist knowledge and battle-tested experience.
This makes site reliability uniquely difficult to hire for, and as a result, many companies have turned to "managed services" to handle the deployment and maintenance of their core infrastructure.
Instead of hiring a team of 8 engineers with appropriate expertise to wrangle a Kafka cluster, they'll buy a managed services package from Confluent or their cloud provider of choice.
In doing so, they're giving up control over which cloud provider supplies the underlying compute and the ability to fine-tune configurations, while also paying a significant premium for that operational expertise. This choice to trade control, portability, privacy, and cost for flexibility and convenience is an understandable one, but one we believe is no longer necessary.
Even more perversely, this issue with distributed system management has warped the way developer effort is rewarded. For example, major cloud providers like Google, Amazon and Microsoft, with their surplus of developer muscle, enterprise relationships, and infrastructure-for-rent, have reaped the majority of the profits stemming from commercialized open source projects. Giving non-corporate developers the ability to deliver applications at low operational burden and into whatever environment a customer desires can outflank this advantage.
With the widespread adoption of cloud for the underlying compute power, near-free managed kubernetes for orchestration, and declarative configuration for specifying the desired state of a kubernetes cluster, we believe that it's now possible to offer companies the best of both worlds – the control and configurability and price of do-it-yourself, without actually having to do everything yourself.
In particular, Plural is building tooling that makes it easy to do things like:
- installing applications in the right dependency order, even if the application requires a mix of cloud services, k8s resources, or anything else
- setting up GitOps best-practices, like secret management
- converting any kubernetes application installation into its own "managed service" by using our extensible admin console
- specifying dashboards, log aggregators, and runbooks for the specific application you might want to deploy via CRD's
All together, this tooling gives us the ability to convert infrastructure tech like RabbitMQ, Etcd, Postgres and Redis, and some operational tools like Airflow or grafana into competitive alternatives to open source managed services.
Currently, there's about a 50% arbitrage between managed K8s + cloud compute+storage cost, which is Plural's setup, and the cost of a managed open source solution on the exact same cloud. Closing that gap will release considerable trapped value back into the marketplace.
A truly scalable, flexible solution to application delivery is the first step towards our long term goal of building a platform that properly remunerates developer communities for their effort. This is the mountaintop – that if you've built a software application that has some degree of traction, we'll give you all the tools needed to build a business around third party installations of that software — licensing, billing, support, observability, multi-cloud installation.
This should enable a new marketplace of portable SaaS applications that allows privacy from the data layer up, process-level control of your core infrastructure, and mobility from provider to provider as costs change.
Legacy constraints currently prevent all the major players from doing this right (Amazon, Google, Microsoft's need to protect their core cloud business, VMWare's lock-in with their core VM business, etc.), but the core technology needed is all there, it just needs to be lassoed together and offered in a holistic manner to the market. We have a unique opportunity to do just that.
If you’re interested in learning more about how Plural works and how we can help your engineering team deploy open-source applications in a cloud production environment, reach out to myself and the rest of the team over at Plural.
Ready to effortlessly deploy and operate open source applications in minutes? Get started with Plural today.
Join us on our Discord channel for questions, discussions, and to meet the rest of the community.
Be the first to know when we drop something new.