How Modeo Utilizes Plural for Their Customers

Ismael Goulani is the CTO and Co-founder of Modeo, a data engineering consulting company based in Paris. The ten-person consulting team helps data-driven organizations set up their modern data engineering stacks. They have worked with companies like L’oreal, ManoMano  Silvr, and Heelio.

According to Ismael, one of the main challenges of running a consulting company is understanding how the different technologies in a client’s data stack work together and how to deploy them swiftly and efficiently into a cloud production environment like Kubernetes.

“We have clients using Kubernetes and there is a lot of complexity when it comes to deploying data engineering tools like Apache Airflow,” added Ismael.

For an application like Apache Airflow, you generally need to deploy the following components:

• A log object storage bucket, which varies based on the cloud provider you choose (A word of caution: Azure currently lacks compatibility with S3)
• A Redis server for celery queue
• A Postgres database
• An Airflow webserver/worker/scheduler

It’s worth noting that all these components need to be installed in sequence. Also, upgrading them requires some sort of version compatibility check to ensure that all the components can function together properly.

Managing applications like Airflow on Kubernetes takes a lot of engineering time. The reality is that most engineering teams don’t have the luxury to hire a team of engineers to manage complex deployments on Kubernetes.

The Modeo Team Was Responsible For Developing Heelio’s Data Platform

Heelio is a financial operations platform that makes it simple and fast to facilitate strategic decision-making in an organization. The Paris-based company connects directly to your software and then consolidates your financial and operational data for easy analysis.

As CTO at Modeo, Ismael was in charge of developing Heelio’s data platform. He needed a solution that would help him get Heelio’s data stack up and running on Kubernetes as quickly as possible. If you are familiar with Kubernetes, you probably know how complex it is to use, set up, and manage.

"I was faced with the task of getting Heelio’s data platform out to the market as quickly as possible, that's where Plural really helped," he said. “I want to focus on getting the data infrastructure online to start bringing value to our clients.”

As a startup, speed is your biggest advantage, and if you can drastically cut down the time to deployment for an application you can put yourself ahead of the competition. However, most startups have to deal with a wide range of constraints. And, you always can’t afford to learn a new technology in a week or hire a person to handle a single task.

For Ismael to get Heelio’s MVP up and running he would need to spend a good chunk of his time working with Kubernetes, which is hard to pick up fast even if you are a DevOps engineer. “I know DevOps, but I couldn’t move as fast as I needed to provide a secure Kubernetes deployment,” he said.

Why Modeo Chose Plural for Heelio

Heelio’s data platform consists of the following technologies:

• Airbyte: Third-party data integration (payment gateway like stripe PayPal, CRM, etc.,)
• Apache Airflow: Used mainly for data pipeline orchestration
• BigQuery: Serves as their Data Warehouse. Data is stored in BigQuery and encrypted at rest.

The Modeo engineering team consists of software and data engineers. Previously, they used to work a lot with Google’s Cloud Composer but found that it was expensive to run. The engineering team also had issues integrating with Airbyte and Airflow due to the lack of authentication with their API.

Over time, they saw their costs were adding up for their clients and were searching for a solution that would allow them to achieve the same results cheaper without sacrificing speed.

“In a small team, you don’t always have time to work on advanced DevOps subjects,” he said.

“We’re a small team with a lot on our plate. For Kubernetes deployments, we would have to think about terraform, security, access control, and a bunch of other things. We just needed to get our data infrastructure to work as quickly as possible.”

Ismael estimates that, it would have taken his team weeks to deploy their application. “Plural was great for our small team as it helped us go from weeks to hours to get our Kubernetes deployments up and running” he added.

In the upcoming months, Modeo is planning to build a data platform as a service with several layers (data ingestion, data transformation, data storage, and data visualization.)

Their storage layer will be on Clickhouse, with a data warehouse deployed in several Kubernetes clusters in their data infrastructure.

Modeo’s goal is to make working with data a much simpler process for even the smallest of companies that don’t have a data engineering presence.

Getting started with Plural

Are you looking to bring value to your users faster? Or are you looking to get your open-source application into the cloud with minimal effort?

Reach out to me and the rest of the team over at Plural to learn more about how Plural works and how we are helping engineering teams across the world deploy open-source applications in a cloud production environment.

Ready to effortlessly deploy and operate open source applications in minutes? Get started with Plural today.

Join us on our Discord channel for questions, discussions, and to meet the rest of the community.