I Decided to Build Multiwoven, an Open-Source Reverse ETL

Hacker News

Share card

I Decided to Build Multiwoven, an Open-Source Reverse ETL

I’m an engineer by heart and a data enthusiast by passion. I have been working with data teams for the past 10 years and have seen the data landscape evolve from traditional databases to modern data lakes and data warehouses. In previous roles, I’ve been working closely with customers of AdTech, MarTech and Fintech companies. As an engineer, I’ve built features and products that helped marketers, advertisers and B2C companies engage with their customers better. Dealing with vast amounts of data, that either came from online or offline sources, I always found myself in the middle of newer challenges that came with the data. One of the biggest challenges I’ve faced is the ability to move data from one system to another. This is a problem that has been around for a long time and is often referred to as Extract, Transform, Load (ETL). Consolidating data from multiple sources and storing it in a single place is a common problem and while working with teams, I have built custom ETL pipelines to solve this problem. However, there were no mature platforms that could solve this problem at scale. Then as AWS Glue, Google Dataflow and Apache Nifi came into the picture, I started to see a shift in the way data was being moved around. Many OSS platforms like Airbyte, Meltano and Dagster have come up in recent years to solve this problem. This has now made life easier for data engineers, especially when I was struggling with ETL pipelines. But later in my career, I started to see a new problem emerge. When marketers, sales teams and growth teams operate with top-of-the-funnel data, while most of the data is stored in the data warehouse, it is not accessible to them, which is a big problem. Then I saw data teams and growth teams operate in silos. Data teams were busy building ETL pipelines and maintaining the data warehouse. In contrast, growth teams were busy using tools like Braze, Facebook Ads, Google Ads, Salesforce, Hubspot, etc. to engage with their customers. The Genesis of Multiwoven At the initial stages of Multiwoven, our initial idea was to build a product notification platform for product teams, to help them send targeted notifications to their users. But as we started to talk to more customers, we realized that the problem of data silos was much bigger than we thought. We realized that the problem of data silos was not just limited to product teams, but was a problem that was faced by every team in the company. That’s when we decided to pivot and build Multiwoven, a reverse ETL platform that helps companies move data from their data warehouse to their SaaS platforms. We wanted to build a platform that would help companies make their data actionable across different SaaS platforms. Why Open Source? As a team, we are strong believers in open source, and the reason behind going open source was twofold. Firstly, cost was always a counterproductive aspect for teams using commercial SAAS platforms. Secondly, we wanted to build a flexible and customizable platform that could give companies the control and governance they needed. This has been our humble beginning and we are excited to see where this journey takes us.