Orchestra – Model and data pipeline monitoring as a service

Hacker News

Share card

Orchestra – Model and data pipeline monitoring as a service

Hi HN-ers, We're Teren and Qiyan, founders of Orchestra (https://orchestrahq.com). We help data scientists/data engineers discover, prioritize and investigate machine learning model performance issues in real-time. We're Datadog for machine learning. We first came across this problem while Teren was leading a team of analysts and data scientists at a global bank. Their main role was to identify opportunities to apply AI/ML to drive business performance internally and externally. When he joined, several models were already in production but upon further investigation, there were a few that were unusable for years. One particular model need significant manual rework to make it usable again. With an early detection system, model performance issues such as the one Teren faced can easily be fixed before it causes further damage not only to the business but the reputation of the data science community internally as a whole. That's the motivation behind Orchestra - to provide robust ML-specific tools to help AI/ML/data science teams build trust and credibility. Our tools are designed for data scientists who are time starved with a million priorities. Let us handle the infrastructure and you can focus on improving the model to actually deliver value for the business. We want to make it easy to use but also flexible enough to meet your monitoring needs for just about any kind of model you develop. In short, a few code snippets allow us to extract, monitor and analyze model inputs/outputs as well as the data pipeline. We want to build a product that solves a problem and loved by customers so we're eager to learn from all the experiences you've had in this area and ideas on how you've built trust and credibility within your organizations. We welcome any feedback so please feel free to share. We're also looking for testers/collaborators to join us on the journey to building high quality machine learning models. Thanks in advance