Open-Source Conversational Analytics

Hacker News

Over the past two years, I’ve developed a toolkit for helping dozens of clients improve their LLM-powered products, which I'm now open-sourcing. First up: a library to bring product analytics to conversational AI. One of the biggest challenges I see clients face is understanding how their assistants are performing in production. Evals are great for catching regressions, but they can’t surface the blind spots in your AI’s behavior. This gets even more challenging for conversational AI products that don’t have a single “correct” answer. Different users cohorts want different experiences. That makes measurement tricky. Coming from a product analytics background, my default instinct is always: “instrument the product!” However, tracking generic events like user_sent_message doesn’t tell you much. What you really want are insights like: - How frequently do users request to speak with a human when interacting with a customer support agent? - Which user journeys trigger self-reflection during a session with an AI therapist? - What percentage of the time does an AI tutor's explanation leave the student confused? This new library enables these types of insights through the following workflow: Analyzes your conversation transcripts Auto-generates a rich event schema Tags each message with relevant events and event properties Sends the events to your analytics tool (currently supports Amplitude and PostHog) Any thoughts or feedback would be greatly appreciated!

Visit product View on Hacker NewsApr 17, 2025

Share card

Actual performance

2points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agent, user, new · Missing: mac, agents, macos

97%97% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: supports · Missing: reddit linkedin, podcasting, created

78%78% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRFits verified-revenue profile · Strong signals: users, way · Missing: mobile apps, ios, personal

55%55% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Hacker NewsMay not resonate with HN audience · Strong signals: io · Missing: https docs, excited, just released

26%26% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: users · Missing: plus, platform, intuitive

22%22% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

15%15% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Typebot69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source conversational apps builder

Indie Hackers6$43,017/mocommunication

Pirsch Analytics - Open-source analytics for developers69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Pirsch Analytics - Open-source analytics for developers

Hacker News4

Trench – Open-source analytics infrastructure72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Trench – Open-source analytics infrastructure

Hacker News155

Trench79%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open source analytics infrastructure

Product Hunt+281Developer Tools

PingPong – Open-source analytics for anything with a URL71%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PingPong – Open-source analytics for anything with a URL

Hacker News1

Intervo84%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The open-source platform for conversational AI

Product Hunt+385Open Source

SlideSelection, an Open Source Hooper Selection Implementation47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

SlideSelection, an Open Source Hooper Selection Implementation

Hacker News4

Piki is now open source73%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Piki is now open source

Hacker News2

Open Source CNC Beehive With Arduino and Sensors58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open Source CNC Beehive With Arduino and Sensors

Hacker News1

An Open-Source Intro to WebApps Curriculum72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

An Open-Source Intro to WebApps Curriculum

Hacker News12

Browse all products Analyze your launch