Relai-SDK – simulate → evaluate → optimize AI agents

Hacker News

What relai-sdk is an open-source toolkit for making AI agents reliable via a complete learning loop: simulate → evaluate → optimize. Why Agent runs are stochastic; tool-calls fail; hard to reproduce, measure, and fix at scale. It’s also hard to align behavior with goals across output quality/format, cost, and latency. We need a loop that integrates user feedback and LLM evaluators directly into the agent code (prompts, configs, models, graphs) without overfitting. How - Simulation: LLM personas, mocked MCP servers/tools, synthetic data; can condition on real traces - Evaluation: code-based + LLM-based evaluators; turn human reviews into optimization-ready benchmarks - Optimization with Maestro: tune prompts, configs and even agent graph for improved quality, cost and latency Try it pip install relai GitHub: https://github.com/relai-ai/relai-sdk Docs: https://docs.relai.ai/ (2-min overview: https://youtu.be/qKsJUD_KP40 ) Looking for feedback on - Where graph-level suggestions help (beyond prompt tuning) - Evaluator signals you rely on for reliability (and what we’re missing) - Simulation setups/environments you’d want out of the box Notes Founder here. Happy to share internals, tradeoffs, and limitations. Works with LangGraph / OpenAI Agents / Google ADK / etc. SDK Apache-2.0 license.

Visit product View on Hacker NewsOct 27, 2025

Share card

Actual performance

5points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agents, agent, model · Missing: mac, macos, cursor

97%97% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

64%64% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: lua, io · Missing: https docs, excited, just released

55%55% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: reviews, calls · Missing: plus, platform, intuitive

45%45% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: google · Missing: mobile apps, ios, personal

39%39% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

18%18% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

SuperOptiX – Evaluate, Optimize, Orchestrate DSPy AI Agents – BDD Style57%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

SuperOptiX – Evaluate, Optimize, Orchestrate DSPy AI Agents – BDD Style

Hacker News2

CodeSandbox SDK – Sandboxes for AI Agents34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

CodeSandbox SDK – Sandboxes for AI Agents

Hacker News3

Scorecard83%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Evaluate, Optimize, and Ship AI Agents

Product Hunt+391Developer Tools

The “Optimize All the Things” SDK64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The “Optimize All the Things” SDK

Hacker News9

Visualizing the Impossible – How to Simulate Electrons63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Visualizing the Impossible – How to Simulate Electrons

Hacker News5

EvalsOne56%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Effortlessly evaluate and optimize your LLM based system

Indie Hackerscommitment-full-time

Chromecast C# SDK56%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Chromecast C# SDK

Hacker News4

Traceo SDK for Java42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Traceo SDK for Java

Hacker News1

I made an SDK that prevents duplicates transactions for Fintechs58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I made an SDK that prevents duplicates transactions for Fintechs

Hacker News2

Zant – A TinyML SDK in Zig70%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Zant – A TinyML SDK in Zig

Hacker News7

Browse all products Analyze your launch