Agent Benchmark Repository and Viewer

Hacker News

We have created a public registry of AI agent benchmarks and agent runtime traces to help everyone better understand how AI agents work and fail these days. Our team and many agent builders we talked to wanted a better way of viewing what agents in these benchmarks do, e.g., how a particular coding agent approaches SWE-bench ( https://www.swebench.com/ ). Right now, there are two reasons why this is difficult: benchmark traces are distributed on many different websites, and they are really hard to read. Often, they are huge raw JSON dumps of the agent in formats that are hard to read. To alleviate this, we build this repository, where it is easy to see what individual agents do and how they solve tasks (or fail to). We hope that putting these agent traces in one place makes it easier to understand and progress on AI agent development in both industry and academia. We are happy to add more benchmarks and agents – let us know if you have something in mind.

Visit product View on Hacker NewsNov 26, 2024

Share card

Actual performance

2points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agents, agent, tasks · Missing: mac, macos, cursor

84%84% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

TrustMRRFits verified-revenue profile · Strong signals: way · Missing: mobile apps, ios, personal

54%54% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Indie HackersIH features products with proven revenue · Strong signals: created · Missing: supports, reddit linkedin, podcasting

49%49% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsMay not resonate with HN audience · Missing: https docs, excited, just released

44%44% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: builder · Missing: plus, platform, intuitive

31%31% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

24%24% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

GitAgent by Lyzr72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Your repository becomes your agent

Product Hunt+230Artificial Intelligence

I made a free Git repository viewer42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I made a free Git repository viewer

Hacker News1

The immutable repository52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The immutable repository

Hacker News1

Unofficial Debian Sid Repository for Hyprland48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Unofficial Debian Sid Repository for Hyprland

Hacker News3

Redu – ncdu for your restic repository41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Redu – ncdu for your restic repository

Hacker News4

WebGL Sprites Benchmark58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

WebGL Sprites Benchmark

Hacker News38

NAB – The Numenta Anomaly Benchmark42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

NAB – The Numenta Anomaly Benchmark

Hacker News17

NAB – The Numenta Anomaly Benchmark42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

NAB – The Numenta Anomaly Benchmark

Hacker News15

Ebola Outbreak Data Repository46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Ebola Outbreak Data Repository

Hacker News2

GitOps Repository Management41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GitOps Repository Management

Hacker News2

Browse all products Analyze your launch