A registry of agent benchmarks (including many OSS agent trajectories)

Hacker News

If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.

Visit product View on Hacker NewsDec 23, 2024

Share card

Actual performance

6points

1comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agent · Missing: mac, agents, macos

80%80% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: created, including · Missing: supports, reddit linkedin, podcasting

52%52% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRFits verified-revenue profile · Missing: mobile apps, ios, personal

50%50% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Hacker NewsMay not resonate with HN audience · Strong signals: including · Missing: https docs, excited, just released

47%47% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

25%25% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

19%19% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

1%1% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Agent Smith Is OSS41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Agent Smith Is OSS

Hacker News1

Thredded forums are minimalistic and OSS62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Thredded forums are minimalistic and OSS

Hacker News5

Dynmgrm – Operate DynamoDB with GORM (Golang OSS)49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Dynmgrm – Operate DynamoDB with GORM (Golang OSS)

Hacker News1

We Put Chromium on a Unikernel (OSS Apache 2.0)63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

We Put Chromium on a Unikernel (OSS Apache 2.0)

Hacker News132

My First OSS as a Teen44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

My First OSS as a Teen

Hacker News4

Benchmarks of UUID bintext codecs in Go34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Benchmarks of UUID bintext codecs in Go

Hacker News1

Crowdfunding forecasts and benchmarks31%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Crowdfunding forecasts and benchmarks

Hacker News1

Capricorn – A browser for aliyun-oss63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Capricorn – A browser for aliyun-oss

Hacker News2

Capricorn – A browser for aliyun-oss63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Capricorn – A browser for aliyun-oss

Hacker News3

Thanks to SMC, firefighters with iPhones can track stairs climbed (OSS)61%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Thanks to SMC, firefighters with iPhones can track stairs climbed (OSS)

Hacker News2

Browse all products Analyze your launch