Verified Multi-Step Synthesis Using LLMs and MCTS

Hacker News

3 months ago, I submitted a hack on using Monte Carlo Tree Search (MCTS) with an LLM guided by a program verifier: https://news.ycombinator.com/item?id=38235407 I just wanted to send this update, because we now have a paper that explains the hack in detail. We also show the use of verifiers with ChatGPT (implemented as actions within GPTs), and our method makes open models like Phind Code Llama competitive with ChatGPT augmented with verifier feedback and step by step instructions. Happy to answer any questions!

Visit product View on Hacker NewsFeb 23, 2024

Share card

Actual performance

1points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, new, models · Missing: mac, agents, macos

87%87% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

67%67% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: llama, ide, io · Missing: https docs, excited, just released

64%64% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoStrong fit for a featured deal · Missing: plus, platform, intuitive

55%55% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: month · Missing: mobile apps, ios, personal

43%43% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

16%16% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

5%5% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Bt is approaching 1.0 – The next step in Java torrents41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Bt is approaching 1.0 – The next step in Java torrents

Hacker News8

Monitor LLMs Using LangChain and Infino47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Monitor LLMs Using LangChain and Infino

Hacker News1

Raink – Document ranker using LLMs60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Raink – Document ranker using LLMs

Hacker News2

ACME Proxy using step-ca46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

ACME Proxy using step-ca

Hacker News2

TSCE – Think Before You Speak (Two-Step Contextual Enrichment for LLMs)53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

TSCE – Think Before You Speak (Two-Step Contextual Enrichment for LLMs)

Hacker News3

Using LLMs and Embeddings to classify application errors80%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Using LLMs and Embeddings to classify application errors

Hacker News65

Neurooo – DeepL clone using LLMs for translations40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Neurooo – DeepL clone using LLMs for translations

Hacker News18

A 64-step drum sequencer I built using React and Redux55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A 64-step drum sequencer I built using React and Redux

Hacker News327

Formally Verified FPSan49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Formally Verified FPSan

Hacker News2

GPTCache – Redis for LLMs69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GPTCache – Redis for LLMs

Hacker News7

Browse all products Analyze your launch