Prompts as WASM Programs

Hacker News

AICI is a proposed common interface between LLM inference engines (llama.cpp, vLLM, HF Transformers, etc.) and "controllers" - programs that can constrain the LLM output according to regexp, grammar, or custom logic, as well as control the generation process (forking, backtracking, etc.). AICI is based on Wasm, and is designed to be fast (runs on CPU while GPU is busy), secure (can run in multi-tenant cloud deployments), and flexible (allow libraries like Guidance, LMQL, Outlines, etc. to work on top of it). We (Microsoft Research) have released it recently, and would love feedback on the design of the interface, as well as our Rust AICI runtime. I'm the lead developer on this project and happy to answer any questions!

Visit product View on Hacker NewsMar 11, 2024

Share card

Actual performance

209points

35comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Missing: mac, agents, macos

85%85% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

AppSumoStrong fit for a featured deal · Strong signals: interface · Missing: plus, platform, intuitive

69%69% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

67%67% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: llama, io · Missing: https docs, excited, just released

57%57% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

43%43% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

17%17% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

2%2% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Timebox, politely timebox unix programs47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Timebox, politely timebox unix programs

Hacker News19

Twitch Programs a Commodore 6443%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Twitch Programs a Commodore 64

Hacker News1

CollegeHippo62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Graduate programs matchmaker

Indie Hackers1$150/moeducation

jevc – compile policy prompts into deterministic verdict programs39%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

jevc – compile policy prompts into deterministic verdict programs

Hacker News2

WSL-Programs; a list of programs' usability on the WSL40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

WSL-Programs; a list of programs' usability on the WSL

Hacker News1

www.RetroBASIC.xyz – Emulate BASIC Programs in the Browser34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

www.RetroBASIC.xyz – Emulate BASIC Programs in the Browser

Hacker News2

Simple logger and debugger for Go programs41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Simple logger and debugger for Go programs

Hacker News32

Applesoft BASIC editor with example programs55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Applesoft BASIC editor with example programs

Hacker News61

SHOW HN: My First Engine and Programs:)44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

SHOW HN: My First Engine and Programs:)

Hacker News1

Komposto – Sketchbook of self-modifying programs45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Komposto – Sketchbook of self-modifying programs

Hacker News4

Browse all products Analyze your launch