I run 30B 22tok/s, 109tok/s not novel,6GB/16GB RAM overcoming llama.cpp

Hacker News

Democratisation of local AI is key. I've been working on pushing the limits of commercial hardware, squeezing any extra bit possible. My Scientific Agentic AI hareness helped me to reallocate every single bit of it. I rewrote the Kernel, I went down the CUDA rabbit hole until I have been able to explain any bit and any ms of computational power involved in the process pushing the Qwen 30B-A3B from 8 tok7s to 19 tok/s with llama.cpp up to 22.2 tok/s with my project and 109 tok/s on not novel content and speeding up the prefill by 5-9X

Visit product View on Hacker NewsJul 29, 2026

Share card

Actual performance

5points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agent, agentic, single · Missing: mac, agents, macos

85%85% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Hacker NewsStrong engagement from HN community · Strong signals: llama, io · Missing: https docs, excited, just released

74%74% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

45%45% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Indie HackersIH features products with proven revenue · Missing: supports, reddit linkedin, podcasting

35%35% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

27%27% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

21%21% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

4%4% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Run Llama 3.1 8B in the browser78%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run Llama 3.1 8B in the browser

Hacker News23

Decrypting Rita, a graphic novel75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Decrypting Rita, a graphic novel

Hacker News4

American Apocalypse (Novel) (2004)64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

American Apocalypse (Novel) (2004)

Hacker News1

Nevertwenty a novel approach to chord synthesis65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Nevertwenty a novel approach to chord synthesis

Hacker News1

Llama or Alpaca?78%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama or Alpaca?

Hacker News6

Llama 3.2 Interpretability with Sparse Autoencoders74%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama 3.2 Interpretability with Sparse Autoencoders

Hacker News579

Run any Llama model finetune and more, instantly63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run any Llama model finetune and more, instantly

Hacker News7

4 yrs ago, I wrote an AI novel, now I published & made it free39%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

4 yrs ago, I wrote an AI novel, now I published & made it free

Hacker News2

TypistStories, new Gothic novel released36%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

TypistStories, new Gothic novel released

Hacker News2

(gr)album – graphic novel + music album44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

(gr)album – graphic novel + music album

Hacker News2

Browse all products Analyze your launch