Moe-Direct – MoE Models far larger than your RAM, on a consumer desktop

Hacker News

I wanted to try using the larger models on my computer (32GB RAM, RTX 5080, Gen5 NVMe), but the best I could do was around 30B. So I started with the idea that it might be possible by taking advantage of the fact that MoE models use only some of the experts rather than all of them. MoE-Direct essentially uses the three layers of SSD, RAM, and VRAM instead of residing entirely in memory, caching only the necessary experts in RAM and making the model usable even with resources far smaller than required. In my environment, I obtained the following decode results: Kimi K2.6: 1.03 tok/s. Qwen3.5-122B: 5.59–5.69 tok/s, with decode performance about 2.3 times better than plain mmap for the same binary. The current project is still far from the intended stage of practical use, and there are still many problems that need to be addressed. Since MoE-Direct is still in its early stages and external usability reviews and testing have not yet been conducted, I am very interested in feedback on my project and participation in testing. (Linux and macOS do not have a test environment available at the moment, so it is only possible on Windows.)

Visit product View on Hacker NewsAug 29, 2026

Share card

Actual performance

1points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: mac, macos, model · Missing: agents, agent, cursor

89%89% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: started · Missing: supports, reddit linkedin, podcasting

83%83% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: ide, io · Missing: https docs, excited, just released

53%53% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: reviews · Missing: plus, platform, intuitive

45%45% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

28%28% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

15%15% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

RunNburn – Run a 295B Moe from a 98GB GGUF on a 64GB RAM Desktop68%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

RunNburn – Run a 295B Moe from a 98GB GGUF on a 64GB RAM Desktop

Hacker News11

Populum37%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Direct-To-Consumer Supplement Brand

Indie Hackers17$70,000/mob2c

Handsy – Direct To Consumer Hand Sanitizer51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Handsy – Direct To Consumer Hand Sanitizer

Hacker News1

I made a tiny MoE/Engram viz tool74%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I made a tiny MoE/Engram viz tool

Hacker News2

Moesnutritioncalculator10%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Moe's nutrition calculator

Product Hunt+5

Moe, a Metric Optimization Engine58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Moe, a Metric Optimization Engine

Hacker News58

Cozee Candles41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Cozee is a direct to consumer e-commerce wellness brand

Indie Hackerscommitment-side-project

LLM Inference Performance Analytic Tool for Moe Models (DeepSeek/etc.)44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM Inference Performance Analytic Tool for Moe Models (DeepSeek/etc.)

Hacker News1

Fin vs Fin54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Direct-to-Consumer (D2C) Wellness Product Reviews

Indie Hackers12$70,000/mocontent

Ichi.moe – translate/romanize any text in Japanese55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Ichi.moe – translate/romanize any text in Japanese

Hacker News1

Browse all products Analyze your launch