Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

Hacker News

Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are included. The trick is to keep the shared part of the model and the KV cache in RAM, then stream only the routed experts needed for each token from SSD. An SSD is way slower than RAM, so the runtime uses a small expert cache and bounded parallel `pread`. While those reads are in flight, the GPU runs the shared part of the layer. I ran more than 100 experiments. Most didn’t work. A few got me here. The experiments are described in the GitHub repo. It currently generates 5–6 tok/s on an 8 GB M2 MacBook Air and 31–35 tok/s on an M5 MacBook Pro. I also added an experimental OpenAI-compatible local server. It supports streaming and tool calls, and reuses one prompt prefix from the KV cache. Try it! The Mac app is easy to install. On the first run, it will download 15 GB of weights from Hugging Face. The model is surprisingly capable. I would love any kind of feedback!

Visit product View on Hacker NewsJul 29, 2026

Share card

Actual performance

919points

344comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: mac, model, openai · Missing: agents, macos, agent

98%98% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: supports, para, compatible · Missing: reddit linkedin, podcasting, created

96%96% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

77%77% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: way, para · Missing: mobile apps, ios, personal

41%41% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: calls · Missing: plus, platform, intuitive

30%30% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

18%18% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Dstack – an open-source engine for running GPU workloads72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Dstack – an open-source engine for running GPU workloads

Hacker News101

Qake WebGL Voxel Engine released open source75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Qake WebGL Voxel Engine released open source

Hacker News5

ioquake3 – An Open-Source Quake 3 Engine72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

ioquake3 – An Open-Source Quake 3 Engine

Hacker News1

Rabbitlol, an open source bunnylol clone running on Google App Engine65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Rabbitlol, an open source bunnylol clone running on Google App Engine

Hacker News5

Beehive – An open source IFTTT powered by Go's templating engine66%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Beehive – An open source IFTTT powered by Go's templating engine

Hacker News247

Open-source web voxel engine80%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source web voxel engine

Hacker News4

Open-source M.U.G.E.N fighting engine using Godot75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source M.U.G.E.N fighting engine using Godot

Hacker News2

TRAINAA71%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-Source Running, Cycling and Triathlon Training App

Indie Hackers1b2c

Scriggo – a Go open-source template engine and embeddable interpreter67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Scriggo – a Go open-source template engine and embeddable interpreter

Hacker News15

HyperCell – Open-source Excel calculation engine for Java52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

HyperCell – Open-source Excel calculation engine for Java

Hacker News3

Browse all products Analyze your launch