InferX – an AI-native OS for running 50 LLMs per GPU with hot swapping

Hacker News

Hey folks , We’ve been building InferX. an AI-native runtime that snapshots the full GPU execution state of LLMs (weights, KV cache, CUDA context) and restores it in under 2s. This lets us hot-swap models like threads. no reloading, no cold starts. We treat each model as a lightweight, resumable process. like an OS for LLM inference. Why it matters: •Run 50+ LLMs per GPU (7B–13B range) •90% GPU utilization (vs ~30–40% with conventional setups) •Avoids cold starts by snapshotting and restoring directly on GPU •Designed for agentic workflows, toolchains, and multi-tenant use cases •Helpful for Codex CLI-style orchestration or bursty multi-model apps Still early, but we’re seeing strong interest from builders and infra folks. Would love thoughts, feedback, or edge cases you’d want to see tested. Demo: https://inferx.net X: @InferXai

Visit product View on Hacker NewsApr 17, 2025

Share card

Actual performance

3points

2comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agent, model, agentic · Missing: mac, agents, macos

94%94% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

75%75% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

61%61% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRFits verified-revenue profile · Strong signals: apps · Missing: mobile apps, ios, personal

52%52% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: builder · Missing: plus, platform, intuitive

38%38% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

23%23% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

1%1% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

wc-GPU: The Unix util `wc` running on a GPU66%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

wc-GPU: The Unix util `wc` running on a GPU

Hacker News15

Running LLMs on CPUs55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Running LLMs on CPUs

Hacker News1

VRAMGlass46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Find the right GPU for running LLMs locally

Product Hunt+1

AI-Native dependency upgrades through codemods23%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

AI-Native dependency upgrades through codemods

Hacker News2

MyLooks.AI12%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Find out how hot you are & become hotter!

Indie Hackers1ai

Hot or not for yachts called Yacht or Knot49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Hot or not for yachts called Yacht or Knot

Hacker News17

Mẫu hàng rào sắt đẹp tuyệt mĩ4%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Mẫu hàng rào sắt hot thị trường

Indie Hackers

Shopeasy.ai64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

AI-Native Commerce OS for Modern Merchants

Indie Hackers1$9,999/mob2b

Running Arch Linux on a $50 Smartwatch (Spreadtrum SC9832E)65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Running Arch Linux on a $50 Smartwatch (Spreadtrum SC9832E)

Hacker News1

RiskRhino64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

AI native GRC tool

Indie Hackerscommitment-full-time

Browse all products Analyze your launch