LLM-Gateway – Zero-Trust LLM Gateway

Hacker News

I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most interesting is the network layer. The gateway and backends communicate over zrok/OpenZiti overlay networks... reach a GPU box behind NAT, expose the gateway to clients, put components anywhere with internet connectivity behind firewalls... no port forwarding, no VPN. Zero-trust in both directions. Most LLM proxies solve the API translation problem. This one also solves the network problem. Apache 2.0. https://github.com/openziti/llm-gateway I work for NetFoundry, which sponsors the OpenZiti project this is built on.

Visit product View on Hacker NewsMar 27, 2026

Share card

Actual performance

7points

1comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, openai, single · Missing: mac, agents, macos

85%85% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: vpn, compatible · Missing: supports, reddit linkedin, podcasting

70%70% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: llama, io · Missing: https docs, excited, just released

53%53% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

36%36% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

29%29% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

19%19% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

4%4% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

VernLLM – LLM fallback, no gateway45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

VernLLM – LLM fallback, no gateway

Hacker News2

NeuralTrust37%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM Gateway and Red Teaming

Indie Hackerscommitment-full-time

Cencurity75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Security gateway for LLM agents

Product Hunt+83

Agnos LLM gateway proxy41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Contain Any LLM Gateway Behind One Control Plane

Indie Hackers1$50/moai

Biboumi – An XMPP-to-IRC gateway60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Biboumi – An XMPP-to-IRC gateway

Hacker News76

Kong Gateway 3.038%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Kong Gateway 3.0

Hacker News1

Fastest LLM gateway (50x faster than LiteLLM)47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Fastest LLM gateway (50x faster than LiteLLM)

Hacker News2

Llmbridge, a C++ LLM gateway with sub-millisecond overhead37%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llmbridge, a C++ LLM gateway with sub-millisecond overhead

Hacker News3

Bifrost53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The fastest LLM gateway in the market

Product Hunt+561Open Source

Tokenhot LLM API Gateway40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

OpenAI-compatible LLM API gateway for 100+ models.

Indie Hackers

Browse all products Analyze your launch