Reduce ChatGPT costs 10x with distributed cache for LLMs

Hacker News

Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache operates transparently, requiring minimal changes to your existing setup. It's like your local content delivery network (CDN), but for LLMs, ensuring that users get the fastest possible access to information. While still in its early stages, we are excited about the potential and are looking to gather feedback from the community. We will share the demo once you subscribe to our mailing list (to control our spending ;)) This could be a game-changer for those using LLMs in prod environments where cost and response time are critical. We’re eager to hear what you think! (launching from the Bunny Coworking House in SF - thanks, Henry, Liyen, and Grace, for having us)

Visit product View on Hacker NewsApr 15, 2024

Share card

Actual performance

22points

3comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: claude, user, chatgpt · Missing: mac, agents, macos

87%87% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

83%83% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

AppSumoStrong fit for a featured deal · Strong signals: users, calls · Missing: plus, platform, intuitive

67%67% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: excited, exist, existing · Missing: https docs, just released, lua

64%64% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRFits verified-revenue profile · Strong signals: users · Missing: mobile apps, ios, personal

52%52% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

18%18% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat, paid · Missing: web3, crypto, cryptocurrency

1%1% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

CloudPouch – See. Understand. Reduce AWS Costs63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

CloudPouch – See. Understand. Reduce AWS Costs

Hacker News7

CloudPouch61%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

See. Understand. Reduce AWS costs.

Indie Hackers2programming

An API to compress images and reduce bandwidth costs54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

An API to compress images and reduce bandwidth costs

Hacker News8

Debuddy

Reduce time and costs of smart contracts building with Debuddy

BetaList

Copyloto14%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Cria conteúdo espetacular 10x mais rápido

Indie Hackerscommitment-side-project

GPTCache – Redis for LLMs69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GPTCache – Redis for LLMs

Hacker News7

prompttest – pytest for LLMs34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

prompttest – pytest for LLMs

Hacker News2

Reduce GitHub Action costs with nested runners48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Reduce GitHub Action costs with nested runners

Hacker News4

ReDoc40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

10x your ChatGPT copywriting output with ReDoc

AppSumo5

Chatclient – Reduce customer support costs by 90%39%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Chatclient – Reduce customer support costs by 90%

Hacker News1

Browse all products Analyze your launch