I Made an API for Persistent KV-Caching (Cache Augmented Generation)

Hacker News

Hi HN! I wanted to demonstrate an easy to use API for Cache Augment Generation. For any open source LLM available on Llama cpp, we can store the KV-cache and model state after it has processed a large corpus of documents, and then load that state in every time we query the document. This leads to a drastic reduction in latency as well as compute/energy used by the model. This demo is part of a larger system that I'm building called DataBridge[0] - with a focus on implementing new and useful techniques for knowledge retrieval - allowing developers to use the latest research in production. I'd love to hear your feedback on DataBridge, and the CAG feature. If you have papers or particular techniques you'd like to see implemented, I'd love to hear about it :) [0] https://github.com/databridge-org/databridge-core/

Visit product View on Hacker NewsFeb 24, 2025

Share card

Actual performance

3points

1comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, new, open · Missing: mac, agents, macos

92%92% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

76%76% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: open source, llama, io · Missing: https docs, excited, just released

65%65% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

38%38% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

27%27% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

12%12% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

I made a simple persistent counting API45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I made a simple persistent counting API

Hacker News54

Persistent Pushd and Popd48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Persistent Pushd and Popd

Hacker News2

Malcolm, a Persistent XMPP/Gtalk Chatroom48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Malcolm, a Persistent XMPP/Gtalk Chatroom

Hacker News4

MyToDoList - Persistent todos built with localStorage53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

MyToDoList - Persistent todos built with localStorage

Hacker News7

Next generation configuration mgmt51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Next generation configuration mgmt

Hacker News3

Stylistic Rhyme-Bound Poetry Generation35%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Stylistic Rhyme-Bound Poetry Generation

Hacker News1

Adversarial Generation of Extreme Samples62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Adversarial Generation of Extreme Samples

Hacker News35

ExGAN-Adversarial Generation of Extreme Samples62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

ExGAN-Adversarial Generation of Extreme Samples

Hacker News54

Perspex – a next generation WPF?48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Perspex – a next generation WPF?

Hacker News3

JavOSize – The next generation of java profiler55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

JavOSize – The next generation of java profiler

Hacker News6

Browse all products Analyze your launch