Butter – A Behavior Cache for LLMs

Hacker News

Share card

Butter – A Behavior Cache for LLMs

Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we built computer-use agents to automate legacy Windows applications. The goal was to replace RPA. But in practice, these agents were slow, expensive, and unpredictable - a major downgrade from deterministic RPA, and unacceptable in the worlds of healthcare, lending, and government. We realized users don't want to replace RPA with AI, they just want AI to handle the edge cases. We set out to build a system for "muscle memory" for AI automations (general purpose, not just computer-use), where agent trajectories get baked into reusable code. You may recall our first iteration of this in May, a library called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Today we're relaunching it as a chat completions proxy. It emulates scripted automations by storing observed message histories in a tree structure, where each fork in the tree represents some conditional branch in the workflow's "code". We replay behaviors by walking the agent down the tree, falling back to AI to add new branches if the next step is not yet known. The proxy is live and free to use while we work through making the template-aware engine more flexible and accurate. Please try it out and share how it went, where it breaks, and if it’s helpful.