Burnless makes LLM context a protocol. Token savings are a side effect

Hacker News

Share card

Burnless makes LLM context a protocol. Token savings are a side effect

I’ve created Burnless because I realized that we were using the context window for things that didn't need to be there anymore. I wanted to stop burning tokens with what had already happened and were no longer necessary for the next step. It was not a discovery that came later. Burnless was born to separate things. What I didn’t know yet was the size of the problem, the size of the gain and how different it was from the path that everyone else was following. What I see nowadays is that everyone wants the context window to be bigger. That helps, of course. But in my opinion, the concept is wrong. A larger window will only allow you to put more things inside it, but it does not solve the fact that we are using the same place for the conversation, memory, decisions, execution, open questions, work history and all the old records of what has already been done. Burnless separates that into layers. Conversations, thoughts, debates, questions, and the history of the work are indexed and stored in your disk. The main context contains only the decisions and information that are still necessary to continue the work. But the real words are still in disk if they need to be checked (not even necessary to be honest). Also when we say something and someone asks us to repeat we never use the same words, why do AI need to do that? (remember, the original is still in the disk dont worry…) Tasks are delegated to independent, lightweight workers. They receive a clean task, write the code, manipulate files, execute commands and register on disk what they have done. And yes, the workers use the same byte cache header (don't worry with this too) Burnless then audits the result. If a worker claims that he created or altered a file, Burnless verifies that it actually exists before marking the task as complete. So, the context stops being the place where everything has to live. It becomes the place where only what needs attention right now is kept. That’s why I compared Burnless to a protocol and, in the beginning, even to TCP/IP. Not because it has the same history dimension, but because the movement is similar: separating responsibilities into layers, into worker capacities, into a useful and clean way of defining the communication between this alive system. Memory is no longer carried throughout the conversation. The execution stops being a role for the main model. The project history no longer permanently occupies the work's memory. The project can go on even after a /clear command or after switching between models. Token's savings were the initial motivation, but it ended up becoming a consequence of the architecture. In a real-world task, the workers processed approximately 1.4 million tokens. But the Brain’s active context had only 1,590 tokens after completing the full task. I’m not claiming that Burnless took the exact same 1.4M tokens of text and compressed it to 1,590 tokens. What really happened was that all of that work was done, recorded, verified and continued without the accumulated history remaining inside the active context. The final operational ratio was approximately 908 to 1. (yeah 908x) When I show you that number, it sounds unbelievable. That was when I realized that the problem was much larger than I had initially imagined. While the general direction is to build increasingly larger context windows, I went in the opposite direction: removed from the context everything that doesn’t need to be considered at the moment. The repository also includes an llms.txt file containing the complete architecture, benchmarks, calculations, limitations and commands that are required to reproduce the numbers. As I cannot properly explain everything in a short comment, the llms.txt file is there for anyone who wants to understand what is happening at a deeper level and verify the claims. Use that to help your AI understand that stuff easily and respond to your questions.