Premortem, a coding-agent-powered airplane blackbox
Premortem, a coding-agent-powered airplane blackbox
A few weeks ago, I was getting random OOMs on my linux box and I had no idea what was causing them. At one point when I realized that the memory was getting sucked up by some process, I kicked off a claude code job to see if it could figure out what was happening in real time. And it did! In real time the coding agent ran through a suite of system commands, figured out which jobs were causing problems, and then even started to dig into the explicit function calls (python and node processes can both be inspected at the function call level by sideloaded processes) before the entire system finally crashed. Besides being extremely cool, I realized that with a few tweaks I could make this a legitimately useful tool. The basic idea: any time certain system vitals cross a threshold, spin up a coding agent and have the agent debug what is going on as aggressively as possible, with all logs being streamed to a third party server (in addition to being stored on disk). This basic abstraction would solve two huge problems: - Most of the time it is very hard to figure out why exactly a machine went down. This tool would effectively act as an airplane blackbox, a sort of last record of what was going on that specifically is focused on debugging the failure as it happened. Massive speed up on figuring out system-breaking issues. - Most of the time there are available interventions that someone could take that would prevent the system from going down at all, if a human was around when the crash was happening. For example, if I see that I’m about to OOM from vitest, I can just kill a bunch of the processes that are spiking memory and prevent the system from crashing that way. We now have premortem running on all of our production machines. Hope this is useful for other folks!
Share cardActual performance
Launch Intel predictions
Analyze your own launch →Incorrect prediction on native model
Similar products
Bestie, a coding agent that respects you
Bailout – The coding agent meant to be deleted
Website Coding Agent
Beating GPT5.5-xhigh for Coding agent security with SLMs and IRM
Hazzel – terminal coding agent, BYOK, undo-everything
Neurogrid terminal coding agent
The coding agent that asks before it builds
The coding-agent harness you can make your own
zot – Yet another coding agent harness
Zot – Yet another coding agent harness