AI-assert – Constraint verification for LLM outputs (278 lines, Python)
AI-assert – Constraint verification for LLM outputs (278 lines, Python)
I built a tiny library that verifies LLM outputs against constraints and retries on failure. The core insight: LLMs don't reliably follow instructions, but you can catch failures cheaply and retry with targeted feedback. This is essentially a lightweight "process reward model" that requires zero training. How it works: 1. Your LLM generates output 2. ai-assert checks it against constraints (length, word count, sentence count, regex, custom predicates) 3. Each constraint returns a score in [0,1] -- composite is multiplicative (zero in any = zero overall) 4. If score < threshold, retry with feedback ("Constraint X failed because Y -- regenerate") 5. Return the best-scoring attempt On IFEval (25 instruction-following constraint types): 69.3% -> 76.2% accuracy. 278 lines. Zero dependencies. Works with any callable that takes a string and returns a string. pip install ai-assert https://github.com/kaantahti/ai-assert
Share cardActual performance
Launch Intel predictions
Analyze your own launch →Correct prediction on native model
Similar products
Hangman in 3 lines of Python
A very simple Lisp interpreter in 250 lines of literate Python
Retirement Calculator in 8 lines of Python
Querying 50k Gameservers in 100 lines of Python
A Python module to steer LLM outputs
Build python ASTs in 502 lines of python without using modules
FuzzyFinder – in 10 lines of Python
2048 in ten 60-width lines of python
Promptic – LLM superpowers for your Python functions (in ~80 lines)
Annotate any tactile sensor in 3 lines of Python.