Otlet – Local LLM inference "inside" Postgres
Otlet – Local LLM inference "inside" Postgres
I’ve been working on Otlet, a Postgres extension for running local LLM inference “inside” Postgres (inference runs in a background worker, while Postgres stores the jobs, inputs, outputs, and receipts) I started it while working on an entity resolution problem (determining which rows in a table refer to the same entity). The data was already in Postgres, but using a model meant pulling rows into another process and then writing the results back. I wanted to see whether I could keep more of that workflow in the database right next to where the data lives Right now Otlet lets you start model work from SQL, require structured output, keep a record of each run, review proposed writes before they touch source data, etc. The example in the repo uses a small model for the first pass and a stronger model for uncertain cases This is just something I've been hacking on, not something I would recommend attempting to use in production yet. There’s a Docker demo if anyone wants to try it. I’m mostly posting it because I wanted to share the approach while it’s still rough to see it get ripped to shreds
Share cardActual performance
Launch Intel predictions
Analyze your own launch →Incorrect prediction on native model
Similar products
Onera – Private LLM Inference Inside AMD SEV-SNP Enclaves
LLM Inference Requirements Profiler
BonzAI – self-sovereign, local LLM inference in the browser
PgModeler – EER Diagramming for postgres
Merging RethinkDB and Postgres with GraphQL
First experiment wih Vertx,Vue,Postgres
Logchain – ClickHouse/Postgres gRPC hook for logrus
pglockanalyze - See the locks your Postgres migrations will acquire
Now supporting MySQL, Postgres, Redis, Mongo
GraphQL for Postgres