Po

Postgres extension for BM25 relevance-ranked full-text search

Hacker News

Postgres extension for BM25 relevance-ranked full-text search

Last summer we faced a conundrum at my company, Tiger Data, a Postgres cloud vendor whose main business is in timeseries data. We were trying to grow our business towards emerging AI-centric workloads and wanted to provide a state-of-the-art hybrid search stack in Postgres. We'd already built pgvectorscale in house with the goal of scaling semantic search beyond pgvector's main memory limitations. We just needed a scalable ranked keyword search solution too. The problem: core Postgres doesn't provide this; the leading Postgres BM25 extension, ParadeDB, is guarded behind AGPL; developing our own extension appeared daunting. We'd need a small team of sharp engineers and 6-12 months, I figured. And we'd probably still fall short of the performance of a mature system like Parade/Tantivy. Or would we? I'd be experimenting long enough with AI-boosted development at that point to realize that with the latest tools (Claude Code + Opus) and an experienced hand (I've been working in database systems internals for 25 years now), the old time estimates pretty much go out the window. I told our CTO I thought I could solo the project in one quarter. This raised some eyebrows. It did take a little more time than that (two quarters), and we got some real help from the community (amazing!) after open-sourcing the pre-release. But I'm thrilled/exhausted today to share that pg_textsearch v1.0 is freely available via open source (Postgres license), on Tiger Data cloud, and hopefully soon, a hyperscalar near you: https://github.com/timescale/pg_textsearch In the blog post accompanying the release, I overview the architecture and present benchmark results using MS-MARCO. To my surprise, we were not only able to meet Parade/Tantivy's query performance, but exceed it substantially, measuring a 4.7x advantage on query throughput at scale: https://www.tigerdata.com/blog/pg-textsearch-bm25-full-text-... It's exciting (and, to be honest, a little unnerving) to see a field I've spent so much time toiling in change so quickly in ways that enable us to be more ambitious in our technical objectives. Technical moats are moats no longer. The benchmark scripts and methodology are available in the github repo. Happy to answer any questions in the thread. Thanks, TJ (tj@tigerdata.com)

Share card

Actual performance

203points
56comments
Made the leaderboard

Launch Intel predictions

Analyze your own launch →
Indie HackersFits the IH revenue-focused audience · Strong signals: para · Missing: supports, reddit linkedin, podcasting
90%90% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.
best fitHighest predicted score across all platforms for this description.
Product HuntOn track for Day 1 leaderboard · Strong signals: claude, using, code · Missing: mac, agents, macos
85%85% predicted probability of success on Product Hunt, based on ML models trained on real launch data.
Hacker NewsStrong engagement from HN community · Strong signals: open source, ide, io · Missing: https docs, excited, just released
70%70% predicted probability of success on Hacker News, based on ML models trained on real launch data.
nativeThis product was originally launched on this platform.
AppSumoMay struggle as an AppSumo deal · Strong signals: soon · Missing: plus, platform, intuitive
40%40% predicted probability of success on AppSumo, based on ML models trained on real launch data.
TrustMRRLess likely to generate early MRR · Strong signals: month, way, para · Missing: mobile apps, ios, personal
37%37% predicted probability of success on TrustMRR, based on ML models trained on real launch data.
Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue
18%18% predicted probability of success on Acquire.com, based on ML models trained on real launch data.
BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto
0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Pg
Pg_CRDT – an experimental CRDT extension for Postgres62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Pg_CRDT – an experimental CRDT extension for Postgres

Hacker News296
BS
BSON Extension for Postgres44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

BSON Extension for Postgres

Hacker News115
Pg
Pg_stat_ch, a Postgres extension to export every metric to ClickHouse63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Pg_stat_ch, a Postgres extension to export every metric to ClickHouse

Hacker News4
Fu
Full Text Search Using Postgres for Startups72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Full Text Search Using Postgres for Startups

Hacker News3
Fi
Five Thousand Novels, Ranked by Vividness43%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Five Thousand Novels, Ranked by Vividness

Hacker News54
Pg
Pg_session_JWT, a Postgres Extension for JWT Sessions42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Pg_session_JWT, a Postgres Extension for JWT Sessions

Hacker News5
Zo
ZomboDB – Postgres extension for indexes backed by Elasticsearch66%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

ZomboDB – Postgres extension for indexes backed by Elasticsearch

Hacker News88
Pg
PgModeler – EER Diagramming for postgres73%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PgModeler – EER Diagramming for postgres

Hacker News9
No
Now supporting MySQL, Postgres, Redis, Mongo60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Now supporting MySQL, Postgres, Redis, Mongo

Hacker News3
Gr
GraphQL for Postgres83%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GraphQL for Postgres

Hacker News195