SearchArray – Text Search in Pandas

Hacker News

I've long worked with lexical Lucene based search engines like Solr and Elasticsearch. Anytime I need to experiment with relevance ranking in these systems, I'm exhausted by needing to set them up. It's not conducive to productivity work with something so disjoint from normal data tooling. Further - the underlying ranking is buried in needless mystique (you know a boolean should query, sums the scores, right?). You shouldn't need to read a book (like Relevant Search ;) ) to unpack mystique that's really basic math. Why not just let people build ranking systems with vectorized math in a numpy/pandas stack? SearchArray lets anyone build a relevance prototype in Pandas. Typically building / experimenting with a smaller labeled dataset. If it works out, you can transfer it relatively easily to Elasticsearch or Solr for implementation and testing. SearchArray is a pandas extension array that creates an underlying search index for BM25 term/phrase based searching. It's not quite done (will it ever be?) but its getting far enough along to be useful. So feedback is very welcome. https://github.com/softwaredoug/searcharray

Visit product View on Hacker NewsNov 19, 2023

Share card

Actual performance

5points

2comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

73%73% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Product HuntOn track for Day 1 leaderboard · Missing: mac, agents, macos

71%71% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

68%68% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

37%37% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

31%31% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Strong signals: arr · Missing: mrr, revenue, profit

16%16% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

RediSearch, a Full-Text Search Redis Module54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

RediSearch, a Full-Text Search Redis Module

Hacker News9

Full-Text search in your Dropbox63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Full-Text search in your Dropbox

Hacker News4

Search London StreetView panoramas by text54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Search London StreetView panoramas by text

Hacker News39

Claid73%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Find video scenes through text search

Product Hunt+3

Search the Pile: An 800GB Dataset of Diverse Text for Language Modeling43%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Search the Pile: An 800GB Dataset of Diverse Text for Language Modeling

Hacker News2

Ruby on Rails' AREL Full-Text Search Support52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Ruby on Rails' AREL Full-Text Search Support

Hacker News1

Full text search Project Gutenberg (60m paragraphs)64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Full text search Project Gutenberg (60m paragraphs)

Hacker News202

Cirrina, minimum span text search library68%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Cirrina, minimum span text search library

Hacker News1

Full text search on 24M filehoster links61%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Full text search on 24M filehoster links

Hacker News11

An NSFW Tumblr Search60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

An NSFW Tumblr Search

Hacker News45

Browse all products Analyze your launch