Sycamore – an LLM-powered semantic data preparation system for search

Hacker News

We’re Aryn and yesterday we came out of stealth: blog.aryn.ai. As part of that, we released Sycamore: https://github.com/aryn-ai/sycamore . Sycamore is an LLM-powered semantic data preparation system for building search applications. It introduces a distributed set-based abstraction, a DocSet, that makes processing a large document collection as easy as reading a single document. Sycamore makes it easy to use LLMs to transform and enrich your unstructured data and prepare it for search. It comes with a scalable distributed runtime, built on Ray, that makes it easy to go from prototype to production. For example, with Sycamore, you can read a collection of PDFs, partition them in coherent chunks, pull out entities like titles and authors, compute vector embeddings, and load them into a local OpenSearch cluster. All with a few lines of code. To learn more, visit the repo: https://github.com/aryn-ai/sycamore , docs: https://sycamore.readthedocs.io/ , and demo: https://www.loom.com/share/53e68b0eb5ab49948111a3fcf6286b7f?... We’d love for you to try it out, give us feedback, and contribute.

Visit product View on Hacker NewsSep 29, 2023

Share card

Actual performance

18points

1comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: single, code, open · Missing: mac, agents, macos

81%81% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

77%77% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

47%47% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Indie HackersIH features products with proven revenue · Strong signals: para · Missing: supports, reddit linkedin, podcasting

47%47% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: para · Missing: mobile apps, ios, personal

28%28% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

13%13% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: introduce · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Semantic programmatic search for computable data60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Semantic programmatic search for computable data

Hacker News24

Biblos – Semantic Search the Church Fathers55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Biblos – Semantic Search the Church Fathers

Hacker News6

Semantic Grep – A Word2Vec-powered search tool57%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Semantic Grep – A Word2Vec-powered search tool

Hacker News356

Semantic Video Search62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Semantic Video Search

Hacker News62

Semantic search for video36%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Semantic search for video

Hacker News77

Queryra47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

AI-powered semantic search API

Indie Hackers1ai

Bumblebee49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Data preparation tool for Data Scientists

Indie Hackers3ai

Codesearch.ai – Semantic Code Search49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Codesearch.ai – Semantic Code Search

Hacker News15

ModelMint26%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Semantic Search made stupidly simple

Indie Hackerscommitment-side-project

PeopleMesh, Semantic Search for People68%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PeopleMesh, Semantic Search for People

Hacker News2

Browse all products Analyze your launch