We're building Apache spark for agents with Rust and Datafusion

Hacker News

Hi HN, BT here. We are trying to build a data platform that's dedicated to serve agents, so we call it the apache spark for agents. The reason why we think we need a new data platform for agents is that we think existing platform is restricting how agents are utilizing data, and we think we should provide agents full autonomy on how to use data and what data to use, and it should have a better leverage for agent's ability. We chose datafusion because it's great extensibility and performance, so our data platform can connect with various different data sources. If you find this project interesting and want to give a try, you can start with the auto knowledge base skill: https://github.com/SkardiLabs/skardi-skills/tree/main/auto_k... which can help you turn your documents into a instant knowledge base that local agents like openClaw or Hermes can ingest instantly. Anyways, happy to answer any questions.

Visit product View on Hacker NewsApr 23, 2026

Share card

Actual performance

2points

1comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: agents, agent, new · Missing: mac, macos, cursor

92%92% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

65%65% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

46%46% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Hacker NewsMay not resonate with HN audience · Strong signals: exist, existing, ide · Missing: https docs, excited, just released

44%44% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: platform · Missing: plus, intuitive, reviews

39%39% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

14%14% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Spark Rewrite in Rust63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Spark Rewrite in Rust

Hacker News4

StatsD in Rust51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

StatsD in Rust

Hacker News1

TF-IDF in Rust51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

TF-IDF in Rust

Hacker News1

Rust FFI in a Wireshark C dissector51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Rust FFI in a Wireshark C dissector

Hacker News1

Rust-wasm-webpack50%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Rust-wasm-webpack

Hacker News6

Risp – Lisp in Rust68%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Risp – Lisp in Rust

Hacker News314

Classical Ciphers in Rust51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Classical Ciphers in Rust

Hacker News2

Screencast: An Introduction to Rust57%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Screencast: An Introduction to Rust

Hacker News3

Handlebars for Rust51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Handlebars for Rust

Hacker News2

Oxischeme – A Scheme implementation in Rust56%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Oxischeme – A Scheme implementation in Rust

Hacker News18

Browse all products Analyze your launch