Li

Librarian a Modern Kafka Connect Toolkit

Hacker News

Librarian a Modern Kafka Connect Toolkit

Hello Everyone. I'm working on improving Kafka Connect. Kafka connect lacks a lot of data visibility, making it difficult to use and operate as a backbone of data pipelines. We're trying to create a kafka connect for modern data. Right now librarian only supports "Snapshoting", but during winter break I'm going to start hacking on the Streaming replication component. This first version of librarian can snapshot postgres tables and save them as parquet. Although duckdb offers this feature too, librarian provides enhanced data observability through its snapshot "catalog". The catalog provides an inventory of the snapshot including duration, source counts and target counts. ------ Do you use Kafka connect regularly? What do you use it for? What would you change about it? What works with kafka connect? What's challenging? Thank you all, Hopefully in a couple weeks I have a more mature product to show!

Visit product View on Hacker NewsDec 16, 2024

Actual performance

1points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Indie HackersFits the IH revenue-focused audience · Strong signals: supports, including · Missing: reddit linkedin, podcasting, created

71%71% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Product HuntOn track for Day 1 leaderboard · Missing: mac, agents, macos

59%59% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsMay not resonate with HN audience · Strong signals: ide, pipe, io · Missing: https docs, excited, just released

46%46% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

42%42% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

27%27% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

14%14% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Rize, a functional toolkit for Ruby48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Rize, a functional toolkit for Ruby

Adventurekeep – A Toolkit for D&D49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Adventurekeep – A Toolkit for D&D

Roppy – A Pwning Toolkit49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Roppy – A Pwning Toolkit

Hexagon Microservices Toolkit (For Kotlin)47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Hexagon Microservices Toolkit (For Kotlin)

I wrote a WebAssembly Interpreter and Toolkit in C71%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

I wrote a WebAssembly Interpreter and Toolkit in C

Go Bindings for Roc Toolkit41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Go Bindings for Roc Toolkit

Rill – Composable concurrency toolkit for Go70%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Rill – Composable concurrency toolkit for Go

OmniTools35%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

All-in-One Revolutionary Toolkit

Indie Hackers1apis

Utilly46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Your All In One Toolkit

Indie Hackerscommitment-side-project

Librarian: A Modern Alternative to Kafka Connect79%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Librarian: A Modern Alternative to Kafka Connect

Browse all products Analyze your launch