Quack-Cluster – A serverless distributed SQL engine with DuckDB and Ray

Hacker News

Hi HN, I'm excited to share a project I've been working on: Quack-Cluster. I love the speed and simplicity of DuckDB for analytics, but I often work with datasets spread across hundreds of files in object storage (like S3). I wanted a way to run distributed queries across all that data without the complexity of setting up and managing a full-blown Spark or Presto cluster. I'm also a big fan of Ray for its simplicity in distributed Python, so I decided to combine them. How it works: You send a standard SQL query to a central coordinator. It uses SQLGlot to parse the query and identify the target files (e.g., s3://bucket/data/*.parquet). It then generates a distributed plan and sends tasks to a cluster of Ray actors. Each Ray actor runs an embedded DuckDB instance to process a subset of the files in parallel. The partial results (as Arrow tables) are then aggregated and returned to the user. The goal is to provide a lightweight, high-performance, and serverless alternative for interactive SQL analytics directly on a data lake. The core tech stack is: Backend: Python, FastAPI Distributed Computing: Ray Query Engine: DuckDB SQL Parsing: SQLGlot The project is open-source and I've tried to make it easy to get started locally with Docker and make. I'm here to answer any questions and would be grateful for any feedback on the architecture, use case, or the code itself. Thanks for checking it out!

Visit product View on Hacker NewsJul 21, 2025

Share card

Actual performance

3points

3comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: user, dock, tasks · Missing: mac, agents, macos

93%93% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: started, para · Missing: supports, reddit linkedin, podcasting

72%72% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: excited, ide, io · Missing: https docs, just released, exist

69%69% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: way, para · Missing: mobile apps, ios, personal

38%38% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

29%29% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Strong signals: arr, active · Missing: mrr, revenue, profit

17%17% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

WhatTheDuck – Browser-Based SQL Engine Powered by DuckDb on WASM73%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

WhatTheDuck – Browser-Based SQL Engine Powered by DuckDb on WASM

Hacker News11

GizmoEdge – Distributed IoT SQL Engine75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GizmoEdge – Distributed IoT SQL Engine

Hacker News4

Typed overlay over SQL now supports DuckDB60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Typed overlay over SQL now supports DuckDB

Hacker News4

GizmoSQL – Run DuckDB as a Server with Arrow Flight SQL71%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GizmoSQL – Run DuckDB as a Server with Arrow Flight SQL

Hacker News17

GizmoSQL58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run DuckDB as a server with Arrow Flight SQL!

Product Hunt+15

Serverless GraphQL78%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Serverless GraphQL

Hacker News14

Go DDOS Yourself (serverless-artillery)58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Go DDOS Yourself (serverless-artillery)

Hacker News2

FaaS-netes – Porting Serverless FaaS to K8s63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

FaaS-netes – Porting Serverless FaaS to K8s

Hacker News5

Serverless sorcery with ImageMagick62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Serverless sorcery with ImageMagick

Hacker News1

Shortly, a serverless pastebin59%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Shortly, a serverless pastebin

Hacker News58

Browse all products Analyze your launch