Nasty, a cross-warehouse, type-checked, unit-testable analytics library

Hacker News

Hey HN - our team wants to open source a project called NASTY (NASTY Abstract Syntax Tree thingY) that we built for ourselves. NASTY was built to maintain testable/composable data pipelines. Our team was ripping our hair out trying to maintain dbt/SQL scripts across different data warehouses (Redshift, BigQuery, Postgres, Snowflake) on top of ever shifting data foundations maintained by our customer's internal data teams. NASTY is the result of our learnings from field experience. We wanted to write abstractions so that we could reuse code. We wanted to bundle those abstractions into libraries. We wanted to statically analyze our models so that we caught more errors before production. We wanted a fast unit test suite that you could iterate on locally without connecting to a data warehouse. In short, we wanted to use all the same practices we used for building our other software. Tools like dbt made a great start at importing these kinds of practices into analytics, but there are many great aspects of the software engineering workflow that still aren't easy to replicate in data. We've found it to be a super productive way to work, and we thought others might want to use it too. It's very early days so we've put up a page for to explain it, and we've made some executable examples for people to have a go. Let us know what you think! FAQ: Is NASTY an ORM? A SQL builder? Neither! NASTY is kinda it's own thing. NASTY is more like a minimal relational algebra programming language shipped as a Typescript library. It borrows a bunch of learnings from other programming languages and applies them to OLAP programming / data engineering. It's an important distinction, because it totally changes the way you can think about building transformations. Why TypeScript? Why not Python? The 100% honest answer is we had a ton of typescript experience and we had a node/react app with pulumi's TS apis. We like typescript and by being a library instead of a full programming language, we can leverage things like vitest, eslint, node, TS's language server, TSdoc, etc.

Visit product View on Hacker NewsMar 8, 2024

Share card

Actual performance

47points

6comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, models, code · Missing: mac, agents, macos

90%90% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

89%89% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: open source, pipe, io · Missing: https docs, excited, just released

74%74% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

39%39% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: builder · Missing: plus, platform, intuitive

23%23% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

15%15% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Analytics44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Pluggable analytics library

Indie Hackers1analytics

Promises – Type safe promise library for go41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Promises – Type safe promise library for go

Hacker News7

A C# library to help you enforce a Given-When-Then structured Unit test32%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A C# library to help you enforce a Given-When-Then structured Unit test

Hacker News24

Unit Converion73%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Simple unit conversion tool

Indie Hackers1$130/moadvertising

Giggly Squad Allegedly Unit Unisex T-Shirt76%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

“For people who are allegedly in the Giggly Squad.”

Indie Hackers1$25/mob2c

Libenvpp - A modern C++ library for type-safe env variable parsing33%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Libenvpp - A modern C++ library for type-safe env variable parsing

Hacker News1

GMP Deserialization Type Confusion Vulnerability [MyBB43%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GMP Deserialization Type Confusion Vulnerability [MyBB

Hacker News1

Co-Occurrence of Type in Pokémon45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Co-Occurrence of Type in Pokémon

Hacker News1

Type someone's name and find out how to pronounce it45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Type someone's name and find out how to pronounce it

Hacker News3

a type-agnostic wrapper of leveldb for Go26%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

a type-agnostic wrapper of leveldb for Go

Hacker News3

Browse all products Analyze your launch