Why Don't Data Engineers Write Tests?

Hacker News

I asked this same question last week in the r/dataengineering subreddit and got some interesting answers: - Unit/Integration testing is often dropped because of tight deadlines and low perceived business impact. - It helps, but they're not a full solution for data quality. - Many Data Engineers don't have a Software Engineering background, so common practices like testing often aren't applied. - Creating multiple input/output tables with realistic synthetic data is complicated. - Tooling and frameworks for testing data transformations are still pretty limited. I'd like to hear Hacker News's perspective on this. I also built an open-source toolkit/framework called Pybujia to help with some of these pains: https://github.com/jpgerek/pybujia/

Visit product View on Hacker NewsSep 28, 2025

Share card

Actual performance

4points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Hacker NewsStrong engagement from HN community · Strong signals: hacker news, io · Missing: https docs, excited, just released

74%74% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

65%65% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Product HuntOn track for Day 1 leaderboard · Strong signals: new, open · Missing: mac, agents, macos

54%54% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: answers · Missing: mobile apps, ios, personal

49%49% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

31%31% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

16%16% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Python Tests That Write Themselves44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Python Tests That Write Themselves

Hacker News131

CrashBreak – Reproduce exceptions as failing tests in Ruby31%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

CrashBreak – Reproduce exceptions as failing tests in Ruby

Hacker News37

PANIC – Distributed Jepsen Tests for Everyone49%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PANIC – Distributed Jepsen Tests for Everyone

Hacker News1

Module for conditionally skipping Mocha tests21%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Module for conditionally skipping Mocha tests

Hacker News1

Documentaion Tests with Vitest30%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Documentaion Tests with Vitest

Hacker News2

Unicorn reporter for AngularJS tests32%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Unicorn reporter for AngularJS tests

Hacker News1

Mocha tests from Vim38%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Mocha tests from Vim

Hacker News1

Bluefin – Groovy DSL for Selenium tests57%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Bluefin – Groovy DSL for Selenium tests

Hacker News1

Find empty tests in your suites30%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Find empty tests in your suites

Hacker News3

DevScreen44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Use realistic tests to hire engineers

Indie Hackers1jobs-hiring

Browse all products Analyze your launch