Phidata – Building Blocks for Data Engineering

Hacker News

Hi HN, I’ve been a data engineer for over a decade and have slowly been working on a way for 1 person to run a data platform (mostly to automate my job) When I was starting, I found the wide range of tools required to build data products overwhelming. Infrastructure setup, app deployment, pipeline development, creating metrics, dashboards and finally, integrating ML into product. No wonder companies have 100s of people working on this. So last year I quit the best job in the world to figure this out. I’d like to introduce Phidata: Building Blocks for Data Engineering It works like this: 1. You start with a codebase that has common data tools like Airflow, Superset and Jupyter pre-configured. Infrastructure and Apps are defined as python objects. 2. Build data products (tables, metrics, models) in python or SQL. Test locally using docker and run production on AWS. 3. Infrastructure, Apps and Data Products live in the same codebase. Teams working together share code and dependencies in a pythonic way. Using phidata, I’ve been running multiple data platforms and have automated most of my boilerplate code using a specially trained GPT-3 data assistant. If you work with data and are looking for a better development experience, checkout [phidata.com]( https://www.phidata.com/ ) or message me, I’d love your feedback. Ashpreet

Visit product View on Hacker NewsJan 23, 2023

Share card

Actual performance

16points

4comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, apps, dock · Missing: mac, agents, macos

93%93% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Hacker NewsStrong engagement from HN community · Strong signals: ide, pipe, io · Missing: https docs, excited, just released

76%76% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

74%74% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: apps, way · Missing: mobile apps, ios, personal

44%44% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: platform · Missing: plus, intuitive, reviews

39%39% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

14%14% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: introduce · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Remotion Bits30%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Building blocks for your videos

Product Hunt+3

Mu – Building Blocks for Life63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Mu – Building Blocks for Life

Hacker News1

Wordcraft40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The Building Blocks of Language

Indie Hackerscommitment-full-time

Openexus – Building blocks for the internet80%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Openexus – Building blocks for the internet

Hacker News38

Anzu41%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Building blocks for software teams

Indie Hackerscommitment-side-project

Casters56%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The building blocks for building (real-time) experiences.

Indie Hackerscommitment-side-project

Thi.ng – open-source building blocks for computational design69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Thi.ng – open-source building blocks for computational design

Hacker News107

HandCalc – A modern tool for building engineering calcs67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

HandCalc – A modern tool for building engineering calcs

Hacker News5

Missing Building-Blocks for the Agent World53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Missing Building-Blocks for the Agent World

Hacker News1

Tutorilla66%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Your Building Blocks for Better Tutorials

Indie Hackerscommitment-side-project

Browse all products Analyze your launch