Re

Reconstruct distributed LLM training traces

Hacker News

Reconstruct distributed LLM training traces

When we train large language models, there are a lot of systems challenges and different sharding schemes one can use. While there are many great resources on scaling LLMs out there ( https://huggingface.co/spaces/nanotron/ultrascale-playbook or https://jax-ml.github.io/scaling-book/ ), I felt like there was still a gap when it comes to visualising different forms of parallelism and building intuition around overlaps and execution order for a distributed training run The idea is to make it easier to visualise FSDP/Tensor Parallel/Expert Parallel/Context parallel and reason about it - you can drag and drop compute kernels and collectives to create DDP/TP/FSDP/EP/CP traces based on real torchtitan profiles.

Share card

Actual performance

2points
Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →
Product HuntOn track for Day 1 leaderboard · Strong signals: model, models, context · Missing: mac, agents, macos
84%84% predicted probability of success on Product Hunt, based on ML models trained on real launch data.
best fitHighest predicted score across all platforms for this description.
Indie HackersFits the IH revenue-focused audience · Strong signals: para · Missing: supports, reddit linkedin, podcasting
70%70% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.
Hacker NewsStrong engagement from HN community · Strong signals: ide, io · Missing: https docs, excited, just released
66%66% predicted probability of success on Hacker News, based on ML models trained on real launch data.
nativeThis product was originally launched on this platform.
TrustMRRLess likely to generate early MRR · Strong signals: para · Missing: mobile apps, ios, personal
42%42% predicted probability of success on TrustMRR, based on ML models trained on real launch data.
AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive
31%31% predicted probability of success on AppSumo, based on ML models trained on real launch data.
Acquire.comPre-revenue stage for this audience · Strong signals: training · Missing: arr, mrr, revenue
27%27% predicted probability of success on Acquire.com, based on ML models trained on real launch data.
BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto
0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Js
Jsonnet Training42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Jsonnet Training

Hacker News1
Ea
Ear Training Exercise63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Ear Training Exercise

Hacker News2
Kibana Training
Kibana Training35%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

kibana training, kibanaonline training, kibana tutorial

Indie Hackerscommitment-full-time
Pi
PilotBuddy – AI for Pilots in Training34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PilotBuddy – AI for Pilots in Training

Hacker News1
Ne
New LLM Pre-Training and Post-Training Paradigms29%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

New LLM Pre-Training and Post-Training Paradigms

Hacker News2
Re
Redis-LLM – Redis module integrates LLM with Redis45%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Redis-LLM – Redis module integrates LLM with Redis

Hacker News2
Li
LitLLM the Spiciest LLM Wrapper34%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LitLLM the Spiciest LLM Wrapper

Hacker News1
LL
LLM Reasonsers46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM Reasonsers

Hacker News2
He
Hegelion – Force your LLM to argue with itself before answering58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Hegelion – Force your LLM to argue with itself before answering

Hacker News1
LLM Hotkey
LLM Hotkey39%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.
TrustMRROther