Chinchilla Scaling Laws Are Not Universal

Hacker News

Hey HN! Chinchilla (DeepMind 2022) tells us that when we scale up our language model training, we should scale the parameters and data equally. Over the last several months I've been hacking on a research project to determine if the optimal compute allocation (scaling law) for training an LLM is sensitive to training data complexity. I found that as data complexity increases, you need even more data than Chinchilla suggests! I released the preprint just yesterday: https://arxiv.org/abs/2405.16684

Visit product View on Hacker NewsMay 28, 2024

Share card

Actual performance

1points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model · Missing: mac, agents, macos

83%83% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: para · Missing: supports, reddit linkedin, podcasting

78%78% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

59%59% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Strong signals: month, para · Missing: mobile apps, ios, personal

44%44% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Strong signals: training · Missing: arr, mrr, revenue

31%31% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

21%21% predicted probability of success on AppSumo, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

4%4% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

utt – Universal Text Transformer54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

utt – Universal Text Transformer

Hacker News127

Erdus17%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Universal converter

Product Hunt+7

Universal Dashboard 2.156%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Universal Dashboard 2.1

Hacker News1

Universal APIs for Xamarin54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Universal APIs for Xamarin

Hacker News2

Prybar, a universal interpreter front-end63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Prybar, a universal interpreter front-end

Hacker News4

Universal Underlined Text Generator51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Universal Underlined Text Generator

Hacker News5

Experimental Universal Translator 0.0.154%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Experimental Universal Translator 0.0.1

Hacker News1

Kiosk – Universal Shopping Cart on Ethereum52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Kiosk – Universal Shopping Cart on Ethereum

Hacker News4

UBPE – a universal BPE tokenizer, optimized and rethought54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

UBPE – a universal BPE tokenizer, optimized and rethought

Hacker News1

InstantParcels68%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Universal parcel tracking platform

Indie Hackers1e-commerce

Browse all products Analyze your launch