Llama 3.2 Interpretability with Sparse Autoencoders

Hacker News

I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1] https://transformer-circuits.pub/2024/scaling-monosemanticit... [2] https://arxiv.org/abs/2406.04093 [3] https://arxiv.org/abs/2408.05147

Visit product View on Hacker NewsNov 21, 2024

Share card

Actual performance

579points

99comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

85%85% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Product HuntOn track for Day 1 leaderboard · Strong signals: new, openai, code · Missing: mac, agents, macos

75%75% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: llama, ide, io · Missing: https docs, excited, just released

75%75% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRLess likely to generate early MRR · Missing: mobile apps, ios, personal

50%50% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

35%35% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

16%16% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

1%1% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Llama or Alpaca?78%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama or Alpaca?

Hacker News6

Llama 2 Uncensored 70B as API79%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama 2 Uncensored 70B as API

Hacker News18

Finetune Llama-3.1 2x faster in a Colab74%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Finetune Llama-3.1 2x faster in a Colab

Hacker News16

Finetune Llama 3.2 Vision in a Colab75%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Finetune Llama 3.2 Vision in a Colab

Hacker News10

TokenHawk, WebGPU Running LLaMA64%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

TokenHawk, WebGPU Running LLaMA

Hacker News2

Llama Running on a Microcontroller76%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama Running on a Microcontroller

Hacker News49

Llama list – todolists are better with a llama companion67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama list – todolists are better with a llama companion

Hacker News3

Python Bindings for llama.cpp with some CLIs60%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Python Bindings for llama.cpp with some CLIs

Hacker News2

Run Llama 3.1 8B in the browser78%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run Llama 3.1 8B in the browser

Hacker News23

Llama 3.3 70B Sparse Autoencoders with API access76%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Llama 3.3 70B Sparse Autoencoders with API access

Hacker News201

Browse all products Analyze your launch