A surprisingly effective way to predict token importance in LLM prompts

Hacker News

We explored a novel method to gauge the significance of tokens in prompts given to large language models, without needing direct model access. Essentially, we just did an ablation study on the prompt using cosine similarity of the embeddings as the measure. We got surprisingly promising results when comparing this really simple approach to integrated gradients. Curious to hear thoughts from the community!

Visit product View on Hacker NewsSep 11, 2023

Share card

Actual performance

14points

6comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, models, using · Missing: mac, agents, macos

74%74% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

69%69% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: io · Missing: https docs, excited, just released

52%52% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoStrong fit for a featured deal · Missing: plus, platform, intuitive

52%52% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRFits verified-revenue profile · Strong signals: way · Missing: mobile apps, ios, personal

52%52% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

14%14% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

5%5% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Recursive LLM Prompts53%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Recursive LLM Prompts

Hacker News97

Wordllama – Things you can do with the token embeddings of an LLM63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Wordllama – Things you can do with the token embeddings of an LLM

Hacker News370

+Prompter63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Your motivational prompts. Your way.

Indie Hackers1health-fitness

Beeminder for Effective Charities63%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Beeminder for Effective Charities

Hacker News4

The SC4-HSM is now a FIDO U2F token46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The SC4-HSM is now a FIDO U2F token

Hacker News3

Jwt.show – show the payload of a jwt token46%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Jwt.show – show the payload of a jwt token

Hacker News1

CryptoTokens.wtf – Token Whitepapers from Highest Grossing ICOs42%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

CryptoTokens.wtf – Token Whitepapers from Highest Grossing ICOs

Hacker News1

Token Thermodynamics52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Token Thermodynamics

Hacker News4

Tokensift, an open-sourced token-efficiency linter for LLM prompts52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Tokensift, an open-sourced token-efficiency linter for LLM prompts

Hacker News6

Predict Then Propagate, ICLR 2019 (PyTorch)44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Predict Then Propagate, ICLR 2019 (PyTorch)

Hacker News2

Browse all products Analyze your launch