Manage LLM providers while factoring in Cost and Speed

Hacker News

Hey y'all! We built this framework for ourselves internally and decided to open source it. It's really similar to open router but allows you to do a couple more things 1. Allows you to factor in 'speed'. If you want your API call to go as fast as possible regardless of cost you can specify that. Otherwise it'll default to the cheapest available provider 2. Allows you to pass in a 'validator' function and 'fallback' models. This way you can ensure the response you get is valid according to your own internal logic 3. Uses the OpenAPI content/role format to interact with models so you can quickly swap between them while you test. We've found a bunch of use cases for this ( https://yc-bot.lytix.co/ , https://notes.lytix.co ) and it's allowed us to move much faster since we don't have to think about credentials between all our providers. Hope you enjoy it and would love to get any feedback on the project

Visit product View on Hacker NewsJul 22, 2024

Share card

Actual performance

3points

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Hacker NewsStrong engagement from HN community · Strong signals: open source, ide, io · Missing: https docs, excited, just released

73%73% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

Product HuntOn track for Day 1 leaderboard · Strong signals: model, models, notes · Missing: mac, agents, macos

70%70% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

70%70% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: way · Missing: mobile apps, ios, personal

47%47% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Missing: plus, platform, intuitive

45%45% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

14%14% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Missing: web3, chat, crypto

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

Django-LLM (manage langchain in Django)38%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Django-LLM (manage langchain in Django)

Hacker News1

A Stateful LLM API that works across providers54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A Stateful LLM API that works across providers

Hacker News1

Open LLM Spec – Standardizing inputs and outputs across providers32%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open LLM Spec – Standardizing inputs and outputs across providers

Hacker News1

Ozempic Providers44%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Compare telehealth providers for Ozempic,Wegovy & GLP-1 meds

Indie Hackerscommitment-side-project

A reliability layer that prevents LLM downtime and unpredictable cost39%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A reliability layer that prevents LLM downtime and unpredictable cost

Hacker News1

Best iptv9809031%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

ALL Providers tested

Indie Hackers

Manage clinics and hospitals with power and speed40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Manage clinics and hospitals with power and speed

Hacker News1

Neuralrouting.io21%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

The Intelligent LLM Gateway: Route for Cost & Speed.

Indie Hackers1ai

infracost - Cost Estimation for Terraform35%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

infracost - Cost Estimation for Terraform

Hacker News2

DripStat – Full Java APM at 1/10th the Cost of NewRelic40%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

DripStat – Full Java APM at 1/10th the Cost of NewRelic

Hacker News9

Browse all products Analyze your launch