Helix apply -f deploy YAML GenAI with RAG and APIs on local open models [video]

Hacker News

Demo starts at 50m into the video. This was a bit terrifying to record because 2am the previous night everything was totally broken after a major refactor (so that we could add external LLM support as well as local GPUs). But pressure can be a useful force :-D We start with a stack deployed on my laptop without a GPU, pointing to together.ai so we can run open source LLMs easily without having to have access to a GPU. We show simple inference through the ChatGPT-like web interface (with users, sessions etc) and then simple drag'n'drop RAG. Then we show some helix apps defined as yaml: Marvin the Paranoid Android (just a system prompt on top of llama3:8b), an HR app that interacts with an API, and a surprise API integration I'd done that morning with the podcast host's own OpenAPI spec for their app Screenly. Finally, we deploy it for real on a DigitalOcean droplet for the controlplane - see https://docs.helix.ml/helix/getting-started/architecture/ - and a $0.35/h A40 on runpod.io. Armed with a real GPU, we can do image inference and fine-tuning as well as all the things described above!

Visit product View on Hacker NewsAug 26, 2024

Share card

Actual performance

8points

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: model, apps, user · Missing: mac, agents, macos

95%95% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Strong signals: started, para · Missing: supports, reddit linkedin, podcasting

73%73% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: open source, llama, ide · Missing: https docs, excited, just released

60%60% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

TrustMRRFits verified-revenue profile · Strong signals: apps, video, users · Missing: mobile apps, ios, personal

60%60% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

AppSumoMay struggle as an AppSumo deal · Strong signals: host, interface, users · Missing: plus, platform, intuitive

34%34% predicted probability of success on AppSumo, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

25%25% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

2%2% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

Smart website search powered by open models58%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Smart website search powered by open models

Hacker News13

GenAi Yule Lights50%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

GenAi Yule Lights

Hacker News1

Orchestra – Deploy machine learning models as APIs in minutes59%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Orchestra – Deploy machine learning models as APIs in minutes

Hacker News1

Open-source RAG App [video]54%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source RAG App [video]

Hacker News1

MiniCPM5-1B82%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

A new SOTA for compact open models on the edge

Product Hunt+96Open Source

Jiss – A community-powered LLM API I built for open models62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Jiss – A community-powered LLM API I built for open models

Hacker News1

Vectorless RAG48%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Vectorless RAG

Hacker News11

PageIndex – Vectorless RAG66%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

PageIndex – Vectorless RAG

Hacker News192

Preprocess65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Preprocess maximises RAG performances

Product Hunt+106API

Danswer APIs – Open-source APIs for building RAG apps over company docs70%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Danswer APIs – Open-source APIs for building RAG apps over company docs

Hacker News3

Browse all products Analyze your launch