Bodhi App – Run Open Source/Weights HuggingFace LLMs Locally

Hacker News

Hi HN, I'm excited to share Bodhi App, a tool designed to simplify running open-source Large Language Models (LLMs) locally on your laptops. While we currently support M2 Macs, we plan to support other platforms as our community grows. # Problem To use LLMs, you typically need to purchase a subscription from providers like OpenAI or Anthropic, or use OpenAI API credits with compatible Chat UIs. These options can not only burden you financially, but also raise data security and privacy concerns. Many laptops are capable of running powerful open-source LLMs, but for non-tech users, setting them up can be challenging. # Solution: Bodhi App Bodhi App allows you to run LLMs on your own hardware, ensuring data privacy and cost savings. Our goal is to bring the power of LLMs to everyone. Built with non-tech users in mind, Bodhi App ships with a simple Chat UI, making it easy to start conversing with an LLM. It also exposes OpenAI-compatible APIs, enabling other apps to use LLM services without relying on external providers. # Features Bodhi App currently supports: 1. Running GGUF format open-source LLMs from the Huggingface repository. 2. An in-built Chat UI to quickly start conversations with an LLM. 3. A powerful and familiar CLI to download and configure models from the Huggingface ecosystem. 4. Exposing LLMs as OpenAI-compatible APIs for use by other apps. -- # Feature Comparison: Bodhi App vs. Ollama ## Bodhi App - Targeted at non-tech users. - Includes a simple Chat UI to get started quickly. - Integrates well with the Huggingface ecosystem: - Use Huggingface repo/filename to run a model. - Use tokenizer_config.json for chat templates. - Currently only supports Mac M2. ## Ollama - Requires some technical insight. - No inbuilt Chat UI. - Requires baking the model using a custom process: - Modelfile - Golang template to specify chat templates. - Supports various OS platforms. Bodhi App leverages the Huggingface ecosystem, avoiding the need to reinvent the wheel with Modelfile etc., and making it easier to get started quickly. # Quickstart Try Bodhi App today by following these simple steps: ```bash brew tap BodhiSearch/apps brew install --cask bodhi bodhi run llama3:instruct ``` # Documentation and Tutorials - Technical docs on GitHub (README.md, docs folder): https://github.com/BodhiSearch/BodhiApp - YouTube playlist covering Bodhi App features in detail: https://www.youtube.com/playlist?list=PLavvg7KIktFI1ZaFc2nLe... # Conclusion We would love for HN to try out Bodhi App and provide feedback. You can reach us through: - Raising an issue on GitHub: https://github.com/BodhiSearch/BodhiApp - Connecting with the developer on Twitter: https://twitter.com/AmirNagri - Leaving a comment on our YouTube tutorials: https://www.youtube.com/playlist?list=PLavvg7KIktFI1ZaFc2nLe... Please show your support by starring the repo on Github. Thank you for your time! Best, The Bodhi Team

Visit product View on Hacker NewsJun 24, 2024

Share card

Actual performance

21points

13comments

Made the leaderboard

Launch Intel predictions

Analyze your own launch →

Indie HackersFits the IH revenue-focused audience · Strong signals: supports, started, compatible · Missing: reddit linkedin, podcasting, created

98%98% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Product HuntOn track for Day 1 leaderboard · Strong signals: mac, model, apps · Missing: agents, macos, agent

90%90% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: excited, open source, llama · Missing: https docs, just released, exist

72%72% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoMay struggle as an AppSumo deal · Strong signals: platform, users · Missing: plus, intuitive, reviews

49%49% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRLess likely to generate early MRR · Strong signals: apps, users · Missing: mobile apps, ios, personal

33%33% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Strong signals: arr, subscription · Missing: mrr, revenue, profit

13%13% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Correct prediction on native model

Similar products

W&B Inference by Weights & Biases82%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Access to top open-source LLMs

Product Hunt+9

Run LLMs locally with WebGPU55%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run LLMs locally with WebGPU

Hacker News2

Poozle – open-source Plaid for LLMs65%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Poozle – open-source Plaid for LLMs

Hacker News132

Leaping – Open-source debugging with LLMs69%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Leaping – Open-source debugging with LLMs

Hacker News15

Kuzco93%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source Swift package to run LLMs locally on iOS & macOS

Product Hunt+216iOS

ClinePass72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Run the best open-weights models in Cline

Product Hunt+249Developer Tools

Lemonade: Run LLMs Locally with GPU and NPU Acceleration83%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Lemonade: Run LLMs Locally with GPU and NPU Acceleration

Hacker News15

Tampon - an open source Buffer app61%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Tampon - an open source Buffer app

Hacker News33

Photoficient (open source Rails 4.0 app)56%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Photoficient (open source Rails 4.0 app)

Hacker News1

My open source poetry app47%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

My open source poetry app

Hacker News4

Browse all products Analyze your launch