Collider – the platform for local LLM debug and inference at warp speed

Hacker News

ChatGPT turns one today :) What a day to launch the project I'm tinkering with for more than half a year. Welcome new LLM platform suited both for individual research and scaling AI services in production. GitHub: https://github.com/gotzmann/collider Some superpowers: - Built with performance and scaling in mind thanks Golang and C++ - No more problems with Python dependencies and broken compatibility - Most of modern CPUs are supported: any Intel/AMD x64 platofrms, server and Mac ARM64 - GPUs supported as well: Nvidia CUDA, Apple Metal, OpenCL cards - Split really big models between a number of GPU (warp LLaMA 70B with 2x RTX 3090) - Not bad performance on shy CPU machines, fast as hell inference on monsters with beefy GPUs - Both regular FP16/FP32 models and their quantised versions are supported - 4-bit really rocks! - Popular LLM architectures already there: LLaMA, Starcoder, Baichuan, Mistral, etc... - Special bonus: proprietary Janus Sampling for code generation and non English languages

Visit product View on Hacker NewsNov 30, 2023

Share card

Actual performance

3points

1comments

Did not reach leaderboard

Launch Intel predictions

Analyze your own launch →

Product HuntOn track for Day 1 leaderboard · Strong signals: mac, model, apple · Missing: agents, macos, agent

91%91% predicted probability of success on Product Hunt, based on ML models trained on real launch data.

best fitHighest predicted score across all platforms for this description.

Indie HackersFits the IH revenue-focused audience · Missing: supports, reddit linkedin, podcasting

88%88% predicted probability of success on Indie Hackers, based on ML models trained on real launch data.

Hacker NewsStrong engagement from HN community · Strong signals: llama, ide, io · Missing: https docs, excited, just released

74%74% predicted probability of success on Hacker News, based on ML models trained on real launch data.

nativeThis product was originally launched on this platform.

AppSumoStrong fit for a featured deal · Strong signals: platform · Missing: plus, intuitive, reviews

58%58% predicted probability of success on AppSumo, based on ML models trained on real launch data.

TrustMRRFits verified-revenue profile · Missing: mobile apps, ios, personal

54%54% predicted probability of success on TrustMRR, based on ML models trained on real launch data.

Acquire.comPre-revenue stage for this audience · Missing: arr, mrr, revenue

21%21% predicted probability of success on Acquire.com, based on ML models trained on real launch data.

BetaListMay not resonate with beta-testers · Strong signals: chat · Missing: web3, crypto, cryptocurrency

0%0% predicted probability of success on BetaList, based on ML models trained on real launch data.

Incorrect prediction on native model

Similar products

LLM Inference Requirements Profiler59%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

LLM Inference Requirements Profiler

Hacker News4

BonzAI – self-sovereign, local LLM inference in the browser62%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

BonzAI – self-sovereign, local LLM inference in the browser

Hacker News5

Launch StableStudio local inference in one commmand51%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Launch StableStudio local inference in one commmand

Hacker News6

Otlet – Local LLM inference "inside" Postgres74%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Otlet – Local LLM inference "inside" Postgres

Hacker News2

Multi-agent LLM editor with local inference via WebSockets61%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Multi-agent LLM editor with local inference via WebSockets

Hacker News2

KTransformers–236B Model and 1M Context LLM Inference on Local Machines67%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

KTransformers–236B Model and 1M Context LLM Inference on Local Machines

Hacker News20

Autotrace – Debug on Steroids52%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Autotrace – Debug on Steroids

Hacker News5

Speeding up LLM inference 2x times (possibly)74%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Speeding up LLM inference 2x times (possibly)

Hacker News419

Open-source AMDGCN kernels for optimizing LLM inference72%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Open-source AMDGCN kernels for optimizing LLM inference

Hacker News5

Onera – Private LLM Inference Inside AMD SEV-SNP Enclaves59%Launch Intel prediction score: how likely this product is to succeed on its source platform, based on its name, tagline, and description.

Onera – Private LLM Inference Inside AMD SEV-SNP Enclaves

Hacker News1

Browse all products Analyze your launch