Zero dependency LLM standby, rollouts, no proxy

Hacker News

Share card

Zero dependency LLM standby, rollouts, no proxy

We believe you should not rely on any company for your critical LLM infra. This does not proxy via another service that might go down or be beholden to whims of some board. We wrote logos-shift for our own needs and are open sourcing it. What it does: - Save input/output data, no proxy, no slowdowns, double buffering, separate thread - Finetune your own model, or we can for a fee. - Auto rollout your model when ready. We recommend keeping your LLM at 80/20 for most cases and rollout to 100 on failure. How to use it: pip install logos_shift_client from logos_shift_client import LogosShift # Initialize with your API key (without if you just want the local copy) logos_shift = LogosShift(api_key="YOUR_API_KEY") # Instrument your function @logos_shift() def add(x, y): return x + y result = add(1, 2) # Optionally, provide feedback logos_shift.provide_feedback(result['bohita_logos_shift_id_123', "success") We can manage all this for you of course, but its not hard to do it yourself. Save data on our infra for free for upto 1GB. $10 per month for after. Download your data anytime. No lock in. Feel free to ask questions and how can we make this easier for you.