Open-source LLM playground for VS Code
Open-source LLM playground for VS Code
Hey HN! We made a VS Code extension that allows developers to play with their prompts right in their editor. It uses Oxc and RustPython to parse and detect the prompts in the source code. Devs can test their prompts against different models and/or variables (models*data matrix). With a CSV dataset, one can test their prompt changes against multiple rows and verify whether the change is good or not. Automated evals are coming soonish. The request/response JSONs are available to inspect, as well as usage stats with projected cost. The extension supports hundreds of models thanks to the Vercel Gateway. If you need more provider options (planned but not priority), please let me know. Users without keys will get their requests served by LM Studio running on my PC. Please don't crash my workstation! It only supports JS/TS and Python for now, but many more languages are coming. Compiling tree-sitter crates to Wasm is PITA, but I'm getting there. I also work on improving the heuristics so prompt strings can be detected from arrays or when passed as arguments to popular LLM APIs. Now you can use "prompt" in the variable name or put a comment with "@prompt" in front of it.
Share cardActual performance
Launch Intel predictions
Analyze your own launch →Correct prediction on native model
Similar products
An open-source registry for VS Code extensions
VSConnect, an open source network client for VS Code
PromptFiddle – Open-source WASM-based LLM playground
CodeSwing – A CodePen-like interactive playground, built into VS Code
Thufir – An open-source AI VS Code extension for SREs
Build your own Vim emulation for VS Code
Seconds to in-cluster microservices mocks in VS Code
Sourcegraph Code Search for VS Code
I am fixing VS Code search
Verilog HDL support for VS Code