ART – a new open-source RL framework for training agents

Hacker News

Share card

ART – a new open-source RL framework for training agents

Hey HN, I wanted to share a new project we've been working on for the last couple of months called ART ( https://github.com/OpenPipe/ART ). ART is a new open-source framework for training agents using reinforcement learning (RL). RL allows you to train an agent to perform better at any task whose outcome can be measured and quantified. There are many excellent projects focused on training LLMs with RL, such as GRPOTrainer ( https://huggingface.co/docs/trl/main/en/grpo_trainer ) and verl ( https://github.com/volcengine/verl ). We've used these frameworks extensively for customer-facing projects at OpenPipe, but grew frustrated with some key limitations: - Multi-turn workflows, where the agent calls a tool, gets a response, and calls another, are not well supported. This makes them a non-starter for any task that requires an agent to perform a sequence of actions. - Other frameworks typically have low GPU efficiency. They may require multiple H100 GPUs just to train a small 7B parameter model, and aren't able to keep the GPUs busy consistently during both the "rollout" and "training" phases of the training loop. - Existing frameworks are typically not a convenient shape for integrating with existing agentic codebases. Existing trainers expect you to call raw text completion endpoints, and don't automatically provide industry-standard chat completion APIs. ART is designed to address these limitations and make it easy to train high-quality agents. We've also shared many details and practical lessons learned is in this post, which walks through a demo of training an email research agent that outperforms o3 ( https://openpipe.ai/blog/art-e-mail-agent ). You can also find out more about ART's architecture in our announcement post ( https://openpipe.ai/blog/art-trainer-a-new-rl-trainer-for-ag... ). Happy to answer any questions you have!