Today we’re launching LangChain Labs, a new applied research effort focused on Continual Learning. Our goal is to advance open, applied research for every agent. We’re working with partners across industries to make sure this technology is useful for the broader agent-building community.
Every agent run contains useful signal. The open problem is how to capture that signal, transform it into usable data, and then applying those improvements.
💡Capturing, transforming, and understanding agent data at scale is exactly what LangSmith was built for. This gives us and our customers a great launching pad for tackling continual learning.
These changes can be applied at different layers of the Agent stack such as the optimizing the agent harness, choosing different models, or fine-tuning models.
We’re starting this work with a few early research partners including Harvey, Nvidia, Prime Intellect, Fireworks, and Baseten.
“We’re excited to work with the LangChain Labs team to push applied research on efficient, self-improving agents for the most complex legal work.”
— Niko Grupen, Head of Applied Research, Harvey
The early research directions we’re tackling are:
Improving Agents by Mining Information from Large-Scale Agent Data: Agents are being integrated into software systems at a rapid rate. Very soon agents will produce more data in months than humans have ever produced in aggregate. Extracting useful signals from that data for eval/environment generation, harness engineering, and post-training is still a difficult problem. Traces are the source of that data and we want to help every team use traces to build better agents.
Efficient Agents at the Pareto Frontier: Agents operate under real organizational constraints around cost, latency, and task performance. For many of the world’s most important tasks, we’re yet to discover the most efficient combination of models harnesses, models, and feedback loops that allow agents to self-improve.
Systematic building of evaluation and simulation environments: To properly evaluate agents, you often need to run them in an end-to-end manner in an environment representative of how they will be used in production. These environments can be difficult and time consuming to create. We’re researching ways to make it easier to create and run environments for evaluation, simulation, and reinforcement learning.