For years, Massive Tech CEOs have touted visions of AI brokers that may autonomously use software program functions to finish duties for folks. However take in the present day’s client AI brokers out for a spin, whether or not it’s OpenAI’s ChatGPT Agent or Perplexity’s Comet, and also you’ll rapidly notice how restricted the expertise nonetheless is. Making AI brokers extra strong could take a brand new set of strategies that the business remains to be discovering.
A type of strategies is rigorously simulating workspaces the place brokers could be skilled on multi-step duties — referred to as reinforcement studying (RL) environments. Equally to how labeled datasets powered the final wave of AI, RL environments are beginning to appear to be a essential ingredient within the improvement of brokers.
AI researchers, founders, and traders inform TechCrunch that main AI labs at the moment are demanding extra RL environments, and there’s no scarcity of startups hoping to produce them.
“All the massive AI labs are constructing RL environments in-house,” stated Jennifer Li, normal companion at Andreessen Horowitz, in an interview with TechCrunch. “However as you’ll be able to think about, creating these datasets may be very complicated, so AI labs are additionally taking a look at third occasion distributors that may create prime quality environments and evaluations. Everyone seems to be taking a look at this area.”
The push for RL environments has minted a brand new class of well-funded startups, akin to Mechanize and Prime Mind, that purpose to guide the area. In the meantime, massive data-labeling corporations like Mercor and Surge say they’re investing extra in RL environments to maintain tempo with the business’s shifts from static datasets to interactive simulations. The key labs are contemplating investing closely too: in line with The Data, leaders at Anthropic have mentioned spending greater than $1 billion on RL environments over the following yr.
The hope for traders and founders is that one in every of these startups emerge because the “Scale AI for environments,” referring to the $29 billion knowledge labelling powerhouse that powered the chatbot period.
The query is whether or not RL environments will actually push the frontier of AI progress.
Techcrunch occasion
San Francisco
|
October 27-29, 2025
What’s an RL atmosphere?
At their core, RL environments are coaching grounds that simulate what an AI agent could be doing in an actual software program utility. One founder described constructing them in recent interview “like creating a really boring online game.”
For instance, an atmosphere may simulate a Chrome browser and process an AI agent with buying a pair of socks on Amazon. The agent is graded on its efficiency and despatched a reward sign when it succeeds (on this case, shopping for a worthy pair of socks).
Whereas such a process sounds comparatively easy, there are a number of locations the place an AI agent may get tripped up. It would get misplaced navigating the net web page’s drop down menus, or purchase too many socks. And since builders can’t predict precisely what improper flip an agent will take, the atmosphere itself must be strong sufficient to seize any surprising conduct, and nonetheless ship helpful suggestions. That makes constructing environments way more complicated than a static dataset.
Some environments are fairly elaborate, permitting for AI brokers to make use of instruments, entry the web, or use varied software program functions to finish a given process. Others are extra slim, geared toward serving to an agent be taught particular duties in enterprise software program functions.
Whereas RL environments are the new factor in Silicon Valley proper now, there’s a number of precedent for utilizing this method. One in all OpenAI’s first tasks again in 2016 was constructing “RL Gyms,” which had been fairly just like the fashionable conception of environments. The identical yr, Google DeepMind’s AlphaGo AI system beat a world champion on the board sport, Go. It additionally used RL strategies inside a simulated atmosphere.
What’s distinctive about in the present day’s environments is that researchers try to construct computer-using AI brokers with massive transformer fashions. Not like AlphaGo, which was a specialised AI system working in a closed environments, in the present day’s AI brokers are skilled to have extra normal capabilities. AI researchers in the present day have a stronger start line, but additionally an advanced purpose the place extra can go improper.
A crowded subject
AI knowledge labeling corporations like Scale AI, Surge, and Mercor try to satisfy the second and construct out RL environments. These corporations have extra assets than many startups within the area, in addition to deep relationships with AI labs.
Surge CEO Edwin Chen tells TechCrunch he’s not too long ago seen a “vital improve” in demand for RL environments inside AI labs. Surge — which reportedly generated $1.2 billion in revenue final yr from working with AI labs like OpenAI, Google, Anthropic and Meta — not too long ago spun up a brand new inner group particularly tasked with constructing out RL environments, he stated.
Shut behind Surge is Mercor, a startup valued at $10 billion, which has additionally labored with OpenAI, Meta, and Anthropic. Mercor is pitching traders on its enterprise constructing RL environments for area particular duties akin to coding, healthcare, and regulation, in line with advertising and marketing supplies seen by TechCrunch.
Mercor CEO Brendan Foody informed TechCrunch in an interview that “few perceive how massive the chance round RL environments actually is.”
Scale AI used to dominate the information labeling area, however has misplaced floor since Meta invested $14 billion and employed away its CEO. Since then, Google and OpenAI dropped Scale AI as an information supplier, and the startup even faces competitors for knowledge labelling work within Meta. However nonetheless, Scale is making an attempt to satisfy the second and construct environments.
“That is simply the character of the enterprise [Scale AI] is in,” stated Chetan Rane, Scale AI’s head of product for brokers and RL environments. “Scale has confirmed its skill to adapt rapidly. We did this within the early days of autonomous autos, our first enterprise unit. When ChatGPT got here out, Scale AI tailored to that. And now, as soon as once more, we’re adapting to new frontier areas like brokers and environments.”
Some newer gamers are focusing completely on environments from the outset. Amongst them is Mechanize, a startup based roughly six months in the past with the audacious purpose of “automating all jobs.” Nonetheless, co-founder Matthew Barnett tells TechCrunch that his agency is beginning with RL environments for AI coding brokers.
Mechanize goals to produce AI labs with a small variety of strong RL environments, Barnett says, moderately than bigger knowledge corporations that create a variety of straightforward RL environments. So far, the startup is providing software program engineers $500,000 salaries to construct RL environments — far increased than an hourly contractor may earn working at Scale AI or Surge.
Mechanize has already been working with Anthropic on RL environments, two sources acquainted with the matter informed TechCrunch. Mechanize and Anthropic declined to touch upon the partnership.
Different startups are betting that RL environments might be influential exterior of AI labs. Prime Mind — a startup backed by AI researcher Andrej Karpathy, Founders Fund, and Menlo Ventures — is focusing on smaller builders with its RL environments.
Final month, Prime Mind launched an RL environments hub, which goals to be a “Hugging Face for RL environments.” The concept is to offer open-source builders entry to the identical assets that enormous AI labs have, and promote these builders entry to computational assets within the course of.
Coaching usually succesful brokers in RL environments could be extra computational costly than earlier AI coaching strategies, in line with Prime Mind researcher Will Brown. Alongside startups constructing RL environments, there’s one other alternative for GPU suppliers that may energy the method.
“RL environments are going to be too massive for anyone firm to dominate,” stated Brown in an interview. “A part of what we’re doing is simply making an attempt to construct good open-source infrastructure round it. The service we promote is compute, so it’s a handy onramp to utilizing GPUs, however we’re pondering of this extra in the long run.”
Will it scale?
The open query round RL environments is whether or not the method will scale like earlier AI coaching strategies.
Reinforcement studying has powered a number of the greatest leaps in AI over the previous yr, together with fashions like OpenAI’s o1 and Anthropic’s Claude Opus 4. These are significantly essential breakthroughs as a result of the strategies beforehand used to enhance AI fashions at the moment are exhibiting diminishing returns.
Environments are a part of AI labs’ larger wager on RL, which many consider will proceed to drive progress as they add extra knowledge and computational assets to the method. A few of the OpenAI researchers behind o1 beforehand informed TechCrunch that the corporate initially invested in AI reasoning fashions — which had been created by means of investments in RL and test-time-compute — as a result of they thought it will scale properly.
The easiest way to scale RL stays unclear, however environments look like a promising contender. As a substitute of merely rewarding chatbots for textual content responses, they let brokers function in simulations with instruments and computer systems at their disposal. That’s way more resource-intensive, however doubtlessly extra rewarding.
Some are skeptical that each one these RL environments will pan out. Ross Taylor, a former AI analysis lead with Meta that co-founded Basic Reasoning, tells TechCrunch that RL environments are liable to reward hacking. This can be a course of through which AI fashions cheat with the intention to get a reward, with out actually doing the duty.
“I believe individuals are underestimating how troublesome it’s to scale environments,” stated Taylor. “Even the most effective publicly out there [RL environments] usually don’t work with out severe modification.”
OpenAI’s Head of Engineering for its API enterprise, Sherwin Wu, stated in a recent podcast that he was “quick” on RL atmosphere startups. Wu famous that it’s a really aggressive area, but additionally that AI analysis is evolving so rapidly that it’s exhausting to serve AI labs nicely.
Karpathy, an investor in Prime Mind that has known as RL environments a possible breakthrough, has additionally voiced warning for the RL area extra broadly. In a post on X, he raised considerations about how rather more AI progress could be squeezed out of RL.
“I’m bullish on environments and agentic interactions however I’m bearish on reinforcement studying particularly,” stated Karpathy.
Replace: A earlier model of this text referred to Mechanize as Mechanize Work. It has been up to date to mirror the corporate’s official identify.
