ACRouter selects the most intelligent AI model for each task, beating the Opus setup only by 2.6x in cost

Model routing becomes a key part of the enterprise AI stack, dynamically sending information to the correct AI model to improve speed and cost. However, current frameworks often treat routing as a static classification problem, which severely limits their capabilities.

A new open source framework called Agent-as-a-Router addresses this problem, treating the router as a dynamic, memory-building agent. It uses a Context-Action-Feedback (CAF) loop to track model successes and failures and update routing behavior.

The researchers also released ACRouter, a portable implementation of this paradigm. In their tests, ACRouter significantly outperformed static routers and an expensive automation strategy on premium models, all without requiring teams to train large models or write endless heuristics.

In real-world applications, this framework provides an option to replace hard-coded AI infrastructure with adaptive systems that can adapt to changes in user behavior and underlying models used in the enterprise AI stack.

Route economy and information scarcity

A single model setup is useful for testing but dangerous when scaling AI applications. AI developers use model routing to map tasks to models that are cheap and open quickly when possible, while retaining expensive models at the border for complex reasoning.

Currently, developers rely on two main methods for this task. The first is heuristics-based routing, which relies on hard-coded rules of thumb. For example, a developer might write a rule that says if the content contains certain keywords, it will be redirected to GPT-5.5. Otherwise, go for a self-hosted open source model like the Kimi K2.7.

The second method is statically trained policies. These are machine learning classifiers trained on historical datasets that look for information embeddings and predict the best model based on previous training data.

Both methods are static. When researchers tested these methods in real-world coding and agent workflows, they found a hard ceiling on accuracy. A key finding shows that static routers suffer from a severe lack of information. Because they only examine the input text and never see if the model actually succeeds in doing the job, they make blind guesses when faced with complex situations.

This causes three different points of failure. First, static routers suffer from a frozen state of information, which means they cannot accumulate new performance feedback during deployment. Second, they fail in output distribution (OOD) generalization. They break during the second day of operation when business data or user behavior changes because their training data no longer matches reality. Finally, they are more vulnerable to model releases. A static phase trained on today’s models may become obsolete if a better model drops next week.

Agent-as-a-Router: An automated system

The basic idea of ​​Agent-as-a-Router is that the actual router should receive and accumulate information based on execution during deployment, basically learning on the job.

Researchers have achieved this through the CAF program. When new information arrives, the router checks the information’s metadata and functionality, such as programming language or complexity. It then searches its historical memory for similar operations to see which models have succeeded or failed in the past. The router uses this context to select the target model and perform the operation. Finally, the system sees a real-world result, outputs a success or failure signal, and writes this feedback back to its memory to inform future routing decisions.

Consider an automated business data analytics pipeline. The router gets the job of generating SQL and sends it to an open source model like Kimi. The model manipulates the column name and fails to compile the SQL. The CAF loop detects the compiler error, registers it as a response, and logs it. The next time the same obscure SQL query arrives, the router checks its context and moves the job to a more advanced model such as Claude Opus 4.8.

ACRouter

The researchers developed ACRouter as a solid foundation for this framework. It is composed of three main components: Orchestrator, Validator, and Memory. This structure is supported by the tool layer to physically implement the CAF loop.

The Memory module enables the core class. Built on a vector store, it retrieves relevant past interactions and updates historical information with new results. The Orchestrator manages the action phase. It processes the user input from the retrieved memory to select the most capable target model from the available pool. The verifier manages the feedback phase by evaluating the output of the selected model to generate a clear success or failure signal.

The tools layer connects Validator to real-world applications, such as a Python code interpreter, an agent sandbox, or a database engine. The tools layer allows the system to run the generated code or query and look at the exact result, providing a verified signal that the router needs to read.

Orchestrator itself is lightweight. Instead of a large, heavy language model, the researchers trained a sub-billion parameter adapter based on Qwen 3.5 (0.8B parameters), which means it can adapt to the device of your choice.

ACRouter is efficient: It is more efficient than border routers

To test the stress framework, the researchers introduced CodeRouterBench, a test environment that includes nearly 10,000 functions with verified points across eight parameter models, including Claude Opus 4.6, GPT-5.4, Qwen3-Max, and GLM-5. The test was divided between internal distribution (ID) testing (which includes nine code dimensions of one curve as algorithm design and test execution) and out-of-distribution (OOD) system testing. The OOD tasks were qualitatively different, requiring multiple steps of programming, file navigation, and repetitive debugging to see if the router could adapt to fundamentally new domains.

Basic results revealed why the single-model strategy is flawed: no single model dominates all categories. For example, while Claude Opus 4.6 achieved the highest average performance, it performed better in algorithm design by GLM-5 (86% relative improvement) and in test generation by Qwen3-Max (111% improvement), despite Opus costing about 12 times than smaller models like Kimi-K2.5.

In benchmarks, static routers consistently failed by sending some niche coding work to a model that was poorly equipped for that syntax. A static router had no way of knowing that the code was failing to run. In contrast, ACRouter adjusted its strategy after receiving a negative feedback signal from the processing area.

According to the researchers’ estimation, ACRouter remains firmly on the Pareto frontier of cost and performance. In both ID workflow and complex OOD agent testing, ACRouter has achieved low incremental regret, metric decisions to measure the best route over time. In a set of in-deployment tests, ACRouter cost $13.21 per workflow, compared to $34.02 per automation on Opus — a 2.6x savings.

It dynamically matches tasks to the best-performing model for that particular niche, suggesting that businesses can achieve or exceed threshold-level accuracy across a wide range of workloads without paying a premium price for every query.

Warnings, limitations, and how to get started

While the Agent-as-a-Router paradigm solves the information shortage, it is not the solution for all AI workflows.

The framework shines in verifiable tasks where the Verifier receives a clear success or failure signal from the environment, such as writing code or retrieving data. It is effective in systems with distribution shifts and domains where different models pass through completely different areas.

Conversely, setup is responsible for trivial tasks where any model will suffice, or for low-volume applications that don’t justify the engineering overhead. It is also not suitable for independent domains, such as creative writing, where the correct response cannot be easily ascertained and the response signals are unlikely to be consistent.

The researchers open-sourced the code on GitHub and released the weights of the orchestrator model in Hugging Face under the Apache 2.0 license. The router is compatible with Claude Code, Codex, and OpenCode.