The team questions that guide the path to the coding expert, logic specialist, and generalist model assume that each will cover the others’ blind spots. A new study examining 67 boundary models across 21 providers shows that assumption is statistically flawed — and the flaw has a name: joint ceilings.
The assumption works like this: as long as two models rarely fail in the same information, combining them should create a safety net against failure.
The real limitation in orchestration is not how often the models disagree, but the percentage of notifications where every model in the pool gives the wrong answer at the same time. By ignoring shared failures, businesses build complex, expensive routing infrastructure to chase missing performance gains. Fortunately, developers can use this same equation to create a free test that determines when multi-model orchestration will really pay off.
The hidden costs of a multi-model strategy
To organize multilingual models, developers often rely on three architectures. Routing models act like traffic police, sending complex questions to expensive models and simple questions to cheap ones. Cascades send all instructions to the cheapest model first, stepping up to the premium model only if the first system shows low confidence. Finally, methods such as Mixture-of-Agents (MoA) combine multiple models by asking them the same question and generating a combined answer from their combined results.
These structures present a "the price of dignity" to determine the cost. Every time a development team uses a router or cascade, they pay a premium in additional system latency, complex infrastructure maintenance, and increased management risks for multiple API providers.
To justify these performance costs, developers rely on “binary error integration” to select their integration model. Imagine a developer with Model A, who writes excellent Python but fails in SQL, and Model B, who writes excellent SQL but fails in Python. Because they fail on different types of information, their pairwise error correlation is low. The developer assumes that by putting a routing layer in front of them, they have created a composite system that rarely fails in coding.
According to research, throwing different models together based on low correlation can harm performance if the models are not equally capable – if you vote on different but unequal models, the weak ones tend to converge and outsmart the most intelligent ones.
Josef Chen, an author of the paper, told VentureBeat that in their analysis, "Most naive voting in all non-equilibrium models has a negative advantage (subtract 10 points from our strong mix): diverse-but-weakest members outnumber strong ones." Practical advice for developers is "only include models within the matched quality band." If you can’t match the quality, take one model as a base and spend your budget on the best model available.
The paper provides one bright spot for this approach concerning MoA structures. When creating ensembles, groups are often used "Self-MoA," where they ask for the same premium model multiple times to form a composite answer. The researchers found that with matched quality, building a diverse set of models with two pairs of connections beats the relative Self-MoA setup.
However, when teams use that pairwise integration metric to predict the overall accuracy of their system as a whole, the statistics fall apart.
"So teams pay for orchestration up front (delays, complexity, multi-supplier tasks) with the assumption that the various dividends come later," Chen said. "Usually it is not possible, because today’s best models agree, and, even worse, they fail in the same questions … the information just carries a small signal of which model will be the correct one when the border does not agree."
Why mathematics fails: the ceiling of collaborative failure
The key findings of the research centers on the metric called "cooperative failure rate" – the official name of the wrong condition described above. No router, polling system, or cascade can achieve more accuracy than the ceiling it sets.
Coding, logic, and the generalist pool show two low correlations in general knowledge – they rarely fail together. But the collective roof represents a more abstract, more complex case that exceeds the limits of current AI architectures. If the data is so complex that all three models fail or fail, it doesn’t matter how smartly the router distributes the work. The entire pool clears simultaneously.
The researchers tested their pool of 67 models, including the GPT-5.5, Claude Opus 4.8, and Gemini 3.1 Pro, on the MATH-500 open math benchmark. Based on two-way correlations, statistical models predicted that the entire pool would clear at once in only 2.3% of queries. In fact, the joint failure rate was 5.2%.
Standard communication metrics underestimated the failure rate by about 2.25 times. Guilt is not just an individual difficulty, but a shared point of failure.
"The driver is what we call the atom of the common mode: a piece of the question where the whole market fails together, for which no parallel figures can be seen," Chen said. "Adding a 20 model to your pool doesn’t buy a tail cover. The tail is divided."
The researchers also found that the work format causes a failure in collaboration. When they took graduate-level science questions from the GPQA benchmark and changed them from multiple-choice to free-response formats, the incorrect tail widened to 12.7%.
Developers can engineer around the roof, though. "What engineering means is not free: the setup of many models buys at least where the parties want it most, in the open generation," Chen said. "Anywhere you can turn generation into validation or constrained choice (structured output, testable responses, synthetic tests), you’re opening the roof again."
Finally, the researchers found these limits of AI applications in two different ways, depending on the domain:
-
Situations involving roofs (eg, open figures): The failure rate of cooperation is high. The task is very difficult, and all models fail simultaneously. No amount of routing can outweigh the lack of under power.
-
Areas bound for achievement (eg, graduate-level science): The cooperative failure rate is close to zero, meaning that at least one model in the pool usually knows the answer. However, the models disagree so subtly that the routing layer cannot choose the correct answer without an omniscient oracle.
$0 sanity check for pre-shipment
Before devoting engineering hours to building a router, teams can calculate their total overhead for free using a mathematical formula called the Clopper-Pearson bound.
The Clopper-Pearson correlation serves as a worst-case calculator. If you flip a coin ten times and get eight heads, you cannot guarantee that the coin will remain heads 80% of the time forever. The bond takes a small sample of test questions and produces a statistically guaranteed ceiling.
Applied to language models, suppose a team tests a group of five agents on 50 sample questions and finds that they all fail together on just two questions. An engineer may think that his multi-agent system will achieve 96% accuracy in production. The Clopper-Pearson formula corrects this expectation. It analyzes the small sample size and provides statistical assurance that the true collective failure rate may be as high as 12%.
To use this in practice, businesses must create a captive dataset. A fintech company, for example, can take 200 complex customer support tickets from the previous quarter and have human agents write the relevant decisions to serve as a benchmark. Although this sounds like a difficult manual project, experienced engineering teams can automate the entire ceiling calculation.
"Compilation is trivial: it is a calculation operation on top of eval logs that teams already generate," Chen notes, "so it runs on the same CI platform as the eval suite and re-triggers whenever the pool model or workload changes."
The engineering team then runs their candidate models against these 200 tickets once and records the results. If they want to evaluate the optimization of multiple models, they can use the average failure rate to predict the maximum accuracy they can get from the system without using additional queries.
One important conclusion reached by the study is that in tasks where answers can be evaluated, combining models is rarely beaten using a single market-leading model, unless the team has a very strong question-level routing signal.
In a business context, the evaluated function has an objective, non-tolerant response. This includes generating a SQL query that should run without error, extracting a specific amount of an invoice from a 50-page PDF, or formatting a JSON upload that exactly matches a robust schema. In these jobs, businesses are often better off paying a premium for the smartest frontier model than putting together three cheaper models and hoping the route picks the right output. The study didn’t examine specific, devolved tasks like writing marketing copy — the authors note that whether these findings hold up outside of your validation benchmarks remains an open question.
Because this statistical check is free, business teams can track their failure rates collaboratively as new models roll out.
"Benchmarking costs nothing, so any team can track their collective failure rate across model generations and see if the tail closes," said Chen. Finally, "Leverage buyers are driven by failure mode variability and market churn, not model numbers."
