Even Combining Multiple Models Has Clear Limits... Challenging the 'Orchestration' Paradigm

Research challenges the common belief that combining AI models improves performance by compensating for each other's weaknesses. A study by KAIKAKU and collaborators reveals that when multiple models fail on the same problem ('co-failure'), simply adding more models or complex routing systems doesn't guarantee performance gains. The analysis of 67 advanced large language models (LLMs) introduces the 'co-failure ceiling' concept, arguing that system-wide accuracy is capped at 1-β due to persistent simultaneous errors across models. While industry previously relied on pairwise error correlation metrics, the research shows this alone cannot predict multi-model system performance improvements.