Microsoft's AI Diagnostic Panel Beats Doctors in Complex Case Simulation

83% is the accuracy rate achieved by Microsoft’s new AI diagnostic system when tackling complex medical cases, a figure that more than double the 36% success rate of generalist physicians in the same simulation. The results, published in Science this Thursday, highlight a significant gap in diagnostic capability that Microsoft CEO Mustafa Suleyman describes as a pivotal step toward “medical superintelligence.” The system, known as MAI-DxO or simply DxO, operates by coordinating multiple AI agents that function collectively like a panel of specialists. These agents question the patient, order necessary tests, and debate potential diagnoses before reaching a conclusion.

The simulation utilized intricate cases from the New England Journal of Medicine, a benchmark often reserved for specialists or physician teams rather than individual generalists. Dominic King, Microsoft AI’s vice president of Health, emphasized the practical urgency of these findings, noting that the healthcare sector is currently grappling with a widening mismatch between the demand for medical expertise and its supply. By publishing these results, Microsoft joins peers like Google and OpenAI in asserting that artificial intelligence can match or exceed human medical performance.

Data from the study indicates that much of the performance improvement stems from the underlying power of the AI models themselves, with Microsoft’s orchestration layer providing additional gains. While Microsoft tested the system using models from OpenAI, Google, Anthropic, Meta, and DeepSeek, the highest scores were achieved with OpenAI’s GPT-5 and GPT-5.5. Cost efficiency was also a key metric; using GPT-5 alone, the system scored 78.9% at an estimated cost of $8,400 per case. DxO reduced this cost by approximately $3,000 while improving accuracy by roughly four points, though the system saw little benefit on routine emergency department cases.

The system proved most valuable for complex cases and when paired with less capable models. For instance, Gemini 2.5 Flash improved from a 46.4% accuracy score on its own to nearly 69% with DxO assistance. Researchers noted that as models approach their theoretical maximum, the marginal gains from orchestration naturally diminish. A Microsoft spokesperson stated that the orchestration system is designed to optimize both accuracy and cost simultaneously, addressing the broader challenges of medical AI deployment.

Suleyman, who leads Microsoft’s frontier AI efforts, revealed that the company is collaborating with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. He anticipates that by 2027, the technology will enable a “coding style breakthrough” for medical records, allowing clinicians to stream highly accurate predictions about how patient records might unfold. Suleyman, a co-founder of DeepMind and former head of Hark, believes this infrastructure is essential for achieving real breakthroughs in clinical care.

To validate these capabilities in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The upcoming study will evaluate DxO’s clinical reasoning across diverse hospital environments, including a Boston emergency department. Patients will receive standard care while consenting to parallel engagement with the AI, allowing researchers to compare the system’s recommendations against those of human doctors. King called this initiative the largest study of its kind, aiming to provide compelling evidence for the scaled deployment of such systems.

The release of the study coincides with a manifesto outlining the path to medical superintelligence, defined as the ability to surpass human-led health systems in reasoning and coordination. Microsoft aims to move beyond simple benchmarking to integrate directly with clinicians for better outcomes. With existing dictation and radiology tools already in the market, the company views DxO as a critical expansion of its clinical portfolio, aiming to hold the state-of-the-art position in medical decision support for the coming years.