Microsoft says its AI tool is better than doctors at diagnosing conditions

retained text

For years, AI leaders have touted healthcare as one of the clearest examples of where their technology will improve lives. Now “it is finally here,” Microsoft AI CEO Mustafa Suleyman tells The Verge. The company claims to have developed an AI-powered tool it says can diagnose conditions up to four times more accurately than human doctors.
Microsoft says its AI tool is better than doctors at diagnosing conditions
‘We’re going to build something that is indistinguishable from human performance … in the next couple of years,’ Microsoft AI CEO Mustafa Suleyman tells The Verge.
The findings, published Thursday in one of the world’s most prestigious journals, Science, are based on tests pitting Microsoft’s AI system against human physicians on a series of scenarios drawn from medical case reports and emergency department data. Microsoft compiled the cases into its own benchmark, SDBench, which it plans to open-source. Designed to function like a panel of medical experts, Microsoft’s Diagnostic Orchestrator uses multiple AI agents to reason through symptoms and medical histories, decide what tests to order, and arrive at a diagnosis, mimicking the process doctors use to investigate cases.
On cases taken from hundreds of reports published in The New England Journal of Medicine, Microsoft’s system achieved a diagnostic accuracy of around 83 percent, compared with roughly 36 percent for generalist physicians using references and 22 percent for those working without them. The Diagnostic Orchestrator also outperformed doctors on emergency department cases drawn from a large database of patient admissions at the Beth Israel Deaconess Medical Center in Boston, scoring 80 percent accuracy compared with an average of 47 percent for human physicians. The AI system also tended to reach its diagnoses at a lower estimated cost than human counterparts.
The researchers say that the system, which is powered by OpenAI’s GPT-5, also outperformed frontier AI models operating on their own, though the peer-reviewed paper does not include newer models like GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon, or Anthropic’s Opus 5.5. The Diagnostic Orchestrator was first developed when GPT-4.1 was the most advanced model available, but the researchers say subsequent testing suggests that “improvements in model capability do not necessarily diminish the benefits” of having multiple agents work together.
Microsoft acknowledges there’s a long road ahead before the technology can be widely deployed. While the tests more closely mirror the way doctors diagnose conditions — asking questions, weighing possible explanations, and ordering tests before reaching a diagnosis — the system has only been tested on past medical cases and it remains unclear how well it would perform in real clinical settings. “It needs clinical validation,” Suleyman says. “That is a leap, but this is a major milestone along that journey.”
The research is part of a broader ambition at Microsoft to develop what the company calls “medical superintelligence” — AI that can outperform the best human-led healthcare systems. Microsoft has used the term “superintelligence” since before Donald Trump and his administration co-opted it as a catch-all replacement for “AI.”
Diagnosis is just part of Microsoft’s health vision, says the company’s AI health lead, Dr. Dominic King, a coauthor of the study. In a blog post published Thursday and an article in the BMJ Digital Health & AI medical journal, the company outlines five other areas it believes AI will need to master, including coordinating clinical care, making treatment decisions, and handling the human side of medicine.
King tells The Verge that Microsoft plans to begin testing the system in hospital settings with the Beth Israel Deaconess Medical Center to see how it performs in real-world clinical environments. Neither King nor Suleyman would commit to a firm timeline, though King says it will likely “get underway in the next couple of months.”
Commercialization could also be on the horizon, though Microsoft has yet to say when or how the technology might become available. The company already has a significant presence in US healthcare through its digital tools, and health is one of the most popular uses for its consumer-facing chatbot Copilot. King emphasizes the need for a responsible rollout, noting the high regulatory and compliance requirements in medicine that would need to be followed.
“Next year we’re going to see extremely impressive deployments of intelligent systems in production,” Suleyman says. “I think that this is just going to change the way care is delivered in a big way.”
Suleyman has spoken previously about his “human-centered” vision for AI, and said medicine is no exception. Even as he predicts the arrival of “something that is indistinguishable from human performance on medical superintelligence in the next couple of years,” he insists that human clinicians should and will always have a central role in patient care.
While he acknowledges AI could end up with doctors being handed more work, he says his hope is that it will take some of “the drudgery” out of medicine, freeing up clinicians to spend more time with patients.
“I’m very optimistic about what’s coming next,” he says. “I think this is why we build technology.”

document fetched 2026-10-23 16:26 · canonical https://theverge.com/ai-artificial-intelligence/1016690/suleyman-microsoft-diagnostic-orchestrator-science-medical-superintelligence