Written By | Torch Media Africa
A peer-reviewed paper in the journal Science says Microsoft's MAI-DxO diagnosed 83 percent of fiendishly complex cases correctly versus 36 percent for doctors working alone.
Search
SCIENCE & TECH
The claim that lit up the internet
Imagine walking into a hospital with a baffling set of symptoms and being seen not by one doctor but by a virtual panel of specialists, each arguing over your case, deciding which questions to ask and which tests to order, and reaching a diagnosis more often than the best human clinicians working solo. That is the picture Microsoft is painting after a paper published on 22 October 2026 in the journal Science.
The study, titled 'Sequential diagnosis with language models,' describes a system called MAI-DxO, short for Microsoft AI Diagnostic Orchestrator. According to coverage of the paper, MAI-DxO correctly diagnosed about 83 percent of complex cases drawn from the New England Journal of Medicine, compared with about 36 percent for generalist physicians working alone with reference materials. On a second set of emergency-department cases, the system scored roughly 80 percent against about 47 percent for physicians.
Microsoft AI chief Mustafa Suleyman, who has branded the ambition 'medical superintelligence,' called the work a big step in that direction. The company is now preparing to test the system in a real hospital. But as the champagne corks pop in Redmond, independent analysts are asking some pointed questions.
Why does any of this matter? Diagnosis is the foundation of medicine. Get it wrong or late and everything downstream, from treatment to cost to survival, suffers. Doctors are human, they are rushed, and the sheer volume of medical knowledge long ago outgrew any one brain. That is the gap AI developers say they can fill, and it is why the race to build a trustworthy digital diagnostician has become one of the hottest contests in technology. Microsoft, Google and OpenAI are all running hard, and each new paper is a shot across the bows of the others. The question is no longer whether such tools will arrive, but whether they will earn the trust of patients and clinicians when they do.
How the digital panel works
MAI-DxO is not a single chatbot. It is an orchestrator that coordinates several AI agents acting like a panel of specialists. One may propose a hypothesis, another may challenge it, and another may weigh the cost of a proposed test. The orchestrator decides what to ask the patient, which scans or blood tests to order, and when it is confident enough to commit to an answer, updating its thinking as new information arrives.
Crucially, it is model-agnostic. It can sit on top of language models from different companies. Reporting on the study says the best results came when it was paired with OpenAI's GPT-5 and GPT-5.5.
To test it, Microsoft built a new benchmark called SDBench. Instead of handing an AI a complete case file and a multiple-choice question, SDBench turns each case into a step-by-step encounter. The participant receives a short opening summary, asks questions, orders tests and finally commits to a diagnosis, which is checked against the known outcome. That is far closer to how real medicine works than a written exam. The main test used 304 New England Journal of Medicine case records, and a second used 325 emergency department cases from Beth Israel Deaconess Medical Center in Boston.
A history that predates the headline
This is not Microsoft's first appearance with the idea. In June 2025 the company published a preprint announcing MAI-DxO, claiming it reached 85.5 percent accuracy on 304 New England Journal of Medicine cases while 21 practising physicians in the United States and United Kingdom managed about 20 percent. GeekWire and other outlets reported on it at the time, and Suleyman said the company was 'taking a big step towards medical superintelligence.'
Back then, Microsoft acknowledged that real doctors do not work in isolation, since they consult colleagues and textbooks, which the test did not allow. It also noted the paper had not been accepted by a journal, and that the tool was not available for clinical use.
The new Science paper is the peer-reviewed follow-up, and the numbers have shifted. Coverage notes the figures from the 2025 preprint differ from the published ones, which is a normal part of peer review, and one reason journalists have urged readers to use the new numbers rather than the old.
The catch behind the headline
Here is where the story gets interesting. Several independent readings of the paper have raised eyebrows.
First, the human comparison is not perfectly like-for-like. According to one critical analysis, the open-book physicians worked a 56-case test set, while the AI's overall results cover all 304 cases. The 83 percent versus 36 percent gap therefore compares two groups that did not attempt exactly the same cases.
Second, much of the credit may belong to the underlying AI models rather than the clever orchestration. TechCrunch's reading of the data, as relayed in coverage of the study, found that with GPT-5.5 the orchestrator raised complex-case accuracy from about 72 percent to nearly 87 percent, a real gain but one that begins from a very high base. In the emergency set, both setups scored around 85 percent. Another analysis noted that GPT-5 on its own scored 76.6 percent against MAI-DxO's 80.3 percent, a 3.7-point improvement, and concluded that 'a large portion of the measured diagnostic capability was already present in GPT-5.' Microsoft's response is that gains naturally shrink as models approach their ceiling.
Third, cost. The original 2025 pitch promised cheaper diagnosis by avoiding unnecessary tests. The published picture is murkier. One analysis found the unconstrained orchestrator's modelled workup cost was higher than physicians': roughly $5,431 versus $4,725 on complex cases and about $4,979 versus $3,400 in the emergency set. Cheaper settings existed, such as 79.6 percent accuracy at $1,303 per case, but they traded away accuracy. These are modelled costs based on standardized U.S. pricing, not real hospital bills.
What the study did not measure
Critics point to a long list of things the paper cannot tell us. It was a simulation using past cases, not live patients. The benchmark was text only, with no raw X-rays, heart traces or pathology slides. The emergency set excluded benign and healthy presentations. And complex cases in real hospitals are usually handled by specialist teams, who were not part of the comparison.
Then there is the patient experience. One analysis noted that the system asked an average of 25.5 questions before ordering its first test, versus 5.6 for physicians. A patient in severe pain might struggle through that interrogation.
No patient outcomes were measured at all, meaning the study cannot say whether treatment was safer or recovery faster. And a 2024 trial in JAMA Network Open of a different AI tool found no statistically significant improvement in physicians' diagnostic performance when they were given AI help, a reminder that strong standalone AI does not automatically make human doctors better.
There is also the matter of authorship. All 15 authors are Microsoft employees, and eight are named as inventors on a related patent application. That does not invalidate the findings, as the critics themselves note, but it strengthens the case for independent replication.
What happens next
Microsoft says it is working with Beth Israel Deaconess to test the system in hospitals. Under the design described in coverage of the paper, patients receive standard care as normal, and consenting patients also interact with MAI-DxO so that its recommendations can be compared with those of clinicians. An earlier report on the Beth Israel plan suggested clinical testing could begin within two months. No commercial release date has been announced.
The company is also working with the Mayo Clinic on a healthcare foundation model, with a release planned next year, and Suleyman expects healthcare AI to improve by an order of magnitude within 24 to 36 months. Microsoft is far from alone: Google and OpenAI have both made similar claims that their AI can match or beat doctors on some tasks.
Microsoft's executives stress that the aim is to support clinicians, not replace them. Dominic King of Microsoft AI has said diagnosis is only one part of the strategy, which also covers care coordination and clinical decision support.
What this means for you
If you have ever waited months for a diagnosis, bounced between specialists or been told 'it's probably nothing,' the promise of an always-available second opinion is intoxicating. Tools like this could one day help catch rare diseases sooner, speed up emergency triage and ease the pressure on overstretched health services.
But for now, treat headlines with care. This is a research result, not a product. It has not been tested on ordinary patients with ordinary symptoms, and no regulator has approved it. Do not swap your doctor for a chatbot. If you use AI tools to research your symptoms, treat the output as a list of questions to ask a professional, not as a verdict.
There are also privacy questions that deserve attention. A system that interviews you at length about your health will collect deeply sensitive data. Who stores it, who can see it and how it might be used are matters for regulators and patients to demand answers on before these tools arrive in the clinic.
The verdict
Strip away the hype and the story is still remarkable. Only a few years ago, an AI that could reason through a hard medical case step by step and rival experienced doctors on a benchmark would have sounded like science fiction. A top journal has now published such a result.
There is a useful way to hold both ideas at once. The achievement is genuine: a system that can ask sensible questions, weigh the cost of tests and reach a correct answer in most hard cases is a significant piece of engineering. The uncertainty is also genuine: benchmarks are not bedsides. Medicine has a long history of promising technologies that dazzled in the lab and disappointed in the ward, and the only way to know which category MAI-DxO belongs in is to test it where it counts.
Yet the fine print is a lesson in how to read AI breakthroughs. The gap with doctors is real but not as clean as the headline suggests. Much of the power comes from the underlying models. Costs are unproven, patients are untested and the authors have a stake in the outcome.
The real test starts when MAI-DxO leaves the benchmark and walks into a hospital. Until independent teams replicate the results and real patients are involved, the honest answer to whether the machine can beat the doctor is: maybe, in a simulation. The next few months should tell us a great deal more.
