The study, published in the journal Science, is one of the most comprehensive to date comparing artificial intelligence with doctors when it comes to clinical decisions. In a number of different experiments, the researchers had OpenAI's o1 review medical data about patients, then make diagnoses and suggest treatments.
The conclusion is that the language model was as good as or better than the doctors in terms of diagnostics and treatment options.
Better in the emergency room
In particular, the model outperformed human colleagues in assessing patients in the emergency department, where healthcare professionals must make quick decisions, often with limited information. In the trial, the model was fed unfiltered information directly from patient records to mimic reality.
It was also allowed to triage patients, that is, to prioritize patients based on medical need.
The language model was better than doctors at triaging patients and making diagnoses even under the messy conditions that prevail in real life," says Peter Brodeur, a doctor and researcher at Harvard Medical School and one of the study's authors, at a digital press conference.
Still just text
Both humans and AI improved as more clinical data became available. Doctors emphasize that they do not think it likely that AI will replace human doctors.
However, doctors can benefit greatly from working with AI, especially when it comes to getting a second opinion and assessing difficult cases," says Harvard colleague Arjun Manrai, another of the study's authors.
The authors address several limitations of the study. Among other things, it only used text-based material, while in real-life healthcare situations, what is seen and heard is also important, areas where AI is not as developed.
AI is already being used today in healthcare.
Diagnostics and image analysis: AI algorithms are used to analyze medical images (for example, mammograms, CT scans) to identify cancer or other abnormalities.
Decision support for staff: The systems help doctors choose the right treatment by analyzing patient data, suggesting dosages, and identifying risks of drug interactions.
Speech recognition: Systems that convert speech into text.
Administrative relief: Used to automate documentation, patient scheduling and billing.
Forecasting and resource planning: AI can predict patient admissions, optimizing staffing and bed space in hospitals.
Sources: Stockholm Region, Skåne Region, etc.
