Researchers measured how effectively MAI-DxO selected diagnostic tests rather than evaluating only final diagnostic accuracy.

Overview Debates Related claims Topics Sources

In the Science study, 304 complex cases published in the New England Journal of Medicine were evaluated without physical examinations or medical imaging available to the AI or human doctors.

Evaluating generative artificial intelligence models that produce full-text radiology reports is challenging because they can represent hundreds of potential findings in an open-ended output.

Cognita Imaging is collaborating with the Center for Devices and Radiological Health's regulatory science team to test using multiple large language models as a jury to evaluate AI radiology reports.

Cognita Imaging will provide the Food and Drug Administration with a final report, software code, guidance for building LLM juries, validation cohort analyses, and radiologist-reviewed discrepancy studies.

Cognita Imaging received a $1.29 million research contract from the Food and Drug Administration to test a new method of evaluating artificial intelligence-generated radiology reports.

Cognita Imaging will build and test its evaluation framework on 1 million patient exams from a large United States cohort to analyze performance across diverse patient groups and equipment.

Cognita Imaging is working with the Food and Drug Administration to design and implement a clinical study to evaluate the accuracy, benefits, and risks of its chest X-ray vision-language model.

Under the Cognita Imaging framework, radiologists will review discrepancies when there are clinically significant disagreements between the AI-generated report, the LLM jury, or the original radiologist's report.