Reporting checklist for foundation and large language models in medical research (REFINE): an international consensus guideline.
To develop the REporting checklist for FoundatIon and large laNguagE models (REFINE), an international reporting guideline for transparent and reproducible reporting of foundation model (FM) and large language model (LLM) studies in medical research, including imaging artificial intelligence (AI) applications.
The protocol was prespecified and publicly archived. A modified Delphi process was conducted to establish reporting standards for unimodal and multimodal FM and LLM applications involving text, imaging, and structured data. The steering committee coordinated protocol development, expert recruitment, all Delphi rounds, and the harmonization phase. Decisions were made based on predefined consensus thresholds. In Rounds 1 and 2, structured ratings and free-text feedback informed iterative revisions. In the post-Delphi harmonization phase, terminology was standardized, and detailed reporting instructions were finalized.
The REFINE development group comprised 57 contributors from 17 countries, and 54 panelists from 16 countries completed Rounds 1 and 2. The harmonization phase was completed by three expert panelists and the steering committee. The entire process produced a 44-item, six-section framework with standardized terminology and detailed reporting instructions, supported by an online platform for practical use (https://refinechecklist.github.io/refine/checklist.html).
The REFINE provides a comprehensive, consensus-based reporting standard for medical FM and LLM research, including imaging AI studies. The online version facilitates practical implementation.
The REFINE enables transparent, comparable, and reproducible reporting of FM and LLM studies, supporting reliable evidence synthesis in medical and imaging-focused AI studies.
The protocol was prespecified and publicly archived. A modified Delphi process was conducted to establish reporting standards for unimodal and multimodal FM and LLM applications involving text, imaging, and structured data. The steering committee coordinated protocol development, expert recruitment, all Delphi rounds, and the harmonization phase. Decisions were made based on predefined consensus thresholds. In Rounds 1 and 2, structured ratings and free-text feedback informed iterative revisions. In the post-Delphi harmonization phase, terminology was standardized, and detailed reporting instructions were finalized.
The REFINE development group comprised 57 contributors from 17 countries, and 54 panelists from 16 countries completed Rounds 1 and 2. The harmonization phase was completed by three expert panelists and the steering committee. The entire process produced a 44-item, six-section framework with standardized terminology and detailed reporting instructions, supported by an online platform for practical use (https://refinechecklist.github.io/refine/checklist.html).
The REFINE provides a comprehensive, consensus-based reporting standard for medical FM and LLM research, including imaging AI studies. The online version facilitates practical implementation.
The REFINE enables transparent, comparable, and reproducible reporting of FM and LLM studies, supporting reliable evidence synthesis in medical and imaging-focused AI studies.
Authors
Mese Mese, Akinci D'Antonoli Akinci D'Antonoli, Bluethgen Bluethgen, Bressem Bressem, Cuocolo Cuocolo, Chaudhari Chaudhari, Tejani Tejani, Isaac Isaac, Ponsiglione Ponsiglione, Meddeb Meddeb, Khosravi Khosravi, Le Guellec Le Guellec, Kahn Kahn, Suh Suh, Pinto Dos Santos Pinto Dos Santos, Koh Koh, Tzanis Tzanis, Kotter Kotter, Colak Colak, Kitamura Kitamura, Busch Busch, Nensa Nensa, Yang Yang, Müller Müller, Kather Kather, Nawabi Nawabi, Kleesiek Kleesiek, Zhong Zhong, Santinha Santinha, Haubold Haubold, de Almeida de Almeida, Lekadir Lekadir, Marias Marias, Reiner Reiner, Maier-Hein Maier-Hein, Moy Moy, Adams Adams, Martí-Bonmatí Martí-Bonmatí, Paschali Paschali, Moassefi Moassefi, Dietzel Dietzel, Huisman Huisman, Ingrisch Ingrisch, Klontzas Klontzas, Papanikolaou Papanikolaou, Diaz Diaz, Kuriki Kuriki, Seeböck Seeböck, Rouzrokh Rouzrokh, Strotzer Strotzer, Park Park, Faghani Faghani, Tayebi Arasteh Tayebi Arasteh, Kim Kim, Venugopal Venugopal, Kim Kim, Kocak Kocak
View on Pubmed