Defining the Clinical Phenotypic Ground Truth: Expert Interrater Agreement in Hyperkinetic Movement Disorders.
Reliable clinical phenotyping is crucial for diagnosis, treatment, and research in hyperkinetic movement disorders, yet its interrater reliability remains uncertain.
To quantify interrater agreement in clinical phenotyping of hyperkinetic movement disorders among experts in a large, phenotypically diverse cohort.
Within the NEMO project, 161 patients with hyperkinetic movement disorders underwent standardized video assessments. Each patient was independently classified by three international movement disorders experts in a structured, stepwise procedure. Experts first assessed videos alone, then received clinical history and examination, and finally diagnostic test results. At each step, they assigned primary and secondary phenotypes and rated their diagnostic confidence. Interrater agreement was assessed using Fleiss' and Cohen's kappa.
Across all phenotypes, interrater agreement on the primary phenotype increased with added information (κ = 0.457 to κ = 0.607). Agreement with the NEMO reference classification similarly improved (κ = 0.478 to κ = 0.656). Among patients with a single dominant phenotype, agreement reached κ = 0.78. In 93% of cases, at least two out of three experts reached consensus. Agreement was substantially lower in patients with mixed or complex presentations, such as myoclonus-dystonia and functional movement disorders. Diagnostic certainty rose throughout, though secondary phenotype agreement remained poor. Experts rarely revised initial impressions, suggesting robust pattern recognition but also potential confirmation bias.
Movement disorders specialists can reliably identify hyperkinetic phenotypes, even without in-person assessment. While complex, mixed and functional presentations remain diagnostically challenging, our findings reinforce the value of expert phenotyping in clinical care, training, and research, and provide a benchmark for future studies and computational approaches.
To quantify interrater agreement in clinical phenotyping of hyperkinetic movement disorders among experts in a large, phenotypically diverse cohort.
Within the NEMO project, 161 patients with hyperkinetic movement disorders underwent standardized video assessments. Each patient was independently classified by three international movement disorders experts in a structured, stepwise procedure. Experts first assessed videos alone, then received clinical history and examination, and finally diagnostic test results. At each step, they assigned primary and secondary phenotypes and rated their diagnostic confidence. Interrater agreement was assessed using Fleiss' and Cohen's kappa.
Across all phenotypes, interrater agreement on the primary phenotype increased with added information (κ = 0.457 to κ = 0.607). Agreement with the NEMO reference classification similarly improved (κ = 0.478 to κ = 0.656). Among patients with a single dominant phenotype, agreement reached κ = 0.78. In 93% of cases, at least two out of three experts reached consensus. Agreement was substantially lower in patients with mixed or complex presentations, such as myoclonus-dystonia and functional movement disorders. Diagnostic certainty rose throughout, though secondary phenotype agreement remained poor. Experts rarely revised initial impressions, suggesting robust pattern recognition but also potential confirmation bias.
Movement disorders specialists can reliably identify hyperkinetic phenotypes, even without in-person assessment. While complex, mixed and functional presentations remain diagnostically challenging, our findings reinforce the value of expert phenotyping in clinical care, training, and research, and provide a benchmark for future studies and computational approaches.
Authors
van der Stouwe van der Stouwe, Vermeulen Vermeulen, Eggink Eggink, Henriksen Henriksen, McGovern McGovern, Smit Smit, Aybek Aybek, Boon Boon, Brandsma Brandsma, Deik Deik, Erro Erro, Espay Espay, Falup-Pecurariu Falup-Pecurariu, Frucht Frucht, Ganos Ganos, Gatto Gatto, Helmich Helmich, Klein Klein, Koelman Koelman, Odekerken Odekerken, Koens Koens, Mahajan Mahajan, Morgante Morgante, MĆ¼nchau MĆ¼nchau, Panyakaew Panyakaew, Peall Peall, PĆ©rez-DueƱas PĆ©rez-DueƱas, Roze Roze, Sadnicka Sadnicka, Schneider Schneider, Schwingenschuh Schwingenschuh, Serranova Serranova, Skorvanek Skorvanek, Stamelou Stamelou, Tamas Tamas, Tinazzi Tinazzi, Valadas Valadas, Vandenberghe Vandenberghe, de Vries de Vries, Zutt Zutt, van der Veen van der Veen, Luiken Luiken, Tuitert Tuitert, Tijssen Tijssen
View on Pubmed