Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Aug;60(8):1032-1040.
doi: 10.1177/10556656221089155. Epub 2022 Mar 28.

Exploration of Caregiver Interrater Agreement and Test-Retest Reliability on the Infant Cleft Observer Outcomes (iCOO)

Salene M W Jones  1 , Todd C Edwards  2 , Brian G Leroux  2 , Kathleen A Kapp-Simon  3 , Donald L Patrick  2 , Laura P Stueckle  4 , Janine M Rosenberg  3 , Meredith Albert  3 , Claudia Crilly Bellucci  3 , Cassandra L Aspinall  4 , Katherine Vick  2 , Carrie L Heike  4
Affiliations

Exploration of Caregiver Interrater Agreement and Test-Retest Reliability on the Infant Cleft Observer Outcomes (iCOO)

Salene M W Jones et al. Cleft Palate Craniofac J. 2023 Aug.

Abstract

Caregiver and observer-reported measures are frequently used as outcomes for research on infants and young children who are unable to report on their own health. Our team developed the Infant with Clefts Observation Outcomes Instrument (iCOO) for infants with cleft lip with or without cleft palate. This exploratory study compared test-retest and interrater reliabilities to inform whether differences in caregiver perspective might affect the iCOO.

This study is a secondary analysis comparing caregiver interrater agreement to test-retest reliability. Twenty-five pairs of caregivers completed the iCOO before surgery, 1 week later for test-retest reliability, 2 days after surgery, and 2 months after surgery. Reliability was assessed using intraclass correlations (ICCs) and t-tests were used to compare ratings between caregivers.

Infants had cleft lip (28%) or cleft lip and palate (72%). Primary caregivers were predominantly mothers (92%) and secondary caregivers were predominantly fathers (80%). Test-retest reliability met psychometric standards for most items on the iCOO (81%-86% of items). Caregiver agreement on the iCOO items was lower than test-retest reliability (33%-46% of items met psychometric standards). Caregivers did not systematically differ in whether they rated infants as healthier or less healthy than the other caregiver (5%-16% of items had statistically significant differences).

Caregivers used the measure consistently, but had different experiences and perceptions of their infant's health and functioning. Future studies are needed to explore mechanisms for the differences in test-retest and interrater reliability. Whenever possible, the same caregiver should provide ratings of the infant, including on the iCOO.

Keywords: ObsRO; child; cleft; infant; interobserver reliability; interrater reliability; reproducibility.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: The authors have no conflicts of interest to disclose.

References

    1. Edwards TE. Infant with Clefts Observation Outcomes Instrument (iCOO): A New Outcome for Infants and Young Children with Orofacial Clefts. Cleft Palate-Craniofacial J Published online 2021. - PMC - PubMed
    1. Benjamin K, Vernon M, Patrick D, Perfetto E, Nestler-Parr S, Burke L. Patient-Reported Outcome and Observer-Reported Outcome Assessment in Rare Disease Clinical Trials: An ISPOR COA Emerging Good Practices Task Force Report. Value Heal. 2017;20(7):838–855. - PubMed
    1. Walton M, Powers J 3rd, Hobart J, et al. Clinical Outcome Assessments: Conceptual Foundation-Report of the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force. Value Heal. 2015;18(6):741–752. - PMC - PubMed
    1. Deater-Deckard K, Mullineaux PY, Beekman C, Petrill SA, Schatschneider C, Thompson LA. Conduct problems, IQ, and household chaos: A longitudinal multi-informant study. J Child Psychol Psychiatry Allied Discip. 2009;50(10):1301–1308. - PMC - PubMed
    1. Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The Brief Infant-Toddler Social and Emotional Assessment: Screening for Social-Emotional Problems and Delays in Competence. J Pediatr Psychol. 2004;29(2):143–155. - PubMed

Publication types