Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jan;56(1):78-85.
doi: 10.1002/jaba.961. Epub 2022 Nov 24.

Interobserver agreement and procedural fidelity: An odd asymmetry

Lindsay Essig  1 , Katarina Rotta  1 , Alan Poling  1
Affiliations

Interobserver agreement and procedural fidelity: An odd asymmetry

Lindsay Essig et al. J Appl Behav Anal. 2023 Jan.

Abstract

We examined articles with experiments published in the Journal of Applied Behavior Analysis and in Behavior Analysis in Practice from 2017 through 2021 to determine how frequently procedural fidelity was assessed. When procedural fidelity was assessed, we determined how often a measure of interobserver agreement for those fidelity data was provided. We also determined how often a measure of interobserver agreement for participants' behavior was provided. Across both journals and all years, 54.7% of relevant articles provided a measure of procedural fidelity. Of them, 17.7% provided a measure of interobserver agreement for procedural fidelity. In marked contrast, 96.4% provided interobserver agreement data for participants' behavior. It is unfortunate that applied behavior analysts frequently fail to provide procedural fidelity data and, when they do, often fail to provide interobserver agreement data for the fidelity data. Reviewers for, and editors of, behavior-analytic journals are encouraged to strongly consider the relative value of procedural fidelity and agreement on procedural fidelity measures when rendering recommendations on the suitability of a given submission.

Keywords: human observers; interobserver agreement; procedural fidelity; treatment integrity.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Percentage of Articles that Provided Procedural Fidelity Data Across Years and Journals Note. BAP = Behavior Analysis in Practice; JABA = Journal of Applied Behavior Analysis.
Figure 2
Figure 2
Percentage of Articles that Provided IOA Data for the Dependent Variable and Independent Variable (Procedural Fidelity) Across Years and Journals Note. IOA = interobserver agreement; DV = dependent variable; IV = independent variable; BAP = Behavior Analysis in Practice; JABA = Journal of Applied Behavior Analysis.
See this image and copyright information in PMC

References

    1. Andzik, N. R. , Cannella‐Malone, H. I. , & Sigafoos, J. (2016). Practitioner implementer functional communication training: A literature review. Research and Practice for Persons with Severe Disabilities, 41(2), 79–89. 10.1177/1540796916633874 - DOI
    1. Ayllon, T. , & Michael, J. (1959). The psychiatric nurse as a behavioral engineer. Journal of the Experimental Analysis of Behavior, 2(4), 323–334. 10.1901/jeab.1959.2-323 - DOI - PMC - PubMed
    1. Baer, D. M. , Wolf, M. M. , & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. 10.1901/jaba.1968.1-91 - DOI - PMC - PubMed
    1. Cooper, J. O. , Heron, T. E. , & Heward, W. L. (2020). Applied behavior analysis (3rd ed.). Pearson.
    1. DiGennaro Reed, F. D. , & Codding, R. S. (2014). Advancements in procedural fidelity assessment and intervention: Introduction to the special issue. Journal of Behavioral Education, 23(1), 1–18. 10.1007/s10864-013-9191-3 - DOI