An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs

Abstract

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5-8, 8-12 and 5-12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Fig 1. Flow chart depicting the various stages of questionnaire development and evaluation, each given with their respective aims and sample sizes.

Fig 2. Theoretical probability plot showing an idealised association between a trait score and later training outcome.

The green box surrounds a range of the score within which only qualified dogs scored, the yellow box indicates dogs who’s scores would be within the range for a greater than 50% chance of withdrawal but not extreme, whilst the red box indicates the range of the score where only dogs that were later withdrawn scored, representing those extremely unsuitable to guiding. Cut-offs were identified in the scores that marked the edges of these zones, and green, yellow and red flags assigned accordingly to dogs that fell within them.

Fig 3. Stacked bar chart showing the percentage of green flags that would have been given out at each assessment, broken down by the dog’s final training outcome.

Breeding, dogs selected as breeding stock; Qualified, dogs that qualified as working guide dogs; W-Beh, dogs withdrawn for behavioural reasons; W-Health, dogs withdrawn for health reasons; W-Health & Beh, dogs marked as withdrawn for both health and behavioural reasons.

Fig 4. Stacked bar chart showing the percentage of red flags given that would have been out at each assessment, broken down by the dog’s final training outcome.

Breeding, dogs selected as breeding stock; Qualified, dogs that qualified as working guide dogs; W-Beh, dogs withdrawn for behavioural reasons; W-Health, dogs withdrawn for health reasons.

Fig 5. Stacked bar chart showing the percentage of yellow flags that would have been given out at each assessment, broken down by the dog’s final training outcome.

Breeding, dogs selected as breeding stock; Qualified, dogs that qualified as working guide dogs; W-Beh, dogs withdrawn for behavioural reasons; W-Health, dogs withdrawn for health reasons; W-Health & Beh, dogs marked as withdrawn for both health and behavioural reasons.

Fig 6. Stacked bar chart showing the percentage of dogs (broken down by training outcome) that would have received either a green, yellow or red flag had the final system been implemented.

Green flags were assigned using the 5, 8 and 12-month data, whilst red and yellow flags were assigned using the 8 and 12-month data only. Breeding, dogs selected as breeding stock; Qualified, dogs that qualified as working guide dogs; W-Beh, dogs withdrawn for behavioural reasons; W-Health, dogs withdrawn for health reasons; W-Health & Beh, dogs marked as withdrawn for both health and behavioural reasons.