Measuring practice element procedural knowledge: How do trainees PERForm?

Overview

Assessment of intervention competence and adherence to evidence-based practices (EBP) has gained substantial attention in recent years. A variety of methods for measuring procedural knowledge and adherence abound; yet many are resource-intensive, may not be feasible across educational and mental health service delivery systems, and are often associated with a specific treatment manual. This demonstration project assessed the feasibility of using clinical vignette methodology to examine EBP procedural knowledge. Learners across three different samples read a case vignette of a youth with either comorbid depression and disruptive behavior or anxiety and then wrote how they would apply a technique (problem solving or exposure) with that youth. Coding these open-ended responses using the Practice Element Response Form (PERForm) yielded excellent interrater reliabilities for problem solving (intraclass correlation coefficient [ICC] M = .80) and exposure (ICC M = .81). Trainees’ scores increased significantly from pre- to posttraining on the PERForm with medium to very large effect sizes. Results suggest that use of clinical vignette methodology such as that used by the PERForm may be an additional approach to evaluating EBP procedural knowledge that offers the benefits of feasibility, reliability, and sensitivity to training. (PsycINFO Database Record (c) 2017 APA, all rights reserved)