Development of Content-Valid Technical Skill Assessment Instruments for Athletic Taping Skills

Journal of Allied Health, Fall 2006 by Butterwick, Dale J, Paskevich, David M, Lagumen, Niko G, Vallevand, Andrea L C, Lafave, Mark R

Background and Purpose: The content validity of technical skill assessment instruments (TSAI) for the skills of athletic taping has not been reported. The purpose of this paper is to outline and present the process of content validation for nine TSAIs for athletic taping. Local and national validators were selected from Canadian Athletic Therapists' Association (CATA)-accredited athletic therapy (AT) programs to serve as content validators.

Methods: The process of content validation began with the creation of a detailed task analysis via mail and simple validation by local validators. Subsequently, the detailed task analysis was committee validated by a group of 10 validators from across Canada. Validators judged the importance and difficulty of each item, and a face-to-face committee-validator meeting established consensus on the majority of checklist items. Through a modified Ebel procedure, frequency distribution was used in the formation of the final TSAIs.

Results: Initial consensus for pre-taping assessment and technical skill performance items was low. Upon committee discussion and lack of agreement, the decision to remove pretaping assessment items was made. Initial results of importance and difficulty for athletic taping technical skills were low prior to the committee meeting. Results of importance and difficulty improved substantially following the face-to-face committee-validators meeting. Consensus on fail points improved from initial to final committee validation.

Conclusion: The process of simple and committee validation can be seen as effective methods to establish the content validity of instruments used for the evaluation of athletic taping. J Allied Health 2006; 35:147-155.

IN THE FIELD of athletic training or athletic therapy (AT), technical skills such as athletic taping, clinical and field injury assessment, exercise rehabilitation, clinical application of therapeutic modalities, and basic and advanced first aid techniques are performed. The use of valid and reliable technical skill assessment instruments (TSAIs), which evolve from a thorough task analysis for the formative and summative assessment of these technical skills, has not been reported in the field of AT. Only one study has evaluated the athletic taping skills of AT students. Herrmann developed a 10-item checklist based on input from several certified athletic trainers in the United States to evaluate the success of two different educational strategies for teaching novice AT students how to tape.1 There is no indication in the literature that Herrmann's checklists are recognized as a criterion or standard for assessment of athletic taping skills.

Recently (in the past two decades), medical and paramedical professions have begun to create valid and reliable tools to evaluate technical skills. Investigators2^7 have shown increasing interest in the effective performance and assessment of technical skills inherent in many medical and paramedical professions. This is particularly important when these technical skills reflect a basic level of competence or professional standards such as minimal performance levels, licensure, and certification examinations in the health professions.8

The literature describes several advances in creating valid and reliable evaluation tools. These include an objective structured clinical examination,4 objective structured assessment of technical skills,2 global rating scales,3 and case-specific checklists.6 Global rating scales evolve from the detailed task analysis. Gorter et al. reviewed the results of 29 articles in an attempt to summarize the methods used to create case-specific checklists assessing physician clinical performance.6 They advocated complete and accurate publication of methods used to create, and subsequently to validate, such checklists.

This background led us to consider both task analysis and TSAI development as goals for a content-valid athletic taping assessment instrument. Hambelton and Novick stated that the process of task analysis is often ignored or is only conducted in an informal basis,9 while Ebel and Frisbie emphasized that expert judgment is an essential step in establishing validity.10 TSAI construction is critical to ensure the validity and ultimately the reliability of the examiner's assessment of the student.6 Potential benefits include the formative assessment of students," structured and focused feedback on the student's performance,12 and encouraging the instructor to clearly define parameters of appropriate performance.5

Well-structured assessment tools can provide the instructor with information as to what individual and group items need to be reviewed and what tasks are consistently being performed well.12 They can also help monitor program curriculum success."

The purpose of this study was to use a thorough athletic taping task analysis to develop the content validity of pretaping assessment and athletic taping TSAIs. The task analysis and subsequently the TSAIs were designed for evaluation of student pretaping assessment and athletic taping skills applied to the ankle, thumb, and elbow. This project received ethical approval from the Conjoint Health Research Ethics Board of the Faculties of Medicine, Nursing and Kinesiology at the University of Calgary.