Comparison of Two Methods of Teaching Early Childhood Professionals to Score a Developmental Screening Test

Journal of Allied Health, Summer 2009 by Mayson, Tanja A, Hayes, Virginia E, Harris, Susan R, Backman, Catherine L

Increasingly, computer-assisted learning is becoming an educational method of choice. This study compared the effectiveness of in-class versus Internet-based training in achieving reliability when administering a developmental screening test, the Alberta Infant Motor Scale. Forty-eight early childhood professionals, including physical therapists, occupational therapists, nurses, and infant development consultants, took part in the study. Participants in this convenience sample were each assigned to one of the two learning groups. We assessed interrater reliability and participants' satisfaction with training method. Disciplines were equally distributed within the two groups, but geographical locations differed significantly. There was no difference in intraclass correlation coefficients for interrater reliability between the two groups. Although there was no difference in overall satisfaction with the quality of the courses, significant differences were found in the trainees' satisfaction with certain aspects of the courses. Although several study limitations existed, Internet-based training provides a feasible option for training practitioners to reliably use developmental screening tests such as the Alberta Infant Motor Scale. J Allied Health 2009; 38:100-105.

MANY METHODS exist for training health care professionals to reliably administer developmental tests, with computer-assisted instruction becoming more prevalent. Benefits of this training mode include increased flexibility, self-paced learning, one-on-one instruction, decreased teaching time, and enhanced accessibility.1'3 In a geographically large country such as Canada, in which pediatrie practitioners may live far from major training sites, accessibility to training is key. Early childhood professionals in rural areas often have difficulty accessing such opportunities due to travel costs to major centers where courses are held, as well as a lack of colleagues to cover their services during these absences. Busy lifestyles may also deter access to special training opportunities, even when locally available.

One method of computer-assisted training involves an Internet-based platform using written text, photographs, video clips, and graphics to enable interactive engagement with conceptual material and skill development. Advantages of computer-assisted courses include accessing the course from any geographical location and self-paced learning. Disadvantages include the need for and cost of technology and Internet access, lack of in-class peer interactions, and delayed response to learner questions.1 Advantages of an Internet-based course (as used in this study) over other non-Internet computerized methods (such as a CD-ROM course) include access to an expert "moderator" through e-mail, access to external links on the Internet, and the ability to collect test scores for reliability purposes on the Web site.

Although access to training is critical for learning, other factors must be considered as well. Training approaches should be evaluated in terms of whether they allow learners to achieve the level of performance required as well as receive timely and accurate feedback.4 Two studies have compared the effectiveness of teaching the administration of pediatrie assessment tools in-class versus by CD-ROM.5,6 Lim et al.5 compared in-class with CD-ROM training for scoring the Gross Motor Function Measure,7 and Liao and Campbell6 studied differences between in-class and CD-ROM training methods for the Test of Infant Motor Performance.8 No differences in reliability scores between the two modes of teaching were found in either study.

No published studies have compared in-class training with Internet training to screen child development or evaluated whether feedback was received in a timely and accurate manner. Therefore, our purpose was to compare and contrast the effectiveness of Internet-based and in-class modes of teaching on assessor reliability and learner satisfaction in participants taught to assess and score infants using a developmental screening test, the Alberta Infant Motor Scale9 (AIMS). We hypothesized that there would be no difference in computer-assisted learners' and in-class learners' ability to reliably score the AIMS, as well as no difference in learner satisfaction between the two training modes.

Methods

SCREENING TEST

The AIMS was selected as the screening test because of its use as an outcome measure in the authors' larger study on training and outcomes of early identification of infants with neuromotor delays.10 Although some tests, such as the AIMS,9 do not require assessors to complete formal training, test users are responsible for determining whether they have the skills and competence required to correctly administer the test.5 When collecting data for research or program evaluation purposes, the trustworthiness of test scores is enhanced by providing review of test protocols and assessing reliability.

As part of our larger project, assessors were trained to administer and score the AIMS, a reliable and valid test of motor development for infants at risk for motor delay. Intended for use by physical and occupational therapists, the AIMS focuses primarily on achievement of motor milestones and can be used to assess infants from birth until independent walking.9 It is a 58-item observational assessment requiring minimal handling and administration time of 10-20 mins9 and was standardized on a normative sample of 2,202 infants living in the province of Alberta.9 The AIMS has strong interrater reliability, test-retest reliability, and concurrent validity with the Bayley Motor Scale11 and the Peabody Developmental Motor Scales.12