|
|
|
Page Last
Updated: Copyright 2005 Worksupport.com©
|
|
|
 Proed Article |
||||||
|
Focus on Autism and Other Developmental
Disabilities, Summer 2003 v18 i2 p75(13) Designing an outcome study to monitor the progress of students with autism spectrum disorders. Joel R. Arick; Helen E. Young; Ruth A. Falco; Lauren M. Loos; David A. Krug; Marilyn H. Gense; Steven B. Johnson. Copyright© PRO-ED, Inc. Reprinted with permission The Autism Spectrum Disorders Outcome Study is tracking the educational progress of 67 students, between the ages of 2 and 6 years, whose primary diagnosis for services is an autism spectrum disorder. This article describes the study, how student outcomes have been measured, and how student progress has been reported to service providers and parents. Outcome data has been collected from performance observations, parent and teacher surveys, and standardized assessments. The strength of this data-collection approach is that it uses a variety of sources and multiple methods to monitor student progress. Initial results, based on the first 16 months of the study, have shown that the majority of the children have made significant progress in the areas of social interaction, expressive speech, and use of language concepts. In addition, they have displayed significant decreases in behaviors associated with autism spectrum disorders. This study began in 1998 and will continue at least through August of 2003. No area of early intervention and early childhood special education (EI/ECSE) has sparked as much controversy in recent years as has the provision of services for young children with autism spectrum disorders. Service providers and parents are often confused about the effectiveness of various intervention practices (Hurth, Shaw, Izeman, Whaley, & Rogers, 1999). This confusion has led to increases in complaints, due process hearings, and legal proceedings, which all sometimes result in highly stressful relations between service providers and parents. Although during the past decade families, their advocates, and professionals have engaged in extensive debates about the efficacy of various treatments and educational strategies, documentation has emerged indicating that intensive early intervention can have significant, positive outcomes for young children with autism spectrum disorders. These positive effects include acceleration of developmental rates, significant language gains, improved social behavior, and decreased symptoms of autism (Rogers, 1996). Several promising programs for children with autism spectrum disorders have been documented (Dawson & Osterling, 1997; Green, 1996; Greenspan & Wieder, 1997; Harris & Handleman, 1994; Lovaas, 1987; Olley, Robbins, & Morrelli-Robbins, 1993; Rogers, 1996; Smith, Groen, & Wynn, 2000). The most well known of these programs (Lovaas, 1987) reported that 9 of 19 children who received intensive early intervention successfully completed first grade and obtained average or above-average IQ scores. This program, which used a behavioral approach, included discrete trial teaching methods, recommended 40 hours per week of one-to-one intervention for up to 2 years, and emphasized remediation of speech and language deficits provided, at least initially, in the child's home (Lovaas, 1996). Advocates of this program have indicated to parents that a substantial improvement in their children's long-term functioning is possible if they obtain intensive behavioral intervention for their children (Maurice, Green, & Luce, 1996). Many programs have documented their effectiveness, and some have even claimed that certain treatments they provide for children with autism spectrum disorders are superior to other treatments, but there have been essentially no studies comparing different comprehensive interventions of equal intensity (National Research Council, 2001). Even though there is disagreement on the best teaching methods, nationally known and validated educational programs for young children with autism spectrum disorders agree that in addition to early intervention, services should include specialized curriculum, individualization, intensity of engagement, systematic instruction, and family involvement (Hurth et al., 1999). Furthermore, it is now widely accepted that programs based on a behavioral model have shown to have the broadest empirical validation for effectively teaching children with autism spectrum disorders (Schreibman, 2000). Many EI/ECSE providers are responding to the challenge of providing the best services possible to young children with autism spectrum disorders. Service providers are initiating programs of more intensive services with varied approaches that have demonstrated promising outcomes for these children. In order to develop and sustain these programs, EI/ECSE programs will need to demonstrate positive outcomes for children with autism spectrum disorders. Even those school and home programs with which parents and service providers have been satisfied must document their results to determine the factors that have contributed to their success. Many researchers have recommended that measures other than the traditionally used standardized tests should be employed to determine the effectiveness of interventions. These measures may include the child's degree of success in communicative exchange, social competence, peer relationships, and competence in natural environments (Prizant & Wetherby, 1998). By using a variety of outcome measures, researchers could discover benefits of interventions or instruction that are not captured by traditional standardized tests. In 1998, the Oregon Department of Education, realizing the importance of documenting outcome results, contracted with Portland State University to design a study to collect outcome data for young Oregon students with autism spectrum disorders. The Autism Spectrum Disorders Outcome Study was developed at that time. The study began in October of 1998 with 67 students and will continue at least through August of 2003. Approximately 50 new students were added to the study in September of 2001, increasing the total number of student participants to 118. This article describes the design of the Autism Spectrum Disorders Outcome Study, including the methods used to measure student outcomes and how student progress was reported to service providers and parents. In addition, some initial results collected through August 31, 2000, are discussed. Method Design of Study This study was designed to track program implementation variables and outcome data for students with autism spectrum disorders engaged in school or home programs. When planning this study, we used evaluation elements recommended in current reviews of research regarding early intervention for young children with autism spectrum disorders. These evaluation elements included 1. a standard protocol of assessments that captures a range of skills and symptoms at specific points in time (Rogers, 1996), 2. clear descriptions of the characteristics of children participating in the programs (Olley et al., 1993), 3. clear descriptions of the components of the programs and how they were implemented (Olley et al., 1993), and 4. the documenting and accounting of variables outside of the study intervention package that is the object of the study (Prizant & Rubin, 1999). During the initial planning phase of the project, the research team (three of the primary researchers each had more than 25 years of experience in the field of autism spectrum disorders) determined that the purpose of collecting the student outcome data would be to provide the following: * objective individual student outcome data for participating programs and parents; * information to assist service providers in planning student programs; * a description of the various instructional strategies being used by programs serving students with autism spectrum disorders; * a comparison of the outcome data results of various school- and home-based programs; * information on the effectiveness of specific instructional strategies; and * a framework for a statewide database of student characteristics, student assessment data, and program implementation strategies that would allow for longitudinal tracking of students and program performance. Numerous data-collection methods were used to acquire the information for this study. Outcome data were collected at baseline and quarterly, biannually, and annually. Combinations of measures were used to collect the data. These measures included standardized tests, non-standardized forms, surveys, and interviews. Participants The participants were 67 preschool students who were between the ages of 2 and 6 (at baseline) and whose primary diagnosis for services was an autism spectrum disorder. This diagnosis was confirmed by the initial assessment results. The primary monitoring instrument used to assess the students in the study was the Autism Screening Instrument for Educational Planning--Second Edition (ASIEP-2; Krug, Arick, & Almond, 1993). Baseline scores from the students in the study matched the students from the national standardization data of ASIEP-2, within one standard deviation of the population mean. All eight Oregon regional programs for students with autism spectrum disorders were invited to nominate students. The participants were located in seven geographic areas of Oregon, in approximate proportion to the state's population distribution. Region I, located in the far eastern portion of the state, nominated no students to be included in the study. This area is located in one of the least populated sections of Oregon. The main criteria for selection was that the child had to be already involved in some type of school and/or home behavioral program. Parents were sent letters by their child's early intervention program asking them to voluntarily include their child in the study. Sixty-seven parents returned permission forms, and all of their children participated in the study. Information collected from baseline demographic surveys and the Student Learning Profile (Arick, Loos, Falco, & Krug, in press), completed by teachers at the beginning of the study, showed that 34% of the students were nonverbal, 79% engaged in self-stimulating behavior, 25% never engaged in imitative play, and 37% did not respond to simple commands, such as "come here." On average, the students received 18.5 hours (15.9 at school and 2.6 at home) of instruction per week. The majority of instructional hours occurred in small or large groups. All of the children received some one-to-one instruction (M = 6.8 hours per week). The most common one-to-one instructional strategies reported were functional routines, incidental teaching, pivotal response training, and discrete trial training. Table 1 shows that the 67 students represented approximately 10% of all the children ages 2 to 6 years in Oregon whose primary diagnosis for services was an autism spectrum disorder. The majority of the children in the study were 3 years old when the study began, and they represented approximately 27% of the 3-year-old children in the state of Oregon whose primary diagnosis for services was an autism spectrum disorder. The nine 2-year-olds in the study represented approximately 50% of the 2-year-old children in Oregon whose primary diagnosis for services was an autism spectrum disorder in the 1998 Oregon Census. Instruments In order to monitor the educational progress of the students, the assessment team administered numerous standardized tests to each student during the 3-year study. Assessments were administered at baseline and then quarterly and annually. In addition, information was collected from teachers, specialists, parents, and consultants. A more detailed description of each instrument follows. Autism Screening Instrument for Educational Planning. This standardized instrument was used to monitor each participant's progress over time. The subtests of this comprehensive instrument have been shown to be good monitors of progress, due to the lack of practice effects (Frye & Walker, 1998). In addition, the ASIEP-2 subtests have been used to reliably and validly assess students with autism (Turton, 1985). Four separate standardized ASIEP-2 subtests were administered:
1. the Autism Behavior Checklist, a screening instrument for nonadaptive behaviors, used to see how an individual looks in comparison to others; 2. the Sample of Vocal Behavior, which evaluates expressive speech at the preverbal and emerging language level and measures communication expressed by vocalizations accompanied by gestures or other means; 3. the Social Interaction Assessment, which elicits an individual's social responses in a controlled setting with stimuli presented in a systematic fashion; and 4. the Educational Assessment, which probes an individual's repertoire of adaptive language concepts and is designed to accumulate information that will be of direct value in curriculum placement. The Autism Behavior Checklist was completed at baseline and then once a year by each participant's teacher or other specialist. The Sample of Vocal Behavior, the Social Interaction Assessment, and the Educational Assessment were administered at baseline and then on a quarterly basis during the school year (fall, winter, and spring of each school year). Expressive One-Word Picture Vocabulary Test (Brownell, 2000). This norm-referenced test provides an assessment of an individual's English vocabulary. Fourteen students in the study reached the ceiling on the ASIEP-2 Sample of Vocal Behavior subtest; therefore, during their fifth assessment in the spring of 2000, those students were given the Expressive One-Word Picture Vocabulary Test as an alternative measure of their expressive language. The project staff administered this alternative assessment to these students on a quarterly basis during the school year. Extended Basic Academic Skills Assessment System (Tindal, McDonald, Crawford, & Tedesco, 2000). This assessment was developed by researchers at the University of Oregon for the state to use to assess a student's emerging skills in reading, writing, and math. Eleven students in the study reached the ceiling on the ASIEP-2 Educational Assessment and, during their fifth assessment in the spring of 2000, were given portions of the Extended Basic Academic Skills Assessment System as an alternative measure of their educational progress in reading, writing, and math. This assessment was administered to these students on a quarterly basis during the school year. Battelle Developmental Inventory: Cognitive Domain Screening Test (Newborg, Stock, Wnek, Guidubaldi, & Svinicki, 1984). This standardized assessment was used to measure each participant's conceptual skills and abilities. If there were previous age-equivalent scores for this assessment in the student's file, they were recorded as baseline scores during a file review at the beginning of the study. The Battelle Developmental Inventory was also administered on a yearly basis by a trained assessment team member who held a doctorate in special education. Vineland Adaptive Behavior Scales: Interview Edition (Sparrow, Balla, & Cicchetti, 1984). This standardized assessment was used to provide a general assessment of each participant's adaptive behavior. The participant's classroom teacher or another specialist completed a survey each school year. Student Learning Profile. This learning profile can be found in The Star Program: Strategies for Teaching Based on Autism Research (Arick, Loos, Falco, & Krug, in press). It was designed to assess and monitor growth in areas taught in the students' curriculum-based instruction. Assessment areas included expressive language, receptive language, daily routines, pre-academics, play behavior, and social interaction behavior. Teachers used the Student Learning Profile to record student progress on written programs. In addition, this profile was used to collect information on how the student requested wants or needs (e.g., verbal, picture systems, voice augmented system, sign language, gestures). This profile was completed at baseline and then on an annual basis by the teacher or other specialist. Program Implementation Checklist. This checklist was developed by the research team to gather detailed information from the teacher regarding each child's specific program. Information collected with this form included the total hours per week each participant received services, how the services were provided (e.g., in a group, pull out, one-to-one), what type of one-to-one or pull out teaching they received (e.g., pivotal response training, discrete trial), and who provided the services (e.g., teacher, educational assistant, related services staff member). In addition, this form was used to collect information on the students' written programs and note whether the student used augmentative or other alternative communication strategies (e.g., sign language, PECS [Picture Exchange Communication System; Frost & Bondy, 1991], picture/visual symbol systems). This form was completed at baseline and then on a biannual basis by the classroom teacher. Information on this form was verified by consultants during annual classroom visits at the end of each school year. Parent Survey. This survey was developed by the research team to gather information from the parents about their child's progress in the areas of communication, social interaction, and behavior. In addition, parents had the opportunity to comment on their satisfaction with their child's program, identify any special treatments they were using with their child, and describe any additional intervention services they were paying for with their own family funds. This survey was sent to parents at the end of each school year. Classroom Observation Form. This observation form was developed by the research team to collect information regarding the child's program, the student's involvement level in classroom activities, and the type of instruction the student received. In addition, the observation form included a section that allowed the observer to use a rubric to rate the appropriateness of six areas: placement, written programs, one-to-one instruction, group instruction, social interaction, and communication instruction. Special education professionals with expertise in designing and implementing programs for children with autism spectrum disorders were employed as consultants to conduct the classroom observations in the spring of each year. Procedures Training of the Assessment Team. Prior to the baseline visits, assessment specialists were trained extensively by special education professionals. They were taught how to complete file assessments, how to obtain and record age-equivalent scores when the Battelle Developmental Inventory had been previously administered, and how to interview the teacher and help him or her complete the Student Learning Profile and the Program Implementation Checklist. In addition to learning how to complete file reviews and interview the teachers to obtain necessary information, the new assessment specialists were taught how to score the subtests of the ASIEP-2 by two of the test's developers. They watched training videos and then practiced scoring by using prior videotapes of students who were given the ASIEP-2 subtests. They were then accompanied by previously trained special education professionals during all the baseline assessments. Baseline Assessment Visits. Baseline assessment visits were scheduled for each participant in the study. A team of assessment specialists arrived at each site and completed file reviews. Information obtained from student files included the participant's date of birth, autism spectrum disorder diagnosis date, and age-equivalent scores from the previously administered Battelle Developmental Inventory. They then assisted the teacher in completing the Student Learning Profile and the Program Implementation Checklist. The assessment specialists also collected the ASIEP-2 Autism Behavior Checklist, which were mailed to teachers prior to the visits. After meeting with the teacher, the assessment specialists administered the ASIEP-2 Sample of Vocal Behavior, Educational Assessment, and Social Interaction Assessment subtests to the student. Each subtest took between 10 and 30 minutes to complete. Specific administration instructions and protocol described in the ASIEP-2 manual regarding materials, stimuli, procedures, and scoring were followed. To ensure the accuracy of the scoring, the ASIEP-2 Sample of Vocal Behavior and Interaction Assessment subtests were videotaped and then watched and rescored by the assessment specialists after their visit. All numbers on the assessments were then rechecked by another specialist before the data were entered into the SPSS computer software data file. In addition, videotapes were randomly selected and scored by independent raters as an additional reliability check. Behavioral Instructional Strategies Training for Teachers. In an effort to strive for fidelity of implementation, training workshops in extensive behavioral instructional strategies for teaching children with autism spectrum disorders were conducted each school year. These workshops were conducted by special educational professionals with extensive experience in teaching instructional behavioral techniques for children with autism spectrum disorders. Each participating region was given at least two full-day training workshops per year. Topics covered included pivotal response training, discrete trial training, and functional routines. The materials were taken from The Star Program: Strategies for Teaching Based on Autism Research. Instruction in data-collection strategies was included in the training workshops. All teachers and staff members who taught students participating in the study were invited and encouraged to attend all training workshops.
The following results are pre-post comparisons of student progress data, survey data collected from parents and teachers, and observational data collected during the first 16-month instructional period. These initial results are provided to show how the Autism Spectrum Disorders Outcome Study was used to monitor the progress of students with autism spectrum disorders. Standardized Tests Table 2 shows the means and standard deviations of the students' baseline assessment scores and their last 1999/2000 school year assessments, completed at 12 to 16 months into the study. The ASIEP-2 Autism Behavior Checklist was completed by the teacher, and all of the other assessments were completed by the research study assessment team. Using paired t tests, the research team found significant differences on several components of each subtest, as well as on the total scores for each subtest. The results of the ASIEP-2 Autism Behavior Checklist showed that there were significant decreases in behaviors associated with autism from students' baseline assessment (M = 70.47, SD = 19.82) to their final assessment (M = 61.60, SD = 25.86), t(59) = 2.476, p [less than or equal to] .05. The total score for the ASIEP-2 Educational Assessment showed that there were significant increases in the number of students' correct responses from their baseline assessment (M = 28.82, SD = 12.63) to their final assessment (M = 37.90, SD = 15.44), t(59) = -7.629, p [less than or equal to] .01. The results from the ASIEP-2 Social Interaction Assessment showed that the students had significant decreases in their total autism social interaction scores (students were displaying more social interactions with the adults present during the assessment) from their baseline assessment (M = 65.21, SD = 15.35) to their last assessment (M = 56.19, SD = 18.60), t(56) = 4.111, p [less than or equal to] .01. Figure 1 shows the language gain or loss for each student. The student who made the most progress started out with a baseline verbal language age of 41 months. This child had a gain of 43 months and, at the 5th assessment, 16 months into the study, had a verbal language age of 84 months. The average language age gain for all students was 10 months in the first 16 months. There was a significant increase between the students' baseline assessment scores (M = 23.21, SD = 8.50) and their last assessment scores (M = 33.51, SD = 16.70) in the spring of 2000, t(55) = -6.848, p [less than or equal to] .01. Seventy-two percent of the students made some language age gain, and 36% gained 16 or more months in the first 16 months (as shown by the dotted line on the graph in Figure 1). Table 3 shows that no correlations were found between the students' chronological age at the first assessment and their language age gain. Students were just as likely to make gains at any chronological age. However, the students in this sample were within the narrow age span of 2 to 6 years. This finding also indicates that maturation does not influence student gains. There were correlations found between the students' language age gain and other variables. Table 3 shows that there was a significant relationship found between the 16-month language age gain and (a) the students' baseline Battelle Developmental Inventory age-equivalent scores, r(27) = .498, p [less than or equal to] .05, taken from their file reviews before the project started; (b) the students' Battelle Developmental Inventory age-equivalent scores, r(53) = .511, p [less than or equal to] .05, from the assessments administered in winter of 2000; and (c) the students' winter 2000 I.Q. scores, r(64) = .469, p [less than or equal to] .05. Parent Survey Information At the end of the 1999/2000 school year, parents were sent surveys and asked to provide input on their children's progress. Forty-one parents (61%) returned their surveys. Table 4 shows the responses from parents when they were asked to advise us of changes in their child's skills and behaviors during the 1999/2000 school year. A skill or behavior was listed on the survey, and parents were asked to rate whether the skill or behavior had decreased, stayed the same, or increased. In each of the 10 areas listed, the majority of parents thought their child's skill or behavior had increased over the school year. The highest percentage of parents agreed that their child's skills had increased in the area of using language or other means (e.g., augmentative or other alternative communication strategies) to communicate. Ninety-three percent of the parents thought their child's skills had increased in this area. In the specific area of labeling, there was a significant relationship found between the 16-month language age gain and the parents' positive responses, r(32) = .473, p [less than or equal to] .05, when asked to rate any changes in their child's ability to label items and pictures in response to questions.
Family Involvement. Family involvement is important to a child's progress and was addressed in two questions on the parent survey. When parents were asked to rate their involvement level in their child's early childhood or school-age program, their mean rating was 7.66, where 1 = not involved and 10 = intensely involved. When asked to rate how satisfied they were with their involvement level, their mean rating was 6.95, where 1 = not at all satisfied and 10 = extremely satisfied. Parents rated themselves above average on both questions. Other Treatments Reported by Parents. Some researchers think outcome research has ignored outside variables that may contribute to a child's progress (Prizant & Rubin, 1999). Therefore, to determine if other factors were affecting their child's educational progress, parents were asked to list any services or treatments they provided to their child that were not paid for by their child's early intervention or school-age program. Table 6 shows all the parents' responses. Common treatments reported by parents included gluten-free/casein free diets, secretin, and vitamins. Classroom Observation Trainings in extensive behavioral instructional strategies for teaching children with autism spectrum disorders were conducted each school year. Instruction on data-collection strategies were included in the workshops. As a follow-up to the trainings, special education professionals with expertise in behavioral instructional strategies and extensive knowledge in designing and implementing early education and school-age programs for children with autism spectrum disorders visited each classroom. They observed the student, viewed data collected on the student, and gave individual consulting advice to the teacher. The Classroom Observation Form was completed during the visit, and a comment sheet was given to the teacher. While in the classroom, the consultant observed the student and his or her program and then completed a form rating the appropriateness of six areas: placement, written programs, one-to-one instruction, group instruction, social interaction, and communication instruction. Table 7 shows that the ratings in these areas ranged from 2.30 to 2.93 on a scale of 1 (not appropriate) to 4 (very appropriate). When examining the area of communication, there was a significant relationship found between the 16-month language age gain and the rating given by consultants, r(48) = .424, p [less than or equal to] .05, when they were asked to rate whether or not they thought the communication instruction in the classroom was appropriate for the child. Validity and Reliability of Instruments
Discussion The initial results collected by the study demonstrate how the design used for the Autism Spectrum Disorders Outcome Study can be followed by others to monitor the progress of students with autism spectrum disorders. The data show that many of the students have made significant progress in the areas of social interaction, expressive speech, and adaptive language concepts. Additionally, two factors (students' initial cognitive level and experts' rating of the appropriateness of students' classroom communication instruction) significantly correlated with the students' verbal language age gain. Student reports that documented each
individual child's progress were generated each quarter and sent out to
parents, programs, and teachers (see Figure 2). Parents and teachers
were extremely cooperative in returning the needed forms and surveys for
collecting data. Many parents voiced appreciation at being able to
provide information on their child's progress, welcomed the opportunity
to have their children monitored, and looked forward to receiving the
quarterly student reports. Many of the teachers expressed that the
outcome data from the study provided them with valuable information to
use in curriculum planning for their students.
* additional family demographic information (e.g., family characteristics, ethnicity, socioeconomic status); * information on whether or not the family receives support, such as counseling and training; * baseline scores for the Battelle Developmental Inventory and the Vineland Adaptive Behavior Scales for all new students (instead of relying on scores obtained from prior records); * a more specific measurement of the students' social interaction with peers; * more detailed information on outside factors that could affect progress, such as diet and other treatments; and * information on the staff members working with each student (e.g., prior training, educational level, degree). This new cohort of students includes only children who, at their baseline assessment, were less than 48 months of age and were going to receive early intervention services for at least 2 years. In addition, they had to have a recent educational diagnosis to receive service for an autism spectrum disorder. Accepting only newly diagnosed children helps control for any interventions used before the student was admitted to the study and enables a clearer analysis of any possible relationships between student outcomes and intervention variables. One of the primary goals of this study was to collect outcome data to help service providers' plan effective programs for students with autism spectrum disorders. The verbal feedback received so far from teachers has shown that service providers have appreciated the information and are using it to help monitor student progress and plan educational programs. At the end of the study, the data amassed will be reported and made available to assist others in planning effective programs for their students with autism spectrum disorders. In addition to giving service providers a way to monitor student progress and to help them with program planning, the information produced by this study could potentially be used to identify effective intervention strategies to use with students with autism spectrum disorders. The strength of the data-collection approach used to collect information for this study is that it used a variety of sources and multiple methods to obtain the outcome data. Progress was measured by observation of performance, surveying of parents and teachers, and use of standardized assessments. Data collected on intervention strategies (e.g., type of teaching, number of instruction hours, group size) and other variables (e.g., quality of instruction, student demographics, outside factors) will continue to be analyzed, we hope to identify factors that could have contributed to positive outcomes, such as the gains found in the majority of the students' verbal language age (see Figure 1). Although the tracking and reporting of student progress is useful, we also hoped that when the outcome data is more thoroughly analyzed at the end of the study, we will be able to identify specific intervention strategies that will lead to effectively educating children with autism spectrum disorders. TABLE 1
Comparison of Participants to all Students in Oregon Whose
Primary Disabling Condition Is an Autism Spectrum Disorder
Age Number of children Percentage of students
group in Oregon represented
(in years) Study (a) Oregon (b) in the study
2 9 18 50
3 23 86 27
4 22 126 17
5 10 196 5
6 3 217 1
All students
(ages 2-6) 67 643 10
(a) As of November 30,1998. (b) Taken from the 1998 Oregon Census
of children whose primary disabling condition
was an autism spectrum disorder.
TABLE 2
Descriptive Statistics of ASIEP-2 Subtests
Range
of
possible
Area assessed n scores
Autism Behavior Checklist
Body/object use 60 0-38
Language 60 0-31
Total score 60 0-158
Educational Assessment
Receptive language 60 0-12
Expressive language 60 0-12
Body concept 60 0-12
Speech imitation 60 0-12
Total score 60 0-60
Social Interaction Assessment
Appropriate social interactions 57 0-48
Self-stimulation/nonresponsive 57 0-48
to adult
Total score 57 0-96
Vocal Behavior
Noncommunicative utterances 60 0-50
Unintelligible utterances 60 0-50
Words used during sample 59 na
Expressive language age score 56 na
Scores at
12 to 16
Scores at months
baseline into study
Area assessed M SD M SD
Autism Behavior Checklist
Body/object use 12.03 7.08 9.90 * 7.87
Language 14.07 6.10 12.23 * 5.97
Total score 70.47 19.82 61.60 * 25.86
Educational Assessment
Receptive language 4.98 3.08 6.87 ** 3.50
Expressive language 2.83 2.78 4.63 ** 4.30
Body concept 4.38 3.80 7.27 ** 4.37
Speech imitation 5.22 3.40 7.37 ** 4.10
Total score 28.82 12.63 37.90 ** 15.44
Social Interaction Assessment
Appropriate social interactions 5.63 5.27 9.18 ** 8.15
Self-stimulation/nonresponsive 22.86 11.88 17.37 ** 12.60
to adult
Total score 65.21 15.35 56.19 ** 18.60
Vocal Behavior
Noncommunicative utterances 35.97 14.03 23.17 ** 18.20
Unintelligible utterances 37.41 14.08 24.68 ** 20.43
Words used during sample 25.39 36.0 52.37 ** 52.32
Expressive language age score 23.21 8.50 33.51 ** 16.70
Note. ASIEP-2 = Autism Screening Instrument for Educational
Planning--Second Edition (Krug et al.,
1993).
(a) Assessment administered at approximately 12 months,
(b) Assessment administered at approximately 16
months.
* p [less than or equal to] .05. ** p [less than or equal to] .01.
TABLE 3
Correlations with Gains in Verbal Age (16 Months Into the Study)
Variable n M Pearson
correlation
Chronological age at first
assessment (in months) 56 51.00 .073
Battelle Developmental
Inventory (cognitive domain) 29 14.70 .498 **
age-equivalent scores given
before the project began
Battelle Developmental
Inventory (cognitive domain)
age-equivalent scores
administered in winter 2000 55 27.89 .511 **
IQ score--Winter 2000 55 44.66 .469 **
Note. Battelle Developmental Inventory (Newborg et al., 1984).
** p = .05.
TABLE 4
Parents' Responses on Changes in Skills or Behaviors of Their Children
Parent response
Stayed
Decreased the same Increased
Skill or behavior (%) (%) (%)
Using language or other means
to communicate 0 7 93
Using spontaneous communication 0 15 85
to request foods, toys,
or activities
Labeling items and pictures 0 27 73
in response to questions
Understanding and responding to 0 10 90
to directions
Imitating other children and 0 34 66
and adults during play
Playing with toys in ways 0 29 71
that are appropriate to
his/her age
Playing with other children 3 29 68
Engaging in imaginative 0 46 54
or pretend play
Demonstrating self-care and 3 29 68
independence in areas such
as eating, dressing, or
toileting
Behaving appropriately 2 25 73
TABLE 5
Parents' Specific Examples of Changes in Their Childrens'
Skills or Behaviors
My child is more aware of surroundings. (6)
My child now wants to interact with others
more. (6)
My child's communication skills have
improved. (5)
Expressive communication has greatly
increased. (3)
My child's generalizing skills learned at
school. (2)
Decrease in inappropriate behavior. (2)
My child has learned to use PECS. (2)
My child screams when he or she doesn't
like something.
Gross motor skills have increased drastically.
Fewer tantrums.
Babbles more.
This has been a positive year, with gains in
all areas.
My child still needs work on regulating self.
Learned structured teaching schedule.
My child is making amazing progress.
Still has trouble transitioning, but it is
getting better.
Can sometimes calm self.
My child has become less tolerant of peers
with disabilities.
Throws self on floor if asked to go to the
bathroom.
Has intense need to be squeezed.
My child seems to be more mature.
Increased show of affection.
Increased inappropriate vocalizations.
Has become dependent upon aide for
prompts.
Self-management skills are growing.
More connected to other people.
Likes to help others.
Attempting consonants.
My child is able to ride a bike.
Responds better to directions.
My child sings to self.
Has greater sense of the rhythm of
language.
Learns very quickly after seeing things
demonstrated.
My child is being silly at inappropriate times.
Very little changes this past year.
My child is developing a sense of humor.
Note. Number in parentheses is the number of parents who gave
this response. Responses that are not noted were given by only
one parent. PECS = Picture Exchange Communication System
(Frost & Bondy, 1991).
TABLE 6
Examples of Services and Treatments Paid for by Parents
Gluten-free and casein-free diet (8)
Secretin (7)
Vitamins (6)
Dimethylglycine (5)
Applied behavior analysis programs (4)
Magnesium (3)
B-6 vitamins (3)
Swimming (3)
Dairy-free diet (3)
Speech therapy (3)
Private preschool (3)
Gluten-free diet (3)
Yeast-free diet (2)
Audio sensory training (2)
Respite care (2)
In-home aide to assist with functional skills (2)
Occupational therapy (2)
Swim therapy
Psychologist
Music therapy
Melatonin
Violin
Naturopathy physician's care
Floor time therapy
Vision therapy
Private therapy
Home program teaching independent tasks
Tutoring for academic skills
Therapeutic horseback riding
Less sugar
Autism Research Project at Oregon Health
Science University
Autism Research Institute
Zinc
Calcium
Naturopath herb regimen
Prozac
Note. Number in parentheses is the number of parents who gave
this response. Responses that are not noted were given by
only one parent.
TABLE 7
Results From Classroom Observations
Area observed n Mean rating
Placement 57 2.93
Written program 54 2.30
One-to-one instruction 41 2.88
Group instruction 53 2.83
Opportunity for social interaction 57 2.66
Communication instruction 57 2.89
Note. 1 = not appropriate; 4 = very appropriate.
TABLE 8
Concurrent Validity Between Assessment Instruments and
Test-Retest Reliability of Assessments Administered Each Quarter
Assessments compared n Pearson correlation
Validity of ASIEP-2 Autism Behavior
Checklist Total Score compared to
VABS Composite Age Equivalent at
4th assessment 57 -.538 *
Validity of ASIEP-2 Social
Interaction Assessment Percentile
Rank compared to VABS Socialization
Domain Raw Score at 4th assessment 61 -.492 *
Validity of ASIEP-2 Sample of Vocal
Behavior Percentile Rank compared
to VABS Communication Domain Raw
Score at 4th assessment 61 -.686 *
Validity of ASIEP-2 Sample of Vocal
Behavior Expressive Language Age
compared to VABS Communication
Domain Age Equivalent at
4th assessment 61 .692 *
Validity of ASIEP-2 Educational
Assessment Percentile Rank
compared to VABS Written
Subdomain Raw Score at 4th
assessment 61 -.534 *
Validity of ASIEP-2 Sample of Vocal
Behavior Expressive Language Age
compared to BDI Cognitive Age
at 4th assessment 66 .686 *
Validity of BDI Cognitive Age
compared to VABS Communication
Domain Age Equivalent at
4th assessment 61 .782 *
Reliability of ASIEP-2 Interaction
Assessment Percentile Rank: 4th
assessment compared to 5th
assessment 63 .647 *
Reliability of ASIEP-2 Sample of
Vocal Behavior Percentile
Rank: 4th assessment compared
to 5th assessment 50 .854 *
Reliability of ASIEP-2 Educational
Assessment Percentile Rank: 4th
assessment compared to
5th assessment 52 .875 *
Note. ASIEP-2 = Autism Screening Instrument for Educational
Planning-Second Edition (Krug, Arick, & Almond, 1993); VABS =
Vineland Adaptive Behavior Scales-Interview Edition (Sparrow,
Balla, & Cicchetti, 1984); BDI = Battelle Developmental Inventory
(Newborg, Stock, Wnek, Guidubaldi, & Svinicki, 1984).
* p = .01.
FIGURE 2. Sample of quarterly student assessment report.
AUTISM SPECTRUM DISORDERS OUTCOME PROJECT--STUDENT ASSESSMENT REPORT
CODED STUDENT NAME: 1 EHCM
LOCATION: City School
REGION: 1
DATE: 6/15/00
Type of Assessment: AUTISM BEHAVIOR CHECKLIST (Annual Assessment)
Description of Assessment: A checklist of autistic behaviors capable of
providing a general picture of how an individual "looks" in comparison
to other children with autism
Date of Assessment 1/7/99
Total Score 74--Score is
typical to that of
others with autism
Type of Assessment: SAMPLE OF VOCAL BEHAVIOR (Quarterly Assessment)
Description of Assessment: Evaluates expressive speech at the
preverbal and emerging language levels through a standardized
language sample observation
Date of Assessment 1/7/99 4/29/99
Number and % of original
utterances 21 of 50-42% 29 of 50-58%
Number and % of communicative
utterances 13 of 50-26% 12 of 50-24%
Number and % of intelligible
utterances 9 of 50-18% 8 of 50-16%
Interpreted expressive
language age 20 months 22 months
Total words used in sample 9 10
Type of Assessment: INTERACTION ASSESSMENT (Quarterly Assessment)
Description of Assessment: Observes an individual's social responses
in a play setting with stimuli presented in a systematic fashion
Date of Assessment 1/7/99 4/29/99
Number and % of appropriate
social interactions 2 of 48-4% 8 of 48-17%
Number and % of appropriate
constructive play interactions 11 of 48-23% 2 of 48-4%
Number and % of no response or
self-stimulation 35 of 48-73% 38 of 48-79%
Number and % of aggressive
negatives 0 of 48-0% 0 of 48-0%
Type of Assessment: EDUCATIONAL ASSESSMENT (Quarterly Assessment)
Description of Assessment: A direct assessment that provides
information about the child's performance in four common
curriculum areas
Date of Assessment 1/7/99 4/29/99
Number and % of correct
receptive language responses 1 of 12-8% 3 of 12-25%
Number and % of correct
expressive language responses 1 of 12-8% 1 of 12-8%
Number and % of correct body
concept responses 1 of 12-8% 1 of 12-8%
Number and % of correct speech
imitation responses 0 of 12-0% 2 of 12-17%
Type of Assessment: AUTISM BEHAVIOR CHECKLIST (Annual Assessment)
Description of Assessment: A checklist of autistic behaviors capable of
providing a general picture of how an individual "looks" in comparison
to other children with autism
Date of Assessment 1/18/00
Total Score 58--Score is
typical to that
of others with
autism
Type of Assessment: SAMPLE OF VOCAL BEHAVIOR (Quarterly Assessment)
Description of Assessment: Evaluates expressive speech at the
preverbal and emerging language levels through a standardized
language sample observation
Date of Assessment 9/29/99 1/18/00
Number and % of original
utterances 37 of 50-74% 29 of 50-58%
Number and % of communicative
utterances 16 of 50-32% 25 of 50-50%
Number and % of intelligible
utterances 13 of 50-26% 20 of 50-40%
Interpreted expressive
language age 26 months 29 months
Total words used in sample 15 35
Type of Assessment: INTERACTION ASSESSMENT (Quarterly Assessment)
Description of Assessment: Observes an individual's social responses
in a play setting with stimuli presented in a systematic fashion
Date of Assessment 9/29/99 1/18/00
Number and % of appropriate
social interactions 1 of 48-2% 5 of 48-10%
Number and % of appropriate
constructive play interactions 21 of 48-44% 27 of 48-56%
Number and % of no response or
self-stimulation 26 of 48-54% 16 of 48-33%
Number and % of aggressive
negatives 0 of 48-0% 0 of 48-0%
Type of Assessment: EDUCATIONAL ASSESSMENT (Quarterly Assessment)
Description of Assessment: A direct assessment that provides
information about the child's performance in four common
curriculum areas
Date of Assessment 9/29/99 1/18/00
Number and % of correct
receptive language responses 6 of 12-50% 5 of 12-42%
Number and % of correct
expressive language responses 1 of 12-8% 1 of 12-8%
Number and % of correct body
concept responses 3 of 12-25% 4 of 12-33%
Number and % of correct speech
imitation responses 6 of 12-50% 6 of 12-50%
Type of Assessment: AUTISM BEHAVIOR CHECKLIST (Annual Assessment)
Description of Assessment: A checklist of autistic behaviors capable of
providing a general picture of how an individual "looks" in comparison
to other children with autism
Date of Assessment
Total Score
Type of Assessment: SAMPLE OF VOCAL BEHAVIOR (Quarterly Assessment)
Description of Assessment: Evaluates expressive speech at the
preverbal and emerging language levels through a standardized
language sample observation
Date of Assessment 5/24/00
Number and % of original
utterances 35 of 50-70%
Number and % of communicative
utterances 27 of 50-54%
Number and % of intelligible
utterances 22 of 50-44%
Interpreted expressive
language age 31 months
Total words used in sample 40
Type of Assessment: INTERACTION ASSESSMENT (Quarterly Assessment)
Description of Assessment: Observes an individual's social responses
in a play setting with stimuli presented in a systematic fashion
Date of Assessment 5/24/00
Number and % of appropriate
social interactions 6 of 48-13%
Number and % of appropriate
constructive play interactions 27 of 48-56%
Number and % of no response or
self-stimulation 15 of 48-31%
Number and % of aggressive
negatives 0 of 48-0%
Type of Assessment: EDUCATIONAL ASSESSMENT (Quarterly Assessment)
Description of Assessment: A direct assessment that provides
information about the child's performance in four common
curriculum areas
Date of Assessment 5/24/00
Number and % of correct
receptive language responses 7 of 12-58%
Number and % of correct
expressive language responses 3 of 12-25%
Number and % of correct body
concept responses 3 of 12-25%
Number and % of correct speech
imitation responses 9 of 12-75%
The Autism Spectrum Disorders Outcome Study is an ongoing study conducted by Portland State University in collaboration with The Oregon Department of Education (ODE). The study is supported by Federal IDEA Funds, Part B, CFDA84.027A., ODE Project 01/ 02-37. Additional information about this study can be found on our Web site (www.autismstudy.pdx.edu). REFERENCES Arick, J., Loos, L., Falco, R., & Krug, D. (in press). The star program: Strategies for teaching based on autism research. Austin: PRO-ED. Brownell, R. (2000). Expressive one-word picture vocabulary test. Novato, CA: Academic Therapy. Dawson, G., & Osterling, J. (1997). Early intervention in autism: Effectiveness and common elements of current approaches. In M. J. Guralnick (Ed.), The effectiveness of early intervention: Second generation research (pp. 307-326). Baltimore: Brookes. Frost, L., & Bondy, A. (1991). Picture exchange communication system. Newark, DE: Pyramid Educational Consultants. Frye, V., & Walker, K. (1998). Review of the Autism Screening Instrument for Educational Planning--Second Edition. Journal of Psychoeducational Assessment, 16, 280-285. Green, G. (1996). Early behavioral intervention for autism: What does the research tell us? In C. Maurice, G. Green, & S. C. Luce (Eds.), Behavioral intervention for young children with autism: A manual for parents and professionals (pp. 15-27). Austin: PRO-ED. Greenspan, S. J., & Wieder, S. (1997). Developmental patterns and outcomes in infants and children with disorders in relating and communication: A chart review of 200 cases of children with autism spectrum diagnoses. Journal of Developmental and Learning Disorders, 1, 87-141. Harris, L. L., & Handleman, J. S. (Eds.). (1994). Preschool education programs for children with autism. Austin: PRO-ED. Hurth, J., Shaw, E., Izeman, S., Whaley, K., & Rogers, S. (1999). Areas of agreement about effective practices among programs serving young children with autism spectrum disorders. Infants and Young Children, 12(2), 17-26 Krug, D., Arick, J., & Almond, P. (1993). Autism screening instrument for educational planning (2nd ed.). Austin: PRO-ED. Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning of young autistic children. Journal of Consulting and Clinical Psychology, 55(1), 3-9. Lovaas, O. I. (1996). The UCLA young autism model of service delivery. In C. Maurice, G. Green, & S. C. Luce (Eds.), Behavioral intervention for young children with autism: A manual for parents and professionals (pp. 241-249). Austin: PRO-ED. Maurice, C., Green, G., & Luce, S. C. (Eds.). (1996). Behavioral intervention for young children with autism: A manual for parents and professionals. Austin: PRO-ED. National Research Council. (2001). Educating children with autism. Washington, DC: National Academy Press, Committee on Educational Interventions for Children with Autism, Division of Behavioral and Social Sciences and Education. Newborg, J., Stock, J., Wnek, L., Guidubaldi, J., & Svinicki, J. (1984). Battelle developmental inventory. Itasca, IL: Riverside. Olley, J. G., Robbins, F. R., & Morrelli-Robbins, M. (1993). Current practices in early intervention for children with autism. In E. Schopler, M. E. Van Bourgondien, & M. M. Bristol (Eds.), Preschool issues in autism (pp. 223-245). New York: Plenum Press. Prizant, B., & Rubin, E. (1999). Contemporary issues in interventions for autism spectrum disorders: A commentary. Journal of the Association for Persons with Severe Handicaps, 24(8), 199-208. Prizant, B., & Wetherby, A. (1998). Understanding the continuum of discrete-trial traditional behavioral to social-pragmatic developmental approaches in communication enhancement for young children with autism/PDD. Seminars in Speech and Language, 19, 329-353. Rogers, S. (1996). Brief report: Early intervention in autism. Journal of Autism and Developmental Disabilities, 26, 243-246. Schreibman, L. (2000). Intensive behavioral/ psychoeducational treatments for autism: Research needs and future directions. Journal of Autism and Developmental Disorders. 30, 373-378. Smith, T., Groen, A., & Wynn, J. (2000). Randomized trial of intensive early intervention for children with pervasive developmental disorder. American Journal on Mental Retardation, 105, 269-285. Sparrow, S., Balla, D., & Cicchetti, D. (1984). Vineland adaptive behavior scales: Interview edition. Circle Pines, MN: American Guidance Service. Tindal, G., McDonald, M., Crawford, L., & Tedesco, M. (2000). Extended basic academic skills assessment system. Salem: Oregon Department of Education. Turton, L. J. (1985). Independent review of the Autism Screening Instrument for Educational Planning. In The Ninth Mental Measurements Yearbook (pp. 120-122). Lincoln: University of Nebraska, Buros Institute of Mental Measurements. Joel R. Arick, PhD, is a professor of special education at Portland State University. Dr. Arick is a co-author of the Autism Screening Instrument for Educational Planning and has many past publications in the field of autism and special education. He is currently working on a comprehensive curriculum for publication that uses research-based behavioral techniques for education of children with autism. Helen E. Young, MS, is a research assistant in the Special Education Department at Portland State University and is the project coordinator for The Autism Spectrum Disorders Outcome Study. She is currently pursuing her doctorate in educational leadership: special education. Ruth A. Falco, PhD, is an associate professor of special education at Portland State University. Her current interests include autism, early intervention/early childhood education, and foundations of self-determination for young children with significant disabilities and their families. Lauren M. Loos, MA, is a behavioral consultant in private practice. She currently provides behavioral and educational consultation for children with autism and conducts workshops and assessments as part of the Autism Spectrum Disorders Outcome Study. David A. Krug, PhD, is professor emeritus of special education at Portland State University and is currently engaged in evaluation of children with autism spectrum disorders. Marilyn H. Gense, MA, is an autism spectrum specialist with the Oregon Department of Education. She currently supports programs for autism spectrum disorders, vision impairments, orientation and mobility, and TBI. Steven B. Johnson, MA, is the associate superintendent for the Office of Special Education at the Oregon Department of Education. Working with other state and local special education program administrators, he has supported the increasing analysis and use of research results for parent education and for program improvement. Mr. Johnson is president-elect of the Association of State Directors of Special Education. Address: Joel R. Arick, Portland State University, Graduate School of Education, Special Education Department, PO Box 751, Portland, OR 97207-0751; e-mail: arickj@pdx.edu
|
||||||
