Reading Recovery in Maine, 2003-2004:
A Qualitative Report (Satisfaction Survey)

 Dr. Valerie Ruhe, Research Associate

As a supplement to the 2003-2004 Reading Recovery evaluation report, the qualitative findings are important for three reasons. First, analyzing the response of individuals affected by a program, such as parents, teachers and administrators, is a reputable approach to program evaluation, known as participatory evaluation, which yields valuable insights into the merit and worth of educational programs (Abma & Stake, 2001; Stake, 2004). Secondly, qualitative findings illuminate the process or “structure at the core of events...a causal description” (Miles & Huberman, 1994, p. 4). In education, not all of these implementation variables, which in Reading Recovery include learner motivation, can be measured quantitatively. Thirdly, qualitative findings “have a quality of “undeniability [and]…a concrete, vivid, meaningful flavor” that may not be provided by tables of numbers (Miles & Huberman, 1994, p. 1). In sum, these qualitative findings shed light on both stakeholder satisfaction and process variables, thereby providing valuable information to supplement the quantitative analysis.

Methodology

In the spring of 2004, Maine Reading Recovery teacher leaders revised the qualitative survey questions to obtain more in-depth information about findings from the previous year. Parents, administrators and teachers accessed the survey of open-ended items over the University of Maine’s Reading Recovery website. Parents and administrators were also surveyed on their general satisfaction with a single item with a scale of 1 to 5, with 5 being the most positive rating.

There were very high response rates from all participant groups. 1,290 parents (62%), 164 administrators (77%), 188 trained Reading Recovery teachers (70%), 18 in-training Reading Recovery teachers (80%), and 306 classroom teachers (78%) responded. These high response rates increase our confidence that the findings represent the views of Grade 1 parents, administrators and teachers across Maine. After the data were collected, Reading Recovery teacher leaders performed the first stage of the analysis—data reduction. The qualitative survey responses were analyzed for themes and salient quotations were listed under each theme. After that, these findings were compared for convergence across items, respondents, samples of raw data and teacher leaders. Responses to the scaled satisfaction item for parents and administrators were compiled by site, and given to teacher leaders.

Administrators

Reading Recovery administrators rated Reading Recovery as a very good program (M = 4.3, N= 164). One administrator described Reading Recovery as “a vital piece”. “Everyone embraces it.” “We highly endorse it.” A recurring theme was that the program provided “a common language to talk about literacy”, and was a “terrific fit” with classroom instruction. “Our school would be lost without it.” “I am really grateful that we have this program. My respect for it has grown over the years.”

Administrators said that Reading Recovery had important long-term effects on classroom practice in grade 1 and beyond. “Reading Recovery concepts and strategies are being adopted by classroom teachers and infused into their regular lessons.” Because children have a more solid grounding in the grade 1 curriculum, the curricula of later grades can move along at a faster pace. “We’ve been able to focus more on reading comprehension in grades 3 to 5.” Administrators also perceive the program as having a beneficial impact on state-wide standardized test scores in later grades.¹ “The positive impact of Reading Recovery is that reading levels have increased. We do very well on the MEA and the NWEA”. Reading Recovery was also credited with reducing the rate of incorrect referrals to more expensive, long-term remedial programs. Another said, “This program has transformed our school”.

Trained Reading Recovery Teachers

Almost all Reading Recovery teacher respondents gave mostly positive comments about the program.² They valued the high-quality professional development provided by a diversity of sources, including conferences, continuing contact, Behind the Glass sessions, school visits, discussions of “outstanding articles” and books such as Carol Lyon’s Teaching Struggling Readers. The videos were especially helpful in giving teachers repeated observations of the same child over time. “We could replay them and carefully analyze what a child was doing.” Finally, the expertise of teacher leaders and their individualized help with decision-making, as well as the literacy team meetings were also highly valued. One teacher said “It is a privilege to be a Reading Recovery teacher.”

The teachers reflected on the specific ways in which their skills in literacy teaching and assessment improved. “Our teacher leader helped our professional development by asking where we wanted to improve.” “I am paying more attention to the language I use, for example, “I like the way you looked at the chunk you knew to get the new word.”” “My knowledge of research-based best practices [has] helped me to extend my students’ reading comprehension to higher levels.” One teacher said she liked being able to observe the same child over time. “Being aware of the emotional side of my students can change students’ attitudes by talking about something they like or can do well so that they are more receptive to learning.”

In-training Reading Recovery Teachers

In-training Reading Recovery teacher respondents gave mostly positive comments about the program, which provided them with “a better understanding of the reading process”. [The program] “has changed my views of the lowest-achieving children because I’ve learned to teach better.” They have seen “changes in the children’s confidence and reading levels”. “I am better at allowing children the time to make their mistakes before I jump in to “fix” it.” “I have seen them grow…they have made reading a part of their day instead of a dreaded hard time; this has helped them to grow in their subject areas.”

I am amazed that it is possible to target so many small components of reading and writing and shift learning in such large ways. I love that the guidance is hidden in the instruction, that the children feel they are doing this by themselves…They crack the code and become better readers.”

I think Reading Recovery is one of the most worthwhile programs that I have been involved in. The gained knowledge combined with the hands-on experience is invaluable.

Classroom Teachers

Classroom teacher respondents also gave very positive comments about the Reading Recovery program, which helped them learn more about literacy and stay informed about new practices. “Reading Recovery teachers become leaders in current ideas and pass this knowledge on to classroom teachers.” Reading Recovery teachers were described as “change agents in the schools.” They appreciated Reading Recovery teachers observing classroom lessons to see how their students performed in context, and found it helpful to work with these teachers in literacy teams.

Reading Recovery skills were perceived as readily transferable to both literacy and the content classrooms. By bringing low performing children up to grade level, Reading Recovery “moves the whole class ahead to comprehension and higher-order thinking skills.” “The findings put to flight the notion that Reading Recovery skills are unrelated to classroom experience. On the contrary, classroom experience is informed by Reading Recovery, and many of the Reading Recovery methods are adopted by classroom teachers.”

Classroom teachers noticed that Reading Recovery children developed better literacy skills, as well as greater self-esteem, motivation and growth. “Reading Recovery provides a solution to the problem of children not meeting the standards.” The program helps “save those kids who could so easily slip through the cracks in reading…these kids are picked up early on before problems become deep-rooted habits.” “I am always amazed and proud of the Reading Recovery children.” “My other students notice changes in the Reading Recovery children and applaud them.” “I would not be able to run a successful first grade classroom without it.”

Parents

Parents rated the Reading Recovery program as an excellent program (M= 4.5, N= 1,290). Several parents said they saw a “big improvement” after Reading Recovery. They described their children as happier, more confident, more independent, and more enthusiastic about reading and school work in general. “She is able to read more instead of saying she can’t read this book.” “He enjoys reading now which before all the help…[he wanted] nothing to do with it.” “He went from not being able to read to slowly progressing to reading very well.” “They...read instead of watching TV.”

Many respondents praised the expertise of the Reading Recovery teachers and expressed gratitude and thanks for the program. “Excellent job.” “This is an excellent program.” “I was very happy with the program/grateful that [my son] got to take advantage of it.” Parents appreciated their visits to teaching sessions, parent-teacher meetings and frequent and detailed correspondence with teachers. “I want to thank Mrs. S. for her dedication to Reading Recovery…[She] magically got [my child’s] attention with reading. She made leaps and bounds with him…which I couldn’t seem to do.”

I really appreciated the thoughtfulness of the instructor, and the way correspondence is arranged on a daily basis. I am certain it is not always easy to write each day a comment or concern about my child, but it is so very helpful and insightful for a parent.

Finally, parents mentioned their own involvement with their children’s reading, which is part of the Reading Recovery program. “He reads to me every evening.” “We have made reading a part of our daily routine for our whole family. Thank-you very much.” “Reading Recovery has made a big difference in how [my son] feels about school and reading. He loves it. Thank-you.”

Concerns

The few concerns expressed about the program tended to reflect implementation issues. Administrators voiced concerns about scheduling, under-implementation, and timelines, specifically that 20 weeks might not meet the needs of all learners. There was also concern about the impact of the No Child Left Behind legislation. “As the bar has been raised for all children, the gap for our Reading Recovery and Title 1 students is getting larger compared to average progressing students.” Teachers were concerned about behaviorally or emotionally challenged children who “fall through the cracks”. There were also concerns that Special Education teachers “come from different philosophies”, and one teacher wanted more discussion of brain research and perceptual difficulties. There were very few concerns expressed by parents, except for concerns about scheduling and continued funding, a theme which recurred across all respondent groups.

Discussion

The most striking finding of this analysis is the high satisfaction levels both within and across respondent groups. The remarkable convergence of perceptions across survey items and respondents, coupled with high response rates, provides empirical evidence that the Reading Recovery program has a broad base of enthusiastic support among parents, teachers, and administrators across the state of Maine. Moreover, the high levels of satisfaction within and across these groups constitute an important indicator of the value of the Reading Recovery program.

Secondly, these data show that the Reading Recovery tutoring process is complex and multifaceted. The “heart” of the program is the theory-grounded expertise and individualized tutoring of Reading Recovery teachers. These skills are honed each year through a surprising array of professional development activities, including individualized guidance from teacher leaders, book discussions, conferences, Behind the Glass sessions, video ethnographic analysis of individual children, and conferences with highly respected scholars. Parents also become actively involved in reading with their children, so that they can refine their skills outside of the classroom. Extensive data is collected on each child and her performance at entry and exit to the program. In short, it is difficult to imagine what additional components could be included to improve the delivery of Reading Recovery services in Maine.

In light of these findings, it is important to note that the critics of Reading Recovery seldom recommend that more be done. Instead, they recommend that less be done, for example, replacing individualized tutoring by small group or classroom instruction, or replacing highly-skilled teachers by commercial reading materials. As such, the alternative models presented by the critics of Reading Recovery could be called “deficit model” programs. Would parents, teachers and administrators be as satisfied with a program which provided less service to children at risk? Traditional outcomes models are not enough to evaluate early remedial reading programs; important supplemental information can be provided by participatory evaluation, which is based on a survey of stakeholders and the use of multiple methods (Stake, 2004).

Conclusions

The strongest finding of this study is the overwhelming support of parents, teachers and administrators for the Reading Recovery program in Maine. There was a remarkable convergence both within and across data from administrators trained and in-training Reading Recovery teachers, classroom teachers and parents that Reading Recovery is a very good program with a highly beneficial impact on students and classrooms in grade 1 and beyond. This broad base of support from across the state is a vital aspect of the overall merit and worth of the Reading Recovery program. There were almost no negative comments, and only a few concerns including a few specific implementation issues. The qualitative data also illuminate the many and varied aspects of the Reading Recovery tutorial process, including ongoing observation and monitoring of students’ performance, rich professional development activities and parents reading to children at home. In sum, these findings show broad base of strong support for Reading Recovery, as well as valuable insights into the program implementation across Maine.

¹This perception of administrators is supported by a state-wide, longitudinal study of the performance of Maine Reading Recovery students on the grade 4 Maine Educational Assessment reading and writing subscales. See: Ruhe, V. and Moore, P. (in press). The Impact of Reading Recovery on Later Achievement in Reading and Writing.

²Scaled data was not collected on teachers, whether Reading Recovery, in-training or classroom.

References

Abma, T. A., & Stake, R. E. (2001). Stake’s responsive evaluation: Core ideas and evolution. In Greene, J. C., & Abma, T.A. (Eds.), Responsive Evaluation, New Directions in Evaluation, No. 92. (pp. 7-21). San Francisco, CA: Jossey-Bass.

Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis: An expanded source book (2nd ed.). Thousand Oaks: Sage.

Ruhe, V. and Moore, P. (in press). The impact of Reading Recovery on later achievement in reading and writing. ERS Spectrum.

Stake, R. E. (2004). Standards-based and responsive evaluation. Thousand Oaks, CA:Sage.