Sit together and read… Oh, and discuss the print to help with spelling.

I remember discovering the joys of storybooks in the 1970’s, as Grandpa Elf and I would sit in the dappled shade of the trees, reading story after story set in the woodland. For human children, tucked up in their beds, tales of deadly wolves and evil grandmothers may well have had a scary edge, but for a tiny elf, the horrors of Little Red Riding Hood were all the greater, with the forest (and the wolves, no doubt!) looming about us. Since the 1970’s, a wealth of evidence has built up that emphasises the importance of shared reading experiences for children.

Project STAR (Sit Together and Read) isn’t simply about sitting down with children and sharing books. It aims for teachers to focus on the print within the text, so encouraging children to pay greater attention to the written elements in storybooks. Children are asked specific questions about the letters and words in the text and are asked to track the words as the book is read aloud. I remember this being a focus of my 1970’s introduction to the alphabet and books, where a character represented each letter. The letter k was Kicking King, who wore a crown and possessed real kicking feet. Such a focus on alphabet and print worked for me, but what of its wider impact on children’s reading skills?

What Works Clearinghouse (WWC) have published a review of a recent randomised controlled trial (RCT) that was carried out to assess the impact of Project STAR. WWC was established in 2002 by the US Department of Education, and aims to ‘provide educators, policymakers, researchers, and the public with a central and trusted source of evidence of what works in education’. WWC identified the original study (PiastaSB et al, 2012) for review due to the media coverage that it received upon publication, as it claimed Project STAR to have significant, long-term affects on a range of reading skills.

What the original study did

For inclusion in the original RCT, the authors selected schools in Ohio USA with a high percentage of children who were from socio-economically deprived backgrounds. 85 nursery (preschool) classrooms were randomly assigned to one of the following 3 groups:

Low-dose intervention (Project STAR techniques used in 2 reading sessions each week)- 116 children in total
High-dose intervention (Project STAR techniques used in 4 reading sessions each week)- 135 children in total
Control group (Regular instruction techniques used in 4 reading sessions each week)- 118 children in total

Teachers in each group were provided with identical sets of books for their class. Over a period of 30 weeks, teachers read the books in the same order, focusing on one book per week. Teachers from all groups took part in training to improve the teaching of reading. Training for those in the control group emphasised high-quality reading practices and reading aloud to children. However, teachers in the low-dose and high-dose intervention groups were trained in instructional methods applicable to Project STAR.

The following primary outcomes (literacy skills) were assessed and measured prior to the start of the project, one year after and two years after the intervention period:

Reading
Spelling
Vocabulary/ oral language
Comprehension

At each assessment time, the authors compared the literacy skills of the children between the:

high dose and low-dose groups
high dose and control groups
low dose and control groups

The study therefore contains 24 tests of the impact of Project Star (3 groups, 4 outcomes, 2 time periods)

WWC evaluations of the original RCT

The study does not meet the WWC standards for student attrition in a randomized controlled trial. The authors were able to demonstrate baseline equivalence for the analytic samples of low- dose Project STAR students and comparison students for three outcomes measured one year after the intervention. The authors also demonstrated the baseline equivalence of the analytic samples of high-dose Project STAR students and comparison students for one outcome measured two years after the intervention. These four comparisons of study groups meet WWC evidence standards with reservations. The other 20 comparisons do not meet WWC evidence standards.

For the 4 comparisons that met WWC standards with reservation, the following results were given:

Children who were taught reading through 4 sessions per week of Project STAR, had significantly improved spelling skills 2 years after the intervention compared to the control group.

At the 1 year follow-up assessment, students in the low-dose group were not significantly different from students in the control group for:
- vocabulary (Effect size ES = -0.05)
- reading (ES = +0.23)
- spelling (ES = +0.23)
At the 2-year follow-up assessment, students in the high-dose intervention group had significantly higher spelling skills than those in the control group (ES = +0.38)

The Education Elf’s View

The authors of the original study, which was published earlier in 2012, made the following conclusion:

Longitudinal results showed that use of print references had significant impacts on children’s early literacy skills (reading, spelling, comprehension) for 2 years following the RCT’s conclusion. Results indicate a causal relation between early print knowledge and later literacy skills and have important implications concerning the primary prevention of reading difficulties.

Clearly the WWC view was significantly less positive about the impact of Project Star on children’s literacy skills. The WWC view is that the original study could not reasonably make the claims that it did on the grounds of student attrition. This basically refers to changes in the groups that may occur because children leave a school, or are unavailable for the assessments, so an outcome variable cannot be measured for some children.

The effectiveness of an RCT works on the principle of randomisation; by randomising groups, it would be expected that control groups and experimental groups should be similar, and therefore comparable at the start of an experiment. However, attrition from groups may create dissimilarities, potentially making the groups incomparable. It is feasible to imagine a situation whereby some children could be purposely removed from a study to swing results. Therefore, it is important to remove any potential for bias by excluding any comparisons where attrition is high.

For a more in-depth look at WWC’s procedures for identifying, reviewing, and summarizing reports, you can read the Procedures and Standards handbook by following the link below. If you want to find out more about the problems caused by attrition bias in educational research, you may wish to look at the article by Dong and Lipsey. For me, I am off to read “The Human and the Shoemakers” to some of our little ‘uns.