Curriculum Decision Makers

Although the scientific evidence base for effective reading has existed for decades, the term “science of reading” has gained traction in the last few years, potentially leading to misunderstandings. As a result, The Reading League convened a group of experts who believed that a common definition would do the following:

• support educators as they discern what is and what is not in alignment with the science of reading
• assist people in becoming informed consumers of instructional materials, professional development, and resources
• impact publishers’ and policymakers’ decisions as they develop materials and policy guidelines

In working together to clarify a definition, the Science of Reading: Defining Guide was created to provide all stakeholders with a clear understanding of what the science of reading is, what it is not, and much more. The Defining Guide also includes 11 calls to action for all stakeholders. Pertinent to this Curriculum Decision Makers content is the following call to action: “We call on curriculum publishers and professional learning providers to create and promote products that are aligned with the science of reading and to eliminate non-aligned products from their offerings.” Unfortunately, some publishers continue to offer materials and professional development that claim to be aligned with the science of reading but are not.

Many states have mandated the use of high-quality instructional materials to be chosen from lists based on alignment to standards, useability, and often one other feature—outdated practices. Alignment with the science of reading was frequently not included in the development of these lists. While some have identified aligned practices within curricula, none identified non-aligned practices. As a result, many states adopt curricula featuring instructional practices that can inadvertently prevent students from developing proficient reading and writing skills.

To assist curriculum decision makers, The Reading League developed its Curriculum Evaluation Tool, later renamed Curriculum Evaluation Guidelines (CEGs) in 2022. The most recent update of the CEGs (February 2023) places red flag practices front and center in the review process to help decision makers steer clear of curricula featuring them.

The practices used to review curricula are those featured in Hollis Scarborough’s reading rope metaphor. These are the elements required for word recognition (phonological and phonemic awareness, phonics and phonic decoding) and language comprehension (background knowledge, vocabulary, knowledge of language structures, verbal reasoning, literacy knowledge). The CEGs also review curricula for practices related to fluency, writing (handwriting, spelling, composition), reading comprehension, and assessment.

The Reading League’s CEGs are derived directly from scientifically-based research; that they are neither arbitrary nor biased is an incomparable strength of this resource.

When using The Reading League’s CEGs, users should carefully examine the following features:

1. Preliminary statements and directions: Recommendations for review teams are provided. Teams should be comprised of educators who have been trained in curriculum-agnostic professional learning to build their knowledge in the science of reading. Decision makers from across tiers of instruction should be included, even if they will not be using the materials, to ensure that students are receiving the same instruction, grounded in the evidence, in general and specialized instruction.
2. Reviewer workbook: This workbook provides space for reviewers to rate the level at which they have found a red flag statement to be true. Reviewers can also note the location and language of the evidence to ensure the review is based on factual evidence and reduce subjectivity.
3. Reference section: An extensive and thorough listing of empirical research and meta-analyses related to each of the aligned and red flag practices in the CEGs is substantiated with a reference from the scientific evidence base.

It is essential to review all components of instruction to ensure your instructional materials provide robust teaching and practice in all components of language and literacy. Empowered with the knowledge of a thorough review, informed educators, coaches, and administrators can leverage this knowledge to ensure their instruction is grounded in evidence.

Stakeholder Uses of The Reading League’s CEGs

• Schools and districts: CEGs can be used to review existing instructional materials or as a walkthrough tool for instructional practices to ensure teachers are not teaching one component of literacy at the expense of another.
• State education agencies: CEGs are useful as a gateway criterion for schools and districts to leverage for curriculum decision-making.
• Institutions of higher education: CEGs are an important part of a framework for course syllabi, first teaching students about the research that substantiates each section, followed by deepening student understanding of the red flag practices and aligned practices. Since many schools and districts are shifting to high-quality instructional materials (HQIM) over teacher-created lessons, at the end of the semester, faculty members will ask students to review HQIM using the CEGs to teach them how to be critical consumers of their instructional materials.
• Publishers: CEGs can be used to develop and refine content to align with the science of reading. This resource is not recommended, however, to be completed by publishers and distributed to schools and districts for marketing purposes.

Have you used the CEGs in another way? We’d love to hear about it! Email info@thereadingleague.org to let us know.

The most common question asked of The Reading League and of many other organizations and social media groups is, “Does my curriculum align with the science of reading?” In 2022, The Reading League developed a process to provide an answer.

Although the CEGs were created to guide teams of curriculum decision makers in the work of evaluating curricula, The Reading League realized that it continued to be intense work. Many consumers lacked the time, resources, or knowledge to be thorough evaluators of instructional materials.

The following are steps taken to formally review curricula:

1. The Reading League launched a pilot review study to develop a non-biased, informational report for some of the most widely used curricula in the United States.
2. Based on this pilot review, small revisions were made to the CEGs and the accompanying workbook.
3. Dr. Matthew Burns conducted an interrater reliability study on the CEGs (see the Interrater Reliability section that follows for more details). The interrater reliability of the CEGs was affirmed.
4. Review teams of three experts were hired and trained.
5. Publishers were contacted and informed that The Reading League wanted to review their curriculum (at no cost). The process was described. If publishers agreed to participate, materials were to be sent directly to the teams. Upon completion of the review, materials were returned to the publishers.
6. Review teams meticulously reviewed all K-5 materials of their assigned curricula. Each person on a team reviewed the programs independently. They then gathered and discussed their findings as a group, and if there were disagreements, a consensus was reached.
7. Data and evidence were gathered and presented to a writer, who drafted them into narrative reports.
8. Reports were then shared with the publishers to ensure nothing was inaccurately represented.
9. Narrative reports were finalized—The Reading League’s Curriculum Navigation Reports are born!

The Reading League does not, and will not, recommend particular curricula. Decision makers can trust that these reviews and those involved in the review process, are entirely free from conflicts of interest and financial ties to publishers. The intent of these reports is to provide useful information on some of the most widely-used instructional materials for schools, districts, and states as they work towards building instruction, assessment, and systems aligned to the science of reading.

Dr. Matthew Burns conducted an interrater reliability study of the Curriculum Evaluation Guidelines. This section provides a link to the pre-print of this study along with the abstract.

Reliability and Validity of The Reading League Curriculum Evaluation Guidelines

Abstract: The current study examined the reliability of The Reading League Curriculum-Evaluation Guidelines (CEGs), which were developed to help school-based teams rate the presence of red flags when considering adopting specific literacy curricula. A total of 30 coders (n = 30) independently used the CEGs to evaluate a free online English language arts curriculum. The coders consisted of 23.3% (n = 7) Ph.D. students in special education with previous teaching experience and master’s degrees, and 76.7% (n = 23) preservice special education teachers completing their final semester of coursework before beginning their student teaching experiences. The results indicated strong internal consistency (a = 0.96), high interrater reliability (Hmean = .91, 95% CI =.89 to .93, p < .01), and an acceptable estimate of content validity (Vmean = .70, 95% CI =.68 to .72, p < .01). Overall, the CEGs demonstrated acceptable to strong evidence for being a psychometrically sound tool for evaluating reading curricula. The data were sufficiently reliable and resulted in acceptable evidence for content validity.

Some publishers declined to provide materials for The Reading League Curriculum Navigation Report review. Those publishers are listed below. The Reading League will continue to reach out to these organizations to request the opportunity to review their programs.

Publishers that declined review or did not respond:

• American Reading Company (Declined 2023 and 2024)
• Benchmark Education (Declined 2023 and 2024)
• Heinemann’s Fountas & Pinnell (Did not respond 2023)
• Savaas’ MyView (Declined 2023 and 2024)
• Heinemann’s Units of Study (Did not respond 2023)

The Reading League’s Curriculum Navigation Reports are the result of years of dedicated work.

To consider before you use the Navigation Reports

1. They are presented to you as informational reports to support decisions on instructional materials.
2. TRL does not recommend curricula, so there is not a mechanism for passing or failing.
3. The first and most important step in aligning a system to the science of reading is to build educator and administrator knowledge. Without professional knowledge about reading instruction related to the science of reading, even the most robust curriculum will not be successful.
4. There is no perfect curriculum.
5. The Curriculum Navigation Reports are intended to help knowledgeable educators understand more about which practices might be avoided, enhanced, or supplemented in instruction and/or determine which curriculum is most suitable for your setting.

To consider as you review the Navigation Reports

1. A Tier I curriculum often consists of an extensive array of materials. A common concern noted by our review teams is that there is insufficient time within the school day to cover all included content. In response to the requests of their consumers, publishers inflate their programs with supplementary materials, add-ons, and extension activities. This makes it challenging and time-consuming to locate necessary or recommended materials for a given lesson across manuals, texts, and digital platforms. Determining what to prioritize or omit due to time constraints can be unclear.
   With these points in mind, it is imperative to review the Navigation Reports with a keen eye toward the CEGs.
   • Address ALL components of the CEGs even if there is no red flag present. For example, if a program did not receive a red flag because it has sufficient writing instruction, be sure to leverage that component of the program! If you take these reports at face value and do not teach and practice each component of the CEGs that was identified, you will be inadvertently creating a red flag of your own by not including it in instruction.
2. Ensure ALL practices are aligned with the science of reading. For example, evidence-aligned word recognition skills are essential but not sufficient. Skilled reading requires word recognition skills and knowledge building, oral language, writing skills, etc.
3. The mere inclusion of a component in the curriculum does not guarantee sufficient practice for successful outcomes. Frequently, the crucial concept of practice (e.g., spaced/distributed, cumulative/interleaved, retrieval practice, corrective feedback) is overlooked in teacher manuals. Therefore, utilizing data to tailor instruction and ensure mastery before progressing is critical.
4. It is crucial to assess the extent to which instructional materials foster and celebrate the cultural and linguistic diversity of students.
5. Note the publication dates of the reviewed curriculum, as the particular edition may feature practices different from the ones featured in these Curriculum Navigation Reports.