Research, Evaluation and Results

Research Tools

• CBE 360 Survey Toolkit from AIR (2017)

 

Reports

• Looking Under the Hood of Competency-Based Education: The Relationship Between Competency-Based Education Practices and Students’ Learning Skills, Behaviors and Dispositions by AIR (2016)
• Reinventing Schools Coalition (RISC): A 2010 report by Marzano Research Laboratory found that the RISC framework significantly impacts student achievement. Researchers compared 7 RISC districts and 8 Non-RISC districts in 3 states. Study findings:
• The odds of a student in a RISC School scoring proficient or above on state tests are 2.3 times greater for reading, 2.5 times greater for writing, and 2.4 times greater for mathematics than the odds of a student scoring proficient or above on state tests at a Non-RISC School.
• Compared to students in Non-RISC Schools, students in RISC Schools are 37% more likely to score proficient or above on state tests for reading, 54% more likely to score proficient or above on state tests for writing, and 55% more likely to score proficient or above on state tests for mathematics.
• In addition, the degree of RISC model implementation was found to relate to the number of students scoring proficient or above on state tests in reading and writing. The odds of a student in a High-RISC School scoring proficient or above in reading and writing were found to be about two times larger than the odds of a student in a Medium-RISC School scoring proficient or above.
• Mastery Learning: Below are some of the findings from a review of 27 evaluations of mastery-based learning in Synthesis of Research and the Effects of Mastery Learning in Elementary and Secondary Classrooms by Thomas Guskey and Sally Gates published in 1986:
• Achievement results are overwhelmingly positive, but vary greatly from study to study.
• Although students at all levels appear to benefit from mastery learning, effects are somewhat larger in elementary and junior high school classes than at the high school level.
• Although applicable across subject areas, effects in language arts and social studies classes are slightly larger than those attained in science and mathematics classes.
• Students tend to retain what they have learned longer under mastery learning both in short-term (2-3 weeks) and longer-term (4 months) studies.
• Students are engaged in learning for a larger portion of the time they spend in mastery classes and require decreasing amounts of remedial (corrective) time over a series of instructional units.
• Students in mastery classes develop more positive attitudes about learning and about their ability to learn.
• Teachers using mastery learning develop more positive attitudes toward teaching, higher expectations for students, and greater personal responsibility for learning outcomes, but may experience diminished confidence in their teaching skills.
• Learning With Others: A Study Exploring the Relationship Between Collaboration, Personalization, and Equity by AIR (2018) explored collaboration as a strategy to personalize learning for diverse student groups. Focusing on grades 9-12 in four high schools, the study looked at connections between collaboration in student-centered learning classrooms and student outcomes and how these differed by race and ethnicity. 
• Student reports of having opportunities for high-quality collaboration were strongly associated with positive classroom experiences and higher engagement, motivation, and self-efficacy.
• Understanding Implementation of Proficiency-Based Education in Maine by Students at the Center (2018) follows up on the 2012 Maine legislation requiring all students to graduate from high school with a proficiency-based diploma. Education Development Center partnered with ten districts in rural Maine to investigate exposure to proficiency-based education, including links to engagement and academic outcomes as well as the nature of implementation.
• Implementation has largely focused on identifying graduation standards and implementing new proficiency-based grading practices, with traditional classroom practices still fairly commonplace.
• This raises a question: If Maine schools had spent more time improving instruction (and culture) rather than just focusing on setting standards and changing grading, would Maine still be on a statewide pathway toward proficiency-based diplomas? 
• Maximizing Student Agency: Implementation of Student-Centered Learning Approaches (AIR 2018) is a partnership with New Tech Network schools to create multiple opportunities to design, test, and revise teacher practices as part of a Networked Improvement Community (NIC). The NIC’s goal was to develop a menu of effective teaching practices in support of student-centered learning geared toward student ownership of learning or agency. The study also looked at how student agency was related to academic outcomes and how these varied for students from different racial and ethnic backgrounds.
• Abolishing the Phrase "I'm Not a Math Person" by Students at the Center investigates how to improve student agency and learning outcomes in math — particularly for students from traditionally marginalized groups. With an improvement science framework, the collaborative tested, refined, and spread “high-leverage” practices that reframed mathematical struggle as learning and engaged students in collaborative problem-solving. 
• There are seven student-centered practices that can make a difference for students. One important point is to make sure that students understand that there are different ways to solve problems, not just one that makes the others wrong. 

 

From Districts

There has been limited research and evaluation on the emerging models of competency education. It's important that districts and schools engage in continuous improvement so that we can continue to produce evidence of effectiveness (especially cost-effectiveness) for low-, middle- and high-achieving students. Also, if you know of people doing research and evaluation please let us know. 

 

• Colorado
  
• Adams 50, Colorado was able to reduce the number of schools identified as lowest-performing from seven to zero in three years. See post. 

 

• California
• Lindsay Unified School District (From District website)   Lindsay Unified School District began the work to redesign the delivery of instruction in their schools in 2007, the actual implementation of the performance based system began with the 9th grade class in fall 2009 at Lindsay High School (LHS) with district-wide implementation in 2011. While we cannot measure the full impact of the PBS K-12, there is promising data that supports the conclusion that PBS is making a difference in the success of our students. The potential impact of the PBS is demonstrated by the growth in student achievement at LHS during the first three years of implementation (2009-2012). Scores on the California Academic Performance Index (API) have grown by 91 points; English language arts (ELA) proficiency rates for 9th graders has increased from 29% (2009) to 41% (2012); for 10th grade students, ELA proficiency has grown from 25% in 2009 to 37% in 2012; the percentage of 11th graders proficient in ELA increased from 21% in 2009 to 42% in 2012. See Education Week article for more information on Lindsay.
• More on Lindsay (2017): Now seven years into the performance-based learning transformation, the district is fully committed to the approach. The district’s graduation rate has risen from below 70 percent in 2010-11 to about 82 percent in 2013-14, the latest data available for the district. The number of students who have completed A-G requirements for entrance to state universities has doubled – from 17 percent in 2008-09 to 34 percent in 2014-15.

 

• Connecticut
• Windsor Locks showed improvements (2016)
  

 

• Alaska
  
• Chugach (From Delivering on the Promise) In 1994, the Chugach School District, serving 214 students over 20,000 square miles in impoverished communities, began a fundamental redesign of how they would educate their students. With the courage to confront the fact that 90 percent of their students could not read at grade level and only one student in 26 years had graduated from college, Chugach focused their mission on ensuring that all students learn to high standards. The district engaged the community in establishing a performance-based approach, developing standards in ten content areas, new assessments, and modified reporting mechanisms. Within five years, Chugach School district saw the following results:  
•  Over a five-year period, average student achievement on the California Achievement Test rose from the bottom quartile to the 72nd percentile.
  
• The percentage of students participating in college entrance exams rose from 0 percent to more than 70 percent by 2000.
  
• Between 1995 and 2000, teacher turnover was reduced to 12 percent; in the previous twenty-year history of the district, turnover was 55 percent yearly.
  
• Chugach’s transformation gained them national attention, including the prestigious Malcolm Baldrige National Quality Award for organizational excellence.

 

• Iowa:
• Iowa's Department of Education recently released a task force report that details their transition to competency education. They were encouraged by Alaska and New Hampshire's significant increase in literacy, graduation rates, and college attendance and success after their switch to competency education. 
• Two school districts, Spirit Lake and Muscatine, have already made the transition to competency education, in different ways. 
• At Spirit Lake, a Department of Education researcher reported that during more than 50% of the observations, students were engaged in the upper three levels of Bloom's Taxonomy, whereas before the switch to competency education, students were performing only on the bottom three levels. The number of excused and unexcused absences, and office referrals dropped significantly. This shows the importance of student success and engagement in attendance and behavior. 
• Muscatine showed similar success. After the district switched to competency education, no students received a D or F in a competency-based education classroom. In the 2011-2012 school year, 38% of students had received those failing grades. In addition:
• Six percent of the students engaged in learning contracts or short-term remediation to reach proficiency prior to the end of a term; 
• Four percent of the students needed intensive remediation, which required additional time beyond the term;
• Three percent of the students were able to accelerate their learning through content or a course; and
• Teacher support for the methodology was rated at 85 percent, as evidenced through a district-wide survey following building presentations in the fall of 2012.

 

Other Reports

• Evaluation of Gates Investments in 3 CBE Districts/Schools/Programs: Rand was hired to evaluate three very different competency-based initiatives in very different stages of development. This is a highly problematic study as one of the initiativies was a pilot program in a school that was not becoming  competency-based in a district that had no intention of becoming competency-based. There is very little reason to believe that this type of a pilot without being seated within a schoolwide competency-based structure would be able to generate many benefits to students within their first year of operation. One of the other examples was a district trying to convert its entire school system.  See Competency Based Education in Three Pilot Programs.
• Personalized Learning: Rand completed a study on the early implementation of new personalized schools as part of Gates efforts to evaluate its own investements. The schools had only been operating two years or less. Some of the schools had some elements of competency-based education (although this was never well-defined or thought about consistently across schools or the study). It is difficult to disentangle implementtion/execution issues from design and very very hard to gain insights about CBE. See Informing Progress (2017)