The 2 Sigma Problem

Blog post by Jeffrey Thayne on February 6, 2013
1Comments

Today, I would like to share my notes from an article I just read by the educator Benjamin Bloom. All quotations are from his 1984 article titled “The 2 sigma problem.” In the article, he describes a series of experiments, in which there was a control group and two experimental conditions. In the control group, students received conventional classroom instruction; in one experimental group, students participated in a mastery-learning curriculum; and another experimental group, students received one-on-one tutorship in a mastery-learning curriculum. Here’s a more detailed description of the groups:

  • Conventional: “Students learn the subject matter in a class with about 30 students per teacher. Tests are given periodically for marking the students.”
  • Mastery Learning: “Students learn the subject matter in a class with about 30 students per teacher. The instruction is the same as in the conventional class (usually with the same teacher). Formative tests (the same tests used with the conventional group) are given for feedback followed by corrective procedures and parallel formative tests to determine the extent to which the students have mastered the subject matter.” The teacher doesn’t move on to a new subject until he or she has determined that 95% of the students have mastered the material.
  • Tutoring + Mastery Learning: “Students learn the subject matter with a good tutor for each student (or for two or three students simultaneously). This tutoring instruction is followed periodically by formative tests, feedback-corrective procedures, and parallel formative tests as in the mastery learning classes. It should be pointed out that the need for corrective work under tutoring is very small.”

The results?

Most striking were the differences in final achievement measures under the three conditions. Using the standard deviation (sigma) of the control (conventional) class, it was typically found that the average students under tutoring was about two standard deviations above the average of the control class (the average tutored students was above 98% of the students in the control class). The average student under mastery learning was about one standard deviation above the average of the control class (the average master learning student was above 84% of the students in the control class).

The variation of the students’ achievement also changed such that about 90% of the tutored students and 70% o the mastery learning students attained the level of summative achievement reached by only the highest 20% of the students under conventional instructional conditions.

There were corresponding changes in students’ time on task in the classroom (65% under conventional instruction, 75% under Master Learning, and 90+% under tutoring) and the students’ attitudes and interests (least positive under conventional instruction and most positive under tutoring).

Wow. That’s pretty dramatic. According to Bloom, “The tutoring process demonstrates that most of the students do have the potential to reach this high level of learning.” However, there’s one problem: one-on-one tutoring is massively expensive. So, Bloom argues, we have got to find a way to recreate these same results without it:

I believe an important task of research and instruction is to seek ways of accomplishing this under more practical and realistic conditions than the one-to-one tutoring, which is too costly for most societies to bear on a large scale. This is the ’2 sigma’ problem. Can researchers and teachers devise teaching-learning conditions that will enable the majority of students under group instruction to attain levels of achievement that can at present be reached only under good tutoring conditions?

In the rest of the article, Bloom explores some of the interventions they’ve researched in their attempts to recreate the gains of tutoring without the tutoring—that is, in their attempts to improve student achievement by 2σ. Some things that have shown to have some effect:

(1) Enhance students initial cognitive entry prerequisites

In other words, many times students begin a class unprepared. They don’t have all the cognitive skills they need to have in order to fully grasp the material. For example, many students starting a Calculus course may not have full grasped geometry or trigonometry, and this impedes their learning. So researchers organized a pre-course intervention, in which students were tested for their understanding of the prerequisites, and gaps in their understanding were patched before the semester began. The results?

  • Conventional instruction: 0 sigma (control group—the group others are compared against)
  • Conventional instruction + enhanced cognitive prerequisites: .7σ
  • Mastery learning: 1σ
  • Mastery learning + enhanced cognitive prerequisites: 1.6σ

The problem with this is that it only works on higher courses, and it requires extensive time and resources prior to the semester course.

(2) Improve home environment

According to Bloom, “We know that the home environment does have great influence on the pupil’s school learning and that this influence is especially effective at the elementary school level or earlier.” There are a number of home factors that have been shown to have high correlations with school achievement:

  1. Work habits of the family.
  2. Academic guidance and support.
  3. Stimulation in the home.
  4. Language development.
  5. Academic aspirations and expectation.

But—how do you change or improve these? Researchers tried bimonthly group meetings between parents and a parent educator (extended over 6 months). The educator would first talk about the kind of home environment that helps students learn, and then parents would share their ideas with each other. Results: 1σ! This is way cool, because, according to Bloom, “Because parent support takes place in the home and [mastery learning] takes place in the school, we expect that these two effects will be additive. The result should be close to a 2 sigma improvement in student learning.”

(2) Improving teaching

According to Bloom,

When we compare student learning under conventional instruction and tutoring we note that approximately 20% of the students under conventional instruction do about as well as the tutored students. … That is, tutoring probably would not enable these top students to do any better than they already do under conventional instruction. In contrast, about 80% of the students do relatively poorly under conventional instruction as compare with what they might do under tutoring. We have pondered these facts and believe that this in part results from the unequal treatment of students within most classrooms.

Observations of teacher interaction with students in the classroom reveal that teachers frequently direct their teaching and explanation to some students and ignore others. … Studies find that typically teachers give students in the top third of the class the greatest attention and students in the bottom third of the class receive the least attention and support.

In other words, the top 20% of students basically receive individualized attention, and this can approximate the tutoring effect. One reason why tutored students do so well is because they all received increased individual attention. “It is very different in a one-to-one tutoring environment where there is a constant feedback and corrective process between the tutor and the tutee.”

How do we fix this? Well, according to Bloom, educating teachers is key: “Teachers are frequently unaware of the fact that they are providing more favorable conditions of learning for some students than they are for other students. … One basic assumption of our work on teaching is the belief that wen teachers are helped to secure a more accurate picture of their own teaching methods and styles of interaction with their students, they will increasingly be able to provide more favorable learning conditions for more of their students, rather than just for the top fraction of the class.”

Making teachers more aware of their interactions with students, by observing them and providing feedback, by training them to look for the right cues in terms of attention and comprehension, led to dramatic gains in student achievement (in one study, 1.5 sigma!). In these studies, “teachers … change their teaching methods to provide more equal treatment of the students in their classes. When this more equal treatment is provided and supplement with the ML feedback and corrective procedures, the average student approaches the level of learning found under tutoring methods of instruction.”

Summary

This fascinates me. I would personally love, instead of trying to recreate the tutor effect in group instruction, to find a way to make individual instruction more feasible.

1
comments so far
  1. “I would personally love, instead of trying to recreate the tutor effect in group instruction, to find a way to make individual instruction more feasible.”

    That’s why homeschool is growing so fast.

Leave a Reply