In the 1200s the word “wicked” meant ugly, coming from “wicca” or “witch”. Then F Scott Fitzgerald ironically used it to mean something wonderful, and the slang stuck.

But what does it mean when you say the kind of learning that takes place in a classroom is bad?

It has nothing to do with either of the previous definitions, but rather describes the reproducibility of research results in the learning environment.

“Nice” or “nasty” learning environments

In his opening speech at World Education Summit at the start of the year, Dylan Wiliam, professor emeritus of educational evaluation at UCL, spoke of “curriculum, pedagogy and evaluation, in that order”.

As part of his presentation, he explored the limits of drawing conclusions from educational research.

Wiliam explained how some learning environments can be called “nice” because it is possible for a person to reliably learn from their experiences there.

Tennis is one example: lessons learned while playing on the court are likely to be replicated the next time you play. Gravity and physics aren’t likely to change, so the way you play a move one day and get a certain outcome is likely to happen the next day as well.

In a “mean” learning environment, however, this replicability cannot be invoked.

What makes a classroom “mean”?

We spoke to Wiliam to get details on what exactly makes a classroom mean rather than nice.

1. Lots of “noise”

This does not refer to actual noise (although that can also be a problem), but rather to a number of factors that can influence the effectiveness of teaching – and, in turn, learning – that has place.

A teacher can teach the same lesson twice, but in one of these lessons the students are distracted by a fire alarm or a bee, or the teacher simply has a day off.

“What we are learning may not be what is true,” says Wiliam. “If we look at a teacher where learning seems to be happening, it might be the teacher who makes the difference, but it could be that you have a group of motivated students and the teacher hasn’t done anything for them. affect. . “

What does that mean? Ultimately, the teacher cannot always trust their own memories or data on what worked and what didn’t.

“The difficulty is [that] we look at the teacher, and we see the results, and we assume that the teacher is causing those results. There is a Latin phrase, ‘Post hoc ergo propter hoc‘-‘ After the event, therefore because of the event ‘. “

Wiliam cautions that if you assume that the things that happen afterward are caused by the things that happen before, then you might consider coincidence rather than cause and effect.

2. Performance vs learning

If you toss a task at your students, don’t raise your hand and they are all able to finish it, that means they’ve learned it, right? Well no.

Performance and learning are two different things, says Wiliam, and we shouldn’t confuse them.

“Performance is the extent to which students are successful in the learning task, and learning is what is held back sometime later,” he says.

“One of the most popular assumptions is that if students successfully complete the learning activity without help, they will learn it. Teachers therefore view performance in the learning task as an indicator of learning. . “

But performance is often inversely linked to learning, continues Wiliam.

“The more students struggle in the learning task, the more they remember, and the less they struggle, the less they learn,” he explains.

But it does not stop there. “If they’re struggling too much, then there’s not enough cognitive ability to create those long-term memories. We need a Goldilocks type sweet spot: we need the students to struggle, but not too.”

3. Short term vs long term

When we think of performance in the classroom, we cannot simply view the results of end-of-term or even end-of-year assessments as an indicator of teaching effectiveness. Wiliam says that a study by Carell and West, involving Air Force Academy students, indicates that an education which obtains good results after one year will not necessarily give good results the following year.

“Teachers who were more effective in the short term were less effective in the long term because they were more focused on short-term goals rather than laying a solid foundation for future learning,” he says.

Therefore, unless you examine the impact of a long-term educational approach, you may draw the wrong conclusions about its usefulness.

4. We sometimes draw the wrong conclusions

Even extensive research can lead experts to draw an incorrect conclusion when a significant variable is neglected, explains Wiliam, offering a story from his own experience.

“I spent four years following students who were placed in sets, and we found that the earlier setting was introduced, the more the results were disseminated, and the students in lower sets made less progress.” , he said.

“At the time, I didn’t understand how important the quality of teachers was. I kind of assumed that the teachers were quite similar. Now I understand that the most effective teachers are much more effective than the less effective teachers. less effective. “

As a result, he no longer feels so confident about the conclusions he drew at the time.

“I don’t know how to interpret these results anymore. It could be the frame that was causing it, or it could be the differential allocation of teachers to groups.”

It can happen to anyone undertaking research, he says.

“Any research result can potentially be overwhelmed when you find out that things you didn’t think were important turn out to be important.”

5. Maintaining randomization can be difficult

Randomized controlled trials are often considered the gold standard in research. But is this still possible to maintain when the research is taking place in the classroom?

Speaking of a research project designed to examine the role of class size, Wiliam recalls how randomization was sabotaged by “drift”.

“Maintaining randomization was impossible,” he says. “Middle-class parents found out about the experience, and if they found out that their child was in the upper classes, they campaigned for their child to be transferred to the lower class.

“Although the children were randomized at the start, there was a drift, and it could be the small classes that made the difference, or it could be that there was a drift of the pupils. the most efficient towards the lower classes. “

This was originally published on April 21, 2021.