Why "Research-Based" is a Great Place to Start...But You Can't Stop There

by Kate Wolfe Maxlow

I admit that I always raise an eyebrow when someone tells me an educational strategy is "research-based." Don't get me wrong, I LIKE John Hattie and Robert Marzano and I use their work all the time...but I also know that there are some pretty big caveats to using educational research to make instructional decisions. Here are some of the biggest:

1. "High yield" usually means "correlated with increased student achievement on standardized assessments."
Remember, standardized assessments are most often multiple choice, and therefore most often written at the Remember through Apply level (with the occasional smattering of Analyze). Therefore, the majority of the research that's out there is looking at student improvement on lower-level skills. But in a good classroom, that shouldn't be all that we're looking to improve.

One of the best examples of this is Hattie's research on inquiry learning. In his book,Visible Learning:  A Synthesis of over 800 Meta-Analyses Relating to Achievement, Hattie does a meta-analysis (which means that he combines the results of multiple studies on inquiry learning) to determine that inquiry learning has a 0.31 effect size.

Okay, so what does that mean? It means that a group of students who were taught using an educational strategy (in this case, inquiry learning) would, on average, do better than a group of students who, all other things being equal, were not taught with this strategy. So, for instance, a group that was taught with inquiry learning that initially scored, on average, at the 50th percentile (i.e., on average, doing better than about 50% of their peers), would improve, on average, 12 percentile points after engaging in Inquiry learning, and therefore the average would now be at the 62nd percentile (i.e., on average, scoring better than about 62% of their peers).

(Want to better understand that? Read this article from Marzano).

While 12 points is nothing to sneeze at, there are other strategies that are easier to implement and have higher effect sizes, such as direct instruction (0.59 effect size/22 points average after engaging in the strategy). Who doesn't want 22 percentile points gain over 12? (Note that direct instruction is NOT the same as lecture.)

Of course, those assessments are mostly using standardized assessment measures of student achievement.

So that's the thing: the point of inquiry learning is to teach those higher-level critical thinking skills. Yeah, if all you want is for students to memorize information, it's direct instruction, all the way. Hattie himself notes that in one 1996 study, Smith found "larger effects from inquiry methods in critical thinking skills (d = 1.02; 35 percentile points average gain), than in achievement (d = 0.40; 16 percentile points average gain). What does this mean? It means that if your goal is to teach actual critical thinking and how to engage in scientific processes, inquiry learning is the way to go. If you're looking just to teach science facts, it might not be worth it.


2. It's not the strategy that's effective; it's how and when it's implemented.
The best example of this is the great homework debate. There are a ton of schools right now that are nixing the homework, especially at the elementary level. According to Hattie, homework has an effect size of d = 0.29, or a percentile gain of 11 points if students engage in the strategy.

So students should do homework, right? Eleven points is eleven points.

Well, it's not that cut and dry. One of the best probes into the research can be found on the ASCD website on "The Case For and Against Homework," but it boils down to this: In elementary school, Cooper (1989a) found that the effect size is 0.15 / 6 percentile points, for middle school it's 0.31 / 12 percentile points, and for high school it's 0.64, or 24 percentile points.

Moreover, other studies have shown that homework is strikingly more effective when teachers grade it and provide comments, and that after a certain number of minutes of homework per night, there are diminishing returns on student achievement gains.

In other words, be careful whenever anyone tries to say something like, "Research shows that homework has an 0.29 effect size." Unless you know how the strategy was implemented and with whom, it's really hard to make overall judgements about the effectiveness of any one strategy.


3. All educational research is based on averages, but students are individuals.
The two biggest swingers in the educational effectiveness game are definitely Hattie and Marzano. They both use meta-analyses to come up with their conclusions about what does and does not work in education. The benefit of a meta-analysis is that it can tell us, broadly, which strategies tend to be good across the majority of students in the majority of content areas and grade levels. What they cannot tell us, of course, is whether a particular strategy is going to work with a particular classroom or a specific student.

Why is that? Because in order to come up with an effect size, most education studies compare two groups of students: one group that has the treatment (the strategy, such as note-taking), and one group that does not (i.e., same instructional methods, but students aren't allowed to take notes). Researchers take the average on the assessment of a group that takes notes and compare it to the average on the assessment of the group that doesn't take notes in order to figure out the effect size (that's overly simplified. Read more here.)

Then, the meta-analyst will pull tons of studies on the same topic, or close to the same topic. For instance, does note-taking in a graphic organizer count...or does that go under the graphic organizers strategy? Can we count the experiment that used a multiple choice test along with the one that used a writing prompt and scoring rubric? And are the studies even rigorous enough to include?

These are all decisions left up to the meta-analyst like Hattie and Marzano. Once they decide what studies to include, they do some fancy math that basically gives them an average of all the study averages.

So, in the end, the appropriate conclusion is less like, "Note-taking has an effect size of 0.99 and therefore yields a percentile increase of 34 points," and more like, "On average, note-taking has been found by some researchers to yield a percentile increase of something like 34 percentile points for many students much of the time."

See the difference?

All that being said, educational research is still worthwhile. It can still tell us really important things (for instance, that retaining students tends to, on average, decrease their student achievement when oftentimes we're retaining students in hopes of raising it). Just remember: read the actual studies as much as possible to find out the actual (not just averaged) results, it's more often about the implementation than the strategy itself, and in any case, your results for a given strategy may vary.

When Students Talk: Motivating Students to Learn More through Collaboration

Compiled by Kate Wolfe Maxlow

Do your discipline issues seem up? Do you ever find yourself wondering if it's because our young people spend more time on screens than they do interacting in-person? And, if that's the case, what are teachers supposed to do about it?

Hampton City Schools has some of the finest teachers around, and they've found ways to help students to learn to communicate and collaborate within the context of their lessons. 

What happens when the teacher takes the time to allow students to work together to learn? Hesitant learners often bloom and learning takes place at an even deeper level.

Here are some ideas for how our amazing teachers have motivated our young people by infusing collaboration into every day tasks:

English-Language Arts

Lakesha Pickering, Middle School
Making Connections with Characters and Classmates: Students joined my Nearpod class and posted their responses to the following question on the "Collaborate" board: "How do our experiences affect our actions? In other words, how does our background influence our behavior?" We had a whole class discussion about several of the responses. After taking notes on the “Making Connections” mini-lesson, students responded to the following poll question: "Of the people/ characters from the articles and stories we've read this year, to which one can you relate most? Be prepared to identify the similarities and differences between you and the character!"

We then, as a class, listed some ways students could draw connections between themselves and the characters. They noted that many of the people we've read about are teenagers and concluded that it was intentional on my part. They stated that I wanted to use people/ characters my students would be able to relate to on some level.  Next, students completed a Venn-like graphic organizer, comparing and contrasting themselves with the character they'd selected. While they did this, I assigned partners (I paired students with another student who'd identified most with the same character, based on the poll results). The students then worked with their partner to create a Triple-Venn diagram...on the desks! They used dry-erase markers to draw the graphic organizer on the tables, talked about what they had in common with each other and the characters, and they talked about how they were unique. When they were finished, they each filled in the paper copy of Triple-Venn graphic organizer and submitted that for a grade. I walked around as students worked, listening in on their discussions and asking guiding questions and providing opportunities for extension, when necessary.

It was, hands-down, my favorite lesson of the year. Everything went smoothly, and we worked from bell-to-bell. Here are two student responses to the exit slip:

"Three important ideas/things from the lesson today was the word schema which affects how we understand things, Text to text which is comparing different text and seeing similarities and differences, and text to world where you can compare the the things are going on now to the text but, the most important thing I learned today is text-to-self which lets you compare your feelings, emotions, and characteristics to another character and I thought that was really cool."

"Three important things I learned today are that you can have something in common with people who seem so different from you. I also learned that not everyone is different from one another and lastly that I have things in common with other people in my class that I didn't know I had."

Nancy Terrell, Middle School
I provide enrichment activities for a 6th grade language arts class.  I plan and prepare literature based Maker-Space activities that allow students an opportunity to work in collaborative groups to solve a problem. The Literature-based Maker Space activities have provided students an extension to their classroom, while working on skills needed for that grade level. One instance was when I shares with students the Robert Frost poem Stopping By Woods on a Snowy Evening. Students had to make inferences as to what the promise was that the traveler made. They were then given a challenge to create something to help the Traveler keep his promise. The collaborative project went well as students shared their ideas and came to a decision as to what to build. Two students had conflict on how to proceed, not giving in to the others ideas. This gave us an opportunity to talk about the difficulties of working in a group and how to problem solve to achieve the goal. 

Kauanza Wilkins-Royal, Middle School
Reciprocal teaching-As a review for several topics we had recently covered, I split the class up into four groups and gave them a topic to teach. The four topics covered were main idea, narrative structure, conflicts, predictions, and inferences. The students had 10-15 minutes to put together a mini-informational session to include examples. Once time was up they had to present to the class and answer class questions. It went well. Students were apprehensive at first but once they started working together they started enjoying the activity.  They were allowed to ask questions which kept them engaged. What worked well was the grouping with having students of different ability levels in the same group. Next time I will provide more of a guide of what is needed from each group. 

Melissa Twisdale, Elementary School
It’s easy to think that adding “turn and talk” to the gradual release process should be a quick and easy strategy that kids should be able to do; it’s also easy to stop offering “turn and talk” when it becomes more like “turn and argue.”  It may help to break this element down for students, so that skills like decision making are built into the process. Students who typically struggle with peer interactions may find more success by starting with paired interactions before working in cooperative groups. The decision for things like, who goes first may be as simple as a quick game of “rock paper scissors. “. Most recently, as a way to build concepts for the cause and effect comprehension strategy in fourth grade, we used a cause and effect matching activity in a partner reading learning experience. The sort was based on social studies content with the cause and effect events for Jamestown listed side by side. The cause and effect events were mixed up so the buddies took turns reading the cause and effect events while collaborating to correctly match the “when-then” events. 

Mathematics

Aimee Foshee, Elementary School
Students had to work together to create shapes using marshmallows and toothpicks. Students used the toothpicks for sides and the marshmallows for the vertices.  Students had to cooperate with their partner in order to assemble the shapes because otherwise they fell apart. Students had to collaborate together to decide how many toothpicks and marshmallows they need to ask me for in order to create the given shape. The project went very well. The students enjoyed creating the shapes and they worked well together. The only thing I may change is use something not a food item like playdough so I can display the item outside the classroom.

Katelyn Harlan, Middle School
My co-teacher and I created groupings to remediate on area and perimeter.  We had at least one strong leader per group, for those groups we were not working with.  Students had to complete all of the activities in sequence, but only one paper from each group for each activity would be graded.  It was important that all students understood the material and kept each other on task.

Social Studies

Annette Crothers, Elementary

The students have to learn the economic terms of Colonial Virginia (barter, credit, debt, and savings). To help the students not only learn the terms, but also apply them, they created skits to act out the meaning of the words. I divided the students into groups of three. I gave them one of the terms, but they were not allowed to share it with other groups. I assigned a different role to each group member (props, writer, timer). Even though they had individual roles, all members were expected to participate in the creation and performance of the skit. The students worked for 25 minutes on creating their script and props. When time was up, they performed in front of the class. When they were finished, their classmates had to guess which economic term they had based on the hints given during their performance.

The activity went GREAT! The students were extremely focused and engaged during their preparation time. I was honestly surprised by how excited they were. I realized that they don't receive many opportunities to share what they know in performance-based activities. I think that assigning roles worked well because students all felt responsible for the performance, which made them more invested in making the final product turn out well. Next time, I would give the students more time to work on their skits. I might stretch the activity over two days because while the skits were fantastic, the finished products took less than one minute to perform. Extending the activity will give the students more time to add artistic details and rehearse.

Music

Wanda Mitchell, Electronic Music Technology (High School)

Students worked in small groups taking on the roles of producer, composer, lyricist, and arranger to brainstorm ideas for a song. They choose the style, the instrumentation and co-wrote the lyrics. They completed a self-assessment and presented their final project with the class, who then gave them constructive feedback. It was very well received. Those students who normally would complain about completing the assignment, feeling they are inadequate, were partnered with more confident students. This helped to boost their self-confidence.

Art

Anonymous, High School

Quizlet Live: Students get placed into random groups for an online game using vocabulary and it forces them to communicate because no two people in a group have the same answer options on their screens and they have to sequentially get correct answers or fall behind in the "race" (kind of like Kahoot for teams).

The Hardest Content to Teach

by Kate Wolfe Maxlow

Which is harder: teaching first graders to read or teaching seniors AP Calculus? Is it more difficult to teach students how to paint in the style of Van Gogh or to understand stoichiometry?

These are trick questions, of course. These are all difficult concepts to teach and for students to learn, but the most difficult content of all to teach is the implicit content in any given standard.

What is the implicit content, you ask? Well, it's the knowledge and skills implied in a standard that students should already have learned...everything they have to already be able to do in order to even begin to learn a new standard.

Let's take a look at an example. Here's a Standard from the Virginia Standards of Learning for English-Language Arts, Grade 2:

2.6 The student will expand vocabulary and use of word meaning.
c) Use knowledge of antonyms and synonyms

• Essential Understanding: All students should understand that knowledge of homophones, prefixes, suffixes, synonyms, and antonyms can be used to read unfamiliar words.
• Essential Knowledge, Skill, and Process: Use knowledge of antonyms when reading; use knowledge of synonyms when reading.

I've underlined what we call the explicit content: it's synonyms and antonyms. That means teachers need to teach the word "synonym" and "antonym" and if they ask for a synonym for "angry," the child can answer "mad," and if they ask for an antonym for "up," the child can answer "down."

Easy enough, right?

Wrong. Think about everything that's implied by this standard. First of all, students aren't just supplying antonyms and synonyms orally when prompted; they're doing this within the context of reading, and the implication here is that the child can already read. What does that mean? It means that they already understand phonics and decoding words and what a sentence is and how to track word-for-word and how to comprehend every other word in a sentence...all in order to decipher the one particular, unfamiliar word.

Here's an example of two sentences where students might have to figure out an unknown word (underlined): "Mr. Bunny was peeved that he did not have a carrot. He would have been happier if he had one."

Then, even if we've taught students what "antonyms" and "synonyms" are, we're counting on them 1) being able to read the sentence; 2) being able to figure out that they don't know the word "peeved," 3) being able to figure out "happier" is the antonym for the unfamiliar word; 4) knowing that "happier" is related to the word "happy;" and 5) knowing at least one antonym for the word "happy."

If any of the above skills are missing, students are going to struggle.

Similarly, think what's implied in this Virginia Physical Education standard:

10.3 The student will demonstrate the ability to apply basic principles of training and scientific concepts and principles to evaluate fitness behaviors and identify strategies needed for health-enhancing fitness for the present and into adulthood.
b) Use a variety of resources, including available technology, to analyze current fitness and activity levels, and to improve physical activity and personal fitness.

Look at all those implied standards! We're assuming that students already know how to use the available technology, which probably includes things like computer or personal-device skills, that they understand what they're currently doing that does or does not apply to their fitness, and that they have an understanding of what they can do to be successful at improving their own physical activity and personal fitness.

I have to admit, some of these are things I'm still struggling with an adult.

When standards come with all this implied content, they often trip teachers up because start to teach the standard, as its written, only to discover that students are lacking tons of the skills needed to even begin.

So, what do we do? How keep the implied content from wreaking havoc with our plans?

One thing we can do is to unpack our standards carefully. Don't just look at what you have to teach, but think about: what else do students have to know in order to be able to actually do this? Unpack standards for the implied content.

Next, pre-assess your students. Pre-assessments are a good idea, but often we only pre-assess on the current level's material to see if students already know it. For instance, if we're teaching Grade 4 mathematics, we might just include problems from the Grade 4 mathematics curriculum.

The problem with this, of course, is that what if students not only don't know how to add and subtract with regrouping...they also don't even have their basic addition and subtraction facts? A pre-assessment should include not only items from the current curriculum, but any previous knowledge or skills that students will need to have to begin learning the current curriculum.