Assessment, One Skill At a Time

I recently realized that I was destroying some of the information that my tests collect.  I was averaging scores of multiple questions together, blending a student’s performance in different areas into a single, summative score. Instead of keeping the information that Johnny could multiply matrices perfectly (100%) but couldn’t really find inverses (50%), I was telling Johnny, “Johnny, you’re at about 75% in this class!”

And so I hit my first personal inflection point of the year.  I stopped averaging scores, and started telling students (and parents) about their strengths and weaknesses in specific skills.  For the most part I am following Dan Meyer’s example, described briefly at http://blog.mrmeyer.com/?p=346, and working with a vague idea of what Hans, a logic teacher here, does.   Instead of getting back a single big score for a month of class, the students get 10 or 12 separate scores.  Check out Dan’s blog for more details.

The new system immediately started helping in three important ways:

  1. Student motivation increased (dramatically in some cases),
  2. Remediation became more informed, and
  3. I started getting feedback that helps me streamline lessons before I give them and assess their efficacy after I give them!

1. Student Motivation

After switching to this new grading system, I have seen an increase in motivation in almost all of my students.  I don’t know if they like the check list, if they like to be recognized as masters of skills (and they are masters!), or if the simple act of breaking the course down into manageable chunks is what is doing it.  But it’s great.

A student that works to improve a single skill, and gets a higher grade in that skill, feels a sense of accomplishment immediately, even if he has five other skills to improve over the next week.  The change in the way I see students catching up is actually astonishing.  They can more easily see that they can do it, and they love it!

Furthermore, students that have earned 80% or 90% already seem motivated to earn the “master” designation in all of the skills on their list.  My experience with students in prior years has almost never been “I have a 95% already – can’t I please take the test again to get a 100%?,” but now it’s practically across the board.

In my old, averaging ways, I wasn’t giving my students this kind of specificity and manageability to work with.  Students with failing grades simply got a big fat “60%” on the top of a unit test.  Now they get “if you work on matrix multiplication, you’ll be at a passing level,” or “you are a master of linear equations – what did you do to get so good with those?”

2. Informed and Focused Remediation

When a student comes in to my office hours now, I can pull out my grade book and see that they aren’t yet passing in skills 13 or 18.  Since these skills are more or less independent of any other skill (an important feature of this program), we can get down to the students’ misunderstanding much faster.  Also, it’s natural to focus on skill 13, and then 18.  There’s no pressure to do everything at once for a single makeup exam that will re-test every skill simultaneously.  Especially for kids who perceive themselves as bad at math, I’ve seen an increased willingness to come to my office hours for help.

3. Formative Assessment

When I give a test, there are a few intro-level questions and a few master-level questions, and also a pretest question.  These are tests of skills that I have perhaps never mentioned before, or mentioned only briefly.  I let the students know that if they score well on these pretest questions they won’t have to take them in the future, but I don’t expect them to know how to do them (how could I?) and that they will absolutely not be penalized for doing them wrong, or just leaving them blank.  I estimate that it takes between 3 and 7 minutes to give a pretest question.  For that price I get a preview of my students’ current knowledge, before I plan a whole class about something they already know or plan to skip over something that they don’t know at all.  I also get a measurement of my skill as a teacher when the students take a test on the same concept after my lesson.  Knowing the difference between “my kids all aced this skill (but they learned it last year)” and “my kids all aced this skill (and they never even thought about it before this class)” helps me rate my lessons.

I recommend it, guys.  The switch is pretty easy, especially at the beginning of a grading period.  Writing tests is easier.  All you have to do is separate the skills you most care about (ok, this is hard), and then stop averaging!

16 thoughts on “Assessment, One Skill At a Time”

  1. I can attest that the increased motivation is palpable among some students – even among some assigned to the structured study hall; you’ve given our students tools by which they can set their own goals and take greater personal responsibility for their learning. As someone who struggled with math, I am particularly excited about the “informed and focused remediation” component of your assessment system. I look forward to observing both classes and office hours under this new system.

  2. I have moved toward giving skills-related quizzes, rather than big tests, so far this year. Tests were the bane of my (and the students’) existence during my first year teaching Spanish. Re-takes sucked so much energy and NObody wanted to take them. I wondered if I was just copping out this year and not making it challenging enough for the kids by letting them demonstrate one skill at a time, and then not rolling it all into a big chapter test or midterm. But you’re right, office hours is easier and kids come willingly. And I have a better sense of the skills individuals and the class in general have learned. Plus, we build upon these skills and “recycle” them– so we’re studying and reviewing as we go, which is what we essentially do before test time anyway.

    There’s lots more I need to do to differentiate for and pretest advanced learners. Some things I’m working on with Nicole and Nordy for next year.

    Again, I appreciate your thoughts and taking the time to write and share.

  3. I too have started to use Dan’s assessment system this year. I teach at an inner city alternative high school, and our focus is on credit recovery. Nearly all of my students have struggled tremendously with math in the past. Well, to be more accurate, they struggled a little bit back in 4th grade after they missed those two weeks and then gave up.

    I have this to say about Dan’s system: I know, right! It’s amazing to see how much it transforms the whole culture regarding assessment in the class. I’ve always wanted to do the very thing I am doing now, but was held back by not knowing how to do the bookkeeping. If I were cooler, I’d have just plunged in long ago.

    In the past, I would pass tests back to students and they would look at the grade and then throw them away. I tried to use all sorts of formative assessments, in order to get them to be more metacognitive. These were a total failure: More work for me, teeny tiny benefit to a couple of kids.

    Now, I have students that come to class focused on learning the math and who come after school for help and who actually look at their tests and try to understand what they did wrong! I actually have students that can think about their own understanding of the topics we teach, because it’s right their on their scorecard in front of them. They can actually figure out on their own what they don’t understand! That “Thunk” sound you just heard? That was me, shitting a brick.

    The more capable students are motivated by the DOMINATE TWICE rule that means they don’t have to do questions on a topic once they have aced it twice. And my struggling students are motivated by the focus and the fact that a zero today doesn’t mean it can’t be an 80 tomorrow. And I’m motivated because all these useful grades (as opposed to the 75% in the class) have turned me into a magpie for data.

    Anyway, enough of the fanboy prattle. I just want to second your recommendation, and add that this has been the smallest change on my part for the biggest change on my students’.

    I haven’t been doing the pretest questions, like you, though. I think that’s a good idea, and I wish I’d have been doing them all along this year. I’ll start doing them soon, because I want that data.

  4. I know, right! Further comment: at our school, we only have three grades, “not passing,” “passing,” and “honors.” So, my scheme is different from Dan’s in another way as well. In order to get a “passing” grade, students must have an average of 75%, and no lower than a 3/5 (60%) in any skill. To get “honors,” a student must have no lower than 4/5 in any skill (80%), and complete one of several choices for an in-depth research or reflection assignment per month (about 3 hours of work for each assignment, or 12 hours for honors for the semester).

    Notice that I never require any student to get a 5/5 on a single skill – and yet I have kids who want to get 5/5 on EVERY skill! There is no possible grade benefit from getting 5/5 on every skill, except to be sure you have the 75% you need to pass… but these kids ask every single week when they can take this masters problem or that masters problem!

  5. I, too, have been following Dan’s assessment system and contemplating how to implement it in my own classroom. Riley and Jason, would you be willing to share any handouts/explanations/examples? What skills did you identify for your classes? With the differentiation/compacting of the curriculum depending on where students are with a skill, how does a normal day in your class look? Outside of office hours to remediate lower skill scores, do you have other suggestions you offer to students as ways to get help? I am trying to pull together information in order to explain this process and how it can positively impact students, teachers and future math classes. Thanks for anything you can offer!

  6. At the end of this semester I’ll be happy to post all of my skills tests, my list of skills (which of course needs revision), and everything else I created. In the meantime, Dan’s blog has suggested lists of skills for a few courses.

    My average class period is not affected very much when some students fall behind the average in a skill or set of skills. I try to make my classes investigative, and to focus on problem finding and solving skills. These are skills on which I actually never grade kids. What I’m trying to say is that if a student is behind in skill 8, he or she is not going to suffer in class until he or she masters it. The skills I *test* on are independent from each other – I try to keep prerequisite skills out.

    Unfortunately, I *don’t* have a great way to get students extra help besides my office hours (only once per week!). This might be an area in which I can improve – any suggestions?

  7. Can anyone share on Advanced Algebra block period model? I am considering following this system, yet want to be sure I have it down before implementing. He mentions putting it into the grading system and do you enter each skill set or the weekly assessment?

  8. I record grades on each individual skill, never an averaged score of multiple skills.

    My Algebra II class meets for four hours in three periods each week, in two classes of one hour each and one class that is two hours. The skills test takes about 30 minutes a week (15% of total class time). I could imagine giving more, smaller quizzes – one skill at a time, every day, if you have only two-hour classes, or something like that.

    I hope to provide my full list of skills and assessments at some point – maybe over winter break?

  9. A suggestion for the not enough office hours problem- could you post practice quizzes and solutions online?

    You could set something up through Google Docs so that it would grade itself and the students could see the right or wrong answers. Or you could link to video explanations of how to solve. Not that you would have enough time to make your own videos but you could link to others on YouTube and the like.

  10. Oh man, the software I envision to support this online. It’s been crippling me. I actually have seriously considered quitting teaching and taking half a year to create the perfect support system (I don’t have the guts to do it (yet?)).

    There’s so much that could be improved with some online solution. Give me some time to try out some options and I’ll get back to you – and if you try it, let me know!

  11. Wow. This is what I want to do. I teach science, so I don’t have skills so much as concepts. How would I assess that? Essay questions? Multiple choice doesn’t seem right. Unless you have five different questions over the concept? We could do some practical assessment. Have you tried anything like that?

  12. Tracie,

    I teach physics. Here are some sample pages from the Learning Folders my students are required to keep in the classroom. I hole punch the sheets and the students keep them in a folder with clasps. Every time they get an assignment back, they update their folder.

    http://www.box.net/shared/7bnkaqk5e6

    Each problem on a quiz is keyed to one of the learning targets. It is not uncommon for a quiz question to link to several topics from different units.

    I hope some of this helps!

  13. A writing teacher at my school has been trying this system out with the concepts she wants to teach, and is also finding that they are impossible to separate as much as a math teacher can separate skills. She uses multiple measures for each concept.

    Even some skills in my Algebra class are too complex for a single question to convince me that a student has mastery of that skill. Sometimes I feel the need to put several questions under a single skill, and that has been working fine so far.

    I imagine the hardest part would be accepting an early, high score in a concept that will be reused throughout the year. For example, if you attempt to test a student to judge their knowledge of “scientific method,” and they ace this test, what do you do in a month when the kid can’t run a decent experiment? In my class I work constantly to use all of the skills that have come before to avoid this kind of problem, but it might be harder in different types of classes.

    Good luck!

Leave a Reply