RSS
 

Archive for the ‘assessment’ Category

Two and a Half Ways to Make Your Next Test More Readable

12 May

Math questions are hard to read.  It’s easy to mix up numbers, mis-attribute modifiers, confuse powers with multipliers, etc.   Unless you’re testing for reading skill, it’s important to put work into making questions as readable as possible.  Here are two (and a half) ideas you can use on your next test.

One: Repeat the Question in an Answer Box

My test questions include space for work and a very specific location for answers: the Answer Box.

The answer box is the dark box you see in the lower-right corner of the question area.  I pre-print that box on every test question I ever give so that, regardless of the sloppiness of a student’s work, I can see what they thought the final answer should be.  I indicate the type of answer I expect right in the box.  My hope is that even a student with low English-reading skills will be guided by that indicator to provide at least the right type of answer.

Two: Provide a graphical representation with your text, where possible.

In this problem I wanted to test students’ ability to transform parabolas.  I accompany the text with a picture of what I mean.

Without the picture, “compressed by a factor of 2″ is ambiguous.  When a student answers y=\frac 1 2 (x+2)^2 +1 I don’t want to have to guess if he doesn’t know what to do or if he just got right and left mixed up.  In the previous example, the MS clipart of a die reminds ELL students what a die is (not obvious from the word itself!).

Two and a Half: Easy Stuff

Actually, more like five tenths of a big idea:

  1. Use large fonts (obviously easier to read)
  2. Include grids on your graphs (the grids above are more obvious in print)
  3. Use the same format every time (students know what to expect even when they don’t command the language)
  4. Number and name questions with the standard they are testing (make double-sure ELL students know what you’re asking)
  5. Leave room between things (not only room for work.  Spaces help us separate ideas)

If you don’t make your questions crystal clear, you can’t be sure you’re testing what you think you’re testing.  Reading is hard, everyone messes it up occasionally, and students who have only been learning English for a couple years need more help than you think!

Of course, I’m just a math teacher, not a design expert or language coach.  What have you used to make your print materials more readable?

 

Team Tests

30 Mar

I administered my skills tests for the week today, but instead of making students complete the tests individually, they were encouraged to work within their team (of 3 or 4 students total).  Each student turned in a separate piece of paper.  I picked problem 1 from student A, problem 2 from student B, problem 3 from student C, and so forth, to determine the grade that all group members would receive.  Interesting.  Overall, I recommend you try it.

Pros

  1. A different format.  The kids were really in to it.  Afterwards, every kid I asked said something between “I liked it” and “It was way better.”
  2. Easy to administer and grade.
  3. I felt like I could put tougher problems on the test.  In fact, all of the problems I put on it were harder than usual, and the scores on this test were 10.6% higher than the scores on the rest of my tests.
  4. Students got practice communicating in a “high stakes” environment.
  5. It didn’t feel like I was spending the whole class on assessment.  There was also learning going on.  I heard things like, “this is the way I’ve started to think about this,” and “can we check this [answer]?”

Cons

  1. Tension was high, particularly in one group, when there were disagreements.  There was some snapping and flustered rustling of papers.
  2. Probably at least one student earned a grade higher than his understanding should indicate.

The pros are great, here.  Compared to the atmosphere during individual tests, today’s vibe was much more collegiate and… educational.  How can I mitigate the cons?  I could alternate individual and team tests.  I could talk with the students about strategies to work under pressure.  Would more time alleviate the tension?

Anyone else have experience with team tests?

 

Percentages don’t have the power to express a grade.

08 Feb

The traditional model for grades in a class lacks the flexibility required to reflect what I really think of a student.  When I used weighted categories (e.g. 50% exams, 30% homework, 20% class participation), I found that some of the students passing my class didn’t really seem to deserve it, and some of the students failing my class really should have been passing.  ”Well, adjust your weights,” you say, and that’s a good idea: I made several improvements and was progressively more satisfied with my results.

But.

One test of the 29 I’ll give this semester deals with simplifying exponential expressions.  If a student gets 100% on each test except the exponential simplification test, on which he gets a 0%, his average will be 96.5%, A+, Honors.  He doesn’t have to worry about exponential simplification at all, and he can just move on and never learn it.  I’m not suggesting that this hypothetical kid be made to retake Algebra 2, of course.  I’m suggesting that he be required to learn exponential simplification.

So, in my class, I require that every student earn at least a 3/5 on every single skill that we study.  Then, I require an overall average of 75% on top of that minimum requirement.  Students get some leeway, and they do not need to master every single subject (I understand that there are time constraints involved in my students lives, and that they may not really care about my class).  However, they can’t just do well on a lot of skills and decide they’re not even going to bother with one.  I am not willing to send a kid who can’t simplify exponential expressions at all to the precalc teacher.

The same philosophy can extend to homework.  If you think homework is vital, make it a requirement of passing.  If you don’t think it’s vital, don’t.  Averaging test scores with homework scores is harmful because it dilutes the meaning of your tests and the meaning of your homework.  Averaging mathematically destroys information!

At my school we only have three grades, Pass, No Pass, and Honors.  Each grade has certain clearly stated requirements that I give the students at the beginning of the semester.  I think that a teacher using letter grades could more clearly define what a C was and what an A was by stating the objectives vital for that award than he or she can by trying to come up with a formula to fit every student.  We shouldn’t be afraid to use some criteria that cannot be expressed with percents.

 

How to create a skills list

10 Jan

My last post focused on three major mistakes I made in my first semester of skill-focused, mastery-based assessment: separating skills into chunks that were impractically small, choosing some skills that were too simple (almost trivial), and neglecting to plan for the end of the semester.  This post will focus on my process creating the skills list for semester two (for Algebra 2).  I’d love to hear your opinions or your own process – leave a comment or a link below!

The place to start is your curriculum map, whether that’s a list of topics, a set of state standards, a final exam, some chapters of a textbook, or whatever.  Find or create the document that describes what you hope to teach this semester.  The list I used last semester I got directly out of my textbook.  I knew what chapters I was going to teach, and I just ripped concepts out of the table of contents.  This process got me a list that I was moderately happy with; click here to download it.  Your list will almost certainly need to be different, since I was planning for 36 class periods.

For semester two, I set out in much the same way.  I went through the chapters in my text book I was planning on studying, and every time something popped up that seemed like it would be a good candidate for a skill, I wrote it down.  This gave me the following list of 36 skills:

Evaluating functions
Analyzing the domain and range of functions
Modeling relationships with functions
Modeling arithmetic sequences with functions
Modeling geometric sequences with functions
Distinguishing between arithmetic and geometric sequences
Recognize exponential growth from situations, tables, graphs, or equations
Understand multiple representations of exponential functions
Represent exponential functions algebraically
Using basic laws of exponents to simplify expressions
Use exponential functions to solve problems involving growth or decay
Find equations of exponential functions through two given points
Identify graphs of quadratic, cubic, square root, absolute value, etc functions
Transform a graph by stretching, shifting, or flipping it
Write a general equation for a family of functions
Use the “completing the square” technique
Model physical situations with quadratic functions
Write equations in graphing form
Invert functions analytically and graphically
Form compositions of functions
Express the relationship between a function and its inverse
Understand logarithms and transform their graphs
Use properties of logarithms
Use logarithms to solve exponential equations
Count possibilities in situations that require a particular order
Count possibilities in situations in which order does not matter
Draw a tree diagram to represent and calculate probabilities
Draw an area diagram to represent and calculate probabilities
Calculate expected value
Using the fundamental principle of counting
Calculate conditional probabilities
Find the value of arithmetic series of arbitrary length
Find the value of geometric series of arbitrary length
Find the value of geometric series with infinite length
Writing a series with summation notation
Using mathematical induction

The next step in the process is to look at each skill from the brainstorm and ask,

  1. How will I test this skill?
  2. Is this skill big enough to be its own skill?
  3. Is this skill small enough to be a single skill?
  4. Does this skill have multiple levels, so that intro level tests will be significantly different from master level skills?
  5. If a student does not understand this skill at all, am I willing to flunk him? (My grades are set up that each student must get a minimum of 3/5 in every skill to pass the class.  If you’re using a simple average, you can ignore this question).

Consider the first skill, “Evaluating Functions.”

  1. How will I test this skill?  What leaps to mind is showing a kid f(x)=3x+6, and asking for f(2).  Maybe f(f(2)) – or is that composition?  They also need to be able to evaluate functions from graphs and tables.  Maybe the question should be a three-parter?
  2. Is this skill big enough to be its own skill?  Hmm… it’s pretty small, isn’t it?
  3. Is this skill small enough to be a single skill? Yes, I am confident that it is.
  4. Does this skill have different levels?  So, I could ask them to evaluate f(x)=3x+6, or f(x)=3x-2/x+(x+5)^-3, but those aren’t different levels of evaluating functions, those are different levels of order of operations or something.  I could ask them to find g(f(2)), but maybe that’s composition.
  5. Is this skill a requirement of passing the course?  Absolutely.  I am not letting anyone who can’t evaluate a function out of Algebra 2.

So this first skill has some complications.  I really like testing f(g(2)) because it requires students to understand the input/output aspect of functions where I feel like a simple g(2) might let them slip by without it.  Since this skill is so essential, I’m leaving it in, though it might be a little bit small.  It may end up conflicting with the composition skill, but there might be flexibility in that skill.  ”Evaluating functions” seems like a solid requirement.

Let’s take another skill, “Modeling arithmetic sequences with functions.”

  1. How will I test this skill?  I’m looking for students to be able to come up with functions that describe arithmetic relationships, like, “write a function that outputs the number of gloves that x people will need,” or something.  I could show a table of inputs 1, 2, 3, 4, and outputs 8, 11, 14, 17, and have them write this function.
  2. Is this skill big enough to be on its own?  You know, I think it’s possible it could be combined with the skill before it, “modeling relationships with functions,” and the one after it, “modeling geometric sequences with functions.”  These three skills are so closely related, with the only difference being the arithmetic skills required.  I’m not trying to test those arithmetic skills – I hope the kids already have them – so I’m going to combine these three skills into just “modeling relationships with functions.”  I’ll answer the rest of these questions for the new skill.
  3. Is this skill small enough to be on its own?  Clearly the title can involve arbitrarily complex functions and relationships, but I think the kinds of simple relationships we’ll study in class can all be combined under one roof.  This skill may be a little bit too big.  If I was the organized man I wish I were, I’d note somewhere that I should revisit this skill after we touch on it to see how I felt.
  4. Are there different levels for this skill?  For intro level questions I can ask the students to model the relationship between Celsius and Fahrenheit, or some other linear relationship.  For master level questions I could ask a geometric volume question which includes an extra level of abstraction.
  5. Is this skill absolutely required for every student that passes the class?  Yes, I think so.  Who wants to teach precalc to students who can’t create their own functions?

Now, I don’t have time to write an essay about each of these questions for each of these skills, and neither do you.  In this post and in my brain I’m deciding to move quicklky here.  I only have so many hours to get this done, and it’s not going to be perfect.  I hope that you can make sacrifices like this – it’s taken me a long time to accept the impossibility of perfection in a finite time frame.  I spent about 30 minutes considering this list, eventually deciding to cut 7 skills and reword several.  Click here for my final checklist.

I hope this post has showed you how easy it can be to come up with a list of representative skills to assess.  It’s an unglamorous process, and the hardest part is coming up with the rough list, but once you do that you can have an effective list in less than an hour.  I don’t recommend using my skills lists wholesale.  I am in the process of trying out several different textbooks and my order is wonky.  My school is exempt from most standardized tests and if you have specific objectives you need to hit you’ll need to take them into consideration, obviously.  That said, here are my skills lists for Algebra II and Calculus this year:

Algebra II, Semester 1

Algebra II, Semester 2

Calculus, Semester 1

Calculus, Semester 2

Also, Dan Meyer has posted his suggestions for Algebra 1, Geometry, and Precalculus (imagine my chagrin when I saw he covered exactly everything but what I needed!).  So, if you’re considering getting started with this system, you at least have a launching point for your class.  If you have comments about my lists or process, I’d love to hear about them!  I’m especially interested what you think of the questions I use on each skill.  What else needs to be asked?

 

Mistakes I Made Creating a Skills Checklist

31 Dec

I made some rookie mistakes with my Algebra 2 skills checklist in semester one this year.  I have invented a word to describe each problem.

Hyperseparation

In my enthusiasm for separating skills, I gave determinants of 2×2 matrices and determinants of 3×3 matrices each their own spot on the list.  They should have been combined.  I also separated multiplication of polynomials from their division and even, astonishing in the euphoric clarity of hindsight, addition of matrices from their subtraction.  This overzealous separation led to inflated grades (students got a 5/5 on matrix addition, matrix subtraction, both kinds of matrix multiplication, AND twice for finding determinants) and tests that felt kind of… stupid.

Unrampability

A more subtle mistake: some skills on the checklist did not have discernible differences between intro-level problems and master-level problems.  For instance, an intro simplification test might look like “3x+2p-x+2x,” and a master might look like “3x+2p-x+2x-p+2p+r,” but for some students it felt silly to bother giving the intro level test first.  If a student can calculate “3+2+8+9,” do you really need to check to make sure they can calculate “3+2+8+9+1?”  In contrast, my favorite skills had some fundamental difference between the intro level and master level tests.  For example, the “dividing polynomials” intro test asked for a division that would have no remainder, and the master test involved remainders.  This was nice because a student could pass the class with a basic understanding of the concept, but would need a more advanced understanding before getting the 100%.

Bad Planning

I did not have an effective way to deliver master-level tests to the students who wanted them.  Early in the year I promised that students would be able to attempt a master-level test on any skill once per day, and early in the year this worked great.  Late in the year, when two thirds of my students wanted six tests a day, it was harder.  I need a better system.  Luckily, this won’t be a problem again until the end of this semester, so I’ll start thinking about it then.  ;)

So, now the task is to create the lists for next semester (my school doesn’t start until January 10!), hoping to avoid these problems and keep the amazing benefits I got from the system last semester.  Stay tuned!

 

Involving students in assessment

13 Dec

The norm is for the teacher to write a test, lead a few lessons, describe what will be on the test, and then administer the test, right?  Students are not involved in making the assessment that will determine their grades.  My form of summative assessment changed drastically this year, but it still does not address this issue.  From what I hear from other teachers (you?), there’s a lot of room in all of our classrooms for more student involvement in assessment creation.

The tempered radical recently wrote about the benefits of explaining to students exactly what the point of each lesson is.  It’s the kind of thing that seems so obvious when you say it like that, but this is relatively new research pointing at this stuff.  In the November 2009 issue of Educational Leadership, “The Quest for Quality” references research from 2006 and 2009 to make the radical claim that “students learn best when they monitor and take responsibility for their own learning.”  It goes on to say “This means that teachers need to write learning targets in terms that students will understand.”  Sam Shah wrote about an experience talking directly with students about what it means to think and act like a mathematician that was so powerful for him that he considers it a genesis for himself as teacher.  And meanwhile, I think I’m totally rad for talking with students about what they want out of my class.

It’s incredible that this stuff is new, right?

Some teachers are on to this already.  I read about teachers developing rubrics with their classes, and others having students write questions from which the teacher will select his favorite three, etc.  These teachers are already reaping the benefits:

  • students feel (are) respected
  • students feel (have) ownership of the assessment, which gives them a new responsibility
  • students know a lot about the assessment before the lessons are all over, which seems, you know, better.

These benefits are obvious and supported by research.

So, if you use an assessment scheme based on written tests (like I do), what are the best ways to get some of these benefits?  I want to experiment with having kids write and critique their own questions for sure, since this seems easy to implement and, at its worst, is a form of review.  I already ask them to assess their progress towards their personal goals.  What else can I do?

I am hereby declaring a new goal for the first month of next semester: I will find a way to include each student in the act and process of his or her own assessment, at least a little.  I’m aiming high – I don’t mean that I will include “the class” in creating “the assessment.”  Whew.  There’s something to think about on the 14-hour drive home for Christmas!

Please leave comments if you have ideas.  I just set this kind of big goal, and to be honest, guys, I don’t know how I’m going to meet it yet.

 

Bag of Tricks #1 – Index Cards

06 Dec

In “Bag o’ Tricks” posts, I’ll give activities that require almost zero prep, but inject a shot of fun, practice, activity, assessment, remediation, or whatever in a small amount of class time.

This post’s focus is index cards.  My students like them – I think they are just nicer objects than sheets of paper.  These are perhaps my favorite no-prep activities.

Memory (20 minutes)

  1. Each student gets two index cards.
  2. On one index card, each student writes an expression of a given type (e.g. an anonymous differentiable function like “2x+sin(x)”).  Every student must use a pencil.
  3. On the other index card, each student writes a corresponding expression after a given operation (e.g. differentiation – “2 + cos(x)”).  After this step each student has two cards that are connected by the given operation, but not by name or any other property.
  4. In pairs, students swap cards and check each other’s work.
  5. Each student gets another two index cards and repeats the process.  Each student now has a total of four cards, two pairs of linked cards.
  6. Students form groups of four, shuffle their combined sixteen cards together, and lay them out upside down.  The cards are (hopefully) indistinguishable.
  7. The students play memory (in teams of two, or not).  A team flips over one card, and then another.  If they match through the operation, they keep the pair, get a point, and go again.  If the cards don’t match, the next team is up.

This activity is great, after you figure out how to make sure students write problems of the appropriate difficulty.  They need to be pretty easy.  Memory is hard when its just pictures of barnyard animals, you know?  I use it to have students practice derivatives over and over again.  Every time they see, for example, “2x,” they have to think “what is the derivative of 2x, and what might have 2x as a derivative?”  You need a problem that’s easy, but takes lots of practice.  Distributing polynomials, finding logarithms, solving linear equations, etc.  The first time I used this activity I put, like, physics word problems on one card and answers on another.  Let’s just leave it at “don’t do that” and move on, please ;) .

Benefits of memory:

  1. A bunch of practice
  2. It’s reasonably fun
  3. Kids write their own problems and solve them
  4. Each student gets the advantage of knowing 2 of the 8 answers right away.  This almost guarantees some success for every student – everyone can feel engaged, even if their skill level is lower than the others’.

Write and Swap (5-7 minutes)

  1. Each student gets an index card and creates an example problem.
  2. Students swap cards at their table (I have tables of two) and confirm that the problems are in the proper form, etc.  Any questions about problem creation are resolved.
  3. The teacher moves quickly and energetically around the room, picking cards swiftly out of kids’ hands and giving them replacement cards from other kids.  This works elegantly – the teacher can move in any pattern, so as soon as problems are written they can be swapped out, but students who need more time may take it as the teacher is passing out cards.
    After this step each student has a new card in front of them, and they don’t know exactly where it came from.
  4. Each student solves the problem on his or her card.
  5. Students swap cards at their table and confirm solutions.  Any questions about problem solution are resolved.

Benefits of write and swap:

  1. Each student gets practice writing a problem, which may involve critical thinking about what is important to include.
  2. Each student thinks about four different problems in a row, but a physical interaction between each problem keeps attentions focused.
  3. Student responsibility is diffused.  Limited responsibility can help students feel safe, which can be important (though students should be fully responsible for at least some work every day).
  4. A peppy teacher can infuse the activity with energy on a slow day by zipping around the classroom in the big card swap.  Carry around a funny container instead of just holding the cards in your hand if you want.

Write and Swap is great for those times when you just want students to practice something kind of boring a few times.  It’s not great for longer problems because the phases get unsynchronized.

Most confusing part (5-7 minutes)

I got this from Science Formative Assessments, by Page Keeley.

  1. Each student gets one index card near the end of the period.
  2. Each student writes, anonymously, the thing about the class that was most confusing, least fun, whatever.
  3. The cards all go in a box and are redistributed, one card per kid.  Page Keeley recommends having the kids literally throw the cards around, but I admit to not being brave enough to try this yet.  It might make this activity really fun… or just add two minutes to its execution.
  4. Kids read their new cards aloud to the class.

The first time I tried this, I wasn’t that impressed with the results, but like any new technique I’ve gotten better at making it succinct and useful.  This activity is mostly to get a quick sense of how your lesson went, if you didn’t have any better way to do it built in.

Benefits:

  1. If a theme emerges, you know, that’s a great piece of information for the teacher.  Write that down on your lesson plan!
  2. You get to hear from every kid in a very low-pressure way.
  3. I imagine that kids who are embarrassed by a lack of understanding are heartened when they (inevitably(!)) hear that someone else had the same problem.
 

Time-independent assessment at the end of the period

06 Dec

This semester I started giving my students small, focused tests that attempt to isolate a single skill.  Among the many things about this method that are astoundingly great is the fact that reassessment is a snap.  A student can earn credit for a skill regardless of whether he understood it immediately or took 2 months of work to master it.  I can easily reassess his skill level in December, even if we studied the concept as a class in September.

But now there are only two weeks left in the semester, and grades will be due.  I want to be able to explore more material, and to test my students’ skill level with it.  The students want more tests, for goodness’ sake.  And if I give a test with only 3 days of class left… a student who earns a low grade at first does not get any time to improve!

It is clear that there is no way to teach new skills at the end of a grading period and give students a lot of time to become comfortable with them.  Is there a way to teach all new skills at least three weeks before the end of a grading period… and still make the last three weeks interesting and curricularly-advancing?

What do you do when a student fails a test at the very end of a grading period?

 

Assessment, One Skill At a Time

14 Nov

I recently realized that I was destroying some of the information that my tests collect.  I was averaging scores of multiple questions together, blending a student’s performance in different areas into a single, summative score. Instead of keeping the information that Johnny could multiply matrices perfectly (100%) but couldn’t really find inverses (50%), I was telling Johnny, “Johnny, you’re at about 75% in this class!”

And so I hit my first personal inflection point of the year.  I stopped averaging scores, and started telling students (and parents) about their strengths and weaknesses in specific skills.  For the most part I am following Dan Meyer’s example, described briefly at http://blog.mrmeyer.com/?p=346, and working with a vague idea of what Hans, a logic teacher here, does.   Instead of getting back a single big score for a month of class, the students get 10 or 12 separate scores.  Check out Dan’s blog for more details.

The new system immediately started helping in three important ways:

  1. Student motivation increased (dramatically in some cases),
  2. Remediation became more informed, and
  3. I started getting feedback that helps me streamline lessons before I give them and assess their efficacy after I give them!

1. Student Motivation

After switching to this new grading system, I have seen an increase in motivation in almost all of my students.  I don’t know if they like the check list, if they like to be recognized as masters of skills (and they are masters!), or if the simple act of breaking the course down into manageable chunks is what is doing it.  But it’s great.

A student that works to improve a single skill, and gets a higher grade in that skill, feels a sense of accomplishment immediately, even if he has five other skills to improve over the next week.  The change in the way I see students catching up is actually astonishing.  They can more easily see that they can do it, and they love it!

Furthermore, students that have earned 80% or 90% already seem motivated to earn the “master” designation in all of the skills on their list.  My experience with students in prior years has almost never been “I have a 95% already – can’t I please take the test again to get a 100%?,” but now it’s practically across the board.

In my old, averaging ways, I wasn’t giving my students this kind of specificity and manageability to work with.  Students with failing grades simply got a big fat “60%” on the top of a unit test.  Now they get “if you work on matrix multiplication, you’ll be at a passing level,” or “you are a master of linear equations – what did you do to get so good with those?”

2. Informed and Focused Remediation

When a student comes in to my office hours now, I can pull out my grade book and see that they aren’t yet passing in skills 13 or 18.  Since these skills are more or less independent of any other skill (an important feature of this program), we can get down to the students’ misunderstanding much faster.  Also, it’s natural to focus on skill 13, and then 18.  There’s no pressure to do everything at once for a single makeup exam that will re-test every skill simultaneously.  Especially for kids who perceive themselves as bad at math, I’ve seen an increased willingness to come to my office hours for help.

3. Formative Assessment

When I give a test, there are a few intro-level questions and a few master-level questions, and also a pretest question.  These are tests of skills that I have perhaps never mentioned before, or mentioned only briefly.  I let the students know that if they score well on these pretest questions they won’t have to take them in the future, but I don’t expect them to know how to do them (how could I?) and that they will absolutely not be penalized for doing them wrong, or just leaving them blank.  I estimate that it takes between 3 and 7 minutes to give a pretest question.  For that price I get a preview of my students’ current knowledge, before I plan a whole class about something they already know or plan to skip over something that they don’t know at all.  I also get a measurement of my skill as a teacher when the students take a test on the same concept after my lesson.  Knowing the difference between “my kids all aced this skill (but they learned it last year)” and “my kids all aced this skill (and they never even thought about it before this class)” helps me rate my lessons.

I recommend it, guys.  The switch is pretty easy, especially at the beginning of a grading period.  Writing tests is easier.  All you have to do is separate the skills you most care about (ok, this is hard), and then stop averaging!

 
 
Performance Optimization WordPress Plugins by W3 EDGE