Tool assesses student work more meaningfully with peer review

In traditional writing classrooms students write and teachers grade their work.  Usually the work isn’t written for an authentic audience but rather what I call the audience of one (the teacher) or perhaps some (classmates).  When we push students to do work that is not worth publishing and has no audience in mind, we are teaching them some bad lessons.  

1. What you have to say is not important enough for anyone but the teacher to read.
2. You are not good enough to have your work published.
3. Audience is not important.
4. Writing is not a tool to connect you with others who share your passions and interests.
5. What the teacher believes about your work is all that matters.

And, let’s face it.  If a secondary teacher has 180 students how much time are they really able to devote to student work?  One way to help this process is by using rubrics which is something I did in my practice as a literacy coach and library media specialist.  Students would self assess, then have two peers assess and turn that in with their papers.  This encourages students to take ownership of their work and have conversations with others about it as well.  

Today there is a cool product that helps automate this process. SWoRD is a free web-based, peer review system that was developed to help teachers organize writing assignments in a way that uses peer review as its backbone.  Students learn a lot from giving their peers feedback, and they learn a lot from getting feedback from multiple peers. SWoRD makes peer review so easy that teachers have the opportunity to assign writing without adding teacher work because the time restraints of teacher reviewing and paper grading is greatly minimized. Instead of assessing student work, teachers can be working with students to develop and grow their work.  SWoRD has a number of novel features over the standard peer review approach to make students take the task seriously and to simplify the teacher’s life. Once a teacher signs up for an account and creates a class w/ assignments it is ready for students to sign up for their accounts and join your class. Students then submit their papers and SWoRD automatically assigns them for review to their peers. As part of the assignment set-up the teacher provides the students with reviewing rubrics. They use this combination of comments and ratings to anonymously evaluate and offer useful advice in revising the paper. This valuable exercise gets student actively involved in thinking about (and articulating) what makes their writing “good.” What is also powerful about SWoRD is that work can not only be shared with peers, but the teacher can set it up so work can also be shared with mentors, family and others whose feedback could be valuable.  

Will students take these peer review tasks seriously? SWoRD not only takes into account if they actually completed the reviewing tasks but it also gives students the opportunity to respond to the reviews through back-evaluation. Students as authors respond to the reviews and provide a helpfulness rating that figures into the reviewer’s reviewing grade.

Once reviewing and back-evaluation are completed SWoRD automatically grades the papers and reviews based on a number of features.  First, SWoRD examines and weighs each reviewer's evaluations through its own unique algorithm. This is used for grading each review activity. Also, this measure is used for computing each writer's writing quality. It helps writers from being penalized by unqualified reviewers.

The teacher determines the % value (typically, Task= 20%, Writing= 40% and Reviewing= 40%) of the following grading categories: task grade (what percentage of assigned reviews and back evaluations were done), writing grade and reviewing grade to determine the overall grade.

The Writing Grade is basically an average of what the reviewers rated a paper, with some minor adjustments. First, the system calculates the reviewing accuracy of each of the reviewers. Rather than an equally weighted average across reviewers, the system weights higher accuracy reviewers more. So, if one of the reviewers is either flipping coins or is very confused about how to do the reviewing, that reviewer's ratings make up a smaller portion. Second, the system does curving----what matters is how well the student wrote relative to others in the class. The teacher determines the curve (what the average grade will be). Third, if the teacher also grade a draft, those ratings are included in the paper grade, but the teacher determines how much their ratings count relative to student ratings. Finally, if a draft was submitted a little bit late (during what SWoRD calls a “grace period”), the student will be docked a few points (also determined by the teacher).

The Reviewing Grade is basically an average across drafts of the following two pieces: the accuracy grade and the helpfulness grade (minus any penalties for submitting reviews during the grace period). The accuracy grade is the degree to which the student rated papers on every reviewing dimension in the same quality order as the average rating given by other reviewers to those same papers. If they loved papers that others hated or if they hated papers that others loved, then their accuracy score goes down. This accuracy number is curved, so what matters is how accurate they are relative to the accuracy of other reviewers. The helpfulness grade is the extent to which authors thought their review comments were generally helpful. The first step is to normalize each author's ratings (some authors might be cranky across the board or overly nice across the board). Then the average rating they got across reviews is calculated. Then the system looks at how helpful reviews were relative to others in the class?

In short, SWoRD does a lot of behind the scenes work that allows teachers to focus on teaching and students to focus on writing.  The final writing is also more likely to writing that is ready for publishing to a real audience because it has been thoughtfully vetted by the writer and his peers.  

SWoRD is a free, educational tool that is available to teachers worldwide. To sign up for an account you can visit: http://sword.lrdc.pitt.edu/sword.  For more information you can visit https://sites.google.com/site/swordlrdc.