Software to support peer review

Despite the benefits of peer review for learning in higher education it is not an approach widely implemented across courses and programmes.  One reason for this is that peer review is administratively complex and time consuming to administer especially when student numbers are large.  Peer review could be made easier and more efficient if there were software support but this is an area of technology that is not well developed although this situation has begun to change.  In reviewing the software to support peer processes it is clear that many of the first tools were developed with the primary purpose of supporting peer assessment, where students marked each other's work (e.g. SPARK, iPeer). However, there are a number of more recent systems that have as their prime focus formative peer review, where the intention is to use peer processes to enhance learning. 

In this section, the purpose is to first identify the features of peer review that a software system must instantiate if the educational benefits are to be realised. A second purpose will be to review some of the most promising currently available software systems for formative peer review and to summarise the features of different software packages.

Desirable feature list for peer review software

Class Lists

  • Extracting lists from institutional databases (e.g. LDAP or LMS)
  • Accommodating students not on institutional databases
  • Specifying subsets of students (e.g. tutorial groups) for particular exercises.

Uploading material to be reviewed

  • Description of the upload requirements
  • Date and time material is expected/required to be uploaded
  • Sending reminders to reviewers
  • Specifying the types of files required for upload (in a form that allows the system to enforce)
  • Support for different kinds of material (e.g. essays, computer programs, photographs, collections of documents)
  • Support for material stored elsewhere (e.g. specified by url)
  • Allowing others (e.g. a teacher) to upload on behalf of students
  • Enabling students to check what has been uploaded
  • Monitoring what has been uploaded, when, and by whom

Allocating material to reviewers

  • Specifying the number of reviews to be completed, and the due date
  • Enforcing dates and time for reviews
  • Sending reminders
  • Option/restriction on students taking part in reviews when they have not uploaded material
  • Allocating assignments to reviewers by topic (e.g. students produce and review same topic, different topic or select topic for reviews) or by ability (e.g. if criteria defined).
  • Support for group reviewing
  • Support for late submissions (e.g. informing reviewers of late submissions, giving extensions to individuals or groups, allocating late submissions)
  • Anonymity: keep author identity hidden, reviewer identity hidden, both or neither
  • Option for self-review, before or after peer reviews
  • Managing mistakes (e.g. where a review is written on the wrong assignment)
  • Monitoring of reviews by teachers (e.g. showing competions, browsing reviews)

Review rubric

  • Provision for reviewing instructions
  • Defining criteria and expectations
  • Rubric creation tools
  • Selection of templates for comments and ratings
  • Specifying text entry (e.g. specifying minimum, maximum words, size of entry)
  • Specifying rating formats if used (e.g. numeric, Likert scale, option list)
  • Formatting of review content (sections, bullet lists, fonts, pre-defined images such as smiley faces)
  • Allowing students to create/select their own rubric
  • Extra: writing feedback directly on submitted assignments (e.g. post-it notes, annotations, editing marks)
  • Automatic back-up of reviews
  • Printing of reviews
  • Rubric libaries for sharing of rubrics amongst teachers

Return of reviews and responses to reviews

  • Feedback can be viewed (e.g. during the review stage, after reviewing is complete for group
  • View reviews written by others on material they also reviewed
  • Printing reviews
  • Means of allowing the quality of the reviews to be commented on or rated by the teacher or student
  • Mechanism to use review-of-reviews in teacher rating of marks
  • Discussion forum, for particular submissions or for the overall activity

Managing Groups

  • Defining group membership for target assignment or review group
  • Specifying constraints on group membership
  • Automatic generation of groups, randomly or based on constraints
  • Support for changing group membership
  • Allowing students to allocate themselves into assignment or review groups, possibly subject to constraints (e.g. different team members from previous activities)

Other administrative functions

  • Deleting or archiving material from previous time-periods or cohorts
  • Guest access to allow others to view but not edit assignments or reviews
  • Setting up now peer review exercises
  • Allocating instructors to new peer review assignments
  • Time recording of how long students spend writing each review

Additional student support

  • Online help; e.g. a step-by-step guide
  • Practice exercise
  • List of dates and calendar
  • Access to reviewing from previous semesters/cohorts

Managing of grades (this is peer assessment, not peer review)

  • Students grading each other's work (note: this kind of grading can undermine the benefits of peer review)
  • Teachers grading peer reviews

Software Systems