Resources to support the JISC Assessment and Feedback programme

Assessment and Feedback: Principles 

Why use assessment and feedback principles?

Here is my thinking about principles. For examples of their implementation see the REAP project at www.reap.ac.uk 

Here is an article that talks about the application of principles in a large-scale project. The title is:  A blueprint for Transformational organisational change: REAP as a case study, David Nicol and Steve Draper, 2009. 

Principles provide: 

  1. A way of operationalizing an important idea in strategy or policy (e.g. assessment and feedback should help students develop the capacity to become self-regulated learners: this university will ensure that all students develop the capacity for critical thinking). The principles unpack and make clear what the concept means (e.g. self-regulation, critical thinking) and they point to a direction of travel for implementation.
  2. A language for sharing and talking about good practice and about the meaning behind proposed changes to curricula.  In the REAP project, having principles resulted in others disseminating the educational ideas behind the project not just the project team.  Principles therefore might help spread the changes suggested by the policy or strategy. The language should be clear as it will also help academics share their views with students. 
  3. Going further, principles provide a framework to make connections across examples of good and innovative practice in assessment and feedback that already exist in the institution. It should be possible to link what is considered as good practice to the REAP principles.
  4. A reference point for evaluating change in the quality of educational provision.  They point to processes which can be used as qualitative indicators of change. For example, if one principle for critical thinking is that ‘students must have opportunities to identify logical inconsistencies in what they and other’s write’ it would be possible to evaluate the number of occasions students have to engage in identifying logical inconsistencies before and after this principle became policy.
  5. A tool for evaluating the benefits afforded by technology applications in education – do technology applications support the implementation of the principles?
  6. A summary and simplification of the research evidence. In the case of assessment and feedback this provides a summary of the important ideas in a complex area for those who do not have time to read all the research literature. Each principle should have research backing and it will often have a strong face logic behind it.

Characteristics of principles

  1. Principles capture an important idea while at the same time they point to how it might be implemented. For example, good feedback practice should 'encourage interaction and dialogue around learning, peer dialogue and student-teacher dialogue' This captures the idea that feedback is a dialogical process and suggests an implementation approach. However, the next point is important.
  2. Principles are action-oriented: they specify an action but do not over-specify it. This formulation recognises that the way they might be implemented will often differ depending on the discipline and context. Consider 'facilitate the development of self-assessment and reflection in learning'. The implementation of self-assessment might take different forms in history as opposed to chemistry but the principle is equally applicable to both disciplines. This is the 'tight-loose' philosophy (Wiliams and Black, 2009). The principles are tight and they should be adhered to (especially if they are in the educational strategy and you wish them to guide implementation) whereas their application is loose - i.e. they must be tailored to the disciplinary and teaching context. 
  3. Principles can be written from different perspectives: 'clarify good performance' is something that it is recommended the teacher should do - note the action verb 'clarify'. All principles start with an action verb. In the student leaflet it says 'find out what is required before you begin an assignment' - this is written from the student perspective and also starts with an action, 'find out'.  If you wish to promote change then think about to whom the message is being communicated.
  4. Reading the principles should have an immediate impact on the reader - the essential meaning should be obvious. This is about how they are written. My best example is perhaps in the feedback as dialogue leaflet I produced with students http://www.reap.ac.uk/TheoryandPractice/Students.aspx. It took me many months to get the wording right but every time I show it to students they recognise that it is saying something quite new about feedback and how they should think about it.
  5. Principles (some people prefer 'guiding principles') are different from procedures: the following is not a principle 'assignments should have a turn-around time of 3 weeks'.
  6. Principles, I believe, should not state the obvious e.g. assessments should be an appropriate test of knowledge and skills or assessments should be ‘validity’.  Almost everyone agrees with this and who would try to design an invalid assessment, it is taken for granted.  In my work I tried to make the principles define a higher aspiration.
  7. In formulating principles you should try to make them as independent and as non-overlapping as possible. They will inevitably be overlapping to the extent that they are building blocks for each other. However, formulating them from this perspective helps one identify what is really important.
  8. The more principles that can be identified in a learning design, normally the better the design is.
  9. I have found that any design can be strengthened by thinking of how another principle might be brought into play.
  10. In a University policy docoument I would recommend that the number of principles should be limited – seven is a good number, ten or more is too many (but see Viewpoints below). This contradicts earlier work by Nicol in 2007 and 2009 where the number of principles grew from 7 to 12. Too many principles will dilute the message and the use of 'principles as a discourse' for change as it makes it hard for staff to remember them.
  11. The principles are essentially headline ideas/statements pointing to what needs to be done, in the case of REAP, what needs to be done if we want students to develop their capacity to be more self-regulated in their learning. However, the headline statements are the starting point not the end point. In constructing a package to support institutional change in assessment and feedback as well as the headline principle, you should produce a compelling argument for that principle, a set of examples of how it might be implemented in both numerate and text-based disciplines and identify the research evidence.  In Nicol and Draper (2009) we called this a set of 'rhetorical resources' and argued that these resources are required if the aim is to persuade the many different stakeholders in an HEI that changing assessment and feedback practices in this way is worthwhile.  Some academics for example will be happy with the headline argument, others will want to know how you might apply this idea in their discipline and still others will want to know what the compelling argument is for that idea and why they should make any change etc.

  • Viewpoints Project: principles as a tool to support module and course redesign

    In the Viewpoints project (wiki.ulster.ac.uk/display/VPR/Home), the University of Ulster have piloted a series of workshops where academic staff rethink the design of a module or a course using the REAP assessment and feedback principles. The Viewpoints team have printed nine principles on cards: each card has a single principle and a question on one side and on the other a set of examples of implementation (these are all drawn from Nicol (2009, see below).  

    Nine principles seems to work well here as course teams are asked to select those that they think are most relevant to the challenge they face and also because the participants are engaging in workshop task. In that task, as participants discuss one principle it leads them to consider another (from the pack) - this confirms the intuition I had when I wrote about them in 2009 saying that they would serve as 'building blocks' for each other or that the use of one 'would call on another'. 

    In the Viewpoints workshop academics also have a timeline of the module or the programme and they are normally encouraged to place the cards/principles on the timesheet. This is a simple poster sized sheet divided into four time-zones - the participants map these out to fit their perception of a module or a course etc. They start by placing the principles/cards on the timesheet - repeating the same principle many times as appropriate - they have many sets of cards. Then when they have the big picture the facilitator encourages them to 'think about how they will implement these ideas'. Members of the course team then turn over the cards and look at the examples, discuss those and modify them or invent their own. The examples are just that - trigger ideas that are expanded upon. Participants record their ideas on post-it notes and place them on the timesheet. At the end of a session the facilitator photographs the output - the timesheet and placed cards and post-its to provide a permanent record.

    The Viewpoints workshop is very simple but also very powerful. More detail about the benefits can be found on the University of Ulster website. I cannot do justice to the value of the process or the benefits here. However, it is worth noting that academics report that the artefacts (the timesheet and the principles) are very effective in facilitating productive discussions about how to improve teaching and learning in a module or course. The visible nature of the artefacts provides a shared representation as new ideas for the course are discussed and placed on the time-sheet. The beauty of the process is that it is the 'principles that facilitate the discussion' (Masson, 2012).   

    Learner proactiveness and implementation of principles

    A key idea behind many assessment and feedback principles concerns learner pro-activeness: what I mean here is that the more active learners are and the more responsibility that they have in the implementation of a principle, the more empowering the educational experience. For example, a teacher might ‘clarify what good performance is’ [principle 1 in the Nicol and Macfarlane-Dick (2006) seven principles] for an essay writing task by providing learners in advance of the assignment with a list of printed criteria. However, what if the teacher instead organised a session where learners were required to examine some example essays (e.g. produced by a previous student cohort) before carrying out the assignment to identify which is better and why. The second approach would usually be more empowering than the first because the learners would be more actively engaged in constructing, internalising and owning the assessment criteria. This pro-activeness dimension is relevant with regard to all the REAP principles. It is recommended, therefore, that in implementing principles consideration is always given to how responsibility might be shared with learners so that they are active participants in assessment processes.

    Finally for now

    This is a page that I will come back to and update. I am currently carrying out an evaluation of the JISC-funded Viewpoints project so I will try to add more as I go along.

    I would be happy however to receive any questions or views on this document and I will try to address these in this document's development.

    David Nicol

    Original version: June 5th 2011

    Update: 14 May 2012

     

    Other references

    Nicol, D. (2009) Transforming assessment and feedback: enhancing integration and empowerment in the first year. Published by the Quality Assurance Agency for Higher Education. Full text

    Nicol, D, (2009), Assessment for Learner Self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3),335-352

    Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.