Re-engineering Assessment Practices in Higher Education

Assessment and Feedback Principles: Rationale and formulation 

Why use assessment and feedback principles?

Here is my thinking about principles. For examples of their implementation see the REAP project at www.reap.ac.uk and the papers cited below.

Here is an article that talks about the application of principles in a large-scale project. The title is:  A blueprint for Transformational organisational change: REAP as a case study, David Nicol and Steve Draper, 2009. 

Principles provide: 

1. A way of operationalizing an important idea in strategy or policy (e.g. assessment and feedback should help students develop the capacity to become self-regulated learners: this university will ensure that all students develop the capacity for critical thinking). The principles unpack and make clear what the concept means (e.g. self-regulation, critical thinking) and they point to a direction of travel for implementation.
2. A language for sharing and talking about good practice and about the meaning behind proposed changes to curricula.  In the REAP project, having principles resulted in others disseminating the educational ideas behind the project not just the project team.  Principles therefore might help spread the changes suggested by the policy or strategy. The language should be clear as it will also help academics share their views with students. 
3. Going further, principles provide a framework to make connections across examples of good and innovative practice in assessment and feedback that already exist in the institution. It should be possible to link what is considered as good practice to the REAP principles.
4. A reference point for evaluating change in the quality of educational provision.  They point to processes which can be used as qualitative indicators of change. For example, if one principle for critical thinking is that ‘students must have opportunities to identify logical inconsistencies in what they and others' write’ it would be possible to evaluate the number of occasions students have to engage in identifying logical inconsistencies before and after this principle became policy.
5. A tool for evaluating the benefits afforded by technology applications in education – do technology applications support the implementation of the principles?
6. A summary and simplification of the research evidence. In the case of assessment and feedback this provides a summary of the important ideas in a complex area for those who do not have time to read all the research literature. Each principle should have research backing and it will often have a strong face logic behind it.

Characteristics of principles

1. Principles capture an important idea while at the same time they point to implementation. For example, good feedback practice should 'encourage interaction and dialogue around learning' captures the idea that feedback is a dialogical process and suggests that this should be encouraged by teachers through their task or course designs. This might be realised through opportunities for peer dialogue, for teacher-student dialogue or even employer-student dialogue if students are on placement.  However, the next point is important.
2. Principles are action-oriented: they specify an action but do not over-specify it. This formulation recognises that the way they might be implemented will often differ depending on the discipline and context. Consider 'facilitate the development of self-assessment and reflection in learning'. The implementation of self-assessment might take different forms in history as opposed to chemistry but the principle is equally applicable to both disciplines. This is the 'tight-loose' philosophy (Wiliams and Black, 2009). The principles are tight and they should be adhered to (especially if they are in the educational strategy and you wish them to guide implementation) whereas their application is loose - i.e. they must be tailored to the disciplinary and teaching context. 
3. Principles can be written from different perspectives: 'clarify good performance' is something that it is recommended the teacher should do - note the action verb 'clarify'. All principles start with an action verb. In the student leaflet it says 'find out what is required before you begin an assignment' - this is written from the student perspective and also starts with an action, 'find out'.  If you wish to promote change then think about to whom the message is being communicated.
4. Reading the principles should have an immediate impact on the reader - the essential meaning should be obvious. This is about how they are written. My best example is perhaps in the feedback as dialogue leaflet I produced with students http://www.reap.ac.uk/TheoryandPractice/Students.aspx. It took me many months to get the wording right but every time I show it to students they recognise that it is saying something quite new about feedback and how they should think about it.
5. Principles (some people prefer 'guiding principles') are different from procedures: the following is not a principle 'assignments should have a turn-around time of 3 weeks'.
6. Principles, I believe, should not state the obvious e.g. 'ensure assessments are an appropriate test of knowledge and skills' or 'ensure validity of assessments'.  Almost everyone agrees with this idea and strives to implement it: indeed no-one would consciously try to design an invalid assessment. Hence it is a taken-for-granted principle.  In my work I have tried to make the principles define and point to a higher aspiration.
7. In formulating principles you should try to make them as independent and as non-overlapping as possible. They will inevitably be overlapping to the extent that they are building blocks for each other. However, formulating them from this perspective helps one identify what is really important.
8. In designing principles it is natural to think that the more abstract the principle the wider its scope. However the REAP principles contradict this idea; they are both general in that their scope is wide yet they are specific in their call for action. This relates to the tight-loose structuring.
9. The more principles that can be identified in a learning design, normally the better the design is.
10. I have found that any design can be strengthened by thinking of how another principle might be brought into play.
11. In a University policy document I would recommend that the number of principles should be limited – seven is a good number, ten or more is too many (but see Viewpoints below). This contradicts earlier work by Nicol in 2007 and 2009 where the number of principles grew from 7 to 12. Too many principles will dilute the message and the use of 'principles as a discourse' for change as it makes it hard for staff to remember them.
12. The principles are essentially headline ideas/statements pointing to what needs to be done, in the case of REAP, what needs to be done if we want students to develop their capacity to be more self-regulated in their learning. However, the headline statements are the starting point not the end point. In constructing a package to support institutional change in assessment and feedback as well as the headline principle, you should produce a compelling argument for that principle, a set of examples of how it might be implemented in both numerate and text-based disciplines and identify the research evidence.  In Nicol and Draper (2009) we called this a set of 'rhetorical resources' and argued that these resources are required if the aim is to persuade the many different stakeholders in an HEI that changing assessment and feedback practices in this way is worthwhile.  Some academics for example will be happy with the headline argument, others will want to know how you might apply this idea in their discipline and still others will want to know what the compelling argument is for that idea and why they should make any change etc.

• Viewpoints Project: principles as a tool to support module and course redesign

  In the Viewpoints project (wiki.ulster.ac.uk/display/VPR/Home), the University of Ulster have piloted a series of workshops where academic staff rethink the design of a module or a course using the REAP assessment and feedback principles. The Viewpoints team have printed nine principles on cards: each card has a single principle and a question on one side and on the other a set of examples of implementation (these are all drawn from Nicol (2009, see below).  

  Nine principles seems to work well here as course teams are asked to select those that they think are most relevant to the challenge they face and also because the participants are engaging in workshop task. In that task, as participants discuss one principle it leads them to consider another (from the pack) - this confirms the intuition I had when I wrote about them in 2009 saying that they would serve as 'building blocks' for each other or that the use of one 'would call on another'. 

  In the Viewpoints workshop academics also have a timeline of the module or the programme and they are normally encouraged to place the cards/principles on the timesheet. This is a simple poster sized sheet divided into four time-zones - the participants map these out to fit their perception of a module or a course etc. They start by placing the principles/cards on the timesheet - repeating the same principle many times as appropriate - they have many sets of cards. Then when they have the big picture the facilitator encourages them to 'think about how they will implement these ideas'. Members of the course team then turn over the cards and look at the examples, discuss those and modify them or invent their own. The examples are just that - trigger ideas that are expanded upon. Participants record their ideas on post-it notes and place them on the timesheet. At the end of a session the facilitator photographs the output - the timesheet and placed cards and post-its to provide a permanent record.

  The Viewpoints workshop is very simple but also very powerful. More detail about the benefits can be found on the University of Ulster website. I cannot do justice to the value of the process or the benefits here. However, it is worth noting that academics report that the artefacts (the timesheet and the principles) are very effective in facilitating productive discussions about how to improve teaching and learning in a module or course. The visible nature of the artefacts provides a shared representation as new ideas for the course are discussed and placed on the time-sheet. The beauty of the process is that it is the 'principles that facilitate the discussion' (Masson, 2012).   

  Learner proactiveness and implementation of principles

  A key idea behind many assessment and feedback principles concerns learner pro-activeness: what I mean here is that the more active learners are and the more responsibility that they have in the implementation of a principle, the more empowering the educational experience. For example, a teacher might ‘clarify what good performance is’ [principle 1 in the Nicol and Macfarlane-Dick (2006) seven principles] for an essay writing task by providing learners in advance of the assignment with a list of printed criteria. However, what if the teacher instead organised a session where learners were required to examine some example essays (e.g. produced by a previous student cohort) before carrying out the assignment to identify which is better and why. The second approach would usually be more empowering than the first because the learners would be more actively engaged in constructing, internalising and owning the assessment criteria. This pro-activeness dimension is relevant with regard to all the REAP principles. It is recommended, therefore, that in implementing principles consideration is always given to how responsibility might be shared with learners so that they are active participants in assessment processes.

  New Paper

  Recasting the educational Discourse in Higher Education: A principles-based model of change

  David Nicol and Michela Clari

  This paper provides a new way of thinking about and managing change in higher education institutions. It makes the case for using research-informed educational principles as a srategic resource to seed new conversations, new mindsets and new teaching and learning practices across a whole higher education institution. The paper draws on recent projects that have used assessment and feedback principles as a framework to guide wide-ranging changes in curriculum practices across higher education institutions (Nicol and Draper, 2009: Draper and Nicol, 2013: Ferrell and Gray, 2013) and on current research in organisational development which highlights the importance of language and discourse in organisational change processes (Grant and Marshak, 2011: Marshak and Grant, 2011). This paper is relevant to change agents and senior managers in higher education who wish to make significant and long lasting improvements in the quality of educational provision. It presents a practical approach to the planning, implementation and evaluation of a large-scale change initiative in higher education. David Nicol June 2015.

  References

  Draper, S.W. and Nicol, D.J. (2013) Achieving transformational or sustainable educational change. In S. Merry, M. Price., D. Carless., & M. Taras (eds) Reconceptualising feedback in higher education: Developing dialogue wth students.  London: Routledge, pp190-203

  Ferrell, G. & Gray, L. (2013) Changing assessment and feedback practice: How to approach large-scale change in assessment and feedback practice with the help of technology. Available at www.jisc.ac.uk/guides/changing-assessment-and-feedback-practice

  Nicol, D (2012) Transformational change in teaching and learning: recasting the educational discourse, Evaluation of the Viewpoints project at the University of Ulster.  Funded by JISC UK.  July 22nd [This document shows how the principles developed in Nicol, 2009 were turned into a toolkit and used to spread a new discourse about assessment and feedback across a whole HE institution.]  Full Text

  Nicol, D. (2009) Transforming assessment and feedback: enhancing integration and empowerment in the first year. Published by the Quality Assurance Agency for Higher Education. [This document provides a list of 12 principle of assessment and feedback, the research rationale for each, an explanation and some examples of how they could be implemented in different disciplinary contexts]  Full text

  Nicol, D, (2009), Assessment for Learner Self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3),335-352

  Nicol, D and Draper, S. (2009) A blueprint for transformational rrganisational change in higher education: REAP as a case study.  In Education through Technology-Enhanced Learning,edited by T. Mayes, D. Morrison, H. Meller, P. Bullen, and M. Oliver, 191-207.  York. Higher Education Academy. text

  Nicol, D, J. & Macfarlane-Dick, D. (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.