Debra Macfarlane
Posts: 21
.jpg)
 |
| 29/05/2007 10:59 |
|
| David was interested in feedback on whether the principles occupied the correct quadrant in his framework diagram, p. 6 of his paper. I raise the point that I was unsure about the placing of the principle, 'encouraging positive motivational beliefs and self-esteem' in the academic/engagement quadrant. For me, motivational beliefs and self esteem span both academic and social arenas, and indeed one can affect the other. In addition, I would also question their position on the vertical axis - doesn't the encouragement of these things enable empowerment and self-regulation? It's a difficult one. Anyone have any views? |
|
|
|
|
Alice Lau
Posts: 2
|
| 29/05/2007 11:25 |
|
Hi Debra,
I agree with you that motivational beliefs and self esteem span both from academic and social arenas. But to me, I can see why David might place it under the academic/engagment quardrant. I would imagine that as a student, that lecturer's positive comments to my work would have a greater motivation effect compared to those from peers. |
|
|
|
|
Debra Macfarlane
Posts: 21
|
| 29/05/2007 11:42 |
|
Hi Alice,
I know what you are getting at re lecturer vs peer feedback. However, I think placing it differently is not just about peceived value of the comments received. This principle is a funny one for me as lots of the other principles have an effect on whether motivational beliefs and self-esteem are being encouraged. As an example, you may affect and encourage motivation beliefs and self-esteem by supporting the development of learning communities or by giving choice in assessment content. In fact, you may affect it by following almost any of the principles. As a result, it seems to underpin a lot of this thinking and doesn't easily rest, for me, in one quadrant. It seems more central to self-regulation and therefore, difficult to position. |
|
|
|
|
sue tickner
Posts: 5
|
| 29/05/2007 11:57 |
|
| yes I agree - my first thought was that it should be a circle in the middle! But isn't it also true that the balance of power lends more weight to the lecturer's feedback, and the lecturer has [and should have?] more control over it? |
|
|
|
|
David Nicol
Posts: 18
|
| 29/05/2007 12:09 |
|
| Thanks for all this - my first thought was to use the two dimensions and I do make the point that the principels could slide up or down the vertical dimension depending on how they were implemented. The other dimension was actually about connecting the academic and social so they are only separated for illustrative purposes - but what it brings out is that structure is even more important in organising academic with social. Steve's thing about the solo-group derives from this. The four quadrants were not meant to be fixed in stone - idea was to think about first year and work my thinking through that - it did get me to think hard. But well spotted I had difficulties with motivation because it is behind and caught up in self-regulation (i say somewhere that SR is a fusion of skill and will). However, the five principles in the lower half of the diagram need to be taken care of in the first year and are related to the first year research. That was all I wished to note. I ended by saying that different permutations would be required for later years. |
|
|
|
|
David Nicol
Posts: 18
|
| 29/05/2007 12:24 |
|
Debra, leaving aside the diagram for a moment what do you think about the other 4 principles given that we collaborated on the original seven. I have collapsed two of the original seven and reordered them in the paper to what is more logical and formulated questions for teachers. I became unhappy that we had not gone far enough in relation to sharing responsibility for learning with students. The tenth principle - engage students in decision-making about assessment policy and strategy would be a proper stakeholder in the learning organisation approach. The one about choice was a useful addition to the original seven and also the one about communities because there was insufficient emphasis on this dimension.
|
|
|
|
|
Margaret Price
Posts: 2
|
| 29/05/2007 12:36 |
|
| Thanks David for an interesting paper. I can see where you are going with this framework as it tunes with our work at ASKe at Oxford Brookes but it does seem to artificially divide the academic and the social whereas the principles when applied can have both academic and social aspects. As you point out in your paper the way the principles are applied may change their position in the framework and by placing them in one quadrant you would not want to lose the meaning they would have if placed in other quadrants. Permeabitlity of the academic and social may be too great to meaningfully divide it. For example our work on feedback is showing the importance of dialogue in engaging students with assessment feedback which in turn enables them to construct their own understanding ( and relating to their own self assessment etc.) that seem to suggest it is both a social and acdemic process. |
|
|
|
|
Margaret Price
Posts: 2
|
| 29/05/2007 12:39 |
|
| As you know david we are placing a lot of emphasis on the importance of community in our work . I would see the the stakeholder principle ( engaging students in decision making ) to be very much part of the building of community |
|
|
|
|
David Nicol
Posts: 18
|
| 29/05/2007 12:52 |
|
| Hi Margaret, I agree with your comments and I certainly didn't mean to separate the academic and social hence the arrows pointing together and comments about academic triggering social and the positive backwash from social on academic. Another option would be to look at each principle in relation to how it could serve each quadrant but this became quite complex. Also, I was thinking about teachers and about how to support thinking about the design of learning and especially with reference to the first year. Diagrams are sometimes constraining but this one did help me to rethink the ideas and especially the community dimension which was highlighted in many of the reap implementatiosn. Look forward to the ASKe meeting later in the year. |
|
|
|
|
Debra Macfarlane
Posts: 21
|
| 29/05/2007 13:14 |
|
Yes, I like the new principles - need some time to digest them and mull over them before I give a considered response, but yes on first view they seem useful additions to our work.
As for collapsing two former principles, not sure where 'providing opportunities to close the gap' has gone. Do you aim to cover this under the principle about 'delivering high quality feedback info that helps learners self-correct'? I think this principle is still valid.
Also, I think P.8 (on policy and practice) is interesting. I can see why it is valid but it would not be top of my priority list (you mention it as the tenth principle in your post, and I think it would be better situated in this position -I may have a different version of the paper?).
I do like the questions that are formulated for teachers that come under each principle. They stick to our original key aim of making what was a theoretical set of principles practical and workable for teachers out there.
As always, I would still like to play about with diagram - it is not yet clear enough to me how it works - perhaps we can get together and debate it!
Debra
Debra |
|
|
|
|
Steve Draper
Posts: 25
|
| 29/05/2007 13:49 |
|
Debra
I’d agree with your views. In my view, the 2 dimensions, either in the form in David’s paper or developed somewhat differently in mine, are independent of the principles. This means any principle can be implemented in different ways, to fit it in any quadrant. So it doesn’t make any sense to say it belongs in one; but it is a great mental exercise to try to think of implementations that would fit it in each in turn. Typically we first think of one place it more obviously belongs.
For example with “close the gap”:
1. BotR: Social-engage = groupwork + low autonomy. Common repeated tasks that all can contribute to and all wish to accomplish. e.g. Baxter. I.e use the entraining effects of social groups to "engage" i.e. get work out of them.
2. BotL: acad-engage = solo + low autonomy. Pure Chickering/Gibbs: address time management and planning by organising a large number of small tasks for Ls. Typical in many science courses.
3. TopL: Acad-empower = Solo + high autonomy. Students set themselves (or find for themselves) examples to practice on e.g. do practice exam essays. They may then seek out staff, and get them to comment on their attempts on the spot.
4. TopR: Social-empower = group + high autonomy. Organise one's own study groups, and do the exercises/tasks with them e.g. set each other quiz questions; do an essay under exam conditions, then swap and critique.
Now can I meet my own challenge for “encourages positive motivational beliefs and self-esteem”? Part of my issue here is that I would now approach what is in some sense the “same” issue very differently (influenced by Dweck). There is a direct emotional, indeed mental health, effect of our interpretations of success and failure (elation/mania, vs. sadness/depression). But there is also a non-emotional functional aspect which may be primary: that whenever we get feedback there is the rational issue of what we take it as evidence of: our traits, or our effort, or our skill (to be adjusted) etc. Still, here goes.
1. BotR: Social-engage = groupwork + low autonomy. Common task that all can contribute to and all wish to accomplish. e.g. Baxter. I.e use the entraining effects of social groups to "engage" i.e. get work out of them. Exercise classes are an example where simply having others there, makes many people more likely to carry on with a task that rationally is solo. This works even more so with academic tasks, where an individual might get stuck if not promptly helped out by peers.
2. BotL: acad-engage = solo + low autonomy. Again, lots of task: Provided the learners “succeed”, this will be reinforcing. It’s noticeab le that in science subjects, students who can do all the problems don’t need a teacher to tell them they are clever: the emotional part of the feedback is intrinsic. Success at the task is, as Bandura showed, the single biggest raiser of self-efficacy.
3. TopL: Acad-empower = Solo + high autonomy. Choice of curriculum (of option courses, of project topics of which question to revise for in exams); agree or choose criteria; choose one's preferred learning activities and/or resources. Here I think we should look at Snyder’s Hidden Curriculum which seems to argue that in reality students are exposed to multiple inconsistent measures of learning success, and must (and do) make a personal choice about which to go for e.g. looking good to teacher vs. focussing on topics best for the next career step.
4. TopR: Social-empower = group + high autonomy. Organise one's own study groups, which can also play a role here: praise from knowledgeable peers on your work is highly valued, and can help free students from what may be relatively narrow staff concerns. For instance, a few of my students write really well, but won’t be rewarded for that in our dept.; but probably will be in many future careers. So staff won’t mention this much; but peers might.
SteveD |
|
|
|
|
David Nicol
Posts: 18
|
| 29/05/2007 13:57 |
|
| I forgot to mention that one thing that influenced me in all of this was the principle of 'encourage time and effort on challenging learning tasks' - also not in the original seven. In REAP I had began to think that it was the designing of learning tasks or activities that is the major challenge in many assessment designs. Also, it is not about one task but a series of tasks of increasing difficulty or challenge. A sequence of low stakes tasks is definitely required in the first year as it helps students learn what is required, it gives opportunities to act on feedback and it stops students from bunching all their work at the end of the semester. So it also helps them learn about time management and can increase confidence and a sense of control over learning. Tasks are given power (but more effort is required in thinking about structure) when they involve peer dialogue (more to go wrong). This principle really derives from Gibbs and Simspon (and from Chickering and Gamson). This principle, along with the original seven was used to analyse some REAP case studies (it can be found in the resources section of the reap website, see 2006 ascilite) |
|
|
|
|
cynthia shedd
Posts: 5
|
| 29/05/2007 14:36 |
|
I think that "encourage time and effort on challenging learning tasks" is a very important principle and I'm glad you added that. And as you note, it is important for first year students to have the opportunity to learn the skills and gain the confidence they will need to progress. I have found, however, that it is not enough to provide these experiences--one has to be explicit about the purpose.
We have a strong (mandatory) first year experience program for all freshmen at our school, with students choosing from a variety of linked courses or taking two stand-alone courses. Some faculty were teaching time management skills by breaking larger assignments, such as research papers, into smaller steps with individual deadlines. In talking with students in focus groups, it was clear that none of them were connecting those smaller steps with time management. Lesson learned: Don't assume students connect their day-to-day activities with a larger framework. Provide a context and be explicit.
|
|
|
|
|
Debra Macfarlane
Posts: 21
|
| 30/05/2007 10:50 |
|
Hi Steve,
I like what you've done here and it makes a lot of sense to me and in a way helps me to like the diagram more. I do think that how you interpret the principle into a learning activity in practice has a great degree of relevance in terms of what quadrant the pronciple fits in. Maybe we shoudn't worry about fitting the principles in at all but look again...
What I would maybe like to see is the use of the diagram to do something slighly different. If you used the diagram but took the principles out of the quadrants and just let them be the foundation for it you could do something different.
What about if the diagram were used to illustrate the development in complexity of a learning activity, underpinned by a principle, over time. For example, lets start with your 'close the gap' example. We might expect points 1. & 2. to happen quite frequently in the first year, progressing to the types of learning activity you describe in 3. & 4. later on.
Again, for your example on “encourages positive motivational beliefs and self-esteem” we might reasonably concentrate efforts on activities like those decsribed in 1. & 2. early in the course (and these types of activities may also link to retention?)and progress to 3. & 4. later.
This might help with our aims of trying to help the learner become more self-regulating over time, allowing us to withdraw and pass more complex learning activities over to the students over time.
David, what do you think? I may be way off with this and if so, I know you'll say ; )
It just seems to me that this framework has loads of potential and could be developed even further.
Also, just a small point. I have problems with the term 'engagement'. In this diagram it seems like almost a negative, but when we use it more generally it's one of the things we want to do - engage the students. That's why it niggles me!
Debra
|
|
|
|
|
Debra Macfarlane
Posts: 21
|
| 30/05/2007 10:51 |
|
Hi Cynthia,
It's a really good point that you make. The students can really like an exercise but not link it to the outcome the teacher has anticipated. Context and explicitiness are really important.
Debra |
|
|
|
|
David Nicol
Posts: 18
|
| 30/05/2007 11:33 |
|
Hi Debra,
Thanks for your comment. I only put the principles in the diagram to illustrate something about what is important in the first year. In an earlier version of the paper, I had made the point that the principles could be put in any quadrant, as subsequently elaborated by Steve. However, I was writing about this topic in relation to the first year experience so it was appropriate to focus on what the teacher did in creating structure, in giving early feedback and in organising group activities (as these are important as suggested by Cynthia above). Also in that paper I was writing as part of a Scottish Quality Enhancement Theme on first year experience where they use the terms engagement and empowerment. I don't think engagement is negative at all. There is a relationship with empowerment: engagement is a necessary but not sufficient condition for empowerment.
|
|
|
|
|
David Nicol
Posts: 18
|
| 30/05/2007 11:52 |
|
| Hi again Debra, I see I didn't fully answer your question when I re-read it. I think that both you and Steve are right that the principles might be better kept out of the diagram. We all agree that it is the dimensions in the diagram that are important and that how the principles are operationalised in practice changes their locus. Indeed that was the original argument in the seven principles paper where each principle could be implemented in ways that could be more or less empowering. The additonal principles beyond the seven about (2) structuring learning, (7) giving choice in assessment content and processes, (8) involving students in decision making about assessment and about (9) and developing learning communities - are the development. They take empowerment to a higher level. A good discussion point here might be: how do these principles relate to the set David Boud proposed? Where is the issue of measurement and its backwash effect in all of this? |
|
|
|
|
Gillian Palmer
Posts: 4
|
| 30/05/2007 17:58 |
|
| David: I really like where this appears to be heading but have a question. As trad. campus students (18+) become, if not less common, certainly more diluted by older, part-time students and lifelong learning becomes genuinely lifelong, how do you see the balance in the model flexing to fit all? Personally, I steer as far from involving employers (possibly part of 'social') as it is possible to do - employers cannot afford it and students do not necessarily want their involvement at that level - but there are national Framework efforts in several countries. |
|
|
|
|
Vanessa Scholes
Posts: 3
|
| 31/05/2007 12:12 |
|
| Thanks very much for the discussion on this so far. I think the stated Ten Principles of Good Assessment and Feedback Practice express highly desirable (and / or necessary) features of good assessment and feedback. I found them to be very clearly formulated, and particularly appreciate the guidance questions attached to each question to provoke reflection on the extent to which one's practices fulfil these principles. In fact, to be honest, I thought Table 1 outlining these principles and accompanying questions to be so clear and useful that I was puzzled by the inclusion of the diagram on dimensions of implementation - I was mystified as to the purpose of this. As a teacher, I could immediately see how consideration of the principles using the questions will help teaching practice in terms of assessment, but I couldn't quite see the use of expressing how to do this in a diagram of this sort. The comments in this forum have helped my understanding on this, particularly Steve's post showing this in practice. I have just one question concerning the comprehensiveness of the 10 principles, although I fear it will seem rather naive: ought there not be more focus on what counts as good practice in summative assessment? I have in mind (for course leaders / lecturers) something along the lines of trying one's best to assess accurately and fairly (have I taken account of subjective influences such as mood, energy levels on my judgement of this assignment?; what have I done to ensure consistency across markers for an assignment? etc. This probably doesn't apply to t/f or multiple-choice asssessment). While I realise it doesn't have the formative, motivational or metacognitive learning focus of the stated principles (which I readily agree are most important in terms of learning), I just thought that, given that providing summative assessment is part of assessment practice, perhaps there should be a specific principle relating to this in a list of principles of good assessment and feedback practice. Apologies if I have misunderstood something (my academic background is not in education, so this is rather new for me). Kind Regards, Vanessa. |
|
|
|
|
Steve Draper
Posts: 25
|
| 31/05/2007 14:33 |
|
Vanessa
Thanks for the question which touches on debates David and I are having.
If you look at my conference paper, accompanying David Nicol's keynote (http://ewds.strath.ac.uk/Portals/2/CSL/keynotes/david nicol/A_momentary_review_of_assessment_principles.pdf) you'll see that I agree with you that some of the missing areas from the 10 or 11 principles are the whole area of accreditation (summative assessment) and of alignment (examining students on what they were supposed to learn, not on something else). However in any attempt to formulate principles we face difficult tensions such as being comprehensive, vs. keeping the issues down to a number few enough that a normal audience can take them in in one go.
As a theoretician I want to be comprehensive.
As someone wanting to influence practice, I want to highlight what will make the biggest difference relative to current practice.
Two or three years ago, and perhaps still today, it seemed clear that HE already worried about fairness etc., but was quite backward compared to schools and educational theory on focussing of formative feedback and more generally, making assessment support learning rather than just accreditation. So one approach to principles is to 'take for granted" issues many, including obviously you, are going to worry about anyway, and focus on those that might make a bigger difference.
I though David Boud's comments in and around his keynote, were particularly clear on the need to maintain (summative asssessment) issues addressed in the past while pushing relentlessly on improving practice on formative assessment.
SteveD
|
|
|
|
|
|
