marija cubric
Posts: 25
 |
| 29/05/2007 10:10 |
|
This is a big dilemma. Wiki work should ideally be used for formative work, but the experience so far is that students will fail to engage with any activity that does not (significantly) contribute to their final grade. Therefore, lots of "potential for research" :) on assessing wiki contributions.
During the two years of trials I have used several different assessment strategies/marking schemas, each of which has been designed to remedy certain deficiencies of the previous approach. I haven't come up with ideal schema (yet:) but merely with couple of guidelines on the following topics:
- assess weekly or at the end? measure quantity or only quality?
Weekly individual assessment would provide more detailed account of students work and enforce regular contributions, however this approach creates significantly increased marking load, especially with groups of 10 and more students. Assessing overall quality of students’ work (at the end of a semester) is a more practical task, but enforces submissions in “bursts”. Therefore, the middle-way approach was used in the last trial (sem b 2007) – a minimum of 7/10 weekly contributions is required and the quality is measured as the overall quality (rather than sum of weekly “marks”)
- assess presentation as well as content
Would encourage everyone to insist on quality of presentation - it makes it easer for everyone to read and contribute to the content that is layed-out according to some pre-set standards (metaphore: wikipedia's "this page does not comply to standards" banner)
- provide early individual feedback on wiki work
Students will find it extremely useful and it will help in orienteering them towards your goals - it does not need to be extensive - three short sentences on strenghts, points for improvement and predicted grade for the work so far
|
|
|
|
|
Lee Asher-Simpson
Posts: 9
|
| 29/05/2007 12:59 |
|
I have been reading through the chat for the Wiki session...... and thinking about the chat in the Peer marking/feedback session...... It occurred to me that a Wiki is the perfect vehicle for developing a rubric for peer marking of individual contributions of each member in the group. A very authentic situation for students and their teacher. The rubric could be carefully designed so that one individual student could not personalize the marking situation and be able to gain some leverage over one or several of the other group members, potentially using this power for their own ends.
Part of the justification for exploring the potential and usefulness of Wikis comes from the fact that collaborative teamwork exists in almost every facet of the real world. However in that real world the evaluation of the team work is measured only by the result or outcome. All the individuals in a team or group are given equal credit no matter what their individual contributions were. So it should be with a Wiki, this result is the item for the teacher to grade/assess.
Individual contributions to the Wiki can only be validly assessed by the group members themselves, they are the ones who should be tasked with the assessment of the contribution of each of the individuals and they are perfectly situated to balance quantity of contribution against quality of contribution.
What do you think?
Lee |
|
|
|
|
Mark Russell
Posts: 8
|
| 29/05/2007 13:01 |
|
Hi Marija,
Some quick thoughts from me.
I wonder if it might be useful to ask a different question? [b]To assess [i]the[/i] learning?[/b] or [b]create assessment to [i]support[/i] the learning [/b]. I stayed away from formative vs. summative since I think the boundaries are a little blurred.
If you move more to the former you can then enhance the effectiveness but lesson the impact on you. You can keep with the weekly tasks (you will of course know my fellings on this already - spreads the load, time-on-task, reduces the issue of cue-seeking students, assessment backwash is all positive etc) and you might get the students to more for you too.
What opportunties does this provide for meaningful and well thought out peer assessment?
why not invite them to produce one piece of work that describes their learning journey - with specific examples and also showing how their work has benefitted from their involvement in the peer assessment process - both as givers and receivers of feedback.
Enough for now - coffee getting cold.
Mark
|
|
|
|
|
Mark Russell
Posts: 8
|
| 29/05/2007 13:04 |
|
Marija,
I've also been doing some work on individualising students scores from group activity. I use a holistic non-negotiated approach. Interestlignly we have compared the results with a negotiated approach - no suprises in the outcomes but its great to have the data to show people. Let me know if this might be useful. |
|
|
|
|
marija cubric
Posts: 25
|
| 30/05/2007 00:32 |
|
Lee, it seems that I read your comments in the wrong order? Never mind, I might repeat the same things here but it does not matter.
I like the comparison with the work scenario, but in my experience, the work of individuals within a team (in an organization) is still assessed by the authority (e.g. manager) who might get informed by peers when making the decision.
I like the idea of splitting the mark into group part (everyone gets credit for a well designed wiki) and understand your point about peer assessment of individual contributions, but not sure how would that work in a real situation i.e. how to implement it, in terms of well & rigorously designed rubric. Another point is that students generally don't like peer assessment (again, based only on my limited experience). What is your experience with the peer assessment?
But, it is definitely an idea worth trying - are you using wikis in your work? If so, ill you be in a position to try it?
|
|
|
|
|
marija cubric
Posts: 25
|
| 30/05/2007 00:34 |
|
| Probably useful, but not sure about the terminology - can you explain? |
|
|
|
|
marija cubric
Posts: 25
|
| 30/05/2007 00:47 |
|
A bit confused - Which one of the two types of assessment (from the first paragraph) is the one that leads to a grade in the student's transcript? Presumably the first one?
In my case, assessment has been is created to engage students with the weekly task, so I guess using your definitions (from the first paragraph) it would be the second one?
Not sure why would the move towards the "assessing the learning" be easer on me (i.e. tutor)?
I am probably misunderstanding some of your points - can't think very clearly - it is late, and I just finihed marking 50 Y1 (non-wiki) exams :)
|
|
|
|
|
Lee Asher-Simpson
Posts: 9
|
| 30/05/2007 01:08 |
|
I agree that students may not like peer marking but I still think if the teacher had already given an overall mark for the task it would make it much easier and more reliable. I think part of the dislike is concerned with thinking that the teacher is the most knowledgeable person and so should be doing all the marking. But in regards to the level of an individuals' contribution to the group work, I feel that actually the students are the experts and are likely to be much more informed than the teachers about the relative merits or otherwise of contributions.
I imagine that if a group has five members, each of the members should assess the other four members individual contributions, and that the individuals' contribution score should be constructed from an average or aggregate of the scoring from the other members. This hopefully corrects for ill will or pique on the part of one or more group member/s. I think there should also be an appeal/veto process available for the student to appeal to the teacher if they think there mark is not justified.
My experience has only been with formative peer marking. I found all the usual problems with peer marking until I spent tutorial time training students how to mark use a marking rubric.In class time we practiced with all the students individually marking the same anonymous piece of work, they were then required to mark using the rubric. When they had finished I gave a demonstration using OHP's [overhead transparencies] of exactly the same piece of work and demonstrated how I would mark it, talking about my criteria and the decisions I made, then answered questions. The students usually found that I marked much more generously than they had and were reassured by this. They learned a LOT about how I marked and what I was needing from them.I have not extended this to a summative situation. The students anecdotally reported that they learned a lot about the subject and about the quality of their own work through this process. |
|
|
|
|
Mark Russell
Posts: 8
|
| 30/05/2007 09:34 |
|
In Peer Assessment (in this context I am talking about distribution of student grades - not marking per se) you have a few choices.
These choices are organised around what they assess and how they distribute the marks.
What they assess.
You could set out the categories that might include questions about product and process. You could also get them to say what categories they most value and hence wish to use as a group.
*Contributed to group meetings
*Came prepared ...
etc
and then you get them to rate each other against the list. This is referred to as catogory-based assessment.
An alternative would be just to ask them to rate their peers. And let them think about what is important. This is 'challenging' and so we also ask for free text justifactions. Don't worry we have technology to do all this for us. This is referred to as holistic assessment.
How do they spread the marks?
You could equally distribute the marks amoungst the group. I don't like that method at all. Or you could get them to split the marks amoungst themself. So what choices are there?
Negotiated - where they all sit around and agree as a group who deserves what. My experience is that the spread is small and the students are not given a chance to give a true picture of what went on. Quiter students are surpressed. But group issues are never faced head on.
Or non-negotiated - where they all score each other without discussing it. There are pros and cons with each approach. But for me I tend to use Non Negotiated and Holistic.
We have published an article on this and the merits of each approach - let me know if you are interested.
The next phase of the work is to undertake a more rigorous content analysis of the students' justifaction statements to see what they prize and what they don't. This will be shared with future groups.
Mark |
|
|
|
|
Mark Russell
Posts: 8
|
| 30/05/2007 09:43 |
|
Marija Cubric WROTE A bit confused - Which one of the two types of assessment (from the first paragraph) is the one that leads to a grade in the student's transcript? Presumably the first one?
MarkRussell WROTE.
I was trying to distinguish between assessment OF learning versus assessment OF learning.
Assessment OF learning leads to grades.
Assessment FOR learning leads to more opps for you to find out what they know, to provide adaptive tecahing, to engage in JiTT, to get them to evaulate, to get them to reflect, and simply to engage in learning. Here the stakes are less high and gives more opps for you to think about what you really want them to do.
Marija Cubric WROTE:
Not sure why would the move towards the "assessing the learning" be easer on me (i.e. tutor)?
MarkRussell WROTE:
When I was thinking about lessoning the load on you, I was thinking about using the students more to do some assessment of their peers. Or even themselves.
Increasingly I am trying to move students away from thinking about me as the knowledge holder - one truth (Perry, Ramsden etc) towards a situation where they appreciate the plurality of the situatioin and the need to develop their own meta-learning.
Mark
Mark |
|
|
|
|
marija cubric
Posts: 25
|
| 31/05/2007 15:49 |
|
Lee and Mark,
I will defintelly try some peer assessment next year and perhaps we can get togehter after the Sem A 2007/8 and re-visit some of the ideas from this forum!
Thanks for reading and commenting on the study
|
|
|
|
|
marija cubric
Posts: 25
|
| 31/05/2007 15:54 |
|
And yes, Lee, I would be very interested in seeing your template for peer assessment, and Mark - your article on non-negotiated holistic approach;
My email address is m.cubric@herts.ac.uk
|
|
|
|
|
Mark Russell
Posts: 8
|
| 31/05/2007 17:35 |
|
Sorry one of my other post should clearly have said Assessment [b]OF[/b] learning or Assessment [b]FOR[/b] learning!
I think wrote OF twice!
Mark
|
|
|
|
|
