Rachel Harris
Posts: 29
 |
| 24/05/2007 11:26 |
|
Welcome to the discussion board for Prof Mantz Yorke's Keynote on 'Assessment in the first year of higher education: old principles in new wrapping?'. Prof Mantz Yorke has provided his keynote as a full paper. This will be available for viewing by Friday 25th May.
Mantz will be joining us for the closing keynote chat session on Thursday afternoon. |
|
|
|
|
Fran Everingham
Posts: 14
|
| 31/05/2007 15:32 |
|
Hi Mantz, Each paragraph of your paper had me ready to bounce back an email. Perhaps I’ll start with ‘wicked problems’ –unbounded and no absolutely correct solution. My students are health professionals completing a masters in education by distance from urban, rural regional and remote Australia and internationally. They apply their learning to workplace projects is diverse cultural and health settings. As a marker I’m not just comparing apples to oranges – at least they are all fruit!
Marking must use high levels of inference, which can also be time consuming. The pay-off is the fascinating immersion into worlds I would otherwise not visit, for example, high fidelity simulation training for surgery teams, paramedic training fort hostage and siege teams, and health worker training for Thai Hills tribesman.
As these are work projects feedback is often used beyond the boundaries of the assignment for personal, professional or political leverage, so the feedback needs to be, well, careful.
Feedback is given via electronic marking templates that increase marker efficiency and enable effective student feedback for complex multi-task assessments. Each assignment has a purpose designed 3,000 word master template using criterion level grade descriptors extended as feedback. The student ends up with about 600 words of feedback. As with action and judgement in the real world, assignment responses are not always black and white, and marker judgement may have to draw on description from above and below the grade band, and not be locked into black and white choices decisions.
|
|
|
|
|
Mantz Yorke
Posts: 10
|
| 31/05/2007 16:49 |
|
Hi, Fran - this sounds exciting for both tutor and course member. Is it written up anywhere? It would make a very interesting case study of how the (perhaps) unpredictable can be brought within a formal summative assessment scheme.
I suspect from what you write that there is some fuzziness about the summative assessment because achievements don't fit neatly into bands of multiple criteria. I would personally be interested in actual examples of the trade-offs that I suspect you have to make across the assessment bands. |
|
|
|
|
Fran Everingham
Posts: 14
|
| 31/05/2007 17:06 |
|
| Hi Mantz, I have given a couple of conference papers about this, and can send you a .ppt which probably gives the essence. The grade descriptors are quite clear, the classic ladder, from Fail, Pass Credit, Distinction, up to High Distinction. The devil is in the detail, as I think Biggs observed, many responses sit inconclusively on the boundary between grades, thus so does the feedback component. So as I cut and paste individual feedback I need to borrow up and down between grades. I don't consider this a design fault of the tool/template so much as its flexability to allow judgement. Sadly I suspect there are colleagues who would insist on an either or position. Whereas I feel I'm doing justice to the students efforts to nuance the feedback with more accuracy at the micro level. |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 31/05/2007 20:14 |
|
Hi, Fran: I'd welcome an example or two (additional to the .ppt if not in it) to show this borrowing, which is what I'd expect you to have had to do. The point I'd make, as you do, is that it's rarely cut-and-dried and that professional judgement is needed to award the grade. The follow-on point is that the students have to accept that there is an element of 'best judgement' going on, and that you as assessor are making these trade-offs.
I'll make a bit of a leap to say that, whether one works in grade-points or percentages, the actual numbers are too fuzzy to do what many do, which is to treat them as if they were measurements rather than as signals of broad achievement and apply all sorts of statistical manipulations to get an 'accurate' overall outcome. |
|
|
|
|
Janet Taylor
Posts: 10
|
| 31/05/2007 23:51 |
|
| Mantz...I wondered if we could continue the discussion related to formative assessment. I understand its merits and function, but have had great difficulty getting first year stduents to participate in it, especially if studying at a distance. They are very time poor and strategic in many ways and are driven to participate by the summative assessment steps. |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 01/06/2007 09:38 |
|
| Hello, Janet. You put your finger on a challenge (aka problem, in downbeat-speak) that I guess is pretty widespread. Would it be possible to run the stepped summative assessments in such a way that each successive step requires a response to feedback from the previous one (as well as a response to the new task)? Not knowing the curriculum content, this can only be a top-of-the-head suggestion. |
|
|
|
|
Janet Taylor
Posts: 10
|
| 02/06/2007 07:47 |
|
| Hi Mantz...wasn't sure if you were still going to be here..we are are a bit out of sinc with the discussion here in Australia. Your suggestion is exactly what I have done...and it worked reasonably well with maths, and I know it has worked in a communication course of a colleague as well...but it is not always easy. I guess I was interested in how summative can a formative assessment be while in the first year |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 02/06/2007 10:01 |
|
| Hi Janet: I guess we're half a day out each time we log in. Following on from the previous posts, I wonder how important the summative (grade, I'm assuming) actually is. Maybe it's sufficient to have shown a passing level of performance, even if ideally you'd like students to exceed that. But if the 'carry-over' work on feedback reaches an acceptable standard each time it's done, I guess that the summative requirements may well have been more or less met on a rolling basis. In other words, the summative bit gets done progressively, and you can give appropriate credit (not talking formal credit-points here) for achievements as they happen. And if it doesn't reach the standard, then you as teacher know where things are going wrong and may be able to do something to sort it out. So back to your question at the end: the formative/summative boundary is rather blurred - but does this really matter? |
|
|
|
|
Janet Taylor
Posts: 10
|
| 02/06/2007 10:36 |
|
| No not to me...but I was getting the flavour from other discussions that in first year the move was away from summative to focus on strictly formative...in my situation I have difficulty seeing how that would work. It's been good talking with you again, especially seeing that the conference is over. best wishes |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 02/06/2007 12:13 |
|
| There has certainly been a shift in thst direction in FY (which I welcome), but in general not to ditch summative completely. Students have to satisfy progression conditions, which implies some degree of summative. It's more a matter of shifting the balance, rather than a black & white either/or. The challenge, as was evident from your earlier postings, is how to be creative about the balance or blend - seems to me you were well on with dealing with this! |
|
|
|
|
Gloria Dunlop
Posts: 1
|
| 03/06/2007 16:02 |
|
I am a great supporter of formative assessment and believe it should feature more within higher education study as this is one way to give learners control of their learning development but it seems to me that there would really need to be radical changes in the way academics think about the purpose of assessment before learners will behave differently towards formative. As long as summative assessment is held as the final marker, summative will be the main focus for learners with formative being less important. I have had some very interesting results with electronic formative assessment which deomstrates learners willingness to voluntarily engage in formative assessment for their own interest and even more so when the learners feels that the formative process can contribute to the summative process. Seems to me that the changes need to be with the academics and not the learners.
Gloria |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 03/06/2007 22:49 |
|
| Hello Gloria: I think that you are right in pointing to the need for academics to rethink assessment, but I'd push a bit further and suggest that there's a need for more of a learning-oriented climate in HE (as opposed to the rather instrumental one which, esp. for students from school, implisitly encourages 'getting the grade'). But this requires a rather different perspective on summative assessment. I have from time to time suggested asking if the model might more appropriately be one in which students make claims that they have fulfilled the expected learning outcomes for their awards - thus allowing different 'profiles' of achievement to be rewarded with a pass. But notice that this undercuts degree classifications and GPAs, and you'll not be surprised that this line hasn't (yet) met with success. |
|
|
|
|
Fran Everingham
Posts: 14
|
| 04/06/2007 04:50 |
|
Hi Mantz,
I looked for an email address so I could send the powerpoint presentation referred to in 'MantzYorke on 31/05/2007 20:14:32' - but it remains illusive.
Could you please post the address here or send to my email address at F.Everingham@usyd.edu.au.
Regards,
Fran |
|
|
|
|
Mantz Yorke
Posts: 10
|
| 04/06/2007 22:29 |
|
| I'll put it here in case anyone else needs it; mantzyorke@mantzyorke.plus.com . |
|
|
|
|
