Derek Rowntree
Posts: 35
 |
| 24/05/2007 18:10 |
|
How would you feel if you were asked to take sole responsibility for teaching more than 500 students in a year? And let's suppose you were given only two 1-hour lecture sessions a week in which to meet them and no assessment opportunities apart from two multiple-choice tests and a final examination? It might well strike you, as it clearly did Jim Baxter, that you'd have no way of giving your students personal feedback on how they were coping with the ideas in the course. And, without such feedback, you might well feel that, although you would be telling people things, you could hardly be said to be teaching. What would you do?
Jim Baxter's approach, outlined in his case study, was to encourage the students themselves to give one another the formative feedback that is necessary for learning to take place. As they went through the course, students were expected to explore the course material collaboratively online, comment constructively on one another's ideas and draft essays, and thereby contribute to one another's development. Sad to say, although the majority of students who finished the course reported learning from other students' contributions, most were reluctant to offer one another constructive criticism. So the hoped-for individual feedback remained in short supply.
Let's be grateful to Jim for giving us a case study that, unlike so many, is not the story of a fait accompli, where everything has gone according to plan. Instead, he has shared with us a work in progress, successful in many aspects -- not least in using assessment to help student focus on key course content -- but with plenty of room for us to exercise our imaginations, not only about the constraints of his situation and how he and his students might overcome problems and build on what has been achieved but also about the implications for teaching and assessment with our own students.
To prime the pump for discussion, I have added below a few questions that occur to me. But it is your conference and I am sure you will have many more urgent issues of your own to raise. I look forward to meeting you online and I wish you some enjoyable and useful discussions.
Derek Rowntree
• Do you have experience of teaching (or studying in) such large classes?
• What do you think about Jim Baxter's approach to working with his students?
• Is there any more detail you would like to ask him for?
• Are there any other relevant approaches you can mention, e.g. ways of assessing large cohort of students without over-burdening staff?
• What can we learn for our own teaching from Jim Baxter's experience?
• Have you any suggestions as to how Jim might modify his course for 2007-8?
Message updated, new thread started 29/05/07 |
|
|
|
|
John Green
Posts: 2
|
| 25/05/2007 02:57 |
|
| I have run similarly large online supported open learning classes at The Open Polytechnic of New Zealand. I found that rather than ask the student to read and comment on another student's work it was more effective to ask the student to make a comparison between his own work and one or more student's work and reflect privately upon any differences. Students were invited to post positive comments highlighting any similarities and where there were differences comment on whether these were additional items missing from their own work or if errors provide a page reference. The reverse"provide feedback to another student" did not work as students felt themselves inadequate to assess another's work. By commenting on their own work the students felt more inclined to read others work and make comments. I felt that making comments was not as important as making the comparison. A lot more commparison took place than comments when stats were examined. Students were placed in regional study team with personal discussion topics which allowed students to set up study buddy relationships and subscribe to an individual's posts which strengthened the bond between the participants. It effectively created a mass blog. |
|
|
|
|
Jim Baxter
Posts: 17
|
| 25/05/2007 10:57 |
|
| Thanks for that John. I like the sound of that a lot. Just the sort of adjustment that the Strathclyde scheme could benefit from. |
|
|
|
|
David Nicol
Posts: 18
|
| 25/05/2007 11:49 |
|
| I note that Jim Baxter has reported elsewhere that the students in his first year class produced work he had not seen before from first years - in fact at a level matching that of second year and even third year students. This would seem to imply that the small group interactive process did something to enhance the learning. If it was not the feedback that students got from making implicit comparisons with other students' work, what created this benefit? It would seem that the students are already doing what John suggests or maybe not? Would John's scheme not just make this more explicit? |
|
|
|
|
Steve Draper
Posts: 25
|
| 25/05/2007 14:48 |
|
I think that's an insight: that the chief underlying process is a student comparing their work with another's. When I've run reciprocal critiquing exercises (in third year), they all report that the biggest interest and benefit is seeing other students' work, and that it is the differences, the variety that makes the impression.
Still, another step on the road towards expressing as well as silently making judgements is if/when I require students to write a preface to their own work saying what they would like feedback on. I find students will usually answer these explicit questions. |
|
|
|
|
James Derounian
Posts: 6
|
| 25/05/2007 15:22 |
|
Following on from Steve's comment, about students usefully comparing work with eachother (not a plagiarists charter you understand!), my current research in to staff-student dissertation relationships is showing how important student-student support is for dissertation survival/progression! Sharing insights, short-cuts, handy tips etc.
James
University of Gloucestershire, UK |
|
|
|
|
David Nicol
Posts: 18
|
| 25/05/2007 21:46 |
|
| Another point in relation to Derek's point 5 is that students do seem to do far more work than the marks awarded for Jim's task or what you would expect under in normal marking circumstances. They are awarded 10% just for participating not for the quality of their participation. Given that they participate at this level (beyond prior teacher expectations) they must be finding the task intrinsically motivating. To me this contradicts all the comments I receive at talks where teachers say students are driven only by marks. Now the real question is how have we been holding students back in the past from this kind of motivation and learning? And how can we build this into other teaching scenarios? |
|
|
|
|
Jim Baxter
Posts: 17
|
| 25/05/2007 22:39 |
|
| Just a slight point of clarification regarding what David says: my students got no marks for participating in their online work this year. They were, however, told that they risked disqualification from the class if they didn't participate. |
|
|
|
|
Derek Rowntree
Posts: 35
|
| 27/05/2007 13:22 |
|
| Reading David's point about Jim's students doing "far more work" than would normally be expected, reminds me that I'm not clear about what proportion of the students' weekly study time might have been taken up by this course. I assume they were taking other courses as well? Perhaps Jim can give us an idea of how many hours a week he would have exected them to spend on psychology and whether they did tend to exceed his expectations in that respect? |
|
|
|
|
Jim Baxter
Posts: 17
|
| 27/05/2007 15:25 |
|
| All students took four other subjects besides psychology. How much time they spent on the online tasks I can't say, but the quality of their work, for the most part, did indeed exceed my expectations. A recent online straw poll, conducted after they sat their final exam, suggests that many found the online programme beneficial in preparing for a conventional, unseen, exam paper. |
|
|
|
|
David Nicol
Posts: 18
|
| 27/05/2007 15:44 |
|
| As I understand it Jim, you asked students to do 12 tasks over the year - 6 light and 6 deeper with the output of the second set usually a group essay of around 800 words. If you got this kind of response back then I would assume that this was more than normal or at least more than before you changed this course. Given that students only had to participate to remain in the course this kind of output would seem rather good. So one question is, how many groups did provide these outputs from your 86 groups and did you monitor these outputs? |
|
|
|
|
Jim Baxter
Posts: 17
|
| 27/05/2007 17:31 |
|
I'd say the great majority of groups produced a higher standard of work, higher in terms of the depth and breadth of reading, than I'd seen before in their exam answers. Of course, they were not writing their online answers under exam conditions, so that comparison is questionable.
That said, although we have not yet collated their final marks, several markers have commented to me that the overall standard of exam scripts marked so far appears to be higher than it was last year. Studetns are quting, paragraph by paragraph, more advanced concepts than their counterparts did in previous years. That's just an impression, at the moment.
What's not questionable, and this is for me the principal positive outcome of the scheme, is that that most of the class started reading about psychology from teaching week one and maintained this habit throughout the year, albeit under some pressure since doing so was now a course requirement and their performance could be monitored. As to the actual monitoring, apart from two early projects, I did this myself, sampling different groups from task to task. This will change next year, Faculty apporval permitting, when it is planned to scrap the existing face-to-face tutorial scheme and devote tutor time to monitoring the performance of each group on each task. |
|
|
|
|
Derek Rowntree
Posts: 35
|
| 28/05/2007 12:14 |
|
Jim has raised an important distinction here -- between assessment and monitoring. It sounds possible to me that what made the difference to the level of commitment is not so much that someone (whether tutor or peers) was able to critique and give feedback on each individual's written work but that someone was able to observe the extent to which each individual participated in the activity.
As far as time-on-task is concerned, I recall some work of Graham Gibbs a few years ago suggesting that 40 hours a week might be what we could normally expect students to devote to their studies (including all class time). Following five courses in one year would thereby suggest a notional eight hours a week for Jim's
course. It would be interesting to know how many hours a week they did spend on it. It would be interesting also to know whether the tutors of the students' other four courses noticed any knock-on effects-- whether for example Jim's students were so hyped up by his course that they put unusually high levels of effort into the other courses also (or were so involved with it that they had less energy to spare for the others). I remember students of one online collaborative course commenting to the effect that they'd enjoyed the experience but wouldn't be able to cope if all their courses were done in that fashion! |
|
|
|
|
Vivienne Armati
Posts: 1
|
| 28/05/2007 13:33 |
|
| I don't know if my reaction to all your discussions fits in here, but I am wondering about my own university undergraduate degree undertaken over 10 years ago. I wasn't one of 500 but I didn't know all my fellow students. I didn't seek feedback other than to recognize the effort I had put into an assignment (good - 85% or a few words at the end of the essay and maybe 70%) I wouldn't say that I felt cheated of any special kinds of learning, but rather, the value of that experience has stayed with me and I know it has enhanced my life prospects and my attitude to the world I live in. I didn't ever consider measuring what I learned as one does when doing a pilot's course or a medical degree, or becoming a teacher, as I was at university. I was a primary school teacher and I recognize the value of benchmarks and levels of achievement as foundational to enduring learning skills, but I always considered that life experience was what provided the school leaver with the ability to succeed. My English Literature degree, as enjoyable as it was, provided me with the foundation to go on and do my postgraduate degree in Library and Information Science. It has more than served the purpose that I expected. I wasn't wanting or expecting the arduous tasks of assessing and comparing and contrasting my flawed assignments with anyone else's. I enjoyed the discussions in tutorials, as students would enjoy these online discussions today, but there was no compunction to engage. One could or one couldn't. Are students at university today expecting something more quantifiable? Or should we perhaps relax a bit and give them a rich and challenging open environment to let their minds open up. Therein lies the benefit of university education from my point of view, not the stacking up of summative and formative assessment. I know this is extreme and more than a little fantastic, but have we lost sight of quality of 'experience' and living in our need for accountability in a capitalist economic rationalization mode of education? |
|
|
|
|
Derek Rowntree
Posts: 35
|
| 28/05/2007 14:26 |
|
| You touch on an important issue here, Vivienne. Many students will feel like you did about how they want their university experience to be and what they want out of it. But many will feel quite differently. Unfortunately, their teachers, and indeed the system, may be more sympathetic to some learning styles and purposes than to others. How far can we expect an assessment system to respond to the variety of needs, goals and preferences among a large group of students? It's quite a challenge. |
|
|
|
|
Jim Baxter
Posts: 17
|
| 28/05/2007 15:18 |
|
| One of the tables in the paper suggests that some students reported that they did indeed devote time to their psychology work at the expense of their other subjects. This could certainly be a problem. The questionnaires were anonymous, however, so I have no way of knowing how many of the students reporting this wished to continue with psychology. My hope in any case is that the change in the Faculty's requirements, reducing the required first-year subject choices from five to four, might help, especially if other subject co-ordinators decide they might wish to operate a similar system. |
|
|
|
|
Steve Draper
Posts: 25
|
| 28/05/2007 19:45 |
|
Quote">Posted By James Derounian on 25/05/2007 15:22
Following on from Steve's comment, about students usefully comparing work with each other (not a plagiarists charter you understand!), my current research in to staff-student dissertation relationships is showing how important student-student support is for dissertation survival/progression! Sharing insights, short-cuts, handy tips etc.
James
University of Gloucestershire, UK
Another point related to this post of James': I've recently grasped from my students that an important source of information for them is reading the personal feedback their friends get in conjunction with their own: the combination tells them a lot more than the supposedly personal one alone.
So while I'm sure emotional support is important to postgrads, simply comparing one's learning with a peer, and the information they get, contains information regardless of "socio emotive" aspects.
SteveD
|
|
|
|
|
Steve Draper
Posts: 25
|
| 28/05/2007 19:46 |
|
This replies to Vivienne’s long post.
I think there are perhaps 3 things here.
Firstly one of Derek’s own principles is that we should assess what students have actually learned, not what we want to impose on them. That is part of the challenge you bring up.
Second, you are not alone in having these liberal ideas of education in the digital age. So do Duguid and Seely Brown. In fact their way of putting it, is that most of the value of a degree as seen by employers is the name/prestige of the university. It is clear that employers like you think a large part of the value delivered is not expressed or even touched on in the official learning aims and objectives.
Thirdly, the distinction between assessed and monitored others mentioned: that indeed is I think important in this case study; and gets at my sense of the real issues behind “time on task” and Gibbs’ principles: that students, particularly first years, need to have some feedback that their (learning) process and work is appropriate: not feedback on its content and achievement at a given week. You too express this. And this is tacitly consistent with the unspoken notion that attendance, or participation or the experience is one of the main things that matters. And that is a very different kind of assessment to an exam.
SteveD
|
|
|
|
|
Derek Rowntree
Posts: 35
|
| 28/05/2007 23:24 |
|
| I agree with Steve that "students, particularly first years, need to have some feedback that their (learning) process and work is appropriate". But I believe it is also encouraging to know that someone else (whether tutor or peer) takes your ideas (or product) seriously enough to comment on them. |
|
|
|
|
Derek Rowntree
Posts: 35
|
| 29/05/2007 00:46 |
|
| I suspect we are in danger of drifting too far away from Jim's case study here -- possibly thanks to some of the questions I rashly posed in my original message; so I have now replaced them. I have also opened a new topic in the Effective Feedback to 550 students forum, inviting discussion on what needs to be special about assessment in students' first year in higher education (regardless of how large the classes are). Vivienne Armati's earlier message would have made a useful contribution to it if it had been open in time. |
|
|
|
|
|
