REAP - Reaching new heights ...
Project Documentation:

Evalution Strategy

Evaluation is critical to the implementation of this project. Evidence of cost savings and/or learning quality improvements are essential to gain and sustain commitment within the three participating institutions and to ensure the uptake of these new educational models across the HE sector. Evaluation is being carried out in three areas to show:

  • The benefits of the re-engineered assessment practices on learning quality and learning outcomes.
  • The costs and benefits of assessment changes in relation to workload and resources across academic and service departments
  • The institutional changes needed to support new models of assessment practice supported by technology

Learning quality and outcomes

Baseline data has been collected documenting assessment practices using a variety of instruments. Instruments used include some new tools developed through the Formative Assessment in Science Teaching Project funded by HEFCE such as the teachers’ assessment review checklist, the student assessment experience questionnaire, the distribution of student effort instrument (Brown, Gibbs, Glover, 2003). Other instruments are being developed to evidence the balance across self, peer and teacher assessment and the relative balance of formative assessments (feedback) and summative assessments (marking) in courses. Some specific measurable benefits sought include:

  • A more even distribution of effort by students over their course of study (in and outside class)
  • Enhanced participation by students in assessment tasks
  • Increased use of formative assessments with opportunities for feedback
  • Increased use of self and peer assessment
  • Proof of better quality feedback provided by teachers to students (i.e. that helps them self-correct rather than tells them where they are wrong)
  • Evidence that feedback is used by students
  • Evidence of contingent teaching – teaching shaped by learning performance
  • Evidence of integration of formative assessment within other teaching and learning processes
  • Improved retention (through early identification of students in difficulty)
  • Improved performance in assignments and examinations
  • Student perceptions of the effects of changes in assessment practices and support systems.

Workload and resource issues

The PREDICT unit and Centre for Academic Practice at Strathclyde have developed methodologies to examine whole institutional changes brought about by e-learning developments and to carry out cost-benefit analyses in relation to these changes. This project will also draw on cost-quality models devised by the National Centre for Academic Transformation, Rensselaer Polytechnic in the US.

As well as applying these cost benefit processes, workload models will be developed and implemented to help identify the time currently spent by departmental staff and support staff on teaching and assessment activities, both formative (to provide feedback) and summative (to mark). This will help produce initial baseline data in order to demonstrate changes in time spent on assessment tasks during the transformational period and the relative balance of time spent on self, peer and teacher assessments. Some specific measurable benefits might include:

  • Reduced time spent by staff on marking of students’ work
  • Increased use of online technologies to provide formative assessments
  • Changes in balance of time spent on developing online materials versus time spent marking
  • Capital (technology) for labour substitution (faculty teaching):
  • Cost savings due to changes in the balance of academic and support staff input to assessment (e.g. Graduate Teaching Assistants)
  • Better understanding of the indirect costs (network infrastructure, IT support) associated with different assessment models.
  • The costs of transition to new assessment models

Institutional changes in support of transformation

An analysis will be carried out to ascertain changes (actual and desired) required to support changed assessment and institutional processes. This will include consideration of the cultural, organisational, pedagogical and technical issues that need to be addressed at departmental and institutional level. Areas of focus would include the following:

  • Policy, culture and support
  • Strategy and Pedagogy
  • Administrative and Technical

 

Last modified:
"-->