Last Monday saw the official launch of the new system for Online Unit Evaluations and I wanted to briefly outline the work which went into developing this new approach to evaluating units.
The existing system hadn’t been updated since 2006, and there were concerns from staff and students about its usefulness, fairness and effectiveness. We wanted to take the opportunity to respond to these concerns and find ways to deliver a more reliable, transparent and effective system.
The improvements were informed by an extensive consultation process with staff and students across the institution, gathering over 200 viewpoints. This included discussion at University committees, cognitive testing of questions, meeting with key individual stakeholders and a wide ranging email consultation on initial recommendations. We also carried out a benchmarking exercise, analysing best practice across the sector and undertaking a comparative evaluation of the external software packages used to support evaluation at other institutions. Finally, a review of the 795 distinct questions previously asked across the institution enabled us to develop a clearer and more reliable question set which better complements NSS and PTES. Working with the Students’ Union also enabled us to capture the ‘student view’ on evaluations which informed all changes. This consultation process was crucial to the changes introduced.
Detailed information on the new system can be found here. Specifically we:
- Reduced the number of questions and improved their wording,
- Refined the operational system, including simplifying the administrative processes, pre-populating unit data on reports and ensuring greater compatibility with mobile devices for students when completing the surveys
- Encouraged mid-term feedback points, enabling staff to identify and (if appropriate) respond to student feedback earlier in teaching,
- Simplified the process for sharing unit evaluation feedback across the University, ensuring it is consistent and clear to staff and students,
- Anonymised all staff teaching data provided to students in the feedback process
- Identified ways to raise awareness of unconscious bias, encourage students to offer constructive feedback, and discourage inappropriate comments
- Developed an expanded, targeted publicity scheme to help raise response rates.
This review was a challenging process. Unit Evaluations have to serve multiple purposes for multiple stakeholders, and different groups and people often have very different – sometimes conflicting - perspectives and needs, which are difficult to balance. Issues such as unconscious (or conscious) bias are vital to consider in all evaluation activities but difficult to address, in an instantly effective manner. We will be continuing to work with staff and students and through business analytics to identify further improvements and solutions to these challenges, and to monitor and evaluate how well the new system is working through a review next year.
I have no doubt that thanks to an extensive consultation and review process, this is a significant improvement on the current system. It is more transparent, more consistent, and more user-friendly. There will no doubt be teething problems, but we will be reviewing the changes with an open mind and I am confident that it will contribute towards ensuring that we continue to provide excellent learning and teaching at the University.
Thank you for helping the new system work. If you have any questions or comments on unit evaluations, please contact firstname.lastname@example.org.