The results of the Research Excellence Framework, or REF, are finally out. This REF was even more eagerly awaited than usual, as Covid delayed the submission and release of the results from 2021 to 2022. Many academic and non-academic colleagues from across the School of Management worked hard to secure a positive result, with the Times Higher Education Analysis ranking the School 7th in the UK for research quality. In this piece Avi Shankar sheds light on the processes behind the result, reflecting on his role assessing research quality for the REF.
Now that the results of the REF 2021 have been released, I can talk about my role as part of the REF as a sub-panel assessor. While the REF has its critics, the purpose of this blog is to reflect on my experience of the REF process itself and to reassure colleagues of its academic integrity.
What is the REF and why is it important?
For those who don’t know, the REF is an audit process that UKRI (https://www.ukri.org/about-us/) conducts on behalf of the Government to judge the quality of research output across the University sector. While our teaching activities are funded by student fees, the research activities of universities are still funded directly or indirectly by Government. The REF audit is important because it determines the amount of money allocated to universities to support their research activities. This is called QR or quality-related funding and is worth in the region of £20 million a year for Bath.
REF divides the outputs of universities into four main panels – roughly; A for medical science, B for science, C for social science, and D for arts and humanities. Each main panel is then divided into sub-panels. The Business and Management sub-panel was part of main panel C.
The Business and Management sub-panel was overseen by a chair and two co-chairs, all of whom had experienced previous REFs, and they were a source of support and encouragement throughout. It fell to them to manage, coordinate, organise and sometimes cajole a motley crew of around 50 sub-panel assessors, output assessors and impact case study assessors, whose expertise spanned the major subject areas of the field.
While the panel membership consisted mostly of academics from across the university sector, we also drew on the expertise of key stakeholder representatives to help audit the impact case studies. I was part of the marketing subject area, and we too were appointed to represent the diverse areas and paradigms of marketing research activity.
So, what did we do?
Well, quite simply, we read a lot. I’m pretty sure that I can speak for all my colleagues on the panel when I say we learned an awful lot too. Not only about the politics and organisational processes of the REF itself, but also about the breadth, quality and importance of the research that is being produced in our field. The Business & Management sub-panel is largest sub panel within social sciences and one of, if not the, largest in the whole REF - over 100 universities submitted REF returns for the Business and Management sub-panel.
Before we read anything we participated in extensive training. This included academic unconscious bias training and rigorous calibration exercises. Here we collectively discussed outputs, or impact cases, or environment statements according to the REF criteria. Due to the size of the sub panel we were further split into three groups to ensure that during these discussions there were no conflicts of interest. Indeed, there was a lot of technology involved to ensure the REF was conducted to the highest standards of integrity, privacy, and confidentiality.
Of course, due to Covid, ten of the twelve scheduled meetings were conducted over Zoom and two were “hybrid”. While collectively we’ve all become very adept at using Zoom or Teams, a three-day Zoom meeting is not something I can recommend.
Members of the Business and Management sub-panel collectively read and re-read, discussed, deliberated, and finally graded over 19,000 research outputs, over 500 impact cases and over 100 environment statements.
This was not an easy task. For example, it meant unlearning embedded academic practices, and adopting new ones; assessing a research paper for the REF is a different proposition to reviewing a paper for an academic journal.
What the REF results mean
Research, from any university, published in any reputable academic journal, that has any combination of originality, rigour, significance, or impact is recognised by the REF process. Our School’s overall GPA means that 93% of our research was deemed to be world class or internationally excellent.
While the REF has value as a reflective exercise, it’s impossible to escape the reality that audits tend to produce numbers and numbers can then be ranked. However, as REF scores become translated into league tables, these effects should not obscure or undermine the integrity of the academic peer review process that underpinned the audit. Indeed, a highly professional, academic service ethos pervaded the entire REF process and I was proud to represent my academic field in this process. Over the next few years I will look forward to sharing my experiences of the REF with the School and more broadly the University, as we collectively prepare for the next one…
Read about the University of Bath School of Management’s REF results.
Find out more about the School of Management’s REF results, and impact case studies.
Respond
