ARQM: Astonishingly Reckless Quality Monitoring

Our research ‘quality’ is being assessed by the ARQM in a way that is astonishingly reckless. The ARQM requires every ‘research active’ member of staff to submit their ‘best’ four publications from the last four years. These are now being rated and assigned a score (supposedly equivalent to the REF star ratings) and each member of staff will receive an ‘average’ between 0 and 4.

There are many problems with the ARQM, including:

  • Even the REF panels do not provide marks for individuals because they acknowledge that at the individual level scores are not reliable. It is only the aggregate score in which they even claim to have confidence. The ARQM therefore claims a level of accuracy that the (much criticised [1, 2, 3], but nonetheless much more rigorous) REF process didn’t claim.
  • During REF two experts, usually from a relevant disciplinary sub-field, review your publications. For ARQM one person (a ‘senior member of staff’) often entirely outside of your field or unversed in your research design or methods, assesses your work. They are being asked to make multiple assessments very quickly and not being granted additional support to do this. The validity of any assessments must therefore be extremely doubtful, something that we know has been raised by many senior staff involved in the rating process.
  • A rolling four year average is used. This means that if you published three great things in 2012 you only need one additional piece since then to have a 4* ARQM rating for the next two years. If you haven’t published sufficient new 4* pieces by 2015, you may then plummet to 1* or 2*. This variability makes no sense; we don’t suddenly become, or stop being, internationally excellent. Research is a process with publication peaks and troughs because research is cyclical. Moreover, so long as we publish four strong things within a REF cycle we will contribute to the University’s research excellence. Because we know this, many of us who already have four good publications, use the later years of a REF cycle to publish things we think are socially useful, even ‘impactful’, but not perhaps very REF-able (reports, chapters, reviews of the field, textbooks). The imposition of ARQM, with its endlessly rolling cycle, does not match the REF cycle, does not allow for lulls and punishes us for being good academic citizens.

ARQM not only produces a score without validity. It is also increasingly being employed in astonishingly reckless ways across the university. To put it bluntly, your ARQM is affecting your job. Already, it is being used, without consultation with the UCU or (as far as we can tell) ratification through Senate, as a means of determining:

  • Promotions and progression;
  • The hours of teaching staff do – workload distribution;
  • The ability of staff to apply for sabbatical leave;
  • The eligibility of staff to supervise PhD students.

Even if the ARQM accurately measures research outputs over the last four years (which as we note above is extremely doubtful), there is no reason to think that staff research will improve if those who performed at a lower level over the last few years now do more teaching and have less research time. Nor will people who score highly continue to prosper if they are required to supervise all our research students (and of course there is no reason to believe that a higher ARQM score makes you a better supervisor). Surely it is in the University’s and our interests if all staff experience the conditions to produce better research?

In short, the ARQM is a worthless process that is being used to remove discretion from departments and schools under a false guise of transparency. It has no place in academic decision making.

An alternative vision for City: The next 10 years

Members of staff from across the university met on 18 March to discuss forming the basis for an alternative vision for City University London in a meeting convened by the City UCU branch. The senior management of the university had invited staff to attend workshops on what the university has achieved and to input into City’s journey over the coming decade. In preparation for these workshops staff members met to start putting together our views on what direction the university should take in the future, and the core principles it should espouse.

Do these ideas match what you would suggest for City’s alternative vision? What principles or values would you like to see the university follow? We look forward to discussing a new way forward with the senior management of the university.

As a place to work, City should

  • Make staff feel valued in an inclusive and collaborative environment
  • Have a sense of community, encouraging staff to meet and socialise
  • Be more collegiate, with a flatter structure, open and transparent communication, and more collaborative, less top-down management
  • Develop its staff with mentoring opportunities, staff events, and fora for discussion
  • Pursue a mission with substantive objectives, not aiming for a number or a percentage

As a place to study, City should

  • Reflect the community it serves
  • Restate its mission as the university for the professions, preparing students for the careers they will embark on in the future
  • Provide opportunities for students to excel
  • Invest in staff and services to add value to the education of its students, whatever their background and previous educational attainment

As an academic institution, City should

  • Recognise the value in all levels and types of research, including research for practitioners as well as four-star research in high-impact journals
  • Give equal value to teaching and research