Libraries and the REF: how do librarians contribute to research excellence?

for the valuable contributions they make to the research life cycle or research environments. In higher education institutions (HEIs), librarians face competition from other professional support services in addition to external organizations and suppliers. This article provides an analysis of submissions made by HEIs to the UK’s Research Excellence Framework (REF) in 2014 and demonstrates that the work of librarians is rarely acknowledged in substantive ways, particularly in STEM disciplines. It provides an overview of the multiple ways in which librarians contribute to research environments as well as how they can practically contribute to the REF 2021 submission process and other research assessment processes. Librarians have developed a wide range of responsive and innovative support services over the last decade and REF 2021 is demonstrated to be an opportunity for librarians to gain recognition for the important role they play in UK HEIs. Libraries and the REF: how do librarians contribute to research excellence?


Introduction
Sector surveys and commentators have asserted that the dividing lines are becoming increasingly blurred between the work done in support of research by librarians and staff in other parts of the university. 1 In addition to this, librarians are occasionally seen as being unable to articulate successfully the value of their contributions or be as influential as staff based in other departments more closely aligned to senior management. 2 Librarians also increasingly face competition from external service suppliers, and external resources, e.g. open access (OA) articles, are likely to greatly outnumber and be easier to access than those subscribed to or otherwise offered by the library. 3 Additionally, external suppliers, including commercial organisations, funders and publishers, increasingly provide open research, research assessment and discovery services, 4 challenging the role of librarians and information science professionals within an institution. An assessment of the quality of an institution's research environment is a major part of the current Research Excellence Framework (REF) assessment in the UK, run by UK Research and Innovation (UKRI), the national funding agency for investment in science and research. In a context where the role of librarians may be perceived as diminished or where librarians are seen to be unable to promote their successes effectively, the REF seems to offer a significant opportunity for librarians to be proactive in articulating the value they provide to an institution.
Initially, a national research assessment was established in the UK to determine the allocation of funding to universities during a period of declining government budgets available for higher education under Margaret Thatcher's government in the 1980s. 5 Throughout the 1990s and 2000s the national Research Assessment Exercise (RAE) expanded in size, complexity and scope, and began to include universities that were known as polytechnics prior to the 1992 Further and Higher Education Insights - 33,2020  'the REF seems to offer a significant opportunity for librarians to be proactive in articulating the value they provide to an institution' Act. When the RAE was rebranded as the REF in 2014, it began to include an assessment of aspects such as the wider societal impact of research produced by each institution and their research environments. Historically, involvement of librarians with the REF (and previously the RAE) has been centred around providing copies of publications for the submission. In the current REF period (2014)(2015)(2016)(2017)(2018)(2019)(2020), the REF OA policy has provided a new focus of discussion for librarians' involvement with REF processes, which is not surprising given the substantial administrative burden this poses and the associated ramifications for what may or may not be submitted to the REF. 6 However, there are many further ways in which librarians are essential to the functioning of good research environments. Some of this work is relatively new and some has been undertaken by librarians for decades or more. This paper assesses how university-based librarians can use the upcoming REF 2021 exercise as an opportunity to position themselves as important stakeholders in the REF preparations, as well as an opportunity to highlight the important contributions they make to their institutions' research environments, and, by extension, the contributions they make to institutional missions which usually involve the laudable aims of benefiting society in multiple spheres, including the economy, public policy, health, culture and the environment. I hope to demonstrate that the REF can act as an opportunity for librarians to more proactively articulate the 'business-critical' nature of the work performed in libraries to university senior management teams.

The Research Excellence Framework 2021
It is first worth outlining in summary how REF 2021 will function. 7 An institution is essentially awarded funding based upon an assessment of the quality of its research, research impact and research environment.

Quality of outputs (60% weighting in overall score)
Universities will submit portfolios of research outputs published between 2014 and 2020 to disciplinary sub-panels grouped into four areas across STEM, the social sciences and the arts and humanities, which are further broken down into units of assessment (UOAs). 8 Following a process of expert review which looks at originality, significance and rigour, the portfolio of work an institution submits to each UOA will be given a quality profile outlining the percentage of work awarded scores ranging from 1-star to 4-star (with estimates calculating that each piece of 3-and 4-star work attracts funding of up to £3,659 and £14,639, respectively, depending on the UOA, and 1-and 2-star work attracting no financial reward). 9 There are various rules about the eligibility of staff for submission, and the number of outputs that may be submitted. For instance, outputs must be submitted for all staff employed on a 0.2 full-time equivalent (FTE) or above research or research/teaching contract. A minimum of one and a maximum of five outputs can be submitted for each eligible staff member, ensuring an average of 2.5 outputs per FTE. The use of citation data, where available and appropriate, may be used as a potential indicator of academic significance by some panels (primarily STEM panels, with no social science or arts and humanities panels receiving citation data, except Economics and Econometrics). 10 It is stressed that this data would not be used as a primary tool of assessment and panels will not use journal impact factors or other kinds of bibliometric analysis.

Impact of research (25% weighting in overall score)
REF sub-panels will assess the 'reach and significance' of research impact on the economy, society, culture, public policy, health, the environment and quality of life. 11 This includes impact at various geographic scales, and involves audiences of different scales, from individuals to larger communities. Impact will be assessed through the submission of case-study documents, including summaries of the impact, descriptions of the underlying research and who has referenced the research (including references beyond printed academic work). Details of impact evidence or indicators of impact beyond academic impact can be included in impact case studies.
'the REF can act as an opportunity for librarians to more proactively articulate the 'business-critical' nature of the work performed in libraries' 'citation data … may be used as a potential indicator of academic significance' Research environment (15% weighting in overall score) Finally, sub-panels will assess the research environment statement submitted by an institution to a unit of assessment, including its approach to enabling impact from its research, and its contribution to the vitality and sustainability of the wider discipline. An institution must prepare an institutional-level environment statement, as well as a more specific unit-level environment statement. The institutional-and unit-level statements require information about: • the context and mission of the institution • the institution's strategy for research and enabling impact for the current assessment period and for the next five-year period (importantly, this includes structures that support open research) • staff (including how staff and students receive support and training, and equality and diversity initiatives) • income, infrastructure and facilities (broadly the resources and facilities available to support research, and this includes structures that support the reproducibility of research, as well as to facilitate impact).

Evidence of libraries in REF 2014
In outlining the REF process above, there are many clear areas where librarians can usefully contribute. This includes providing advice about bibliometrics and indicators, for example advising research office staff and academics about what indicators are appropriate and how they are calculated and may be used. There could be an important role for librarians in advising which are potentially the best performing papers in terms of citation-based metrics, noting this is one possible indicator that could be used by some sub-panels and which may sway sub-panel decisions in practice. 13 In the area of research impact, there are further areas for librarians to contribute. Librarians are not only becoming more involved in helping researchers to measure impact, but are also becoming more involved in helping to generate that impact. In addition, within the environment section of the REF, there is a chance to highlight recent library-based innovations, particularly services developing around open research and scholarly communications in general.
However, looking at the REF 2014 unit-level environment statements, we might be forgiven for saying that the common adage that 'the library is at the heart of the university' is not true. 14 The complete corpus of REF 2014 unit-level environment statements (n = 1891) was analysed and allocated to one of three categories depending on how extensively they referenced libraries and librarians' contributions to the research environment ( Table 1). The categories were:

• No mention
The institution's libraries or librarians were not referenced directly

• Brief mention
The institution's libraries or librarians were referenced in no more than two sentences and without further substantiation There are clear disciplinary differences shown in Table 1, with around two thirds (67.1%) of submissions to Panel A (STEM-focused) making no reference to library support services, and only 7.2% offering a substantive mention or discussion of librarians' contributions to the research environment. This is a stark contrast to Panel D submissions (arts and humanities-focused), which saw 44% of submissions including a substantive mention of libraries or librarians and less than a fifth (17.6%) having no mention. These figures may reflect disciplinary differences in the perception of the value of the institutional or departmental library and in the relationship of researchers to libraries or librarians. A greater reliance of some subjects in the arts and humanities, such as history, on physical resources based in libraries may explain some of these differences. For instance, some institutional submissions offered an extensive list of the special collections or archival material held by the institution's libraries. However, it is not true to say STEM researchers do not use institutional library resources or support services, so this is an area requiring further research.

Ensuring discoverability and accessibility
The work of librarians in supporting the REF goes well beyond the provision of copies of articles for submission to the REF panels. Of course, a traditional function of the library is that of content provision, and this is still important. Librarians ensure that research resources and collections are both discoverable and accessible, including helping to make them intellectually accessible. Librarians work actively to facilitate access to all staff and students beyond the basic provision of a library catalogue, and many librarians should be able to articulate what they have done beyond the norm to support research. For instance, work has been done in many libraries to make acquisitions and collections policies more responsive to researcher needs, 19 including innovations in the development of the library management system or discovery layer, 20 and decolonisation initiatives, 21 efforts in improving delivery and discovery services including through inter-library loan and reciprocal access schemes, and integration of plug-ins like the Open Access Button into systems and workflows. 22 When REF officers start asking what the library does to support research, this is a good start.

Improving research practice
Librarians aim to improve research practice through teaching good information and organizational skills, and they may also offer general skills and writing workshops. 23 Some offer support for literature reviews and perhaps more in-depth, time-consuming systematic review support across the life cycle of a project, from the point of grant application and then during the project lifetime. 24 They will at least offer training in the use of databases and various software, such as referencing software, to ensure researchers can select and evaluate the most appropriate research in an information-rich environment. Some libraries go well beyond the provision of basic introductory courses, with librarians offering one-to-one support to research students and staff. Librarians are actively improving the quality of funding applications and contributing to research outputs, sometimes even as authors on systematic reviews. Research data management (RDM) services are also implicated in the funding process, and OA and scholarly communication teams support researchers through the project lifetime and beyond. 25  varying applicability of metrics to particular disciplines and the responsible use of metrics.
In the context of REF, it is important that librarians make clear that the most cited is not necessarily the best research. Librarians can also highlight the metrics that will not be used in the REF, including journal-level metrics, and guide selection processes accordingly. Bibliometricians offer an important voice in explaining the workings and biases of university rankings more generally. 32 Librarians are increasingly working with researchers not just to measure impact but to develop impact, in collaboration with other staff around the university, including research and knowledge exchange officers and press officers. Guidance on the use of social media to improve impact and how to measure impact via altmetrics is provided by many university libraries, and workshops on establishing online profiles on academic and general social networking sites is also provided by some. 33 OA teams work across the impact space, with metrics increasingly visible on institutional repositories. 34 Again, this is all work that can be highlighted in the REF, with the environment statement guidelines recommending discussion of 'How the unit has sought to enable and/or facilitate the achievement of impact arising from their research and how they are shaping and adapting their plans to ensure that they continue to support the vitality and sustainability of the unit's impact in the future'. 35

Equality, diversity and a critical voice
In working to develop responsible research metrics, there is an opportunity to highlight their pertinence to equality and diversity in the research environment, which are important issues that should be discussed in REF environment statements (i.e. in relation to researchrelated recruitment, promotion and reward procedures, support for funding applications and access to internal funds). Within many universities, the development of responsible research metrics continues to provide the chance for library staff to work with various academic and professional service departments, including human resources, the research office and training and development departments. Some universities have implemented responsible research metrics in practice, going beyond the simple signing of DORA or similar initiatives like the Leiden Manifesto to roll out detailed guidelines and training; 36 responsible metrics initiatives should lead to a greater level of accountability and processes of transparency in practice, not just in policy.
Progress towards equality and diversity is demonstrated through library work in various other ways, including in the creation of accessible spaces and accessible copies of resources, as well as advising about related software and copyright issues as they pertain to accessibility. Moreover, a growing body of work has begun to analyse critically the ongoing issues raised by colonialism in universities, reflecting on the fact that libraries and their collections may be intellectually inaccessible or biased and actively exclude many researchers. 37 The REF may be used as a focus to begin a critical discussion about the merits and problematic aspects of prevailing research assessment practices more broadly, 38 including in hiring and promotion processes, and lead to further involvement with staff in other departments.

Developing open research initiatives
Not ignoring the effort of researchers in producing outputs in the first place, OA teams have exerted great effort in order to increase compliance with the REF OA policy, ensuring that papers can be submitted to the REF. 39 A great deal of cross-institutional, collaborative and innovative work has gone on in libraries over this decade to enable this. Work around the REF OA policy has gone well beyond uploading manuscripts to the institutional repository and providing financial administration of article processing charges (APCs). Library staff have developed technical skills to help ensure the discoverability and accessibility of research outputs through institutional repositories, ensuring they are well indexed on Google Scholar and other aggregators like Core. 40 The impact of repositories can be measured through various metrics -download counts being the most basicwhich in many institutions show tens of thousands, if not hundreds of thousands, of downloads per month (though it is harder to demonstrate that downloads lead to downstream academic or societal impact). Support for OA publications will continue to be vital as the OA policy will extend into the next REF period and probably become more stringent. 41 Policy-led OA initiatives provide opportunities for librarians to be at the heart of research support services in universities. This includes the development of staff and student skills and capabilities through various methods, including formal training and interactive workshops. 42 OA teams have delivered training via doctoral training programmes, and focused further efforts on doctoral students, such as the provision of e-thesis policies and PhD thesis digitisation programmes. 43 This is often not driven by the REF, but by an aim to ensure that research students are prepared for future careers in research in which they are engaged with the concept of openness.
In addition, librarians continue to provide copyright education, helping to ensure that researchers can retain control over their research outputs where possible. Practitioner groups like LIS-copyseek and the London and South East Copyright Community of Practice show the varied discussions around leveraging copyright literacy to enable research to be conducted effectively and outputs to be reused and disseminated widely. Increasingly, copyright-literate librarians have helped researchers to navigate and negotiate publisher policies, 44 and are beginning to get to grips with emerging issues like text and data mining for research. 45

Open research
Librarians are increasingly moving towards offering holistic open research support services, with support across the research cycle, and are developing a range of core skills in open research. In addition to promoting open access to research publications, support includes RDM advice services: reviewing and offering help with data management plans, providing advice on securely storing and transferring data during a project and at the close of a project, and providing specialist advice for both quantitative and qualitative research data. Some are intervening at the beginning of the grant proposal process, encouraging researchers to think about legal and ethical issues before they become barriers to open sharing. 46 Furthermore, an increasing number of librarians are contributing to the resurgence of the university press and the development of scholar-led initiatives. 47 This is at varying scale, depending on the availability of funds, from establishing publishing advice services and contributing budgets to open source infrastructure and publishers like the Open Library of Humanities, to running self-hosted platforms based on systems like Open Journal Systems (e.g. University of Edinburgh Library, University of Leicester) or Janeway (e.g. University of Huddersfield), to investing funds in commercial suppliers to deliver publishing services (e.g. LSE Press, UCL Press, University of Westminster Press). Press activity increasingly enables librarians not just to be service providers or partners, but innovative leaders in the university. 48 The process of running and establishing a press might better enable library staff to interact with and influence various stakeholders within the university 49 and contribute directly to the delivery of institutional strategy.

Challenges for library staff
There are challenges to taking a more proactive approach toward cross-institutional working, not least adding to staff workloads. We should also recognize that a focus on research may compete with other functions and services, such as teaching and user experience. 53 Library workers may require support and skills development to be able to effectively work with people across the university. A new set of skills for the librarian may include technical skills and competencies in both general and discipline-specific areas of research support, as well as negotiation, strategy, leadership and management skills. 54 Sensitivity needs to be shown towards burdening teams or specific individuals, particularly in smaller institutions, though a movement towards shared services and collaborative efforts may help. 55 However, librarians are already skilled in techniques of engagement and communication with various audiences (e.g. engaging academics in the REF process when they may not particularly want to engage), so involvement in REF processes may not go against the grain of existing skill sets.

Conclusion
Librarians are essential to one of the primary outputs of universities: research. They have always supported world-leading research and will continue to do so, while they also respond to both internal and external pressures like funding and policy environments, as well as developments in the scholarly communications landscape. They have been proactive in contributing to research processes and engaging researchers, in turn helping to contribute to positive changes in the world. Over the next few years, librarians' expertise around scholarly communication issues in particular is likely to offer further opportunities to position them as influential partners in the research process and potentially as leaders 'ensure library strategy responds to each strand of the broader university strategy' 'a focus on research may compete with other functions and services' within the university. However, engaging with the REF process can be about promoting and getting recognition for the things librarians do in spite of the REF. It offers an opportunity to reposition the library and librarians as an important part of the university (if that position has been lost), and offers the library sector more broadly a platform to demonstrate its importance to investors in research, not least the government, other funding organizations and the public. 15. It is recognized that a focus on the words 'library' and 'librarian' may disregard librarians' unacknowledged contributions to staff training and development programmes organized by other professional service departments but it was not within the scope of this study to undertake a detailed investigated of the research support set-up in every UK higher education institution.
16. Examples include: "Physical library access for students and staff is 24/7 365 days a year"; "The University provides truly world-class library facilities"; "We benefit from the extensive collections of the University's libraries, with a dedicated funding stream"; and, "Researchers and PhD students are also supported by an extensive library with well-resourced online journal access to Web of Knowledge through Primo".
17. Examples include: "There is also a specialized Research Support Services team which co-ordinates and develops tailored services for researchers … the UOA is supported by a dedicated, professionally qualified Academic Support Librarian who provides advice and support on information sources, scholarly communication issues, including intellectual property rights and compliance with open access mandate requirements, reference management tools, and the measurement of research impact through citation analysis and application of bibliometric techniques … Training is provided through a six-week information and research skills course specifically for PhD students. This covers literature searching, using the internet, finding specialist research materials, data, reference management, sharing research and building network s … In addition, Academic Support Librarians offer one-toone consultations to all research students in the departments they support"; and, "… In the REF period Research and Innovation Services with the Library & Student Support services commenced a programme of Research Café events (held on average four times each semester), to encourage interdisciplinarity and networking, and to increase the visibility of research among student populations. They achieve this through the delivery of high impact, speed presentations by staff and students in an informal and student-centred setting".