Introduction

Sector surveys and commentators have asserted that the dividing lines are becoming increasingly blurred between the work done in support of research by librarians and staff in other parts of the university. In addition to this, librarians are occasionally seen as being unable to articulate successfully the value of their contributions or be as influential as staff based in other departments more closely aligned to senior management. Librarians also increasingly face competition from external service suppliers, and external resources, e.g. open access (OA) articles, are likely to greatly outnumber and be easier to access than those subscribed to or otherwise offered by the library. Additionally, external suppliers, including commercial organisations, funders and publishers, increasingly provide open research, research assessment and discovery services, challenging the role of librarians and information science professionals within an institution. An assessment of the quality of an institution’s research environment is a major part of the current Research Excellence Framework (REF) assessment in the UK, run by UK Research and Innovation (UKRI), the national funding agency for investment in science and research. In a context where the role of librarians may be perceived as diminished or where librarians are seen to be unable to promote their successes effectively, the REF seems to offer a significant opportunity for librarians to be proactive in articulating the value they provide to an institution.

Initially, a national research assessment was established in the UK to determine the allocation of funding to universities during a period of declining government budgets available for higher education under Margaret Thatcher’s government in the 1980s. Throughout the 1990s and 2000s the national Research Assessment Exercise (RAE) expanded in size, complexity and scope, and began to include universities that were known as polytechnics prior to the 1992 Further and Higher Education Act. When the RAE was rebranded as the REF in 2014, it began to include an assessment of aspects such as the wider societal impact of research produced by each institution and their research environments. Historically, involvement of librarians with the REF (and previously the RAE) has been centred around providing copies of publications for the submission. In the current REF period (2014–2020), the REF OA policy has provided a new focus of discussion for librarians’ involvement with REF processes, which is not surprising given the substantial administrative burden this poses and the associated ramifications for what may or may not be submitted to the REF. However, there are many further ways in which librarians are essential to the functioning of good research environments. Some of this work is relatively new and some has been undertaken by librarians for decades or more.

This paper assesses how university-based librarians can use the upcoming REF 2021 exercise as an opportunity to position themselves as important stakeholders in the REF preparations, as well as an opportunity to highlight the important contributions they make to their institutions’ research environments, and, by extension, the contributions they make to institutional missions which usually involve the laudable aims of benefiting society in multiple spheres, including the economy, public policy, health, culture and the environment. I hope to demonstrate that the REF can act as an opportunity for librarians to more proactively articulate the ‘business-critical’ nature of the work performed in libraries to university senior management teams.

The Research Excellence Framework 2021

It is first worth outlining in summary how REF 2021 will function. An institution is essentially awarded funding based upon an assessment of the quality of its research, research impact and research environment.

Quality of outputs (60% weighting in overall score)

Universities will submit portfolios of research outputs published between 2014 and 2020 to disciplinary sub-panels grouped into four areas across STEM, the social sciences and the arts and humanities, which are further broken down into units of assessment (UOAs). Following a process of expert review which looks at originality, significance and rigour, the portfolio of work an institution submits to each UOA will be given a quality profile outlining the percentage of work awarded scores ranging from 1-star to 4-star (with estimates calculating that each piece of 3- and 4-star work attracts funding of up to £3,659 and £14,639, respectively, depending on the UOA, and 1- and 2-star work attracting no financial reward). There are various rules about the eligibility of staff for submission, and the number of outputs that may be submitted. For instance, outputs must be submitted for all staff employed on a 0.2 full-time equivalent (FTE) or above research or research/teaching contract. A minimum of one and a maximum of five outputs can be submitted for each eligible staff member, ensuring an average of 2.5 outputs per FTE. The use of citation data, where available and appropriate, may be used as a potential indicator of academic significance by some panels (primarily STEM panels, with no social science or arts and humanities panels receiving citation data, except Economics and Econometrics). It is stressed that this data would not be used as a primary tool of assessment and panels will not use journal impact factors or other kinds of bibliometric analysis.

Impact of research (25% weighting in overall score)

REF sub-panels will assess the ‘reach and significance’ of research impact on the economy, society, culture, public policy, health, the environment and quality of life. This includes impact at various geographic scales, and involves audiences of different scales, from individuals to larger communities. Impact will be assessed through the submission of case-study documents, including summaries of the impact, descriptions of the underlying research and who has referenced the research (including references beyond printed academic work). Details of impact evidence or indicators of impact beyond academic impact can be included in impact case studies.

Research environment (15% weighting in overall score)

Finally, sub-panels will assess the research environment statement submitted by an institution to a unit of assessment, including its approach to enabling impact from its research, and its contribution to the vitality and sustainability of the wider discipline. An institution must prepare an institutional-level environment statement, as well as a more specific unit-level environment statement. The institutional- and unit-level statements require information about:

  • the context and mission of the institution
  • the institution’s strategy for research and enabling impact for the current assessment period and for the next five-year period (importantly, this includes structures that support open research)
  • staff (including how staff and students receive support and training, and equality and diversity initiatives)
  • income, infrastructure and facilities (broadly the resources and facilities available to support research, and this includes structures that support the reproducibility of research, as well as to facilitate impact).

The REF guidance suggests that institutions include supporting quantitative indicators where applicable and appropriate in line with guidance from the UK Forum for Responsible Research Metrics.

Evidence of libraries in REF 2014

In outlining the REF process above, there are many clear areas where librarians can usefully contribute. This includes providing advice about bibliometrics and indicators, for example advising research office staff and academics about what indicators are appropriate and how they are calculated and may be used. There could be an important role for librarians in advising which are potentially the best performing papers in terms of citation-based metrics, noting this is one possible indicator that could be used by some sub-panels and which may sway sub-panel decisions in practice. In the area of research impact, there are further areas for librarians to contribute. Librarians are not only becoming more involved in helping researchers to measure impact, but are also becoming more involved in helping to generate that impact. In addition, within the environment section of the REF, there is a chance to highlight recent library-based innovations, particularly services developing around open research and scholarly communications in general.

However, looking at the REF 2014 unit-level environment statements, we might be forgiven for saying that the common adage that ‘the library is at the heart of the university’ is not true. The complete corpus of REF 2014 unit-level environment statements (n = 1891) was analysed and allocated to one of three categories depending on how extensively they referenced libraries and librarians’ contributions to the research environment (Table 1). The categories were:

  • No mention
    The institution’s libraries or librarians were not referenced directly
  • Brief mention
    The institution’s libraries or librarians were referenced in no more than two sentences and without further substantiation
  • Substantive mentions
    More than two sentences where the institution’s libraries or librarians are referenced or discussed explicitly in relation to their contribution to the research environment.

Table 1

Mentions of libraries and librarians in REF 2014 unit-level environment statements

REF PanelNumber of submissionsNo mentionBrief mentionSubstantive mention

Panel A (n)3042047822
Panel A (%)67.125.77.2
Panel B (n)39624311241
Panel B (%)61.428.310.4
Panel C (n)612164279169
Panel C (%)26.845.627.6
Panel D (n)579102222255
Panel D (%)17.638.344.0
Total (n)1,891713691487
Total (%)37.736.525.8

It is striking that over a third of environment statements (37.7%) made no direct reference to the institutional library or librarians, not even just to mention the library collections. A similar number (36.5%) only briefly referenced the library or librarians. However, this was often in a very superficial way, such as a passing mention to the existence of the library or a subject librarian and the amount spent on library resources. In some cases, it was strange that an HEI emphasized its proximity to other public or university libraries over a discussion of its own research support offerings. Substantive mention was only made in about a quarter of submissions (25.8%). In many cases this simply consisted of a longlist of the electronic and physical collections made available by the library, but there were some good examples of HEIs extensively discussing the research support services offered by librarians, including in the areas of doctoral student support, open research, literature searching and systematic reviews, copyright and responsive acquisition procedures.

There are clear disciplinary differences shown in Table 1, with around two thirds (67.1%) of submissions to Panel A (STEM-focused) making no reference to library support services, and only 7.2% offering a substantive mention or discussion of librarians’ contributions to the research environment. This is a stark contrast to Panel D submissions (arts and humanities-focused), which saw 44% of submissions including a substantive mention of libraries or librarians and less than a fifth (17.6%) having no mention. These figures may reflect disciplinary differences in the perception of the value of the institutional or departmental library and in the relationship of researchers to libraries or librarians. A greater reliance of some subjects in the arts and humanities, such as history, on physical resources based in libraries may explain some of these differences. For instance, some institutional submissions offered an extensive list of the special collections or archival material held by the institution’s libraries. However, it is not true to say STEM researchers do not use institutional library resources or support services, so this is an area requiring further research.

A factor impacting inclusion or exclusion of library services in environment statements may have been how closely library staff worked with the research office or the various REF administration officers throughout central departments and academic faculties in developing environment statements and participating in other REF processes. Speculatively, the complexity or size of an institution, as well as physical proximity, may also be important factors. Strategic decisions may have been made about emphasizing certain aspects of support for research over others, valuing contributions made by other professional service departments at the expense of the contributions made by the library. Yet some universities’ environment statements clearly show the value of library services, mentioning not just collections but also contributions like establishing new open research initiatives, aiding in the set-up of research journals, delivering systematic review support, and a wide range of other support services offered to various groups of researchers and doctoral students.

Overall, however, in the documented record of the UK research environment as it existed in 2014, librarians appear to be hardly more than a footnote, offering little of substance to the research environment. Around three quarters of submissions made no mention or only superficial mention of the library. Another study has made similar observations about the lack of recognition of library activities in the context of the UK Teaching Excellence Framework. Is this really how we want librarians to be represented to various audiences, including the government, research funders, academic staff and the public? There are clearly missed opportunities for internal and external recognition, which should be taken up as REF 2021 is approached.

How can librarians participate in the REF?

There are various ways in which librarians can participate in the REF process. Importantly, there is an opportunity for librarians to position themselves as vital to administrative processes around the REF, which may have longer-term implications for how business critical the library is perceived to be within an institution. However, there are also distinct opportunities for explicit recognition of this value in externally-facing documentation, particularly REF environment statements. Below is a summary of some of the research-related work librarians have been involved in since (and in many cases before) 2014. Some of this work may seem obvious, but it is worth outlining and being explicit about our many contributions.

Ensuring discoverability and accessibility

The work of librarians in supporting the REF goes well beyond the provision of copies of articles for submission to the REF panels. Of course, a traditional function of the library is that of content provision, and this is still important. Librarians ensure that research resources and collections are both discoverable and accessible, including helping to make them intellectually accessible. Librarians work actively to facilitate access to all staff and students beyond the basic provision of a library catalogue, and many librarians should be able to articulate what they have done beyond the norm to support research. For instance, work has been done in many libraries to make acquisitions and collections policies more responsive to researcher needs, including innovations in the development of the library management system or discovery layer, and decolonisation initiatives, efforts in improving delivery and discovery services including through inter-library loan and reciprocal access schemes, and integration of plug-ins like the Open Access Button into systems and workflows. When REF officers start asking what the library does to support research, this is a good start.

Improving research practice

Librarians aim to improve research practice through teaching good information and organizational skills, and they may also offer general skills and writing workshops. Some offer support for literature reviews and perhaps more in-depth, time-consuming systematic review support across the life cycle of a project, from the point of grant application and then during the project lifetime. They will at least offer training in the use of databases and various software, such as referencing software, to ensure researchers can select and evaluate the most appropriate research in an information-rich environment. Some libraries go well beyond the provision of basic introductory courses, with librarians offering one-to-one support to research students and staff. Librarians are actively improving the quality of funding applications and contributing to research outputs, sometimes even as authors on systematic reviews. Research data management (RDM) services are also implicated in the funding process, and OA and scholarly communication teams support researchers through the project lifetime and beyond. All this is relevant in the REF environment statement as the guidelines suggest discussing ‘support for early career researchers and career development at all stages in research careers; support mechanisms for, and evidence of the training and supervision of, postgraduate research (PGR) students’.

Research information services

An increasing range of research-focused roles have been developed in response to changes in the way research is conducted and communicated, as well as in response to policy developments. Open research roles existed in some universities at the time of the last REF, but there have been many developments since then, and partnerships between librarians and researchers across the research life cycle are increasing.

One of the major contributions librarians have made during the current REF cycle is in the management of current research information systems (CRISs) or similar databases, which are of vital importance as they contain information about what the university actually published during the REF period, and can allow identification of the publications brought by new staff and those left by previous staff. Librarians with oversight of the institutional CRIS play a vital role in capturing and understanding the institution’s information assets. Library staff often generate and analyse reports on these outputs, using them to highlight groups who may require targeted outreach about the importance of OA beyond REF and funder mandates. Even where a university does not manage a CRIS, it is likely that an OA repository functions in similar ways, with plug-ins that bring CRIS-like functionality into play, or librarians will have developed other bespoke spreadsheets or databases. The REF submission process is a good opportunity for librarians to demonstrate these business-critical services.

OA librarians have worked creatively with open source as well as proprietary software to ensure that institutional outputs can be discovered and accessed by various audiences, and they contribute to making the university’s research more visible with possible effects for downstream impact. An increasing number of universities have worked with digital object identifier (DOI) services like DataCite to ensure non-article outputs, particularly doctoral theses and data sets, are more discoverable via repositories. Librarians based in universities with greater numbers of arts outputs have worked particularly creatively with repositories to better represent non-text-based outputs on repositories, and to ensure they can be discovered, curated and, in turn, reported on.

Support for research indicators and metrics

Librarians can help to connect various disparate research support silos in universities, through leading on institution-wide initiatives and policies, including the implementation of the San Francisco Declaration on Research Assessment (DORA) and other responsible metrics policies, and, in a small number of universities, work towards rollout of the UK Scholarly Communications Licence (UK-SCL). This can help develop a higher profile for library workers within universities, and particularly get them known to university management and various stakeholders across the institution. The number of institutions rolling out responsible metrics statements has increased significantly since 2016, but some librarians may consider their contribution to these initiatives to be significant enough to highlight in REF environment statements.

A clear role for librarians in the REF is in support for bibliometrics and altmetrics (alternative metrics), not least in offering support for selecting articles for the REF, which may be partially guided by peer-review processes alongside citation-based indicators where appropriate. More generally, librarians can use bibliometrics to broadly identify research strengths, and to measure some aspects of impact, such as policy citations and mentions in the media via services such as Altmetric. This offers an important opportunity to work with researchers and senior management to explain concepts in bibliometrics and altmetrics, the varying applicability of metrics to particular disciplines and the responsible use of metrics. In the context of REF, it is important that librarians make clear that the most cited is not necessarily the best research. Librarians can also highlight the metrics that will not be used in the REF, including journal-level metrics, and guide selection processes accordingly. Bibliometricians offer an important voice in explaining the workings and biases of university rankings more generally.

Librarians are increasingly working with researchers not just to measure impact but to develop impact, in collaboration with other staff around the university, including research and knowledge exchange officers and press officers. Guidance on the use of social media to improve impact and how to measure impact via altmetrics is provided by many university libraries, and workshops on establishing online profiles on academic and general social networking sites is also provided by some. OA teams work across the impact space, with metrics increasingly visible on institutional repositories. Again, this is all work that can be highlighted in the REF, with the environment statement guidelines recommending discussion of ‘How the unit has sought to enable and/or facilitate the achievement of impact arising from their research and how they are shaping and adapting their plans to ensure that they continue to support the vitality and sustainability of the unit’s impact in the future’.

Equality, diversity and a critical voice

In working to develop responsible research metrics, there is an opportunity to highlight their pertinence to equality and diversity in the research environment, which are important issues that should be discussed in REF environment statements (i.e. in relation to research-related recruitment, promotion and reward procedures, support for funding applications and access to internal funds). Within many universities, the development of responsible research metrics continues to provide the chance for library staff to work with various academic and professional service departments, including human resources, the research office and training and development departments. Some universities have implemented responsible research metrics in practice, going beyond the simple signing of DORA or similar initiatives like the Leiden Manifesto to roll out detailed guidelines and training; responsible metrics initiatives should lead to a greater level of accountability and processes of transparency in practice, not just in policy.

Progress towards equality and diversity is demonstrated through library work in various other ways, including in the creation of accessible spaces and accessible copies of resources, as well as advising about related software and copyright issues as they pertain to accessibility. Moreover, a growing body of work has begun to analyse critically the ongoing issues raised by colonialism in universities, reflecting on the fact that libraries and their collections may be intellectually inaccessible or biased and actively exclude many researchers. The REF may be used as a focus to begin a critical discussion about the merits and problematic aspects of prevailing research assessment practices more broadly, including in hiring and promotion processes, and lead to further involvement with staff in other departments.

Developing open research initiatives

Not ignoring the effort of researchers in producing outputs in the first place, OA teams have exerted great effort in order to increase compliance with the REF OA policy, ensuring that papers can be submitted to the REF. A great deal of cross-institutional, collaborative and innovative work has gone on in libraries over this decade to enable this. Librarians have used tools like Unpaywall, Cottage Lab’s Lantern, Wellcome Trust Compliance Tool and Jisc’s Publications Router, among others, and have helped to develop national and regional communities like the Open Access Scotland Group, the London Open Access Network and UKCORR. It may be worth stating in REF environment statements where librarians have clearly played a key role in developing these communities, which help to improve staff knowledge and solve particular issues around open research, in turn improving research environments for both their own and other institutions.

Work around the REF OA policy has gone well beyond uploading manuscripts to the institutional repository and providing financial administration of article processing charges (APCs). Library staff have developed technical skills to help ensure the discoverability and accessibility of research outputs through institutional repositories, ensuring they are well indexed on Google Scholar and other aggregators like Core. The impact of repositories can be measured through various metrics – download counts being the most basic – which in many institutions show tens of thousands, if not hundreds of thousands, of downloads per month (though it is harder to demonstrate that downloads lead to downstream academic or societal impact).

Support for OA publications will continue to be vital as the OA policy will extend into the next REF period and probably become more stringent. Policy-led OA initiatives provide opportunities for librarians to be at the heart of research support services in universities. This includes the development of staff and student skills and capabilities through various methods, including formal training and interactive workshops. OA teams have delivered training via doctoral training programmes, and focused further efforts on doctoral students, such as the provision of e-thesis policies and PhD thesis digitisation programmes. This is often not driven by the REF, but by an aim to ensure that research students are prepared for future careers in research in which they are engaged with the concept of openness.

In addition, librarians continue to provide copyright education, helping to ensure that researchers can retain control over their research outputs where possible. Practitioner groups like LIS-copyseek and the London and South East Copyright Community of Practice show the varied discussions around leveraging copyright literacy to enable research to be conducted effectively and outputs to be reused and disseminated widely. Increasingly, copyright-literate librarians have helped researchers to navigate and negotiate publisher policies, and are beginning to get to grips with emerging issues like text and data mining for research.

Open research

Librarians are increasingly moving towards offering holistic open research support services, with support across the research cycle, and are developing a range of core skills in open research. In addition to promoting open access to research publications, support includes RDM advice services: reviewing and offering help with data management plans, providing advice on securely storing and transferring data during a project and at the close of a project, and providing specialist advice for both quantitative and qualitative research data. Some are intervening at the beginning of the grant proposal process, encouraging researchers to think about legal and ethical issues before they become barriers to open sharing.

Furthermore, an increasing number of librarians are contributing to the resurgence of the university press and the development of scholar-led initiatives. This is at varying scale, depending on the availability of funds, from establishing publishing advice services and contributing budgets to open source infrastructure and publishers like the Open Library of Humanities, to running self-hosted platforms based on systems like Open Journal Systems (e.g. University of Edinburgh Library, University of Leicester) or Janeway (e.g. University of Huddersfield), to investing funds in commercial suppliers to deliver publishing services (e.g. LSE Press, UCL Press, University of Westminster Press). Press activity increasingly enables librarians not just to be service providers or partners, but innovative leaders in the university. The process of running and establishing a press might better enable library staff to interact with and influence various stakeholders within the university and contribute directly to the delivery of institutional strategy.

Beyond OA publishing, there are many other involvements in open research, including: advice on preprints, open peer review, research software (as well as contributing to the development of open source projects and software) and digital humanities projects like digitization, 3D digitization and AV production. Looking at the REF environment statement guidelines once more, it suggests an institution discusses ‘How the submitting unit is progressing towards an open research environment, including where this goes above and beyond the REF open access policy requirements… Consideration of reproducibility should also be included where relevant to the discipline’. Librarians clearly contribute to these areas.

Demonstrating outcomes and the contributions of librarians

I have outlined how librarians contribute to various elements of the university research environment, which may be directly rewarded as part of the REF. However, in REF 2014, library work was not well represented in environment statements. It has been asserted by some that librarians may not be well positioned to communicate successfully about their successes and contributions and have perhaps adopted a role as service provider rather than leader. However, it is clear that REF 2021 provides an opportunity for university-based librarians to contribute in varied and essential ways to the REF submission process, as well as to highlight the wide range of initiatives that support excellent research.

Taking a longer-term view beyond the REF, a good first step would be to ensure library strategy responds to each strand of the broader university strategy so that any initiative can be directly demonstrated to be relevant to university management. More practically, if REF officers are not proactive in reaching out to library staff to help contribute to the preparation of institutional-level and unit-level environment statements, then it is important to reach out to them. Given the disciplinary variations shown in REF 2014, greater attention may need to be paid towards STEM unit-level environment statements, though the institutional-level environment statements requested for REF 2021 may help to offset this need. Closer involvement in REF processes may or may not follow, but it is likely, as we approach 2021, that REF officers are thinking about drafting environment statements, so even preparing a few paragraphs of how librarians support the research environment would be useful.

Challenges for library staff

There are challenges to taking a more proactive approach toward cross-institutional working, not least adding to staff workloads. We should also recognize that a focus on research may compete with other functions and services, such as teaching and user experience. Library workers may require support and skills development to be able to effectively work with people across the university. A new set of skills for the librarian may include technical skills and competencies in both general and discipline-specific areas of research support, as well as negotiation, strategy, leadership and management skills. Sensitivity needs to be shown towards burdening teams or specific individuals, particularly in smaller institutions, though a movement towards shared services and collaborative efforts may help. However, librarians are already skilled in techniques of engagement and communication with various audiences (e.g. engaging academics in the REF process when they may not particularly want to engage), so involvement in REF processes may not go against the grain of existing skill sets.

Conclusion

Librarians are essential to one of the primary outputs of universities: research. They have always supported world-leading research and will continue to do so, while they also respond to both internal and external pressures like funding and policy environments, as well as developments in the scholarly communications landscape. They have been proactive in contributing to research processes and engaging researchers, in turn helping to contribute to positive changes in the world. Over the next few years, librarians’ expertise around scholarly communication issues in particular is likely to offer further opportunities to position them as influential partners in the research process and potentially as leaders within the university. However, engaging with the REF process can be about promoting and getting recognition for the things librarians do in spite of the REF. It offers an opportunity to reposition the library and librarians as an important part of the university (if that position has been lost), and offers the library sector more broadly a platform to demonstrate its importance to investors in research, not least the government, other funding organizations and the public.

Data accessibility statement

Data underpinning this article is available at https://doi.org/10.17037/DATA.00001457. Environment statements submitted by institutions for REF 2014 can be accessed at https://results.ref.ac.uk/DownloadSubmissions.