Introduction
Research data management (RDM) refers to the ways in which researchers organize, structure, store and care for the information used or generated as they carry out their research. Recognition of the important issues surrounding RDM has grown over the last decade, and RDM has been described as a ‘wicked’ problem: it requires a wide range of skills and technologies, alongside collaboration and understanding across significantly different stakeholder groups.
Discussions about RDM have become increasingly important not only because data is growing rapidly in volume and complexity, but also due to regulatory requirements concerning data protection (e.g. the GDPR). Data is increasingly viewed as a critical asset for universities, and recent hacking attempts have brought institutional RDM provision to the public’s attention. At the same time, major research funders have developed policy requirements around RDM, and good data handling practices are essential for access to key datasets from partners in government, the National Health Service and industry. At the other end of the research process, data sharing plays a key role and touches on many of the above considerations: the State of Open Data 2020 report notes that, ‘over the past five years, the science ecosystem of researchers, librarians, publishers, institutions, funders, and others have embraced improving data sharing’ in the context of broader open science policies.
It is, therefore, clear that today’s research performing organizations would struggle to deliver research to the highest standard without appropriate investment in RDM services and support. The Covid-19 pandemic has focussed attention even further on the importance of research data and software, as the ways research is carried out have been revolutionized: a large portion of academics and researchers are working from home, and the impact of cutting-edge research on people’s everyday lives is more apparent than ever. Institutional services and support had to adapt, too, to mirror the changing landscape of practices, ‘data professionals in academic libraries sprang into action to help. They shared resources, developed workshops, helped find alternative methods of carrying out research, and found ways of coping with the influx of Covid-related data.’
Generally speaking, RDM is supported by digital infrastructures of varying sophistication, from standard hard drives to integrated cloud solutions. The FAIR (Findability, Accessibility, Interoperability and Reusability) data principles have been increasingly acting as a useful focal point to assess the maturity of research data infrastructures and have been recently described as a tool to help build campus infrastructure and change culture.
This case study seeks to describe the review of RDM services designed and carried out at the University of Oxford in partnership with external consultants between November 2019 and November 2020.
Designing the RDM review
Laying the groundwork
Direct support for RDM at the University of Oxford operates at multiple levels, with researchers’ own efforts being supported (i) at the local level by their departments or institutes and divisions; or (ii) centrally by the Bodleian Libraries, IT Services and Research Services, where the Oxford-based authors of the present article are based. RDM activities are critical to enabling the quality, reproducibility and transparency of Oxford’s world-leading research, and this is reflected in Oxford’s 2018–2023 Strategic Plan priorities and in the University’s IT Strategic Plan 2019–2024.
In 2018, the Bodleian Libraries, IT Services and Research Services carried out an internal scoping exercise to gather preliminary information regarding current service provision and potential gaps for RDM delivery across the University. The exercise stemmed from a recognition that evolving funder mandates, including an increased emphasis on open scholarship in the UK’s research excellence framework (REF), were likely to place increasing demands on RDM support services over time.
The scoping exercise made clear that RDM was a cross-cutting issue that required input from multiple institutional support services, and that there were areas of potential overlap and duplication between these services, as well as some gaps in existing provision. The results of this exercise were summarized in a briefing paper, which informed an internal funding bid for independent consultants to undertake a more in-depth review. The aims of the review were to:
- position the University to ensure that its systems, staffing and support services are best placed to support academics as they carry out their research, and
- ensure that future investment in infrastructure and services is optimized and targeted, in line with global principles and standards for RDM and reproducibility of research, and in line with the University’s new IT Strategy.
Following the preparation of the internal briefing paper, a detailed Request for Information (RFI) document was drafted in consultation with multiple stakeholders across the University. The Request for Information was issued by the University in February 2019, and the objectives of this RFI process were to:
- understand to what extent the University’s needs could be met by a third-party consultant
- inform the budgetary requirements for the review, and
- pre-qualify suppliers in advance of the tender stage.
The RFI was followed by interviews with consultancy providers. A partnership led by Research Consulting was selected as the preferred supplier, with technical lead on RDM from Charles Beagrie Ltd and on university libraries from Tracey Clarke Consulting. Such a partnership appeared to be optimal for addressing the desired scope of work, as it included experts in all the key areas of focus identified in the RFI.
Scope of work
The RDM review focussed on Oxford’s divisions and central provision of RDM services. In addition, the review sought to compare Oxford’s current practices in the domain of RDM with those of national and international peers. It considered national and international best practices and guidance in RDM (including the FAIR Principles), external approaches to charging models and recent policies and reports on RDM. A benchmarking cost survey by the Russell Universities Group IT Directors forum (to the planning of which Research Consulting and the University of Oxford have contributed) was delayed by the Covid-19 pandemic, and its findings, which were expected in 2021, could not be incorporated into the report.
To inform the preparation of a concise report and road map, the scope of work was structured around five pillars (Figure 1), which were developed in close collaboration with the University during a project scoping phase (Figure 2). Particularly, the review used the five pillars as a tool to organize and rationalize the evidence assembled, analyse the findings and summarize the actions required to meet the University’s strategic objectives.
The five pillars approach
Project methodology and time frames
Although the topics and areas investigated in this review were specific to the University of Oxford and relevant at a given point in time, we believe that the overarching five pillar framework developed will be more broadly applicable to other higher education institutions seeking to review their provision of RDM services.
Review timeline
The review was delivered as a multi-stakeholder consultation, including an online survey, interviews, focus groups and workshops (Figure 2): a total of 237 University stakeholders across the academic divisions and central services contributed to this work.
Three key milestones were included in the project plan to allow for scheduled deliberation time: University stakeholders at different levels of seniority and from different organizational units were given the chance to engage directly with emerging issues and to consider the draft outputs.
We note that this review took place during the Covid-19 pandemic. Although the project started via in-person engagement in late 2019, all subsequent engagement activities (interviews, focus groups, workshops) were delivered online using tools such as Zoom, Microsoft Teams and Mentimeter. The impact of online delivery was minimal, and project participants were able to engage effectively with the consulting team via digital tools. However, some activities had to be rearranged or rescheduled to allow for the engagement of key stakeholders who were orchestrating responses to the pandemic.
Delivering the RDM review
Stakeholder engagement
As the RDM review took shape, the breadth and depth of RDM activities carried out at Oxford quickly became apparent: the review needed to acknowledge an extremely multi-faceted and diverse community, engaging different management levels, committees, sub-committees and working groups with either primary or tangential responsibilities about RDM.
As a result of this complexity, the RDM review had to employ different tools to engage different audiences (Figure 2), as follows:
- An online survey was used to gather an initial body of information from the academic community and to inform the development of questions for investigation later on in the project.
- One-to-one or one-to-few interviews were used to engage senior decision makers involved in strategy development and with a leadership role, to investigate the University’s ambition, the appetite for given levels of investments and risk, and to understand their strategic priorities.
- Focus groups and workshops were used to engage academic divisions and teams providing direct support (including the Bodleian Libraries, IT Services and Research Services), including to assess collaboration across professional services and with academic groups. Some of these conversations informed the preparation of divisional case studies, which reflected disciplinary complexities from the points of view of both academics and those supporting them.
To facilitate the interactions between these stakeholder groups and the consulting team, the University appointed a project co-ordinating group, which included representatives of the Bodleian Libraries, IT Services and Research Services. The co-ordinating group acted as the first point of contact for the consulting team and received fortnightly progress updates to monitor the status of the review.
Senior-level buy-in
The review sought to develop actionable insights for the University. Therefore, it was essential to allow sufficient time to secure organizational buy-in for the findings and recommendations arising, and to identify and involve appropriate owners at both senior committee and operational levels.
To facilitate this, the University engaged a range of senior decision makers in the review:
- a project Governing Board, including the above-mentioned co-ordinating group and representatives of the academic divisions
- the chairs of the Research and Innovation Committee, IT Committee and the Research Information Management and Technology Sub-Committee
- the Divisional Secretary and Chief Operating Officer of Gardens, Libraries & Museums, and
- the University’s Chief Information Security Officer and the Head of Information Compliance (and University Data Privacy Officer).
These individuals and groups were engaged particularly around project milestones (Figure 2), but the Governing Board was engaged more regularly to ensure representation of the concerns of individual academic divisions.
Developing actionable insights
Service mapping
The range of activities in the domain of RDM requires cross-service and cross-organizational working. This, combined with the range of RDM services and projects carried out in a very large research-intensive university such as Oxford, means that orchestrating resources and RDM efforts across the University, and maintaining awareness of them amongst an ever-changing academic community, is an inevitable challenge.
A mapping of RDM services was a key deliverable from the review. Our aim in preparing this was to develop an infographic (Figure 3) that could easily convey:
Infographic of RDM services and projects mentioned by stakeholders in the review
- current and proposed services for RDM across the University
- whether these services were provided centrally or locally by the University or provided externally, and
- the key functions or data life cycle stages that the services addressed.
The mapping was based on services and projects mentioned in the survey or individual or group interviews, so it did not showcase the complete universe of services – particularly local or external services. Services were analysed and mapped into five life cycle/activity sectors (Active Data, Semi-active/Living Data, Archived Data, Catalogues and Registries, Advice, Guidance, Consultancy), and zoned as central (university-wide services provided centrally by the Bodleian Libraries, IT Services, or Research Services), local (services provided at divisional, departmental, institute or team level), or external. Services bridging more than one zone were shown on the boundaries.
Note that some services shown in Figure 3 may also be used for other purposes not identified by interviewees.
Definitions used in the study and mapping of services were as follows. ‘Archived Data’ covered both medium-term retention (i.e. beyond the life of project funding but not permanent retention) and preservation and access for the long term (potentially permanent, e.g. in the Bodleian). For ‘Semi-active/Living Data’ we noted there is no widely accepted terminology for this type of data, which is often very long-lived and worked on intermittently by individual scholars or research groups. Our working definition was ‘used data currently held for possible future re-use or further development’, e.g. corpora or databases between periods of grant funding or unfunded. ‘Active Data’ was used in its conventional sense of data in its first phase of being generated and analysed by researchers.
Synthesis of findings
As the project gave rise to a complex set of information, the consulting team carried out thematic coding via NVivo to summarize and prioritize findings. The coding of findings was complemented by a series of workshops within the consulting team, where information pertaining to different areas of the University (e.g. academic community, Libraries, IT Services, Research Services) could be effectively rationalized and transferred between team members. The output of this process was the creation of a coherent longlist of recommendations (used as a way to organize actions by theme or area) and specific actions that could support the University’s ambition with regard to RDM. Recommendations were mapped (Figure 4) to the five pillars in Figure 1 to ensure that all strategic concerns identified at project inception had been addressed.
Recommendations and pillars addressed
The development of a prioritized shortlist of actions under each recommendation was informed by the identification of underlying key issues and requirements. The benefits, impact and feasibility of emerging actions in terms of costs, dependencies and timescales were also assessed. This approach helped the consulting team build the grading matrix shown in Table 1, which underpinned the preparation of the project road map.
Table 1
Grading criteria for actions
| Grading | Description |
|---|---|
| Green | Actions that deliver operational improvements or efficiencies. These can be largely delivered within existing resources, or the necessary funding has already been secured. |
| Amber | Actions that contribute to achievement of the University’s strategic priorities or enhance the quality of research. Some additional investment and/or changes to institutional governance processes may be required. |
| Red | Actions that directly support achievement of the University’s strategic priorities or address strategic risks. Additional investment and/or changes to institutional governance arrangements are required. |
Road map
The findings of the RDM review were used to inform the preparation of a future road map. This was created as a dynamic working document in spreadsheet form (Table 2) and, once validated, it was turned into an infographic for internal dissemination within the University (Figure 5).
Table 2
Fields included in the spreadsheet version of the road map
| Column header | Description | Value |
|---|---|---|
| Recommendation # | Number of the recommendation | 1, 2, 3, … |
| Recommendation | High-level recommendation | Text (see Figure 4 for examples) |
| Action # | Number of the action grouped under a recommendation | 1.1, 1.2, 2.1… |
| Action | Specific action grouped under a recommendation | Text |
| Oversight responsibility | Individual responsible for overseeing the implementation of the action (e.g. Chair of committee) | Text |
| Primary responsibility for implementation | Individual(s) or group primarily responsible for implementing the action | Text |
| Additional responsibility for implementation | Other individual(s) or group responsible for implementing the action | Text |
| Cost | Estimated cost of the action, if available | Currency or Category (Low/Medium/High) |
| Type of cost | Likely type of cost arising from the action | Category (e.g. recurring staff costs, project staff, infrastructure and service provision) |
| Time frame | Time frame for the implementation of the action | Category (Short/Medium/Long term) |
| Priority | Priority of the action, based on the outcome of the review | Category (Low/Medium/High) |
| Implementation mechanisms | Specific implementation mechanisms, if known (e.g. ongoing project, new role within the University) | Text |
| Notes | Any other notes or details that cannot be captured in structured form | Text |
Road map in infographic format
The following considerations played a key role in the preparation of the road map:
- the road map should include information at different levels of granularity to be understood by different audiences
- the road map should be a dynamic document that is easy to edit, both during the review and after
- the road map should include standard fields and categories wherever possible, to allow for streamlined tracking during the implementation stage, and
- responsibilities should be allocated for both oversight (named individuals) and implementation (named individuals and/or groups).
The infographic in Figure 4 represents a snapshot of the road map at a given point in time and caters to decision makers who may not need detailed information at the action level but do require a clear view of the proposed strategic priorities. Notably, Figure 4 includes a significant amount of contextual management information: actions are grouped by recommendation and time frame, red/amber/green (RAG) colour-coding is used to indicate the priority of actions (based on the gradings in Table 1) and an icon communicates whether actions require investment. In addition, Figure 4 is part of a two-page infographic, where the second page (not shared for reasons of confidentiality) provides a list of all numbered actions and acts as a legend.
Final validation and socialization
As noted above, the project’s findings and road map were discussed and validated at various points throughout the review. However, the final validation and socialization stage was key to ensuring the future implementation of the road map: it allowed the consulting team to gather feedback from the stakeholder groups responsible for implementing and financing the recommendations and actions arising from the review.
Focus groups with senior University leaders, including the top administrators in IT Services, Research Services and Libraries were used to achieve the above. These meetings also allowed the consulting team to finesse the messages and language of the report to ensure it would speak to both the academic community and central and divisional support services. Some changes were made in response to the focus group findings, including an increased focus on the budgetary and resourcing implications of the road map and associated recommendations in their final form.
The report and road map in final form were presented by the consulting team to the Research Information Management and Technology Sub-Committee and to the Curators of the University Libraries. They were then taken over by the University for further discussion and implementation.
Next steps
The road map provides a recommended RDM action plan for the University that will continue to evolve and be iterated in the light of additional internal costings, available resourcing and reprioritization.
At present, the road map is owned by the University’s Research Information Management and Technology Sub-Committee, who have commissioned an RDM review Task and Finish group to develop a costed business case for a programme of work to implement the road map. In order for the original aims of the review to be realized, it is seen as important to maintain a coherent programme with a clear narrative and sense of strategic direction.
Academic awareness and engagement are essential to the success of the road map. The Task and Finish group members have made a concerted effort to ‘socialize’ the review outcomes at a broad range of committee meetings, and to link up with local expert research groups such as Reproducible Research Oxford and FAIRsharing.
A key area of the review will be focussed on joining up services in alignment with the research data life cycle and developing local repository offerings in a manner that facilitates this. The RDM landscape at Oxford is complex – but this should prove to be one of its greatest strengths.
Lessons learned
Reflecting on the review process, we would make the following recommendations to other institutions seeking to enhance their support for research data management:
- Secure high-level sponsorship – The support of senior leaders is critical to the overall success of any review of RDM services. In Oxford’s case, overall leadership was provided by the Pro-Vice-Chancellor for Research, and the resulting road map is owned by the Research Information Management and Technology Sub-Committee. A range of influential stakeholders, including the senior administrators within IT Services, Research Services and the Bodleian Libraries, were identified early in the review process and given the chance to shape the final outcomes.
- Prioritize engagement with the academic community – The use of multiple consultation methods, including an online survey, interviews, focus groups and engagement with relevant committees, allowed a wide range of stakeholders to be involved and was crucial to legitimizing the findings in the eyes of the academic community at Oxford. Engaging researchers through a range of techniques helps ensure that the findings are ultimately driven by their needs, rather than those of the administration.
- Recognize the cross-cutting nature of RDM – The collaborative approach taken by the Bodleian Libraries, IT Services and Research Services was a significant strength of the review and allowed for a holistic approach to be taken to the project. Other institutions embarking on a similar project are advised to recognize the cross-cutting nature of RDM and ensure all relevant professional services are engaged from the outset.
- Acknowledge the respective roles of central and distributed support services – In a large institution, critical RDM services and support staff will be present at multiple levels, including within departments, at division or faculty level, and within central services. Researchers will also rely on a range of externally provided services. The most appropriate level for provision of RDM services will vary according to local context and needs, with no single ‘right’ answer.
- Identify unifying themes or pillars to guide the review process – Our review used the five pillars outlined in Figure 1 as a tool to organize and rationalize the evidence assembled, analyse the findings and summarize the actions required to meet the University’s strategic objectives. Given the complexity of the subject matter, having a comparable set of themes or pillars for a review of this nature reduces the risk of scope creep and makes it easier for other stakeholders to grasp the review’s purpose and implications.
- Understand and embrace different disciplinary cultures for data sharing – Disciplinary communities, and their organizational units, have very different needs and expectations of RDM services. This needs to be explicitly acknowledged from the outset of any review process, with a recognition that there will be few, if any, one-size-fits-all solutions.
- Recognize the value and limitations of using independent consultants – The use of independent consultants allowed the review to be progressed much more quickly than would have been possible through internal resources alone. It was also of value in reaching a synthesized outcome that did not privilege the interests of any particular stakeholder or constituency. However, there remained a need for significant engagement and input from Oxford staff to guide the consultants. It is therefore important that effective knowledge exchange between consultants and staff members occurs throughout the review process.
- Be prepared to adapt to changing circumstances – The Covid-19 pandemic meant that the review at Oxford took on a very different shape than was originally expected. While disruptive in some respects, the pandemic also served to highlight the critical importance of good data management practices and information security, which was ultimately helpful in securing buy-in for the review’s recommendations.
- Be realistic about institutional planning and budgeting timelines – A key lesson learned is that completion of the review report is best seen as the beginning rather than the end of the process. While the review itself took just over 12 months to complete, further work continues to be required to embed the recommendations in operational plans and budgets.
- Recognize that the journey is as important as the destination – While there is a tendency to see the final report and road map as the primary output of a review of this nature, the real value is likely to lie in strengthened internal relationships, improved institutional awareness and cultural change in favour of good research data management practice. A well-managed review should aim to deliver all of these benefits, irrespective of the findings of the final report.