Background
The Adelaide and Meath Hospital incorporating the National Children's Hospital (AMNCH) is a major acute public sector hospital in Dublin, Ireland. It is a teaching hospital for Trinity College Dublin (TCD) and this relationship is governed by a teaching agreement between the Hospital and the University. AMNCH Library serves a diverse range of users: AMNCH and Acute Psychiatric Unit staff; and those TCD academic staff and students that are based in the Hospital. The substantive library for TCD staff is Trinity College Library, which does not have a role in providing a service to AMNCH staff. AMNCH Library is a hospital department staffed by two librarians and two part-time library assistants.
In 2009, the Library provided a collection of 300 journals to hospital staff. In 2010, when its budget was reduced by 11%, the Library cancelled a database and 14 journals that usage statistics compiled from COUNTER JR1 reports showed were little used. User input was not sought and did not inform these decisions. In 2010, the Library provided 286 journals and in early Autumn of that year it was signalled that library budget would be further reduced for 2011.
Within this context, a journals review project was initiated by AMNCH Library in October 2010 with a twofold purpose: to review the journals collection utilizing both user evaluations and journal metrics; and to communicate with users. The consultation and communication component of the project was run in November 2010. Key users, comprising 500 senior clinical and management staff, were surveyed to rate journals relevant to them using an evaluation scale. Cost and usage metrics, where available, were compiled.
In January 2011, the Library was advised that its budget was to be reduced to 75% of the 2010 level. This was a larger cut than had been expected and higher than the generally applied level of 7% across the Hospital. It required a more thorough and rigorous review of the journals than had been anticipated. Having conducted the user survey in November 2010 and compiled journal metrics, the components needed in order to base decisions on robust evidence were in place.
This was the third journals review project in which the Library sought user input. Previous projects in 1997 and 2003 had identified the requirements of hospital staff, and the Library realigned the journal collections using this information. In contrast, the 2010 project was run with the explicit purpose of cancelling titles.
Literature review
Budget reductions are a primary driver for libraries in undertaking a journals review,,,. Libraries are keen to include users in the reviews to safeguard good working relationships with users and ensure the relevance of collections,,,. Journal cancellations can damage the relationship between users and their library if communication is poorly or incompletely executed. The University Libraries of University of Nevada conducted a Serials Assessment Project in 2003 and were concerned to develop an effective communication campaign with users because previous cancellations had left them ‘confused, angry, and disappointed’. Carey et al, concerned from the start about maintaining a good relationship with users, describe a Serials Cancellation Project which explicitly sought to achieve a 15% reduction in journal costs in the ‘most equitable, expedient and affable manner possible’. Their project objective was to conduct ‘a quick, cooperative cancellation campaign that minimizes damage to the integrity of the journal collection while maintaining excellence in service to patrons’.
“Journal cancellations can damage the relationship between users and their library if communication is poorly or incompletely executed.”
Day advises that in seeking feedback it is vital for the librarian to decide how much information to give the user: they need ‘enough information to make informed decisions, but you don't want to overwhelm them with information’ and ‘a serials and database review cannot and should not be entirely data-driven’. Rating scales were commonly devised for users to rate journals and a variety of methods and communication channels were used to seek user input,,. Although libraries were keen to seek user input, most retained the ultimate decision-making authority,,. Interestingly, this is not the default stance of all librarians, for instance in the University of Nevada. ‘the library signalled that it was placing the decision-making process in the hands of the faculty’.
Objectives
The journal review project aimed to meet budget targets by cancelling titles on the basis of an evidence-based approach, and to prepare hospital staff for reduced access to journal content. Additionally, AMNCH Library sought to update its insight into the continuing relevance of its collections to hospital staff to ensure that the most pertinent titles were retained.
Previous projects had built users' trust in AMNCH Library, and the librarians wanted to retain this in less positive economic times by consultation and a transparent decision-making process around cancellations. Transparency to users was to be achieved by seeking their input, and using it as one strand of evidence in the decision making and by reporting back the project results, including the retention and cancellation decision for each journal.
“Transparency to users was to be achieved by seeking their input, and using it as one strand of evidence in the decision making…”
Methods
Identifying the journals
In 2010, the Library's collection of 286 journals was coded by department and 50 titles were identified for retention for one of three reasons: either because they were high-impact journals, or they were paid for by secure funds, or they formed part of a full-text database package (see Figure 1). Four codes were devised for general-interest journals: management, medicine/surgery, nursing, and allied health. 236 journals remained for purchase consideration. A master list of the 286 journals was created that recorded journal title, departmental codes and a retention status of ‘Keep/core title’, ‘Keep/secure funding’, ‘Keep/package’ or ‘For review’.
Breakdown of retention status of journals in review
User survey
The survey adapted and developed the ratings scale devised by Carey et al. The master list of 286 journals was copied and sorted by department code. Columns were added to record the rating scale and journal format (print, online, combined print/online). A customized list was then created for every discipline/department based on the master list and 43 customized lists were devised in total (see Figure 2). Thus, each respondent was required to rank journals relevant to their discipline, management journals, and where applicable, general medical or nursing journals. Users were asked to rank each title as either 1) essential; 2) cancel only if necessary; 3) may be cancelled or 4) cancel.
Many hospital staff do not use their hospital e-mail, so a print survey form was mailed to ensure a valid response rate. In order to represent the range of disciplines and specialist staff, 500 senior clinical and management staff were identified to survey. The customized lists were also available for printout on the Library intranet and staff were invited by e-mail to participate. A reminder was sent one week before the deadline for submission. The survey was conducted in November 2010.
Journal metrics
A separate Excel sheet was created to record usage statistics, cost including VAT and cost-per-use for each title. The cost-per-use was calculated for each journal by dividing the price including VAT by the number of successful full-text article requests in 2010. Unlike the user survey, this journal listing included only the 236 titles assigned for review.
Customized list for Department of Anaesthesia
Budgets and targets
To maintain the level of library service as it was in 2010 would have required an increase in the budget for 2011 of 6.25%. The cost of maintaining the journal subscriptions was in line with this increase, rising by 6.4%. A figure for journal expenditure was calculated by deducting the cost of database and other expenditure from the 2011 budget. This set the financial target to meet for journal subscriptions and represented an absolute reduction of 25%, or a 33% reduction in relative terms, over 2010. There was a considerable gap between the cost of renewing all journals and the available budget.
Results
User survey
Hospital staff returned 202 forms (see Figure 3). Of these, 180 were from targeted staff who had received the print form. This is a 36% response rate. A further 22 staff returned website forms.
Number of respondents by category
Information from the returned forms was compiled into an Excel sheet based on the survey form with columns added to record the purchase decision, response rates for each of the four options, a check column for number of respondents, total population of respondents for each title and an overall response rate for each title (see Figure 4). The Excel sheet was sorted by discipline/area and sub-sorted alphabetically by title.
Sample of summarized results of user survey for the Department of Anaesthesia and the Department of Age-Related Health Care (ARHC)
The rate of response varied widely across departments, from 100% in Rheumatology to 12% in Surgery.
Decision-making process
The Head Librarian and the Collection Development Librarian undertook the decision making governed by the principles that departments would have equitable coverage and that the most used and valued journals would be retained. A guide of one journal to be cancelled per discipline was set. This corresponded to the global cut of 25% as the average provision was four journals per department, although the distribution ranged from one title for 14 single-handed disciplines to 12 for large departments or those with a national remit.
The decision-making process started in earnest with journals in the first two rounds reviewed globally. At the conclusion of each round, the purchase decision was noted in the master Excel sheet, costs recalculated and reviewed against the financial target. In the first round, the emphasis fell on renewing titles for reasons beyond the scope of the metrics and evaluations. Accordingly, 30 journals that were either sole titles in a particular department/ discipline or high-impact high-use general titles were selected for reordering. High use was defined as usage set above a particular figure.
In the second round, the emphasis shifted to identifying potential cancellations using either price above a set value or usage below a set figure. Journals exceeding either criterion were not automatically designated for cancellation. For example, one journal far exceeded the value set for price but its cost-per-use was well below the lower cut-off point and it was reordered. In this round, 44 titles were tagged for probable cancellation and nine titles survived for retention.
From the third round, the focus shifted to assessing journals at department level. Some departments emerged from the second round with journals tentatively tagged for cancellation. First, the metrics for each department were reviewed. Then the evaluations were reviewed to see if they supported emerging retention/cancellation candidates. Where the metrics did not indicate an obvious candidate for cancellation, the evaluations played a more significant role as arbiter. There was some variation in the responses from staff, but in most departments a clear consensus emerged about the journals judged relevant. There were a few departments where no consensus emerged and the larger a department, the less a consensus pertained. By the end of this round, a gap still remained between the target and the recalculated costs.
Clear-cut candidates for cancellation now proved more elusive and as it became progressively more difficult to decide between journals based on the data sets, the librarians' judgement became increasingly necessary. Previous decisions were revised in light of the emerging spread of departmental/discipline coverage. In order to meet the target, a second title for cancellation in some departments had to be selected. It was necessary to revisit some of the larger departments, key specialisms of the hospital, which had more than four titles. In all, 73 journals were identified for cancellation. When the budget target was reached, the decision-making process had served its purpose and 213 journals were ordered. The cancellation of the identified 73 titles was put into effect immediately and these journals were not renewed.
“There was some variation in the responses from staff, but in most departments a clear consensus emerged about the journals judged relevant”.
Communication and feedback
The project report was published on the Library's intranet site in May 2011. It included classified and alphabetical lists of the purchase decisions for every title, and staff were invited to give feedback. The lists enabled staff to quickly see what journals had been cancelled or retained and to see the spread of titles by department. Six hospital staff gave feedback. “This is very interesting” was a typical response. One journal was reordered based on user response.
Discussion
The evidence-based approach supported decision making about journal retention or cancellation, and provided a basis on which to evaluate journals similar in price, impact and coverage which otherwise would have been difficult to differentiate. However, the evaluations were not as useful in the decision making as had been anticipated. The evidence-based approach demonstrated that it was not possible to evaluate journals using solely the metrics and evaluation data. The librarians' knowledge played a crucial element in the decision making because they had a disinterested overview of the collection which no individual or department shared and, because they retained the purchasing-decision power, were able to ensure equitable coverage.
The data sets were referred to from the first round but they increased in significance at the third, when the decision making narrowed focus to department level. Departmental consensus in evaluations was helpful where there was agreement about what title could be cancelled. But where it indicated that all journals were deemed to be equally essential, it could not inform the decision and in these instances the librarians made the final decision. It may be useful in any future project to inform users about how to give actionable evaluations.
The response rate, the size of the department and the share of the budget were not explicitly taken into account. Although it is uncertain how non- responders would have evaluated the journals, it was decided to accept the evaluations for a department as representative of the whole to enable them to be used in the decision making. However, even in those departments lacking consensus about titles or with a low response rate, the evaluations were useful in any insight they gave about how departments rated the journals.
The low level of response to the project report in May 2011 is noteworthy. It is not possible to ascertain if it is a result of the project's transparency or a lack of interest, complacency or disheartedness.
“The librarians' knowledge played a crucial element in the decision making because they had a disinterested overview of the collection which no individual or department shared…”
Conclusion
Although 73 journals represent a 25% reduction in titles, the librarians are confident that the most relevant journals have been retained because of the evidence-based and transparent consultative approach with hospital staff. The project appears to have successfully managed expectations by engaging staff in evaluating the journals and reporting the results.
AMNCH Library is actively considering a fundamental change in how it meets the information needs of hospital staff. With value for money and ease of use being key considerations, it is likely that the future lies in developing services that blend journal provision with alternative means of document delivery that include just-in-time supply of journal articles. As part of the project the Library surveyed hospital staff about the acceptability of journal article supply at the point of need. Of respondents, 46% agreed in principle, 26% disagreed and 28% did not express a view.
The journal review project has indicated that engaging staff by seeking their evaluations both manages their expectations and aids the decision making. This may be considered by any library, and for libraries with similar levels of journal provision, the decision-making process may be useful when faced with budget reductions.
At the time of writing, a further reduction to the Library's budget has been set for 2012. AMNCH Library is again engaged in the processes this paper describes and it is likely this will now become an annual event.