The University of Sussex Library (UoSL) supports the learning and teaching of over 13,000 students. For courses with large student numbers or where students are expected to go on a work placement and may need to access resources remotely, the policy is to purchase core reading as e-books1. The UoSL currently subscribes to a number of different e-book collections and in September 2012 started to use Ebook Library (EBL) to provide a platform for patron-driven acquisition (PDA). This gives students a more prominent role in selecting the books that form the collection2 and has proved successful at other institutions, with books purchased through PDA circulating more than those acquired in the traditional way3. More recently, EBL has also been used to fulfil requests for inter-library loans. The introduction of these services, together with pressures on space, means that the number of e-books available from the Library is set to rise. With the student experience becoming increasingly important4, assessing the impact of growing e-book collections is paramount.

EBL provides logs containing data about the e-books that have been accessed along with measures such as duration of view and whether the item has been downloaded. As usage data informs decisions about collection development it needs to provide the greatest possible insight into usage. This is equally true of decisions made regarding study skills teaching since, if user behaviour is not understood, appropriate content cannot be included in the sessions being delivered. This investigation looks at the use of e-books through student outputs (in this case coursework) in an attempt to further our understanding of the way that e-books are being used.

“… usage data informs decisions about collection development …”

Why bother?

Guthrie5 explains that “libraries and publishers both have much to gain from investing in analysis to gain a deeper understanding of the ways that students and scholars use electronic books”. Martin and Quan-Haase6 agree, arguing that the key to this lies in gaining an awareness of scholarly habits and how e-books fit into these practices. Previous studies7 have found that students browse e-book content and tend to read particular chapters. This is supported by data gathered from the University of Sussex which found that 70% of sampled e-book users looked for a ‘Download PDF of a chapter’ function, 20% more than those who reported looking for an option to download the entire title8. This may be true but still leaves a large number of students who are not using e-books in this way.

It is increasingly important for every library to evaluate its own impact, and if this is to be done successfully it must include assessing the impact of the resources that are being provided. The UoSL Strategic Plan includes goals that evaluate services based on a key performance indicator (KPI) to ensure that expenditure on electronic resources provides value for money. In the current financial climate the rising cost of acquiring scholarly resources is becoming one of the biggest drivers of change9. As more of the acquisitions budget is spent on e-books, libraries have to measure the extent of the impact of e-books to see if they are a worthwhile resource and determine whether this is money well spent10.

“It is increasingly important for every library to evaluate its own impact …”

The popular approach

Understandably, usage has become the most popular metric to assess e-book impact as it is relatively cheap and easy to collect11. This is because usage has become synonymous with access counts such as number of downloads and duration of view. However, Thelwall12 warns that certain measures of usage based on access may not reflect reader numbers, as books may be downloaded but not read. It may be convenient for libraries to measure usage as access counts but unintentional misrepresentation of this data can have a damaging and lasting effect13.

Nicholas et al14 investigate methods of assessing reading behaviour more accurately by using deep log analysis to identify repeated viewings. Although these measures indicate increased user interest in a particular item, they are essentially still based on access data which only tells one side of the story. Tenopir15 argues that we should move beyond these quantitative measures of usage and that satisfaction and outcomes should be the metrics against which we measure the impact and success of collections. By analysing the citations contained within student coursework we can start to do this.

Something different

Much research concerning the impact of electronic journals has involved citation counts but citation analysis has not yet been applied to the study of e-books. Consequently, it was not the intention of the investigation to take a representative sample of the student population at Sussex but instead an exploratory sample. This method of sampling is useful as “a way of probing relatively unexplored topics and as a route to the discovery of new ideas”16 and was appropriate for exploring a new method of measuring e-book usage involving citation analysis. Therefore a purposive sample of 240 global studies taught postgraduates (GSTPGs) was selected as these students are currently the most prolific users of EBL e-books at Sussex.

“… citation analysis has not yet been applied to the study of e-books.”

Citation analysis was performed on the 480 pieces of coursework submitted by GSTPGs during the spring term and subsequent assessment period. This was done to identify the e-books referenced in coursework, which were compared with those accessed through EBL in the same period. In addition to providing a measure of the e-books used by students, citation analysis enabled identification of the e-book collections from which cited items had been accessed through cross reference with the Library Search discovery tool.

Both the e-books accessed and the e-books cited were compared with Talis Aspire reading lists for the global studies modules taught during the Spring term to determine whether appearing on a reading list was a driver of usage. Five in-depth student interviews were also conducted in an attempt to better understand this usage.


Data collected from EBL logs and citation analysis found that e-books are not being used extensively, with 22.5% of students viewing EBL items and 11.6% citing some type of e-book in their work. Only 14% of the EBL e-books viewed were subsequently cited in coursework. Interviews suggested that this was due to e-books being used for purposes other than completing coursework, such as background reading for seminars and for signposting to other resources. For example, one participant said:

“I'll use them all in some way either by discussing them in seminars or citing them in my references.”

This was supported by another:

“When I take a book out for my essay I use it nearly always but with e-books because you can just click around you can find all sorts of things and then it maybe leads to something else… I end up using them less perhaps as an actual citation or as an actual reference in my essay just because it's more flexible.”

Citation analysis found that in the 480 pieces of work submitted by GSTPGs, a total of 12,072 items were cited including 44 e-books (see Table 1).

Table 1

Summary of citations by item type

Total no. of citations No. of citations to books or book chapters No. of citations to e-books

12,072 3,718 44

Citation analysis was able to provide information on aspects of e-book usage that are not available from transaction logs, including use of non-library resources. By following the URLs within the citations and cross checking with Library Search and Google, the source of the cited e-books could be identified (see Table 2).

Table 2

Source of cited e-books

Available from the Library Available through the web Kindle Unavailable from Library or web

13 17 2 12

Open access e-books available through the web made up a large percentage of the e-books cited (39%). This is greater than the number accessed from library collections and has implications for study skills teaching: it will become increasingly important for sessions to discuss authority and equip students with the skills needed to critically evaluate the information they are viewing online. The relevant resources held by the Library should also be demonstrated clearly in training sessions to ensure that students are not reverting to web-based e-books because they unaware of, or cannot use, appropriate library resources. Further analysis of URLs would be needed to identify the specific web collections used by students (HathiTrust17, Project Gutenberg18, etc.) and was beyond the scope of this investigation.

“… ensure that students are not reverting to web-based e-books because they unaware of, or cannot use, appropriate library resources.”

As shown in Figure 1, 13 e-books were cited from five different library collections.

Figure 1 

Number of e-books cited from different library collections

GSTPGs took a total of 32 modules during the spring term, 11 of which had reading lists on the Talis Aspire reading list system. These lists contained 891 total items but only 15 e-books (none of which were cited in coursework), indicating that they are not currently a driver of usage.

Interviews were conducted to assess the feasibility of using citation analysis as a measure of e-book usage (which largely depends on whether students are citing e-books correctly). Although four of the participants explained that they follow the guidelines provided by their school office or the Library's online tutorials, one of the students took a different approach, describing how the print equivalent was cited to ensure that anyone will be able to access the cited item, not just staff and students at Sussex who have access to that particular e-book collection. This is an interesting point and demonstrates an unexpected student consideration in using resources that are not open.


Clearly, at just 0.36% of total citations, GSTPGs are not citing e-books widely in their coursework. Similarly, only two of the cited e-books were accessed from Kindles, indicating that e-readers are not being used extensively. Although information on e-readers can be obtained from citation analysis, this is reliant on students citing correctly and using a referencing style such as Harvard, which notes whether an e-reader has been used19. For investigations solely concerning e-reader usage, a different methodology may be more appropriate.

Citation analysis found EBL to be the most popular library collection for these postgraduate students due to a high proportion of titles in their general subject area of social science20. No student cited e-books from more than one library collection in their coursework, revealing narrow usage of e-book collections by individual students in this particular department.

Data collected from the semi-structured interviews indicated that there may be a spectrum of usage containing:

  • ‘frequent users’ who read online and make use of search and annotation functions
  • ‘occasional users’ who are aware of these functions but unsure how to use them and often choose to print instead
  • ‘non-users’ who are unaware of what e-books can potentially offer and see no reason to access them.

For this small number of GSTPGs, the way in which text could be searched and annotated was the most important factor in choosing whether to use a book or an e-book. The reason given by frequent users for not accessing more e-books was simply because many of the titles they needed were unavailable electronically. Further research is needed to see whether this is the case for other student groups.

The lack of e-books on reading lists can also be explained by the fact that many academic titles are only available in print21. Together, libraries should lobby publishers in an effort to address this for the researcher. Even if students know how to use e-books, they will not be satisfied if the titles they require are unavailable. A campaign promoting the benefits of e-books could even have an adverse effect on satisfaction if students then feel that they are missing out because the titles they need are not available electronically.

“… many academic titles are only available in print. Together, libraries should lobby publishers …”

Moving forward

Although citation analysis provides insights into user behaviour and scholarly habits, it would be unreliable as a quantitative measure of e-book usage. However, instead of defining users by quantitative measures, it may be more helpful to think about usage in the qualitative sense, putting the emphasis on the user instead of the e-book. Instead of ‘power users’22, who view e-books for a certain amount of time, consider an ‘empowered user’. This would be a student who is able to access items from their reading list (or find them using a resource discovery tool), search within an e-book using inbuilt functionality, evaluate the information (annotating if appropriate) and finally make use of the information, whether for seminars or coursework. In this way we can start to move towards a more sophisticated way of assessing e-book usage, one that is not based on duration of view.

Creating these empowered users is something that all staff can work towards, from colleagues delivering skills sessions to those managing collections. Key to this will be effective marketing: identifying the collections that particular students are likely to need, followed by targeted promotion and training. Furthermore, by collaborating with tutors to embed e-book skills in teaching for specific subjects, the most relevant collections can be shown to students. Although this could lead to the narrow usage of collections, it will also help to identify those no longer providing value for money (in line with KPIs).

The results of this small-scale investigation may have limited transferability but this is not true of the methodology: citation analysis can be used by other institutions hoping to better understand the impact of their e-book collections. Quantitative measures can be used to benchmark against other institutions and fulfil business planning requirements, while qualitative success can be used at a local level to assess student satisfaction and inform decisions about collection development and teaching. In the same way that e-books complement print resources, data collected from citation analysis can complement the data gathered from access logs.

“Creating … empowered users is something that all staff can work towards”