Abstract
The University of Massachusetts Amherst Libraries recently conducted usability studies on EBSCO’s Discovery search platform, which is to be the default search tool on the UMass Amherst Libraries’ website beginning on July 2017. During the spring of 2017, Information Resources Management utilized surveys, focus groups, and hands-on testing of students and faculty to assess how library patrons interacted with the new discovery services (EDS) and other related library services. The following report documents this usability study, findings discovered, and recommendations hitherto. The following recommendations are broken into three categories: dataload (e.g. bibliographic records), design, and library services.
Goal/Question
The University of Massachusetts at Amherst libraries intention for this usability study was to identify ways to enhance the interface and the way in which search results were displayed so that the new search tool, Discovery Search, was as easy to use as possible for the majority of our community.
Methods
The Usability Team set out to explore various usability testing methods, theories, and procedures using best current practices. Working with the goal of running a usability study before the July 2017 rollout of EDS, it was necessary to plan and administer our study during the spring 2017 semester. A literature review of other academic libraries’ usability testing of EDS provided informative and instructive guidance on how best to design our testing schema. Common themes that have arisen during other academic libraries’ usability studies were: issues accessing full text versions of e-resources, poor searching practices exhibited by 1st and 2nd year students, inadequate ability using limiters and expanders, patrons rarely looking beyond the first page of results, and confusion about “library-ese” terminology (Woods, 2015; Clark et al., 2016; Cassidy et al., 2014; Lawrence, 2015).
Usability study methodology most often used by other academic libraries included: use of surveys, use of focus groups, and traditional usability testing (i.e. hands-on testing of the library website). We decided to incorporate these three methods into our usability study. By using a three pronged approach; we would gather data from observing actual usage of the EDS platform; get follow-up reactions from the surveys; and hear feedback during the focus groups.
Statistics/Data
Sample size: 70 students and faculty
Dates: 01/01/2017 – 02/02/2017
Campus/College: The University of Massachusetts at Amherst
Contact: Kathleen Berry, Discovery and Resource Management Services Coordinator kgberry(@)umass.edu
Methods Used: Focus groups, hands on testing, survey
Recommendation
Dataload Related Issues
- EDS search results for e-books show all e-books held by the 5 Colleges, but it is not always clear which ones are accessible from UMass Libraries. This creates confusion and frustration with patrons.
Resolution: Work with EBSCO to create an all-encompassing UMass only/UMass E-Resources limiter/subsection in EDS listings of library locations.
Resolution: Review and overhaul existing bibliographic records in Aleph and other relative datasets so UMass resources are more easily distinguishable.
Resolution: Recommend that Five Colleges Discovery/Aleph Committees review the number of clicks it takes to get from Discovery to an e-book, and see if that number can be reduced.
Resolution: Configure UMass only/UMass E-Resources limiter so UMass content displays at the top of search results. - UMass e-books should be ranked at the top of relevance results in EDS, as of now, they are not.
Resolution: Editing of bibliographic records to accurately reflect what is available via UMass Amherst Libraries.
Resolution: Work with EBSCO to customize how EDS pulls records from the shared 5 College Catalog.
Resolution: See 1 d. - Online videos available via UMass are not always obviously accessible – patrons have issues identifying what resources are available via online access (e.g. Kanopy, UM Internet Films on Demand, Alexander St. Press, Gale videos, etc.).
Resolution: See Recommendation 1a.
Resolution: Have EBSCO configure limiters to display UMass content more prominently. - Use of video limiter can skew results showing that videos are not available online.
Resolution: Work with EBSCO to create an all-encompassing UMass only/UMass E-Resources limiter/subsection in EDS listings of library locations. - Using a location (e.g. Du Bois only) limiter for videos does always not show all available video resources, if any.
Resolution: See Recommendation 1a and 2b. - Redundant records and records for items are not accessible via UMass.
Resolution: Revise the way our MARC content is displayed on EDS. By reimagining the way our Aleph MARC content is sent to and used by EDS will prioritize UMass content and reduce duplication of inaccessible Five Collage content.
Design Related Issues
- The New Search button is in a poor location (upper left corner). Users do not see it and their subsequent searches are skewed because limiters/expanders and other sorting preferences are still enabled from the prior search.
Resolution: Existing Clear button will be reengineered into a New Search button that will redirect a patron to the advanced search page, which also clears any limiters/expanders set during prior searches, giving the patron a fresh start on their new search. - SFX does not always show when a digital item (e.g. online video) is available, which creates confusion.
Resolution: Either eliminate SFX from EDS display, or modify records so SFX displays if and when a digital resource is available.
Resolution: Customize links on EDS to enable ease of access for patrons. - Clickable tabs to access items are not easily seen by many patrons, (PDF Full Text, Connect to Resource, Full Text at IEEE Xplore, JSTOR Full Text, etc.). Pair down the number of tabs that link users to various resources to increase accessibility.
Resolution: Research what other institutions have done, consult with EBSCO about how to customize/design a single link for full text access.
Resolution: Include instruction on how and what tabs provide access to full text versions of e-resources. - Create a custom ILL tab.
Resolution: This is a customizable tab that will be in place by the July 2017 rollout of EDS. - Redesign the Ask Us tab so it has more visibility and usage.
Resolution: Have quick links to research resources, not just an email link on left handed menu.
Resolution: Communicate from the main page that the Ask Us tab has a chat function with a real live person.
Resolution: Address in next round of usability testing. - Less dropdown menus on the library homepage.
Resolution: It was mentioned in focus groups that students rarely use them, or are willing to explore what is contained within them. Consider other options that can have the same functionality and deliver comparable service.
Library Services Issues
- Promote the library services more.
Resolution: Create more mouse hover over tutorials for various features on the library homepage.
Resolution: Engage in more outreach about library instruction like: “Demystifying Library Research”, how to customize course guides, and easier access for “how to” videos. - There is a need for EDS information literacy skills for UMass Library staff, faculty, and students (i.e. many patrons do not know how to use EDS’ many facets, limiters, Advanced Search/Boolean, and other related capabilities.
Resolution: Library staff training scheduled to take place in August 2017 before the start of the semester. - Many 1st and 2nd year students have trouble identifying what type of resource they are looking at, as well as understanding if the resource is available to them via UMass Libraries.
Resolution: This can be addressed with more in-depth information literacy instruction. - There is a wish for increased collaboration with subject liaison librarians.
Resolution: Consider different outreach approaches to promote more cooperation with faculty. - Need for more frequent and in-depth information literacy instruction, particularly for undergraduates.
Resolution: Assessment of current approaches to library instruction to look for more opportunities.
Action Taken
The Discovery and Resource Management Systems team worked to implement the recommendations listed above over the summer of 2017. The enhancements were completed for the beginning of the fall semester. The services recommendations were passed onto the appropriate departments of the library for consideration.
References
Blakiston, R. (2015). Usability Testing: A Practical Guide for Librarians, No. 11. London, UK: Rowman and Littlefield, i-131.
Calvert, K. (2015). Maximizing Academic Library Collections: Measuring Changes in Use Patterns Owing to EBSCO Discovery Service, College and Research Libraries, 81-99. Available at: http://dx.doi:10.5860/crl.76.1.81
Cassidy, E. D., Jones G., Mc Main, L., Shen, L., Vieira, S. (2014). Student Searching with EBSCO Discovery: A Usability Study, Journal of Electronic Resources Librarianship, 26:1, 17-35. Available at: http://dx.doi.org/10.1080/1941126X.2014.877331
Clark, A. R., Erdmann, A., Ferguson, J., Gambrell, K., Shaw-Munderback, J. (2016). Taking Action on Usability Testing Findings: Simmons College Library Case Study, The Serials Librarian, 71:3-4, 186-196. Available at: http://dx.doi.org/10.1080/1941126X.2014.877331
Krug, S. (2009). Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems, Berkeley, CA, i-163.
Lawrence, K. (2015). Today’s college students: Skimmers, scanners and efficiency-seekers, Information Services and Use, 35, 89-93. Available at: http://dx. doi.org/10.3233/ISU-150765
What is EBSCO Discovery Service? (2012). Available at: https://www.latech.edu/library/tutorials/others/eds2.pdf
Wood, J. (2015). Discover: Survey, Usability Testing and Focus Group Report, The University of Liverpool Library, 1-95. Available at: http://www.liv.ac.uk/library