New Discover Usability Testing: Bento-Style

Abstract

The Smith College Libraries conducted usability testing in 2016 during local development of a new interface and “bento box” style of display for search results from EBSCO Discovery Service (EDS).

Research and usability tests were conducted by a team made up of staff members from the Smith College Libraries Web Coordination Team and Discovery & Access and Teaching, Learning & Research departments. Planning for the project was informed by results of EBSCO user experience studies.

The Bento group met to review usability test results and discuss changes — in content, functionality, design, and language — after each successive round of testing. The “New Discover” was launched on the Smith Libraries website in January, 2017.

Goal/Question

How do users enter searches in Discover?
Do users know enough to know that each of the resource types in a bento box-style display serve different purposes?
Are our data stores differentiated enough? Will we end up with the same results in different parts of the bento display?
How good/bad are each system’s search engines? Will the steamed turnips in one part of the bento display make the sushi in the other unappetizing (as it were)?

Methods

Analysis of EDS search logs (July 1, 2015-June 30, 2016)
Moderated usability testing including:
Scenario-based questions – complete specific tasks;
Discussion – features, content, terminology, presentation of the citations; “product value” questions
Staff sandbox testing
Beta release feedback

Statistics/Data

Sample size: 4 rounds of testing, 20 tests
(16 undergraduates, 1 Ada Comstock non-traditional aged undergraduate student, 2 faculty members)

Approximately 189,000 EDS (Discover) searches on approximately 5,000 search terms between July 1, 2015-June 30, 2016

Dates: 06/01/2016 – 12/01/2016

Campus/College: Smith College

Contact: Rob O’Connell, Directory of Discovery & Access, Smith College Libraries

Files Used

Approaches to Search: Bento vs Tabbed

Outcome

Discover Search Logs: Key Findings

Overwhelming majority of searches were for a known item
Significant number of search terms were VERY Broad — whole disciplines (history, psychology, chemistry) and very general topics (therapy, feminism, women, Africa)
Most “subject” (as opposed to known-item searches) were one word or a short phrase expressing the concept

Key Usability Test Findings

Proof of the Bento+ concept
Contain the discovery and access within Bento+
Limit citation information to essentials
Limit to “availability at Smith”
Enable sort by date option
Library-ese (jargon) remains a problem

Results

User testing through iterations of system development allowed the Smith College team to design an optimized discovery experience interface. Expanded functionality makes it easier for users to scan search results, connect to online content, and refine searches. New Discover provides access to millions of resources in the library catalog, journal article databases, and local primary source collections.

Recommendation

Continue to study terminology, functionality, and content types that are problematical. Consider “guerrilla testing” for terminology. Conduct additional user studies prior to launch of new developments.