Download Usability Issues in Metasearch Interface Design

Usability Issues in Metasearch Interface Design
Presented by the LITA Human-Computer Interface Interest Group
Renaissance Orlando Resort
Friday, June 25, 2004 2 pm to 5 pm
The program began with an introduction by Lisa Martincik (University of Iowa Libraries), Co-Chair
of the Interest Group with Paul Soderdaul. Martincik suggested we follow the interface usability
rule of computer game developers: is the interface intuitive and can it be learned quickly?
The preconference was organized around four expert panelists, each speaking for 20 minutes.
There were opportunities at the end of each presentation and at the end of the session for
questions and answers. The participating audience was made up of over 40 attendees from
academic and public libraries as well as content-providing vendors. The format and diversity of
the speakers, as well as the audience, made for a lively and provocative afternoon.
“Metasearch and the Public Portal”
Brenda Bailey-Hainer, Colorado State Library
Interface as Portal. The Colorado Virtual Library provides an interface for library information and
catalogs, public records, geospatial data, historic documents and web sites. Its complexity points
to the importance of the portal as a tool for libraries in helping the public make sense of
information. The functionality of metasearch is critical to this interface/portal. The public portal
can provide access to resources of two types: those that are relatively static – OPAC, websites,
GIS maps, digitized materials from local collections, and knowledge databases from reference
transactions. These are the resources we most often think about when developing metasearch
interfaces. But we might also incorporate interactive services into our portals such as online
classes or online homework help. Public libraries, in particular, have challenges not only in
presenting a broad range of collections and services but also in serving a community diverse in
age and language.
Usability has two faces. We tend to think most about the public usability face of a portal, but the
back end component is an important aspect of usability for librarians, content managers and web
designers. There may be a different measure of success for different users. The public just wants
it fast. They don’t care about the technical challenges of metasearch, they just expect the
behavior and speed of Google. The back end workers need flexibility and easy maintenance.
Increasingly, the capacity for statistical output (numbers) as well as facility for conducting
outcome-based evaluations (a survey instrument) may be required.
Future development. What we expect of metasearch has grown and evolved. Challenges for
metasearch engines and interfaces are speed, consistent application of standards, aggregating
and presenting results, differing record structures, and interoperability. We need to crosswalk
between different levels of specificity. And how do we integrate searching with services? How do
we include new viewpoints for interfaces – GIS, temporal, special access? For example, the teen
view varies quite a bit from the genealogists’ view and that is different from the K-12 educators’
view. In addition to the standard information gathering for assessment (comments submitted via
web site, focus groups and usability testing) Colorado has developed a pool of “cultivated
reviewers” who regularly provide feedback on the portal as it develops.
“Usability and Metasearch at Rochester”
Stanley Wilder, University of Rochester
Serial Failure. The University of Rochester is driven by its awareness of the phenomena they call
serial failure. This is the single most important access issue today. Metasearch is the most
obvious tool to reduce serial failure. Examples from Rochester demonstrate how usability testing
influenced the metasearch interface design. Other insights from usability testing were that the
library interface was too complex with two many choices to be made, including confusion over
when to use the catalog and when to use databases—these became assumptions for further
development.
Developing the Out-of-the-Box Interface. There were several problems with the out-of-the box
metasearch interface provided by Endeavor. The initial changes made by Rochester were to
reduce branding and to reduce the steps in the selection process for databases. The process for
choosing a database from a subject-oriented list is a task that does not correspond to any other
experience that students have on the web. Students need to be reassured that they are on the
right track. To respond to that need, Rochester named screens for associated tasks.
Current work. Rochester is currently working on distributing the metasearch box throughout the
web site. In course pages, the course name is important since students don’t necessarily
associate their class work with a course number. Current tab marked Articles is still problematic,
since students expect that the articles will be displayed at this point. The MyLibrary type of
functionality is a distraction that doesn’t fit into the needs of the student, requiring them to
reserve a corner of their brain for this specialized function.
Partnerships. Rochester and Endeavor have a partnership; currently they are working together
to develop a different “out of the box” product. An IT divide is developing. That divide is
characterized by the libraries who use search tool as it and the modifying library. It is imperative
that coalitions be built between libraries together and with vendors.
Questions
Q. How do you choose databases for the metasearch subject areas?
A. This is determined to some extent by what you CAN you link to and what databases can
provide you with results quickly. Broad subject coverage is good, as is provision of full text
quickly.
Q. Were reference librarians resistant to the metasearch interface?
A. Rochester went from strong opposition to support from public services librarians. Wilder
described how they show it off now. Their involvement in the design was critical to this buy in.
Rochester sees it’s top audience as undergraduates and their top task, to find articles.
“Usability Issues in Metasearch Interface Design: perspective of an information provider”
Oliver Pesche, Chief Strategist, EBSCO
Current challenges for vendors. What does metasearching mean for vendors? Practically,
there is an increase on system load. Although serendipitous content may be discovered for end
users, vendors have gone from supporting one user on one database at a time to supporting 10
users on 20 databases all at the same time. This may be possible for the larger information
provider but not sustainable for the small database provider. The library gets to decide what
databases get searched and what is displayed from those databases. Faster searches get
displayed first, but is their ranking consistent?
Relevancy is ranked on a number of elements, including, frequency of words within a single
document, frequency of words relative to the document and to the database as a whole. Other
databases may take into account the fields keywords appear in and factors such as proximity. But
relevancy between databases is not necessarily comparable, creating problems when results of
multiple databases are presented together.
How standards can help. Standards are important in addressing problems of presentation,
system load and usage statistics. How can vendors maintain the integrity of presentation in order
to comply with copyright and licensing agreements? The vendor wants and needs to retain some
control over the user experience and presentation to users. Usage statistics present another
challenge. Can you isolate metasearch activity so that it doesn’t inflate your usage statistics?
They need to be able to isolate metasearch traffic. If vendor knows metasearch is coming in, they
can get rid of certain overhead of computing resources, like authentication or setting up of user
profile. They can tailor responses to the specific needs of the metasearch users. Finally, what
direction should investments take to enhance content for users?
Questions
Q. Where will work on classification schemes go if the market is not there for specialized subject
schemes and functions?
A. Librarians need to speak out for that kind of valued-added data classification.
“Getting to Know U”
Erza Schwartz, President, Art & Tech Inc. – The User Interface Practice
Balancing differing needs. The challenge for vendors is in designing an interface to satisfy the
needs of many clients: management, marketing, finance and developers. Librarians are not the
end user.
User needs are evolving to include new requirements for functionality: portability, direct
manipulation of results, sharing capability, abilities to organize and present information in different
ways. These requirements present a conundrum for vendors that need to balance the provision of
“just in case” functionality but avoid an interface that becomes too complicated for end users with
too many buttons. In designing the user interface, we need to think broadly and balance these
concepts.
User mental models. What is the mental model for searching? There are new search engines
all the time. Do students know what metasearch means or do they care how it works? They just
want to move on. They don’t want to understand the search; they just want the results. We as
librarians want to understand the process. An agricultural metaphor helps us to understanding the
complexity of the mental model for searching: If you are a magnet, finding the needle in the
haystack is not hard; what is hard is finding the particular piece of hay in the stack that you want.
If it is there at all how do you distinguish it from the others?
Barriers to searching. Barriers make it hard to get the user from here to there. Permission for
access is denied, or the incorrect keyword is used. It’s hard to describe what you want. Dealing
with search results presents other barriers. Results may look promising, but the student wonders
about the authenticity and value of the results. Is this the real thing? Perhaps it is valuable but
needs massaging or cleanup. Perhaps it needs to be shared. Vendors are trying to build to
interact that satisfies the needs of everyone. To solve that is challenge, they need to have real
people use the system.
Final question and answer session
Q. How much of the burden of designing the user interface should be taken on by libraries and
how much from the vendor?
A. Librarians need to be involved because they are the intermediaries; vendors are trying to
anticipate the needs of librarians and end-users.
We are in a transitional time. Eventually, we as libraries will be able to give clear signals to
vendors as to what we need. Standards are the tool for overcoming challenges.
Q. If metasearch engine skims results, you run the risk of losing those 51-100 records that where
the best one might have been found?
Q. Can relevancy ranking be standardized?
A. That information may be proprietary. How to normalize “bests” against the others? How to
compare such various databases like WorldCat, Google, and EBSCO? Index the database as a
single thing; return results that are a amalgam that analyzes the results in a value-added way.
Need to meet the student at where they are and contextualize the search for students.
Q. Where does the OPAC fit into that? Will metasearching make cataloging go away?
A. The role of the library in standardizing metadata, particularly for resources like institutional
repositories, is critical. Provide better access through mapping of subject heading and semantic
indexing. How can we get our print collections back in front of students?
Q. What are some ideas for getting public service librarians on board with metasearch?
A. Get reference librarians to watch usability testing; they need to see this in a context where they
are not in a position to “help”. Have public services staff find attempt to find articles on their web
site in fields with which they are unfamiliar in order to experience what students may experience.
Notes provided by:
Nancy Turner
Electronic Resources Librarian
Syracuse University Library