Download To: Victoria Pifalo, Chair Electronic Resources Quadrant

To:
Victoria Pifalo, Chair
Electronic Resources Quadrant
From: The Website Evaluation Task Force
Deborah Blecic (Convener)
Martin Brennan
Courtney Greene
Ellen Starkman
Mircea Stefancu
Date: May 30, 2003
Re:
Report
Enclosed is the report of the Website Evaluation Task Force about ongoing and needed
studies for the evaluation of the Library’s web site.
Web Site Evaluation Task Force Report: Ongoing and Needed Studies for the
Evaluation of the Library’s Web Site
Ongoing Studies
The Website Evaluation Task Force surveyed the Library faculty and Staff via an e-mail
posting to the lib-staff listserv, asking the following:
“The Electronic Resources Quadrant has appointed a Web Site Evaluation Task
Force. The initial charge of the task force is to conduct an inventory of what
projects are currently underway in the Library that in any way involve evaluation
of any aspect of use of the Library Web Site. Projects may include usability and
use studies of the website, portal, UICCAT/Voyager, electronic reserves, digital
library, electronic resources (journals, databases, books) and any other aspect of
the web site. Methods of study may include but are not limited to Webalizer
statistics, transaction logs, vendor supplied statistics, and patron surveys or
interviews.
If you are involved in a such a project, either as research or library work, please
send a brief description of the project to Deb Blecic ([email protected]) by
Monday, April 21, 2003. If you are working with others, please include the
names of all who are involved in the project. Thank you for your attention to
this. “
The Task Force received six replies reporting seven ongoing projects:
1) Courtney Greene is planning a usability study for Fall 2003. This will build on a
previous study conducted in Spring 2001 and will be a controlled study of the new
website. Eight participants will be asked to complete a series of twenty tasks designed to
test the design and organization of the UIC Library Web Site.
2) Ruth Holst reported that UIC will be participating in NLM’s upcoming Internet
Performance Evaluation which will involve a vendor downloading the UIC Library web
page once per hour on a 24/7 basis for a period of time. This will slightly inflate statistics
during this time period.
3) Peter Hepburn, Priscilla McGlaun, Ana Ortiz, Wenona Rhodes, and Nina Williams are
using Webalizer statistics to aid in reviewing and revising the Daley Circulation
Department web pages, trying to integrate lesser used pages into other pages and remove
redundant pages.
4) Krystal Lewis, Kavita Mundle, and Mircea Stefancu are evaluating the ways in which
the UIC user community accesses e-journals. They will be using webalizer statistics and
exproxy logs in this research.
5) Steve Brantley, Krystal Lewis, and Annie Armstrong have completed a usability study
of the MyLibrary Portal. The study involved 8 participants performing nineteen tasks.
The results will be presented at the 2003 ALA conference in Toronto and are being
written up for publication.
6) Sandra DeGroote is using web site statistics to study if the use of help guides has
increased now that new links have been added to the LHS Gateway.
7) Sandra DeGroote, Jo Dorsch, Carol Scherrer, and Scott Collard are comparing
statistics generated from the Digital Services web pages with the number of questions
actually asked.
There are numerous studies underway that involve vendor supplied use statistics but not
the library web site. These studies are not reported here.
Logs Being Collected
The Task Force also determined which statistics/logs were currently being collected and
archived in the Library. These logs might serve as data for future projects.
Currently Collected:
Webalizer Statistics, archived in tigger (/usr/local/etc/httpd/htdocs/depts/lib/webalizer/)
• Current 12 months Stats are available online at:
http://www.uic.edu/depts/lib/webalizer/ (which includes summary stats and ability
to “View All URL’s”)
• Past month Stat reports are available from April, 2000 in the same directory and
named “usage_yyyymm.html”—eg, to display the 12 month report from April,
2000 use the url: http://www.uic.edu/depts/lib/webalizer/usage_200004.html
EZProxy Logs, archived currently in outreach (/WebSTAR Server Suite 4.2/uicstats/)
• See http://outreach.lib.uic.edu/uicstats/ for inventory and summary reports
• Daily logs are available for analysis for most days since January 2003
• January 31 to March 12, 2001 reports are archived in tigger, see
http://www.uic.edu/depts/lib/stats/webproxy/
UICCAT Apache Logs, archived currently in outreach (see above)
• See http://outreach.lib.uic.edu/uicstats/ for inventory and summary reports
• Monthly logs are available for analysis since December 2002
qUICsearch - Webfeat Usage Tracking (http://66.167.20.178/newwut/)
• Activity by Day, Hour of Day, Database, Database and Day
Those that might be helpful:
UICCAT logs – Ellen Starkman is looking into what logs are available to us given that
much is centralized at ILCSO (ie, OPAC Log Export and Bib Usage Log Export
Programs)
Suggestions for areas meriting further research or benefiting from formalized evaluation
Based on discussions at Task Force Meetings, we make the following recommendations
for future projects:
1) A formal evaluation should be conducted with the next redesign of the Library Web
Site, including both a pre- and post-redesign assessment. Faculty currently involved in
web site usability studies should be consulted regarding design and implementation.
2) All of the logs currently collected, as well as future UICCAT logs, offer research
possibilities. The Task Force recommends that archiving continue and that faculty be
made aware of the existence of the logs for research purposes.
3) The ARL New Measures Project E-Metrics Initiative has asked participating libraries
to collect the following statistic during the beta test : % of library visits that are remote.
This involves separating out remote use of the library website and catalog from in-house
use. It is not known at this time is this will become a permanent part of the ARL
statistics, but if it does we will need to find a way to collect it. The statistic definition
says that multiple web pages and images may be used by a user in a single visit, so the
reporting of this statistic would involve software that would track when a visit by a user
begins and ends.