Download Emerging factors in user evaluation of the World Wide Web

Information & Management 38 (2001) 373±384
Emerging factors in user evaluation of the World Wide Web
John D'Ambraa,*, Ronald E. Riceb,1
a
b
School of Information Systems, University of New South Wales, Sydney 2052, Australia
School of Communication, Information and Library Studies, Rutgers University, 4 Huntington St., New Brunswick, NJ 08901-1071, USA
Received 1 December 1999; received in revised form 14 August 2000; accepted 2 September 2000
Abstract
This study develops an integrative model and conceptually-based scales for evaluating the extent to which Web services
satisfy information needs that arise outside the traditional organizational/work domain. Three streams of literature are
considered: usage of the Web, user satisfaction with the Web, and individual performance and the impact of information
technology. Based on this literature, as well as focus groups and pilot surveys, questionnaire items were developed and
analyzed across three surveys. Predictors of performance included greater weekly usage, ®nding information on hobbies and
interests, ability to ®nd information on the Web that is current, reduced shopping cost and travel, ®nding otherwise dif®cult-tolocate information, and fun/entertainment. # 2001 Elsevier Science B.V. All rights reserved.
Keywords: User evaluation; World Wide Web; Task±technology ®t; Scale development
1. Introduction
World Wide Web (Web) services for satisfying
personal day-to-day needs are growing at a rate that
will have substantial in¯uence in the larger information infrastructure. Yet it is unclear how the Web will
realize that in¯uence. In parallel with the unprecedented rate of growth in the Web presence by businesses, institutions and information providers is the
rapid growth of access and usage of the Internet at
home, in school, and while traveling [4]. The Web is
now competing with traditional business and information services by providing an alternative way for
*
Corresponding author. Tel.: ‡61-2-9385-4854;
fax: ‡61-2-9662-4061.
E-mail addresses: [email protected] (J. D'Ambra),
[email protected] (R.E. Rice).
1
Tel.: ‡1-732-932-7381; fax: ‡1-732-932-6916.
individuals to satisfy their needs, whether work- or
business-related or not. The Web is a computermediated environment that allows consumers to overcome some of the problems/barriers of traditional
media: accessibility, bottlenecks, interaction and identi®cation [24]. It is not clear what level of success is
being experienced by users in utilizing the Web for
information-based activities and how user behavior
has changed and what bene®ts, if any, have been
derived from the change [14].
Web-based consumer services are generally perceived as being successful, but there has been little
evaluation of how well the Web meets its users'
primary information requirements. There has been
much research performed on the level of usage, satisfaction and effectiveness of IS within organizations
(see [16]). The Web, however, is an IS that has a vast
number of users who are not con®ned by an organizational context and for whom use of the Web is optional
[27].
0378-7206/01/$ ± see front matter # 2001 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 8 - 7 2 0 6 ( 0 0 ) 0 0 0 7 7 - X
374
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
This vast and rapidly growing number of new users
has lead to the provision of a multitude of new
possibilities in providing IS and services, implying
a number of new aspects to consider when designing,
and designing with, those systems and services. User
pro®les may be less homogenous than earlier and
therefore their needs are harder to de®ne. Providers
aiming to build successful Web systems must attract
voluntary users to visit and revisit the site. These
requirements make the users' perception of the quality
of the Web system critical for the success of the system
[19].
2. Review of relevant literature
2.1. Web usage
Several important issues emerge from the literature on the Web from a user perspective. It may be
goal-directed or experiential, the concept of ¯ow
may in¯uence usage and evaluation of the Web, and
the Web is used primarily as a communication
medium.
Much of the study investigating user perceptions
and experience of the Web has been undertaken from a
user perspective. Hoffman and Novak [15], in addressing the role of marketing in a hypermedia computermediated environment, propose a broad structural
model of consumer navigation of the Web, including:
the nature of the task (Web usage is both experiential,
e.g. net-sur®ng and goal-directed, on-line shopping),
the concept of ¯ow, and Web usage. In an investigation
of the effect of the individual characteristics of playfulness on Web usage, Atkinson and Kydd [3] also
distinguished between intrinsic and extrinsic factors
affecting Web use with differential usage for entertainment or research purposes. In explaining the
experiential nature of Web usage, Hoffman and Novak
replace the notion of play with the theory of `¯ow'.
Flow experiences are those optimal and enjoyable
experiences in which we feel ``in control of our
actions, masters of our own fate. . .we feel a sense
of exhilaration, a deep sense of enjoyment'' [5] (p. 4).
The concept of ¯ow has been adapted to the human±
computer interaction experience [22]. In that context,
¯ow incorporates the extent to which: (1) the user
perceives a sense of control over the computer inter-
action; (2) the user perceives that his or her attention is
focused on the interaction; (3) the user's curiosity is
aroused during the interaction and (4) the user ®nds
the interaction intrinsically interesting.
Eighmey [10] considered two questions about the
bene®ts delivered by commercial Web sites, as well as
the approach that delivers the greatest bene®t. As part
of that study, a scale of 44 items among six thematic
areas (marketing perceptions, entertainment value,
informational value, ease of use, credibility, and interactivity) was used to measure perceived bene®ts from
visiting 28 commercial Web sites. Three factors
emerged: (1) playfulness; (2) clarity of purpose and
(3) timeliness and the approach to presenting information. Overall, Eighmey found that Web users are
assisted by information placed in an enjoyable context. Liu and Arnett [20] concluded that four factors
are critical to Web site success: (1) information and
service quality; (2) system use; (3) playfulness and (4)
system design quality.
The `Homenet' study [17] found that electronic
mail (e-mail) use was more popular and more stable
than use of the Web. Thus e-mail drove continued use
of the Internet overall. In the study participants used email in at least 49% of their Internet sessions, but Web
used only 38% of them. An investigation of usage
patterns of the Internet in Singapore [26] found that
messaging and browsing are performed more frequently than downloading or purchasing activities.
2.2. Influence on usage, task±technology fit and the
nature of usage
Two traditional models help to provide the basis for
identifying possible in¯uences on Web usage. The
theory of planned behavior and a synthesis of usage
and task±technology ®t (TTF) models.
Klobas [18] tested the effectiveness of three models
of information resource use in explaining the use of
the Internet: information use [1], the technology
assessment model [6] and the theory of planned
behavior [2]. Klobas found that the theory of planned
behavior best explained the use of the Internet, indicating that information resource use is motivated by
similar factors as those in¯uencing other human behaviors. This is an interesting ®nding, as, unlike IS used
at work, the use of the Internet at non-work locations is
not mandatory and therefore may be in¯uenced by a
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
wider range of social and motivational factors than are
typically studied in organizational settings.
The theory of planned behavior is a model designed
to explain speci®c human behavior. It has been used to
explain why people participate in different recreational activities and health related behaviors. The most
direct in¯uence on behavior is intention to perform an
activity (`behavioral intention'). In turn, this is in¯uenced by: (1) attitude to outcomes Ð the person's
attitudes to the results of performing the action; (2)
social pressure Ð the in¯uences resulting from the
individual's environment and (3) perceived behavioral
control Ð the extent to which a person believes he or
she has control over his or her behavior. The theory
of planned behavior is the foundation of models
examining people's intentions to utilize organizational
systems [6].
Various measures of the three general categories of
in¯uence on technology use are possible.
1. One measure of attitude toward using a specific
information technology, such as the Web, would be
the extent to which an individual believes that using
the Web will improve his or her performance.
Another would be perceptions of how much using
the Web is interesting, enjoyable, or a productive
experience.
2. Social influence may be measured by identifying
sources from which potential users may experience
pressure to use the Web (possibly weighting
beliefs according to the individual's motivation
to comply with these pressures). Pressures to use
the Web may come from peers, the workplace and
the media.
3. Perceived behavioral control may be measured by
identifying potential costs of, and barriers to,
Web usage, such as the cost of using the Web,
accessibility to a computer with an Internet connection, understanding the Internet, legal implications, fear of being monitored and poor response
time.
However, one important construct is not included in
this model: the tasks for which the Web is used.
Insofar as usage of the Web is optional, the decision
to use it may be based on an individual's expectation
that the Web may have some impact on the task or that
using the Web to solve an information-based task may
375
be a satisfying experience. There must be some degree
of ®t between the task and the technology that has been
chosen to accomplish it. In a response to unresolved
issues in studying user evaluations of systems, Goodhue and Thompson [13] proposed a user-speci®c
construct: TTF. The essence of this model is the
assertion that IS will have a positive impact on individual performance if the system is used, and it is a
good ®t with the tasks it supports. The authors derived
this model by analyzing the limitations of two streams
of research that have proposed models of technology
use: utilization and TTF. While each of these perspectives provides insight into the impact of IS on performance, each alone has some important limitations.
First, usage may be more a function of how jobs or
tasks are designed than of the quality or usefulness of
systems, or of the attitudes of users toward using them.
Mathiesona and Keil [23] similarly argue that TTF
issues may override interface design and system
accessibility issues. Second, DeLone and McLean's
[7] study investigating the many aspects of IS success
lends support to the TTF model, but also concluded
that there is not one, but many, measures of success,
including: system quality, information quality, use,
user satisfaction, individual impact and organizational
impact. Other studies show that user satisfaction may
be a more powerful in¯uence on performance outcomes than technology utilization [11,12]. Dishawa
and Strong [8] also emphasize the value of integrating
multiple models, such as the technology acceptance
model, with its emphasis on attitudes developed
through perceptions of usefulness and ease of use,
and the TTF model, with its emphasis on matching IS
functionality with user needs. Thus, the ®t model can
bene®t from the addition of this richer understanding
of the nature and forms of utilization.
2.3. An integration
Our research argues that the focus of the Goodhue
and Thompson [13] model matches the environment
of Web usage. This is a technology for which use is
optional; at the same time usage is dependent on user
perceptions of the impact of the Web on the task, as
well as a host of social and contextual factors. The
following integration of the prior constructs suggests a
model for explaining usage and evaluation of the Web
in voluntary situations, and thus identi®es potential
376
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
concepts and scales that should be used to study Web
usage.
Tasks are broadly de®ned as the actions carried out
by individuals in turning inputs to outputs in order to
satisfy their information needs. Characteristics of the
individual (knowledge, expertise, motivation) could
affect how easily and well he or she will utilize the
Web. Technologies are tools (hardware, software and
data) used by individuals in carrying out their tasks,
and the technology's attributes (accessibility, response
time) can affect usage. TTF is the correspondence
between task requirements, individual abilities, and
the functionality of the Web. Social norms are the
external factors that in¯uence use of the Web (peer
pressure, others using the Web at their workplace,
using the Web as an educational tool, or media coverage). Control factors may limit the use of the Web,
such as cost, accessibility of hardware and software,
local, institutional or legal restrictions, and concerns
over monitoring by others of usage and sites visited.
Utilization is the behavior employed in completing
tasks (®nding information, entertainment, extrinsic or
intrinsic). Performance impact involves accomplishing a portfolio of tasks by an individual. High performance implies a high level of TTF, and satisfaction
with the IS [12] (here, the Web). Feedback through
actual use is of course an important aspect of the
model, as it may change users' perceptions of possible
consequences, and thus both their future utilization
and performance outcomes [12].
This model leads to the following research questions.
Q1. What constitutes valid and reliable scales measuring user evaluation of the Web for non-work based
activities?
Q2. How are such scales related to the performance
impact of Web usage for non-work based activities?
3. Method
The derivation of a scale measuring user evaluation
of the Web must be ecologically valid, incorporate
diverse concepts and identify underlying dimensions
[9]. To that end, we developed the ®nal survey items
based on
an initial set of focus groups;
a small pilot survey;
a first large sample survey;
a second large sample survey;
a third sample survey using the same survey as the
second sample.
We report results from the combined second and
third samples. Details from the other samples are
available from the second author.
3.1. Focus groups
Focus groups provide ecological validity and
insightfulness in identifying salient attitudes or perceptions [21]. The focus group method involves bringing together one or more groups of subjects to discuss
an issue in the presence of a moderator. To this end,
four focus groups were convened. The participants of
the groups were undergraduate and postgraduate students of a major Australian university. This kind of
sample is reasonable for this study, as a study at that
time period reported that 42% of Internet users were
18±24-year-old and 56% of people with a bachelor's
degree used the Internet [4]. The gender distribution
across the four focus groups was 70% male (18 of 26).
The focus groups were facilitated by the researcher
and were managed in accordance with standard procedures [21]. Questions asked about the in¯uences on
Web usage summarized above, and dimensions of the
Goodhue and Thompson TTF scale that are relevant to
Web usage, including: quality (Is it current enough to
meet needs? Does is maintain necessary data?); locatability (Is it easy to determine what data is available?
Is it easy to interpret?); compatibility (data from
different sources can be consolidated or compared
without inconsistencies); system reliability (system
access is dependable and consistent) and ease of
use/training (How easy is it to do what you want to
do using the system hardware and software?).
Questions speci®cally emphasized non-work based
tasks.
The transcripts of the focus groups' comments were
analyzed using both ex post facto and a priori content
categories. The 70 items that emerged from the content analysis demonstrated that the Web usage and
TTF dimensions were of interest to the participants in
the focus groups. Duplicate items were dropped, and
statements were reworded to re¯ect common issues.
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
377
3.2. Pilot survey
3.5. Third sample survey
Based on the items from the focus groups, a pilot
survey was constructed. The following items were
added, to include other concepts from the task±
technology to performance chain model: compatibility, system reliability, ease of use/training and
individual performance impact. The following controls were also included: Web usage, location where
the Web was accessed, respondent's sex, number of
years using a PC, overall expertise with PCs/applications/Internet/Web and respondent's age. The draft
instrument was pre-tested using 11 master's students
at a major American north eastern university. Based
on comments and responses to the draft, further
duplicate items were dropped, `double barreled'
questions were modi®ed and ambiguities were
resolved.
To assess the robustness of the results, we administered the survey to a smaller section, with different
students, of the initial large ®rst-year course. We
3.3. The first sample survey
We administered the revised survey to a large freshman course; this was part of an extra credit opportunity with 100% of the students (295) attending that
class completing the survey.
3.4. The second sample survey
Based on analyses of the ®rst sample survey data,
we again revised the survey. To address the problem
with disappointing reliabilities of the 52 TTF items, a
second version of the scale was developed. It only
included items that loaded highly on principal components, and were involved in signi®cant correlations
with performance.
In an attempt to improve the validity and reliability
of the items, we generated additional items representing locatability, control, and anonymity. We also
added two items to the performance scale, as well
as items re¯ecting the social in¯uence construct, to
allow for further testing of the task to performance
model. These changes resulted in a smaller survey
consisting of 50 items. We then administered the
revised survey to a large course for juniors, again
as an extra credit opportunity, using consent forms (in
all cases, we also returned to summarize the class
responses). All participated, for a ®nal sample size
of 121.
Table 1
Percentages for usage and demographic items for combined (2 and
3) samplesa
Used
Ever used the Web
Web access location
Where you live
School computer lab
School library
At work
At parents' home
At friend's place
Sex
Male
Female
97.8
68.5
86.0
79.8
34.8
65.2
74.2
28.7
71.3
Percent of people you know who use Web
0±20
21±40
41±60
61±80
81±100
1.1
10.7
16.3
36.0
35.4
Hours per week using Web
<1 h
1±2 h
2±4 h
4±6 h
>6 h
22.5
24.2
35.4
9.0
9.0
Years experience using PC-Internet
<1 Year
1±2 Years
2±4 Years
4±6 Years
>6 Years
2.8
6.7
22.5
33.7
34.3
Computing expertise
None
Novice
Somewhat familiar
Familiar
Very familiar
0
6.2
24.2
46.1
23.0
Age
<18
18±19
20±21
22±25
>25
0
11.8
61.8
17.4
9.0
a
N ˆ 178.
378
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
received responses from 57 of the 67 students. After
analyzing the third sample, and ®nding no signi®cant
differences from the second sample, we combined the
second and third samples for the combined analysis.
4. Results
Tables 1 and 2 provide descriptive statistics for the
combined samples. The respondents most strongly
agreed with the following statements:
I see that other people benefit from using the Web;
I can find information related to my hobbies and
interests on the Web; the information on the Web is
current enough to meet my needs and Web search
engines are useful in helping me to find the
information I need. The two most highly rated
performance items were I have increased my
knowledge about topics of interest to me because
of my Web use and using the Web has had a
positive impact on my ability to get things done.
Table 2
Item and scale descriptive statistics for combined (2 and 3) samplea
Number
Variable
Mean
S.D.
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
I am more likely to find specific kinds of information on the Web than from other information sources
Web search engines are useful if I do not know which sites to go to directly for specific information
The information on the Web is current enough to meet my needs
It is easier to find information I need on the Web than in a library
I can find information related to my hobbies and interests on the Web
I see that other people benefit from using the Web
Web search engines (Yahoo, Lycos, etc.) are useful in helping me find the information I need
By using the Web I can avoid going to a store
It is easy to know which Web sites to go to directly to find the information I need
Free information is not as reliable as information that you pay for
I use the Web because it is necessary for my work or classes
By using the Web I can avoid in-store sales people
The Web is useful for finding information that is difficult to locate elsewhere
I have complete control over what sites I visit on the Web
It is not easy to find very specific information on the Web
It is not easy to find very specific information on the Web
I would use the Web even if none of the people I know were using it
I have very easy physical access to a computer with Web connections
I prefer visiting sites on the Web that do not require me to identify myself
I have complete control over how I use the Web
I need to develop my skills more to use search engines on the Web better
Being anonymous on the Web is important to me
More training would make my Web use more effective
I use the Web because many of my friends do
I use the Web for entertainment
I would use the Web more if I had greater computer or Internet knowledge
Using Web is fun
The Web gives me access to information that I cannot find elsewhere
If my Web usage was monitored by the system administrator I would not visit certain sites
I use the Web because of all the attention it receives in the media
Performance
Using the Web has a positive impact on my ability to get things done
Because of my Web use, the number of people I communicate with has increased
The quality of my work has improved because of using the Web
I make better decisions because of information I get from the Web
I can accomplish things more quickly because of my Web use
I have increased my knowledge about topics of interest to me because of my Web use
Because of my Web use, I am better informed in general
3.90
1.62
4.42
4.03
4.61
4.62
4.37
2.64
2.90
2.15
3.76
2.57
4.14
3.85
2.97
4.13
1.73
4.49
4.17
3.92
3.45
3.97
3.89
2.24
4.06
3.23
4.13
3.82
3.24
1.98
1.06
0.87
0.76
1.06
0.68
0.59
0.87
1.34
1.15
1.09
1.15
1.30
0.84
1.16
1.15
0.85
0.99
0.85
0.96
1.01
1.33
1.02
1.15
1.18
1.00
1.30
0.89
1.05
1.33
1.06
3.85
3.37
3.16
2.97
3.77
3.91
3.70
1.07
1.46
1.18
1.11
1.08
1.12
1.14
44
45
46
47
48
49
50
a
Item ranges were 1 (strongly disagree) to 5 (strongly agree).
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
Table 3 shows the factors (emerging from a principal components factor analysis), loadings and resultant scale reliabilities. Usage and experience loaded
on a single factor but with low alpha of 0.51. Thus, we
chose to use just the single-item measure of number of
hours per week using the Web. The access variables
loaded on two factors: access from residences and
from school sites and friends' place. The six performance items loaded on one factor.
Table 4 shows the factors, loadings and resultant
scale reliabilities for the TTF variables. The scales
included: training (F1); interests (F2); information
(F3); shopping cost (F4); use Web to locate dif®cult
to ®nd information (F5); fun and entertainment (F6);
social in¯uence (F7); identity control (F8) and use
control (F9).
Table 3
Factor loadings (principal components analysis) and reliability of
computer use/expertise, access, and performance items for
combined (2 and 3) samples
Usage
Hours per week
Years using PC
Computing expertise
Eigenvalue
Percent variance
Alpha
Access
Where you live
School computer lab
School library
Where you work
Parents' home
Friends' place
Eigenvalue
Percent variance
Alpha
Performance
Positive impact on my ability to get things
done
Quality of my work has improved
Better decisions because of information
from Web
Accomplish things more quickly
Increased knowledge about topics of interests
Better informed
Eigenvalue
Percent variance
Alpha
School
Home
0.14
0.76
0.69
0.09
0.24
0.69
1.60
0.27
0.53
0.83
0.26
0.03
0.02
0.78
0.20
1.40
0.23
0.51
0.81
0.78
0.84
0.79
0.85
3.85
0.65
0.89
Table 5 shows the correlations among usage, access,
task technology, and performance scales. Bivariate
predictors of the mean scales in the combined sample
include: for F1 (training): less weekly usage, less
expertise; F2 (interests): greater expertise; F6 (fun):
greater weekly usage, greater expertise; F7 (social
in¯uence): less expertise and F9 (use control): more
people I know use the Web. Bivariate predictors of
performance included: greater weekly usage, F2
(interests), F3 (information), F4 (shopping cost), F5
(dif®cult information) and F6 (fun).
The ®nal regression predicting performance explained 28% of the variance (F ˆ 11:2, P < 0:0001).
Signi®cant predictors (with standardized beta coef®cients) were weekly use (0.19, P < 0:01), F5 (dif®cult
information, 0.19, P < 0:01), F6 (fun, 0.13, P < 0:1),
F4 (shopping cost, 0.17, P < 0:01), F3 (information,
0.17, P < 0:05) and F2 (interests, 0.15, P < 0:05).
However, the reliabilities of some of these scales are
low. Fig. 1 summarizes the signi®cant relationships.
5. Conclusion
0.66
0.66
0.84
1.60
0.53
0.51
0.73
379
This study has presented some perspectives and
research issues about the evaluation and impact of
the Web for non-work based information tasks. Some
consideration should be given to the nature of the
sample. The samples used in the ®nal surveys consisted entirely of college students, though the items
were developed using a variety of samples including
professionals taking master's courses. Moreover, the
tests were given to a large number of subjects. Crosscultural issues may also affect the validity [25], as the
focus groups were held in Australia while the pilot and
surveys were administered in both Australia and the
United States.
This exploratory work demonstrates that it is possible to develop preliminary scales for measuring user
evaluation of the Web, and that such scales are multidimensional. The dimensions that emerge indicate that
users of the Web need to be addressed by developers of
Web based systems if those systems are to be successful. Factors identi®ed by this research may be of
interest to other researchers and providers interested
in the evaluation of Web-based services. The reliabilities for some of these scales were low, however, so the
scales need further development.
380
Table 4
Factor loadings (principal components analysis) and reliabilities of task±technology items, combined (2 and 3) samplea
Number
Item
Factors
F1
More likely to find information on the Web
Search engines useful if do not know sites
Web information current enough to meet my needs
Easier find information Web than library
I can find information hobbies and interests
I see other people benefit from using Web
Search engines useful in helping find information
Can avoid going to a store
Easy find Web sites go to directly
Free information not reliable as information paid for
Use Web because necessary work or class
Can avoid in-store sales people
Useful finding information difficult to locate elsewhere
Complete control over sites I visit
Not easy find specific information on Web
Web provides information that is important to me
Would use Web even if no-one I knew was
Very easy access to complete world Web connection
Prefer visit sites where I do not identify
Complete control over how I use Web
Develop skills more use engines
Anonymous on Web import to me
More training make my use more effective
Use Web cause many my friends do
Use Web for entertainment
Would use Web more greater complete/information knowledge
Using Web is fun
Web access information cannot find elsewhere
Web usage monitor would not visit certain sites
Use Web cause attention receives in media
Eigenvalue
Percent variance
Cronbach's alpha reliability
a
0.08
0.05
0.00
0.07
0.04
0.08
0.00
0.06
0.08
0.04
0.02
0.03
0.06
0.05
0.45
0.17
0.18
0.08
0.01
0.19
0.81
0.07
0.81
0.13
0.01
0.71
0.08
0.06
0.04
0.17
4.5
15.1
0.74
0.09
0.17
0.12
0.35
0.70
0.83
0.30
0.05
0.04
0.06
0.20
0.15
0.44
0.06
0.01
0.30
0.15
0.44
0.08
0.21
0.02
0.02
0.05
0.22
0.14
0.05
0.03
0.01
0.04
0.13
2.7
9.1
0.61
F3
0.66
0.09
0.65
0.34
0.04
0.03
0.15
0.04
0.39
0.02
0.59
0.03
0.22
0.14
0.08
0.37
0.12
0.03
0.01
0.06
0.07
0.00
0.05
0.00
0.01
0.07
0.07
0.41
0.29
0.00
2.1
7.0
0.45
F4
0.30
0.06
0.09
0.13
0.12
0.03
0.17
0.85
0.31
0.09
0.20
0.84
0.12
0.09
0.15
0.20
0.02
0.09
0.12
0.05
0.09
0.06
0.05
0.17
0.01
0.03
0.23
0.06
0.12
0.09
1.9
6.2
0.78
F5
0.17
0.15
0.02
0.19
0.08
0.06
0.00
0.12
0.13
0.08
0.01
0.07
0.57
0.08
0.01
0.39
0.43
0.01
0.00
0.11
0.03
0.05
0.08
0.06
0.09
0.18
0.39
0.69
0.61
0.10
1.6
5.5
0.58
F6
0.14
0.01
0.10
0.17
0.16
0.05
0.35
0.07
0.14
0.08
0.10
0.04
0.21
0.09
0.13
0.21
0.26
0.40
0.07
0.04
0.20
0.02
0.01
0.29
0.85
0.09
0.64
0.00
0.08
0.08
1.4
4.8
0.61
F7
0.06
0.20
0.11
0.13
0.05
0.02
0.13
0.03
0.17
0.10
0.30
0.01
0.01
0.06
0.14
0.10
0.55
0.16
0.13
0.10
0.01
0.05
0.12
0.69
0.05
0.31
0.04
0.07
0.20
0.78
1.4
4.8
0.58
F8
0.03
0.12
0.08
0.01
0.13
0.03
0.16
0.13
0.26
0.07
0.16
0.05
0.12
0.02
0.20
0.04
0.04
0.01
0.84
0.04
0.03
0.90
0.04
0.08
0.08
0.07
0.05
0.01
0.12
0.02
1.3
4.4
0.78
F1: training; F2: interests; F3: information; F4: shopping cost; F5: difficult information; F6: fun; F7: social influence; F8: identity control; F9: use control.
F9
0.14
0.01
0.17
0.06
0.13
0.08
0.01
0.00
0.08
0.04
0.22
0.03
0.01
0.85
0.07
0.21
0.16
0.18
0.11
0.84
0.13
0.06
0.03
0.04
0.03
0.07
0.15
0.07
0.01
0.00
1.2
4.1
0.73
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
F2
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
381
Table 5
Correlations of usage, access and demographics with task±technology factors, and correlation of all variables with performance, for combined
(2 and 3) samplesa
Variables
F1
F2
Correlations with task±technology factors
Sex
0.03
People
0.12
Weekly hour
0.18*
PC experience
0.13
Expertise
0.41**
Access home
0.08
Access school
0.02
0.01
0.07
0.03
0.12
0.23*
0.13
0.11
Correlations with performance
Sex
People
Weekly hour
PC experience
Expertise
Friends
Access home
Access school
F1 training
F2 interests
F3 information
F4 shopping cost
F5 difficult information
F6 fun
F7 social influence
F8 identity control
F9 use control
0.06
0.00
0.29**
0.03
0.10
0.00
0.07
0.10
0.02
0.29**
0.31**
0.30**
0.35**
0.29**
0.10
0.13
0.17
F3
0.02
0.03
0.04
0.01
0.03
0.14
0.08
F4
0.02
0.10
0.12
0.14
0.16
0.07
0.04
F5
0.05
0.04
0.12
0.08
0.00
0.09
0.03
F6
0.08
0.11
0.36**
0.09
0.23*
0.14
0.06
F7
0.00
0.00
0.09
0.03
0.20*
0.00
0.03
F8
0.01
0.03
0.17
0.00
0.16
0.02
0.11
F9
0.00
0.20*
0.06
0.01
0.12
0.12
0.09
a
N ˆ 165.
P < 0:01.
**
P < 0:001.
*
Over a quarter of the variance in reported performance
impacts of using the Web was explained by several
task±technology factors, along with the number of
hours per week spent using the Web. Several in¯uences are of particular interest. First, greater use of the
Web appears to be directly related to both using the
Web for fun/entertainment and to performance outcomes. Thus, Web usage has both intrinsic and extrinsic purposes and both are associated with positive
performance outcomes. Further, greater expertise
increases perceived fun/entertainment and ®nding of
information related to hobbies and interest, indicating
that intrinsic purposes require technical knowledge.
Further, the inherent power of Web expertise is reinforced by the fact that greater expertise is associated
with less need for training, as well as less susceptibility to social in¯uence; thus greater expertise pro-
motes greater independence of motivation, as well as
increased use for intrinsic purposes. While social
in¯uence is diminished by greater expertise, the number of people you know who use the Web increased the
perception of one's control over use of the Web. This
implies two different kinds of social in¯uence: one
based on susceptibility to in¯uence from others
because of one's limited expertise, and one based
on increased independence and con®dence because
of a social context of fellow Web users. Note that the
issue of anonymity, or of controlling one's identity
online, while a notable social concern, seems unrelated to either in¯uences or outcomes, implying this is
a personal trait not much in¯uenced by external
factors. Also note that general personal computer
expertise, as well as the location where one accesses
the Web, have not in¯uence on TTF factors or
382
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
Fig. 1. Summary of significant relations among influences, TTF factors, and performance outcomes of Web usage.
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
performance outcomes. This implies that Web expertise is qualitatively different from computer expertise
per se, and that the virtual experience and results from
the Web are largely independent of the access location.
Concerning performance outcomes, they are in¯uenced directly by usage of the Web, implying that
some kinds of bene®ts are not dependent on a TTF.
Rather, simply using the Web more generates positive
outcomes. However, several TTF factors do themselves directly increase the performance outcomes.
Intriguingly, the two more intrinsic purposes, fun and
hobbies, mediate use and expertise, while the three
more extrinsic purposes Ð ®nding information, avoiding shopping costs, and ®nding hard to locate information Ð in¯uence performance outcomes
independently of Web usage or Web expertise. This
implies that intrinsic purposes Ð often seen as secondary or less instrumental applications of the Web Ð
require greater use and expertise in order to convert
that activity into a productive one. So `sur®ng' or
using the Web for fun can be of two types: `mindless',
infrequent, or uninformed, and thoughtful, frequent,
and knowledgeable. Finally, perceived need for training, usage due to social in¯uence, and concerns for
control of one's identity online and of control over
how one uses the Web, do not seem to affect performance outcomes. Training would seem to be relevant
to performance outcomes, but perhaps its in¯uence is
captured by the prior in¯uence of Web expertise,
mediated by intrinsic purposes. Social in¯uence
may serve personal needs for acceptance or social
conformity, but does not seem to have much to do with
actual performance outcomes. Need for control of
one's identity seems to be a more personal trait
unrelated to in¯uences or performance outcomes,
while perceived control over Web site usage appears
to be more of a sense of social ef®cacy built up by a
social context of other users, but not a real indicator of
how well one uses or applies the Web.
Overall, then, the results support a model whereby
Web usage as well as Web expertise in¯uence several
intrinsic and extrinsic TTF factors, and those factors,
along with Web usage, directly in¯uence positive
performance outcomes. The intrinsic TTF factors
seem dependent on prior usage and expertise, while
the extrinsic, instrumental factors seem to be more
general information purposes. Finally, some factors
seem to be primarily socially- or individually-oriented
383
traits unrelated to performance outcomes, and possibly related to greater susceptibility due to low levels of
prior expertise. A simpler model (and set of survey
items) would remove the factors of training, social
in¯uence, identity control and use control. Further
analysis needs to be undertaken to extend and re®ne
the relationships and the tentative model suggested in
Fig. 1. In the end, however, this study indicates that
real performance bene®ts accrue from non-work
related Web usage and intrinsic purposes, and that
TTF factors substantially mediate the in¯uence of
Web usage and Web expertise on those bene®ts.
References
[1] T.J. Allen, Managing the Flow of Technology: Technology
Transfer and the Dissemination of Technological Information
within the R&D Organization, MIT Press, Cambridge, MA,
1977.
[2] I. Ajzen, Attitudes, Personality and Behaviour, Open
University Press, Milton Keynes, UK, 1993.
[3] M. Atkinson, C. Kydd, Individual characteristics associated
with World Wide Web use: an empirical study of playfulness
and motivation, The DATA BASE for Advances in Information Systems 28 (2), 1997, pp. 53±61.
[4] Australian Bureau of Statistics, http://www.abs.gov.au/websitedbs/ (1998).
[5] M. Csikszentmihalyi, Flow: The Psychology of Optimal
Experience, Harper & Row, New York, 1990.
[6] F. Davis, R. Bagozzi, P. Warshaw, User acceptance of
computer technology: a comparison of two theoretical
models, Management Science 35 (8), 1989, pp. 982±1003.
[7] W. DeLone, E. McLean, Information systems success: the
quest for the dependent variable, Information Systems
Research 3 (1), 1992, pp. 60±95.
[8] M. Dishawa, D. Strong, Extending the technology acceptance
model with task±technology fit constructs, Information and
Management 36 (1), 1999, pp. 9±21.
[9] W. Dolla, G. Torkzadehb, Developing a multidimensional
measure of system-use in an organizational context, Information and Management 33 (4), 1998, pp. 171±185.
[10] J. Eighmey, Profiling user responses to commercial Web sites,
Journal of Advertising Research 37 (3), 1997, pp. 59±66.
[11] M. Gelderman, The relation between user satisfaction, usage
of information systems and performance, Information and
Management 34 (1), 1998, pp. 11±18.
[12] D. Goodhue, B. Klein, S. March, User evaluations of IS as
surrogates for objective performance, Information and
Management 38 (2), 2000, pp. 87±101.
[13] D. Goodhue, R. Thompson, Task±technology fit and individual performance, MIS Quarterly 19 (2), 1995, pp. 213±236.
[14] S. Hill, Technology and organisation culture: the human
imperative in integrating new technology into organisation
design, Technology in Society 10, 1988, pp. 233±253.
384
J. D'Ambra, R.E. Rice / Information & Management 38 (2001) 373±384
[15] D.L. Hoffman, T.P. Novak, Marketing in hypermedia
computer-mediated environments: conceptual foundations,
Journal of Marketing 60, 1996, pp. 50±68.
[16] IS effectiveness page, http://dmsweb.badm.sc.edu/grover/isworld/isoehom3.htm.
[17] R. Kraut, V. Lundmark, S. Kiesler, T. Mukhopadhyay, Why
people use the Internet, http://homenet.andrew.cmu.edu/progress/purpose.html (1997).
[18] J.E. Klobas, Beyond information quality: fitness for purpose
and electronic information resource use, Journal of Information Sciences 21 (2), 1995, pp. 95±114.
[19] K. Lindroos, Use quality and the World Wide Web,
Information and Software Technology 39, 1997, pp. 827±836.
[20] C. Liu, K. Arnett, Exploring the factors associated with Web
site success in the context of electronic commerce, Information and Management 38 (1), 2000, pp. 23±33.
[21] P. Lunt, S. Livingston, Rethinking the focus group in media
and communication research, Journal of Communication 46
(2), 1996, pp. 79±98.
[22] J. Martochio, J. Webster, Effects of feedback and cognitive
playfulness on performance in microcomputer software
training, Personnel Psychology 45 (3), 1992, pp. 553±578.
[23] K. Mathiesona, M. Keil, Beyond the interface: ease of use and
task/technology fit, Information and Management 34 (4),
1998, pp. 221±230.
[24] R.E. Rice, Computer-mediated communication and organiza-
tional innovation, Journal of Communication 37 (4), 1987, pp.
65±94.
[25] R.E. Rice, J. D'Ambra, E. More, Cross-cultural comparison
of organizational media evaluation and choice, Journal of
Communication 48 (3), 1998, pp. 3±26.
[26] T.S.H. Teo, V.K.G. Lim, R.Y.C. Lai, Users and uses of the
Internet: the case of Singapore, International Journal of
Information Management 17 (5), 1997, pp. 325±336.
[27] H. Wan, Opportunities to enhance a commercial website,
Information and Management 38 (1), 2000, pp. 15±21.
John D'Ambra (PhD, The University of New South Wales, 1996)
is a senior lecturer in the School of Information Systems,
Technology and Management at The University of New South
Wales, Sydney, Australia. His research interests include, user
evaluation of web based information services, computer mediated
communication and systems development.
Ronald E. Rice (PhD, Stanford University, 1982) is Professor, and
Chair of the Department of Communication, School of Communication, Information and Library Studies, Rutgers University, New
Jersey, USA. He has conducted research and published widely in
communication science, public communication campaigns, computer-mediated communication systems, methodology, organizational and management theory, information systems, information
science and bibliometrics, and social networks.