Download Paper Title (use style: paper title)

Conceptual Models for Better Quality of Portfolio
Web Site Experience
Jesenka Pibernik, Bojan Kanižaj, Lucija Frljak
University of Zagreb Faculty of Graphic Art, Getaldićeva 2, Zagreb, Croatia
[email protected]
Abstract - The article analyzes the quality of user experience (QoE)
for particular category of websites, i.e. Portfolios. Portfolios represent
an interesting blend of creativity and development techniques.
Graphic designers blend great design practices and good coding
techniques in order to provide an aesthetically pleasing usable web
portfolio. In trying to impress possible employers graphic designers
are in some cases more concerned with showing off their expertise
rather than focusing on the user experience. The article explores the
correlation between visual interests and computer competences of the
user with the quality of experience of the particular website. The
research is based on a subjective survey research of users and
discusses the results of what type of portfolio web site’s conceptual
model/programming tool is more suitable with regard to user’s
interest and competences. It was shown more competent users are
generally more critical about the quality of experience, and value
pragmatic over hedonic quality of the product.
Keywords - Conceptual Models, Portfolio Web Site, User Experience
I.
INTRODUCTION
The aim of a web portfolio is to present the designer’s best
works, which will in turn help them attract clients and find
work [1]. In the context of web portfolios for graphic
designers, the target audience are most often other graphic
designers, students of graphic design, employers at marketing
agencies and design studios, and any other employers in need
of design. We can conclude that the primary characteristic of
this target audience is a relatively good grasp of computer,
especially graphic, programs and that we are talking mostly
about people who appreciate visual art and have personal
attitudes towards it.
After the goal for the web page and the intended audience
have been established, work can begin on designing a personal
web page which, in order to be successful, must adhere to
certain guidelines. In presenting a personal web page there are
numerous keys to success, from clear communication,
functionality and usability, through the selection of works and
easy access to contacts, to creativity and personality [2].
The aim of this article is to prove that the addition of
personality to a digital portfolio is advisable and useful in
making the designer stand out, but it may also result in a
completely different effect, depending on the set goals, the
target audience and the choice of program tools for the creation
of the web portfolio [3]. Although many web pages, especially
personal portfolio pages, appear completely unique at first
sight, they possess an inner structure that can be totally
traditional. Using known structures increases the likelihood
that users will intuitively connect with the web page .
In order to analyze user experience differences between
different conceptual models and programming tools for web
portfolios, two web portfolios of identical content and
structure, but different concept, were created [4]. The web
pages were made with the same goal and for the same target
audience. Two different program tools and two content display
modes were used in the making of the web pages:
•
gallery display: begins with the main display which
contains smaller images – so-called thumbnails – which
represent individual work. Gallery display is common, flexible
and easy to maintain and upgrade. It provides good balance to
represenation, which is professional and easy to use. The
program tool Html/CSS was used, accessible at
http://lucijafrljak.net63.net/
•
conceptual display: requires a good concept for the
result to be original. The web page created for the purpose of
this article is based on photo montages that are partly animated.
The photo montages were assembled from parts of previous
works by the author, while the links on the initial page, which
are conceptual, are also user-interactive photo montages. The
conceptual display of the author’s personal web page also
boasts atypical navigation incorporated in the design of the
page. Such navigation is interactive, mouse-triggered photo
montage which reduces the page’s usability for a certain type
of audience. Disadvantages could be reduced usability,
maintenance and upgrading, but if designed well, such a page
will stick out among the competition. The Adobe Flash
program tool in combination with the Action Script programing
language was used, accessible at http://lucijaf.pondi.hr/
The structure of both web portfolios consists of the initial
page which leads onto four main categories; about the author,
contacts, news and portfolio, which is further subdivided into
graphic design and illustrations. This structure makes up the
skeleton of the web page and all other elements are later
upgraded in detail around it with the help of mental mapping,
i.e. developing the idea of design through conceptual or mental
maps.
II.
METHOD
There are three main perspectives in the estimate of web
pages in relation to the user [5]. One of the perspectives is
esthetics or presentation, that is, the graphic design of the web
page which defines the impression the page gives. The design
of the web page needs to be attractive while allowing users the
perception and use of its contents. User satisfaction is only one
measure of usability, and for usability to be properly evaluated,
effectiveness (whether users are able to perform tasks/goals)
and efficiency (how long the user takes to perform task/goal)
also need to be taken into account [6]. It is widely agreed on
that QoE is specific to the user and to the application under
consideration, among other things. There are several definitions
of user experience. They all suggest that for a given application
in a given usage context, the quality will vary for different
users or user groups.
There are several user-centered models for web site design
[7] and numerous works on user experience evaluations [8] [9]
[10]. This article seeks to explore if there is a connection
between the user’s visual interests and their computer skills
with the quality of experiencing a certain web page. It is
expected that the obtained results will serve to better
understand the target audience and their needs, as well as to
help with further upgrades and selection of more acceptable
technologies in the design of personal web pages. We therefore
hypothesized that:
Hypothesis 1_ users who prefer classic art are visually
more inclined to a traditional web portfolio display (gallery
display), while users who prefer modern art give better ratings
to a conceptual display (hedonic quality).
Hypothesis 2_ users with stronger computer skills visually
prefer the conceptual display over the traditional one because it
brings freshness and visual surprise in comparison with the
traditional display (hedonic quality).
Hypothesis 3_ users with average computer skills will
prefer gallery display over conceptual display because good
usability is important to them (pragmatic quality).
III.
EXPERIMENTAL DESIGN
The experiment consisted of two parts: a questionnaire and
usability measure with the system usability scale (SUS) [11].
The experiment was run in a computer laboratory, on Intel
Pentium personal computers, Microsoft Windows operating
system, 17” monitors, Mozzila web browsers, with groups of
15-20 participants who worked independently. The participants
were asked to answer a questionnaire and then to inspect one
portfolio web site followed by SUS questionnaire. After that,
they had to visit the other web site and answer the SUS
questions for that web site.
A. Questionnaire – User Skills
The entrance poll consisted of three types of questions
according to the type of answer sought, and these were factbased questions – general information about the user (age, sex,
occupation) and their technical skills (how long they have
been using the Internet and how many hours per day), as well
as questions about the user’s interests (type of art they prefer:
classic or modern). The questions in the questionnaire itself
are such as to offer users answers in advance. The user’s
artistic preferences are determined according to which type of
artwork they like (paintings by famous artists, film genres,
book genres).
B. Measuring Usability with the System Usability Scale
(SUS)
For research purposes the questionnaire method was
chosen as the method of subjective evaluation of the web
page’s usability from the user perspective. Subjective
evaluations from the user’s perspective (questionnaires,
interviews, focus groups) show what the user feels like when
using the web page / user interface. The SUS is a 10 item
questionnaire with 7 Likert scale response options, developed
by John Brooke at Digital Equipment Corp. in 1986 as a tool
in usability engineering. Data from over 500 different studies
show that SUS is a valid questionnaire for evaluating web
page and interactive system usability which also provides the
most valid results [12]. The questionnaire contained questions
which examine attitudes (attitude type) according to the SUS
(system usability scale) questionnaire:
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I found the various functions in this system were well integrated.
5. I would imagine that most people would learn to use this system
very quickly.
6. I felt very confident using the system.
7. I found the system very cumbersome to use.
8. I found the system visually attractive.
9. I needed to learn a lot of things before I could get going with this
system.
10. I thought there was too much inconsistency in this system.
IV.
EXPERIMENTS
The questionnaire was completed online, with 65
participants, 23 of which were men and 42 were women. Of
these, 43 were between 19 and 30 years of age, 4 were between
31 and 45 years old, while 3 were over 45 years old.
Respondents were a target group for this type of product:
students and employees in the area of graphic design and
graphic technology. The validity of the results, as well as the
success of the evaluation via questionnaire, alongside wellconstructed and meaningful questions, depends on the number
of completed questionnaires - also known as the response rate,
or questionnaire feedback rate. The response rate to the
questionnaire was sufficient for a statistically valid analysis.
A. Respondents’ technical skills
Results showed that most respondents have been using
computers and the Internet for a long time: 61% of
respondents have been using the Internet 4-9 years, and 34%
of respondents for more than 10 years; they have been
spending several hours a day on the Internet: 39% of
respondents spend 3-5 hours a day on the Internet and 34% up
to 2 hours a day, while 21% of respondents spend 6-10 hours a
day on the Internet. Respondents use different programs,
mostly Word, Powerpoint, Excel, Photoshop and Illustrator.
For the question what they do actively and occasionally, 25
respondents answered that they do photography, 17 do visual
art, 30 do graphic design, 20 do web design, 9 do film or
moving graphics, and 12 do not do anything of the above.
TABLE I.
GRADE AVERAGE FOR THE GALLERY VS CONCEPTUAL WEB PAGE ACCORDING TO
ASSERTIONS BASED ON YEARS AND HOURS OF INTERNET USE.
SUS
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
SUS
GALLERY
CONCEPTUAL
- 3 years
4‐9 years
10+ years
- 3 years
4‐9 years
10+ years
4.6667
1.3333
6.6667
6.0000
6.6667
6.3333
2.3333
5.3333
1.6667
6.6667
3.9286
1.9286
6.2500
6.0000
5.8929
5.2143
1.5357
4.7857
1.6667
5.8214
3.3529
2.5294
5.7059
4.1176
5.0588
4.2941
2.7059
3.4706
3.5882
5.5882
4.3333
3.0000
6.0000
5.0000
5.3333
5.6667
4.6667
6.0000
4.3333
6.0000
3.9355
3.2258
5.0968
5.0000
5.3226
4.8065
5.0645
5.5806
3.0645
5.4194
3.2941
3.0588
4.6471
4.5882
4.7647
4.2941
4.7059
5.0588
2.8235
5.4118
GALLERY
CONCEPTUAL
-2h
3-5 h
6-10 h
10+h
-2h
3-5 h
6-10 h
10+h
1.
4.0000
3.1667
4.2727
4.5000
4.0588
3.6957
4.0000
3.1667
2.
2.0000
2.0556
2.3636
2.0000
2.5882
2.8696
3.5455
4.1667
3.
6.3529
5.8333
6.0000
6.5000
5.4118
5.0870
5.2727
4.5000
4.
5.4706
4.2778
5.1818
4.5000
4.6471
5.0435
4.8182
5.1667
5.
6.0000
5.5000
5.2727
6.0000
5.1176
5.2609
5.1818
5.1667
6.
5.5294
4.5000
4.7273
5.5000
4.8824
4.6087
4.6364
4.8333
7.
1.4118
1.9444
2.3636
5.5000
3.2941
5.1304
4.1818
5.1304
8.
5.0000
3.7222
4.4545
4.0000
4.8824
5.5652
5.0909
5.5000
9.
2.4118
3.4444
4.3636
3.0000
3.3529
2.4348
3.7273
3.3333
10.
5.8235
5.5000
6.0909
6.5000
5.0588
5.5217
5.9091
5.5000
Figure 1.
Grade averages for SUS assertions for two portfolios
Respondents’ artistic inclinations
On the basis of their choice from offered works of art, book
genres and film genres, respondents were divided into two
groups: modern art aficionados (p1) 55% and classical art
aficionados (p2) 45%.
V.
RESULTS
For analysis purposes, in questions with multiple answers
(sex, time spent daily on the Internet, years of Internet use,
artistic preferences) and assertions with the Likert scale, the
answers were numerically coded. For 10 SUS scale assertions
average values were calculated, depending on the variable.
In order to determine whether the average SUS scale
assertion grade differed significantly, from a statistical point
of view, depending on variables, a variance analysis was
conducted. The dependence of SUS scale assertion grade on
the type of web page (gallery and conceptual display) was
checked through a chi-square test. All statistical test were
conducted on a 0.5 significance level. The variance test
showed there were no statistically significant difference
between assertion grades based on respondents’ sex.
VI.
DISCUSSION
Generally, users found conceptual web site system visually
attractive but unnecessarily complex and very cumbersome to
use (Fig 1.). In order to verify whether the assertion grades
depended on the web page (gallery/ conceptual), a chi-square
test was conducted . The test showed that there was no
difference for assertions no. 4, 6, 9 and 10. depending on the
years of Internet use (Table 1): Anova variance test showed a
statistically significant difference for assertions no. 4 and 9
between users who have been using the Internet less than 3 and
over 10 years for an gallery web page, and for assertions no. 3
and 9 between users who have been using the Internet less than
3 and over 10 years for a conceptual web page. Users with
longer experience in using the Internet have a lower opinion of
content integration and learnability for both web portfolios than
those with less experience.
Depending on number of hours spent daily on the Internet
(Table 1) Anova variance test showed a statistically significant
difference for assertion no. 7 between users who have been
using the Internet less than 2 and more than 10 hours a day for
a gallery web page, and for assertions no. 2 and 7 between
users who have been using the Internet less than 2 and more
than 10 hours a day for a conceptual web page. Users who
spend more time on the Internet per day (6-10 hours) have a
better opinion of the usability of both web portfolios than those
who spend less than 2 hours a day on the Internet.
It is also interesting to observe that most respondents who
use only basic programs such as Word and Powerpoint
consider a conceptual web page unnecessarily complex, unlike
respondents who use a wider range of programs. The same
holds true for respondents who spend more than 10 hours a day
on the Internet: 40% partly agree that a conceptual web page is
unnecessarily complex, while 54% of respondents who spend
up to 2 hours a day on the Internet think the page is not
unnecessarily complex.
Depending on visual preferences, persona 1 (p1 ) > modern
art, persona 2 (p2) > "classical" art results show that modern art
aficionados rate more highly the visual attractiveness of both
web portfolios than classical art aficionados. There is statistical
difference between answers to questions no. 1, 7 and 9 between
users who like modern art and users who prefer classical art.
Given the very few differences in the grade average for both
web portfolios we can conclude that the respondents’ artistic
preferences did not exert a significant influence on the
assessment of hedonic quality.
From the above stated measurement results we can conclude
the following:
Hypothesis 1 - was partially confirmed.
Hypothesis 2 - was confirmed.
Hypothesis 3 - was proven wrong.
VII. CONCLUSION
The research is based on a subjective survey of users and
discusses the results and what type of portfolio web site
conceptual model/programming tool is more suitable with
regard to users and their computer competences. Users with
longer Internet experience who are spending more hours daily
using their computers are more competent and use both
technologies easier than average users. They are generally
more critical of the quality of experience. They found
conceptual system cumbersome to use, unnecessarily complex
and at the same time visually attractive. They would prefer
gallery over conceptual system. However, if average user is
analyzed, conceptual wins in more categories. Average user
would like to use conceptual system frequently, found the
various functions in this system were well integrated, felt
confident using the system and found the system visually
attractive. Research showed that Flash technology is still a
good choice for creating a web portfolio, provided that the
design is intended for a wider audience. If, however, the
intended public is a group of experts with ample experience in
using the Internet, it is advisable to use newer technologies but
keeping in mind the hedonic element of user experience. This
paper has presented a new experimental approach to
understanding the diversity in website perceptions between
people. The proposed method has several advantages. First, it
follows a pragmatic approach to user modeling by
distinguishing users based on their computer competences and
how they perceive products rather than on abstract
demographic and psychographic information. Second, the
method can be used in different phases of the product
development, for instance using concept sketches, but also for
benchmarking between competitor products.
REFERENCES
[1] I. Clazie, Creating Your Digital Design Portfolio, Switzerland:
RotoVision SA, 2010.
[2] C. Eccher, Professional Web Design: Techniques and Templates, Course
Technology, USA, 2011.
[3] J.W. Palmer, Web site usability, design, and performance metrics.
Information Systems Research, Vol. 13, No.2, 2002, pp. 151-167.
[4] J.J. Garrett, The Elements of User Experience: User – Centered Design
for the Web and Beyond. USA, New Riders, 2011.
[5] S. Lee, R. J. Koubek, The effects of usability and web design attributes on
user preference for e-commerce web sites. Computers in Industry, Vol.
61, No.4, 2010, pp. 329-341.
[6] Finstad, K. The Usability Metric for User Experience. Interacting with
Computers, Vol. 22, No.5, 2010, pp.323-327.
[7] J, Wiedemann , Web Design: Portfolios, Taschen, Köln, 2005.
[8] A. Sears, A. Jacko, J. (ed), The Human – Computer Interaction Handbook.
Lawrwnce Erlbaum Associates, Taylor &Frances Group, 2008.
[9] P. R. Ross, & S. A. G. Wensveen, Designing aesthetics of behavior in
interaction: using aesthetic experience as a mechanism for design.
International Journal of Design, Vol. 4, No.2, 2010, pp. 3-13.
[10] M. Hassenzahl, N. Tractinsky, User experience – a research agend.
Behaviour and Information Technology, Vol. 25, No.2, 2006, pp.91-97.
[11] J. Brooke, SUS - A Quick And Dirty Usability Scale.URL:
http://www.itu.dk/courses/U/E2005/litteratur/sus.pdf. (9.4.2012.)
[12] J. Sauro, Measuring Usability with the System Usability Scale .
URL:http://www.measuringusability.com/sus.php
and
procedure
(9.4.2012.)