Download HOT_evaluation_CHECKLIST

Heuristic Online Text (HOT) Evaluations
Jonathan and Lisa Price
Want to know how well your prose meets Price and Price’s standards? Use these
evaluation questions to measure your pages.
CONTENTS
1. Brevity
pp. 1-2
2. Scannability
pp. 3-5
3. Hot Links
pp. 6-7
4. Chunky Paragraphs
pp. 8-9
5. Cognitive Burdens
pp. 10-11
6. Menus
pp. 12-14
1. Brevity
Strategy 1: The language is terse.
1. Is nothing repeated except where the context demands repetition? Did the running
text seem free of repetition and/or repeated words or phrases?
2. Is the text generally free of unnecessary adjectives and adverbs?
3. Does the next take more than a phrase to establish context at the beginning or end
of a paragraph or sentence?
4. Are there any junk phrases that could be replaced with single words?
5. Do one- and two-syllable words made up the majority of the text?
6. Does the text use words that refer to tangible, visible, common objects, or
activities.
7. Does the text use familiar or frequently used word over the rare or unfamiliar
ones?
Strategy 2: The paragraphs are short.
1. Does each paragraph have four sentences or less?
2. Does each paragraph focus on only one main idea?
3. In the online layout of the site, does each paragraph have no more than six lines?
Strategy 3: Marketing yes, fluff no.
1. Is the copy reader-centered, and not focused on the writer, or the organization the
writer is representing? Do any paragraphs seem particularly self-centered?
2. Does the text avoid broad claims, full of boastful bride?
3. Does the writer avoid slogans and marketing phrases?
4. Does the writer restrain their enthusiasm, avoiding exaggerated claims about the
company, or its products or services?
5. Are noun clumps avoided, those concentrated noun strings that marketing people
use to summarize their pitch (“world-class enterprise solutions”)?
6. If discussing features and benefits, does the writer use far more nouns and verbs
than adverbs and adjectives?
7. Does the writer provide evidence behind its claims? Does the text link to or
include elements that customers regard as trustworthy, such as quotes from trusted
sources, links to reviews, statistics, datasheets, or detailed descriptions?
8. Is the tone neutral, almost objective?
Strategy 4: Tangential material is moved out of the way.
1. Is background or supplementary material linked to, not forced on the users? Does
the writer unfold information gradually, rather than putting it all on one page?
2. When the topic is complex, is information broken up into many paragraphs,
sections, or pages? Does the writer divide material into bite-sized chunks?
3. When there are multiple links to related stories, are the links pulled out of the
regular running text? If there is a set of links to other stories, do the links appear
in a separate sidebar, a distinct list, or a separate section?
4. Are all facts and ideas relevant to the main topic of the paragraph, not extra?
1
Strategy 5: Brevity is not carried to the point of ambiguity.
1. Does the write avoid achieving brevity by condensing phrases into clumps of
letters? Overall, does the writer appear willing to take the text to explain a term,
or spell out a phrase before showing its abbreviated form, or its acronym?
2. Does the writer leave articles in the text? The, a, and an help people identify the
nouns and noun phrases, distinguishing them from verbs. The articles also make
explicit whether you mean this particular item, or just any item like this.
3. Does the writer preserve relative pronouns like who, which, that? These help
people see the relationship between the subordinate clause and the main clause.
4. Does nothing stand between the subject and the verb? Do any sentences seem
confusing because a phrase or clause slips between the subject and verb?
Strategy 6: Repeating categories of information appear in tables
1. Does the writer break repeating information out into lists, and if each item
contains two or more categories, a table? Do any paragraphs seem crowded with
information that could be turned into rows and columns?
2. If there is a table, are all the components labeled?
3. Do the tables avoid horizontal scrolling? Can you see the whole row at once?
In a few paragraphs, summarize your most important observations—both positive and
negative.
Major Recommendations
List the top three problems with the text, and in a sentence or two, summarize what you
would recommend as solutions.
1. Problem 1:
2. Problem 2:
3. Problem 3:
2
2. Scannability
To determine the extent to which the text on this site achieves the scanability needed for
effective online presentation. Scannability opens up the text, allowing people to see how
it is organized, and encouraging them to jump in at the spot they find most interesting.
Spotting a Title, Headings, and Subheads
Title
Major Heading (#1 Level)
Here is a paragraph of running text. Perhaps it
introduces the content of the page.
Minor Heading (#2 Level) aka Subhead
Here is a another paragraph of running text. It
deals with the topic announced in the secondlevel heading. Any heading below the major
heading is known as a sub-head.
Really Minor Heading (#3 Level)
A title appears only in the top bar of the window.
Headings appear inside the window. They come in various sizes, indicating their level
within the hierarchy. Sometimes people talk about major headings and subheads.
A subhead is simply any heading that is not the biggest one on the page.

Strategy 1. The Titles Signal the Content, Fast.
1. Did the titles start off with the text that indicates the most important topics on this
page? Does critical information appear first, in each title?
2. Do the titles stand on their own, describing the content of their pages, even if you
did not know what section of the site these pages appear in? In looking at these
titles in a search results list, or a menu, I could tell what they are talking about?
3. Do the titles start off with the text that distinguishes this page from others? In
each sample title, the first few words are unique. They do not appear in any other
titles.
3
4. Does the major page heading echo the page title? In each sample title, the
repeated identification (of company, department, section) appears after the
information that is unique to this page, so users can spot the differences between
pages in search results and menus.
5. Do the titles show parallelism of structure and phrasing, when the pages are
similar?
6. Does each titles describe the pages in a way that make sense before you read the
pages—and afterward?
Strategy 2. Headings provide meaningful information about the way you have
organized the material.
1. Does a heading announce every change of topic? In each sample page, a heading
of some kind appears whenever the subject matter changes. Headings, then, act as
advance organizers, tipping the user off to the transition, and the next topic.
2. Do subheads introduce subtopics? In each sample page, if there are subtopics,
they are introduced by subordinate headings.
3. Do the pages use no more than three levels of headings? Overall, did your eye
perceive the difference between levels of heading easily?
4. Do headings accurately and concisely sum up the content of the following
sections? Each heading expresses the content of the section that follows. The
headings are not puns, jokes, metaphorical excursions, or phrases that make you
guess what the authors meant.
5. Do headings signal the structure of each page, showing what the page is about,
and how the page is organized? In each sample page, the whole set of headings,
taken as a sequence, expresses the basic content of the page. A user, looking only
at the headings, gets an outline of the content.
6. Do the headings remind the user of some familiar pattern of organization, such as
chronological order, familiar to unfamiliar, important to unimportant, cause and
effect, problem and solution? In each sample page, the headings indicate the way
the page has been organized, and, if that follows a familiar pattern, the headings
are phrased in a way that reminds the user of that schema.
Strategy 3. Important text stands out.
1. Do paragraphs include some text that stood out because of its formatting? In most
paragraphs, at least one critical word or phrase has been boldfaced, or otherwise
highlighted. (A link counts as an emphasized phrase).
2. Do links speak meaningfully to the topic of the paragraph? In a paragraph
containing one or more links, the linktext says something meaningful about the
topic, and therefore helps the user figure out what the paragraph is about, as a
whole.
3. Did all paragraphs have no more than three or four boldfaced terms and links. In
most paragraphs, the emphasized words, phrases, or links stand out, because they
are not surrounded with a lot of other emphasized text.
4
Strategy 4. Sets of items appear in bulleted or numbered lists.
1. Did pages light up with significant words and phrases, tipping you off to the point
of the page, and, possibly, its organization? In most pages, the emphasized text
could be used as a digest of the major points, or topics.
2. Does any set of three or more items in a row appear in a list? Whenever there are
three or more items in a row, they are broken out in a list, so people can scan
through the material quickly.
3. Do most lists appear at the end of the paragraph? When paragraphs include lists,
the lists always appear at the end of the paragraphs.
4. Did sequential lists have numbering, rather than bullets? Whenever there are
items that must be read or done in order, the list items begin with numbers.
5. Do lists of options appear with bullets? Whenever there are optional items in a
list, they begin with bullets.
6. Are most lists parallel, or consistently phrased? When a set of items are similar,
they begin with the same kind of phrase, or use the same grammatical form, to
help people compare one with another.
7. When list items contain two elements, can the reader quickly distinguish one from
the other? When each list item has two parts (a term and definition, a link and a
description, say), the two parts are separated by punctuation such as a dash, a
colon, or a paragraph return, or by other formatting.
8. Are long lists organized into sections? Whenever a list extends beyond nine or ten
items, the items are grouped by object, topic, or action, and if the list is
particularly long, those groupings are indicated by headings. (Not necessary if the
list must be arranged in alphabetical or numeric order).
Major Recommendations
List the top three problems with the text, and in a sentence or two, summarize what you
would recommend as solutions.
4. Problem 1:
5. Problem 2:
6. Problem 3:
5
3. Links
To determine the extent to which the links on this site do their jobs. To make links work
well for users, though, the authors must follow a dozen strategies, and within those broad
approaches, a number of tactics.
Strategy 1. The linktext makes clear what I will get if I click and go.
1. Is Linktext fairly short, but descriptive?
2. For each link, is supplementary description of the content on the target page, or
the reason to click, provided? Next to each link, I found extra information about
the target, in ordinary text, or a label. Can readers tell where they’re going, before
they click?
3. Do the links echo the titles or major headings? With each link, I found that the
text of the link contained words or phrasing similar to the text in the title or major
heading on the target page.
Strategy 2. In a sentence, the link appears at the end.
1. Does the linktext act as the emphatic element in the sentence or paragraph? When
a link appears within a sentence, it goes at the end.
Strategy 3. The text focuses on the subject, not the links.
1. Does all linktext make a meaningful contribution to its sentence? In each link
within running text, I found the text made sense.
Strategy 4. The pages offer plentiful links to related information on the site.
1. Does the page not refer very often to other pages within the site? There is a
section of related links, with descriptions of the target info.
2. Does the running text seem thickly related to other content on the site? The
running text contains pointers to related pages on the site.
Strategy 5. The site establishes credibility by offering outbound links.
1. Do the pages offer links to other sites? There are links to other sites on every
page, or most pages.
Strategy 6. The target page indicates where I am in the overall structure.
1. If part of a larger site, can the reader can tell where the page fits into the larger
structure? There are breadcrumbs, highlighting, graphics, or text to show where
this page is within the larger structure. The page has a logo or site identifier on it.
Strategy 7. The page offers meta-information.
1. Does the page contains at least three of the following: status information, date,
phone number, comment link, author name?
Strategy 8. The URLs make sense to a human.
1. Do the page URLs contain words or text that the reader can understand where
they are or what they are looking at?
6
Strategy 9. The links are accessible.
1. Does the page consider the special needs of some audiences? When I choose to
View the Source of the page, I see that there is alternate text for every image,
animation, sound, or video. The headings are conventional H1, H2, H3. The page
uses stylesheets, relative font sizes, few or no frames, a single table, if any.
Strategy 10. The site describes any downloadable object.
1. Does the site make an effort to help the user decide whether or not to download
the files? If the page offers a file for downloading, they tell me how large the file
is, how long it will take to download at a particular speed, what format it is, and
what software I can use to view it.
Strategy 11. The site announces the new, and identifies what has changed.
1. Are there some pages that alert the reader to the new, or the changed? Content
that is new, or changed, has some identifying information, to alert me to the fact
that the info is new or updated.
Strategy 12. The pages contain content and meta-information that increases the
likelihood that the pages will show up as relevant in searches.
1. Do page Descriptions and keywords match or echo the title, headings, running
text, and links? When I choose to View the Source, I see that the meta tags for
Description and Keywords contain words and phrases that reappear in at least two
of the following: the title, the major heading, the first sentence on the page, the
running text, and the linktexts on the page.
Major Recommendations
List the top three problems with the text, and in a sentence or two, summarize what you
would recommend as solutions.
7. Problem 1:
8. Problem 2:
9. Problem 3:
7
4. Write chunky paragraphs.
Strategy 1: Design each paragraph around one idea.
1. Is each paragraph a distinct object that has one main purpose? Figure out what
that purpose is, and highlight that throughout.
2. Are long complicated paragraphs broken up into a series of short ones? Go into
big paragraphs and press return.
3. Are there any irrelevant topics that can be removed? Throw out irrelevant topics.
If they are important, put them in their own paragraphs. If trivial, discard.
4. Does the real point of a paragraph appear in a distinct sentence within the
paragraph? If you cannot figure out what the real point is, your reader cannot
either. If it does not, your paragraph has no core.
5. To all the sentences cohere? Each should contain words that relate to your main
topic. Skip the thesaurus, too. If you are referring to the same thing over and
over, use the same word.
6. Is the flow of ideas recognizable and signaled? For instance, if you are working in
chronological order, use words such as first, next, and finally.
7. Do sentences proceed from the familiar to the new, throughout? Start with the
known, and end a sentence with the new. Then start the next sentence with that
new idea, creating a chain.
8. If you are writing in XML, treat each paragraph as an element. What is the
subject? What are you trying to do with this element? Why are you including it?
Strategy 2: Put the idea of the paragraph first.
1. Does the writer lead off with an idea, a complete sentence, not just a topic?
2. Does the writer avoid a slow introduction?
3. Does the writer lead paragraphs with the main point? To guarantee that people
can comprehend the whole paragraph, give them the main point right away. It
serves as a framework for the rest of the content. Users understand better. Be
blunt, even if you feel a bit impolite. In a conversation, you can sidle up to the
point. But online, directness is a courtesy. People expect the point to be first.
That is their conceptual model of a paragraph. Accommodate them.
Strategy 3: If you must include the context, put that first.
1. If the text calls for a little context before the reader can understand the paragraph,
does the writer put the orientation session at the start? But don't take more than a
phrase or two. Keep the context to a sentence or less.
2. If referring back to the previous paragraph, does the writer use the same words he
or she used to introduce toward the end of that paragraph, to make the
connection?
3. Does the text explain why we should bother to read on? Does it motivate us, set a
vision of our goal dancing before our eyes? Tie this paragraph back to the one
before...if you are sure that everyone saw that one. If you introduced an idea
there, start with that, and move on.
Strategy 4: Put key conclusions, ideas, news, at the start of the article.
8
1. Does the writer start with the summary? At first, this seems backward. Reach
your conclusion, and then put it at the beginning. Let visitors know what is in the
article for them--right away. Write the first sentence last. You don't know what
you think until you write the whole article. Then, when you know what you are
saying, sum that up in a first sentence. Let people get the point without the
argument and evidence. If they want details, they will read on. For many, the
summary of the news is enough.
9
5. Cognitive Burdens
To determine the extent to which the text on this site reduces cognitive burdens on the
visitor. Don’t make me think, when I use or read your text.
Strategy 1: Most sentences have one main clause, and no subordinate clauses.
Do the sentences seem simple, because they contained very few subordinate clauses, or
none?
A clause contains a subject and a verb. Examples:

The batter hit the ball.

When I go to the store

If I like the dress

Who is my best friend

That I chose

Which is the brightest red of all
If the clause is introduced with a word like when, because, while, who, that, which, or a
similar word, the clause is subordinate to the main clause. Tip: If you can remove the
clause and still have a complete sentence, you have spotted a subordinate clause.
Strategy 2: There are very few nouns made up to replace perfectly good verbs or
adjectives.
Do most nouns seem simple, not made up out of verbs, to avoid actually using the verb? I
found only one or two sentences that contained nominalizations—nouns made out of
perfectly good verbs or adjectives, to make the prose sound more elegant, or formal.
Strategy 3: There are no strings of three or more nouns in a row?
Was the text was free of noun strings that make readers wonders how to put the nouns
together, to figure out which nouns formed pairs modifying some other noun? Avoid
noun trains, where all the nouns build up to a single idea (as opposed to a list of items).
Strategy 4: There are no ambiguous phrases that the reader must ponder and
debate.
1. There are no familiar terms used in a strange new way?
2. For every pronoun, the reader can find the noun referred to, right before the
pronoun, without any other nouns in between?
3. There are no sentences that began with a solo “This” or “That” referring vaguely
to the whole previous sentence, or an idea suggested, but not spelled out, earlier?
4. There are no references to other elements within the page, or on other pages, that
assume the reader is reading sequentially (first, second, before, after), or that the
10
element is in a particular spatial relation to the mention (above, below, to the
right, to the left)?
Strategy 5: The agent—the person, program, or thing carrying out the action—
appears as the grammatical subject of the sentence.
Overall, does the text follow the standard English order: subject, verb, object? The
subject of each sentence should be the person, program, or thing that performs the action
described in the main verb. There should be no verbs in passive voice (“The ball was
hit.”). If the agent acts on an object, the object appears after the verb.
Strategy 6: Make all statements positive.
Is the text straightforward and declarative? The writer avoids explicit negatives, such as
not, no, never. The writer avoids implicit negatives, in words such as impossible,
unlikely, and inattentive. The writer avoids putting together several explicit or implicit
negatives in the same sentence.
Strategy 7: Keep scrolling to a minimum.
Do the pages put key content into the first window, without making the user scroll a lot to
find important information? The user should never be in doubt about what was out of
sight, lower down, on the page. The user should rarely have to scroll down more than one
or two screens, to see all the content on the page, and never have to scroll horizontally.
Most important content should appear above the fold, at the top of the first window. Even
when some content is out of sight, users should have a good idea what they would find,
when they eventually scroll down to that text.
Strategy 8: Long documents even if broken up into a series of short pages should
also appear on a single page, or in a single file, for saving or printing.
If a page might extend over several printed sheets, does the site offer the user the
opportunity to get an HTML or PDF file formatted for printing? Long documents should
appear in a printer-friendly version. Long documents should appear in a downloadable
PDF version.
Major Recommendations
List the top three problems with the text, and in a sentence or two, summarize what you
would recommend as solutions.
10. Problem 1:
11. Problem 2
12. Problem 3
11
6. Menus
To determine the extent to which the text of menu items articulates the structure of the
site, and clarifies what the target pages are about.
Outline
Top Level (title of home page)
Main Menu Items
Subtopics
Sub-subtopics
Strategy 1: A title or a heading is an object you reuse in many menus, search
results, bookmark lists, and See Also lists.
1. Does each menu item reflects the actual title or major heading on the target page?
Does the linktext of menu items seem similar to that text of the title or major
heading on target pages? Clicking the linktext of each menu item should take the
user to a page in which that text reappears, verbatim, as the title or major heading
on the page, confirming that users have gotten where they expected to. The page
title or major heading is the same object used in the menu, as a menu item.
2. Is each menu item distinct from the others? Comparing all the items in a single
menu, can users tell the difference? There should be no items that seem as if they
might be describing the same topic. When users think of a topic they would like to
find through the menu system, they should be able to figure out which menu item
might lead me toward that information. (Try 5 topics).
3. Are the menu items accurate and complete enough to let users gauge what content
will appear on the target page? Reading almost any menu item, users can correctly
guess what information will appear on the target page.
4. Can users distinguish between menu items because the important words appear
early in each item? In looking at ten menu items that contain more than a few
words, users should see that the keyword always appears early in the item, not
late.
5. Does the site offer shortcut menus to popular items? The site is helpful, offering
menus leading to important pages that lie deeper within the site (not at the next
level down). On the home page, and major departmental pages, users can find
links to popular pages that exist deep within the site.
Strategy 2: Each menu offers a meaningful structure.
1. Does each short menu follow at least one easily recognizable pattern? Each menu
of less than seven items contains menu items written so that users can tell why
12
these items appear in a group, and why they appear in this sequence. No short
menu should be arranged in alphabetical order, unless it is a list of items that
usually appear in that arrangement (states, authors’ names, book titles).
2. Does each long menu chunk menu items in groups, arranging the groups in
familiar patterns? Can users spot a meaningful arrangement of the menu items in
groups? Each menu of more than seven items is broken into groups. When menu
items appear in a group, users should see that the items all refer to the same kind
of object, category, service, product, or activity. Users should see why all, or
almost all of them belong together. Within each group of menu items, users
should see, from the text, why they are arranged in this sequence. Users should at
least grasp why one item is first, others are in the middle, and another comes in
last. When users look at the sequence of groups, they should see some familiar
pattern emerging. Users should not be puzzled by the order of the groups. No long
menu is arranged in alphabetical order, unless it is a list of items that usually
appear in that arrangement (states, authors’ names, book titles).
Strategy 3: The site offers multiple routes to the same information.
Do the multiple menus reflect the different interests, perspectives, or mental models of
typical audiences? Are there several different menus that users could follow down the
same page? There are distinct menus for several different audiences. But the content is
not entirely separate. Following the trail of one audience, users should find the same page
they already viewed when they posed as another audience. There are several menus set
up using different categories (toys by age, toys by type, toys by character, toys by
vendor). Users should be able to get to the same page from several different menus. (Not
necessarily from every menu).
Strategy 4: The site displays several menu levels at once.
Can users see several levels of content in the menu items displayed on most pages? The
second-level items appear at the same time as the main items, and stay put as long as I am
on the page. If the second-level items appear after I select a major item, they stay put, and
do not disappear when I move the mouse, or release my grip on the mouse button. They
remain stable. Second-level menu items never appear in cascading menus (temporary
dropdown menus that go away if you slide your mouse the wrong way). Second-level
menus never appear when I hover over the right areas, disappearing when I jiggle the
mouse. The site allows me to explore more than two levels of menu items, all on the
same page, so I can get a sense of the full content of the section before proceeding.
Strategy 5: The target page echoes the menu item I clicked to get there.
Can users confirm that they have arrived at the page they expected to? Users should
know that they have arrived at the right page, because the text at the top of the page is the
same text they just clicked. Some or all of the words in the title or major heading should
13
appear in the menu item. The introductory sentence echoes those words, or the same idea.
The caption of the first big picture echoes those words, or the same idea.
Strategy 6: The target page shows me where I am within the site hierarchy.
Does the page contain some indication of the users location within some hierarchy? Can
users see where they are within the larger structure? The page should display
breadcrumbs, or some other text or graphic device showing where users are in the larger
structure, for instance, displaying the major choices users would have made if they came
down from the top, following a common path.
Major Recommendations
List the top three problems with the text, and in a sentence or two, summarize what you
would recommend as solutions.
13. Problem 1:
14. Problem 2
15. Problem 3
14