Dont make me think

Don’t Make Me Think!
a common sense approach to web usability
SECOND EDITION

Steve Krug

New Riders Publishing
Berkeley, California USA


t h e h om e pag e i s b e yo n d yo u r co n t ro l

The actual Welcome blurb statement ("Our
experts provide you with the information
you need…") is underneath the promos, and
it needs to come before them. And, as usual,
it’s too long. I have to work hard to find the
crucial information: editors select products
without any influence from manufacturers
or advertisers.

DO YOU KNOW
WHERE TO START?
There are three clear starting points on the
page:

> Type something in the prominent
search box.

> Click on one of the categories in the

Yahoo-style directory.

> Click on one of the three featured
products (if that’s what they are).
The only problem is, if I’m unclear on
what the site is, how do I decide what
to search for or what category to choose? A
successful Home page has to tell me what
the site is and show me where to start.

[ 119 ]


c h a pt e r 7

THEIR REVISED VERSION
While I was writing this chapter, Productopia
redesigned their Home page, improving it
substantially.
They eliminated the stray tagline on the
right, and put a much better tagline ("We
Help You Find the Products You’ll Love") at
the top of the area on the left.
And they shortened the crucial explanation
("Our experts offer unbiased advice to help
you choose the product that’s right for
you") so that it now stands a chance of
being read. But it’s still buried at the
bottom of what still looks like the featured
products section.

And they moved the Utility links (Editorial
Policy, User Reviews, and so on) into a new
area at the bottom of the page, but they
lumped them together with promos like
"Women’s Spring Fashion" and "Do You
Cook?" It took me a while to figure out that
the two columns were different.

[ 120 ]


t h e h om e pag e i s b e yo n d yo u r co n t ro l

MY VERSION
I’d start by moving the tagline to the
top of the page with the Site ID, making
it clear that it’s a descriptor for the
entire site.
I’d also move the Welcome blurb above the
promos, and make it more prominent.
I’d separate the Utility links and the promos
at the bottom of the page, grouping the
promos with the "featured products" above
them on the left side.
And I’d reformat the awards icons. Unlike
most Web awards, these four are actually
meaningful.(The Digital Time award puts
Productopia on a short list of e-commerce
sites with Amazon and eBay.) But lining
them up across the bottom of the page

makes them look like they’re "Bob’s Cool
Site of the Day" icons. This is a case where
you want to be sure you don’t follow a
convention.

[ 121 ]


6789
8
c h a pt e r

“The Farmer
and the Cowman
Should Be
Friends”
why most web design team arguments
about usability are a waste of time, and
how to avoid them


One man likes to push a plough
The other likes to chase a cow
But that’s no reason why they can't be friends
— oklahoma! , oscar hammerstein ii

L

eft to their own devices, web development teams
aren’t notoriously successful at making decisions about usability questions.

Most teams end up spending a lot of precious time rehashing the same
issues over and over.
Consider this scene:

WEB DESIGN FUNNIES

Today’s episode: “Religious Debates”

featuring…
Rick from
Marketing

Kim the
Project
Manager

We could use a
pulldown menu for
the product list.

Bob the
Developer

I hate
pulldowns.

Caroline makes a suggestion…

Caroline the
Designer

Well, I don’t think most
people mind them.
And they’d save us a
lot of space.

People don’t like
pulldowns. My father
won’t even go near
a site if it uses
pulldowns.

Besides, have
you got a
better idea?

continued…

[ 123 ]


c h a pt e r 8

…but Bob plays his developer’s trump card

Do we know if there’s
any research data
on pulldowns?

I think there might

be a problem using
pulldowns on the
ASP pages from our
remote servers.

Rick attempts an appeal to a higher authority…
So, what does
everybody think?
Should we try
using pulldowns?

I hate
my life.

Two weeks later…

Did we ever make
a decision about
pulldowns?

I usually call these endless discussions “religious debates,” because they have a
lot in common with most discussions of religion and politics: They consist largely
of people expressing strongly held personal beliefs about things that can’t be
proven—supposedly in the interest of agreeing on the best way to do something

[ 124 ]


t h e fa r m e r a n d t h e cow m a n

important (whether it’s attaining eternal peace, governing effectively, or just
designing Web pages). And, like most religious debates, they rarely result in
anyone involved changing his or her point of view.
Besides wasting time, these arguments create tension and erode respect among
team members, and can often prevent the team from making critical decisions.
Unfortunately, there are several forces at work in most Web teams that make
these debates almost inevitable. In this chapter, I’ll describe these forces, and
explain what I think is the best antidote.

“Everybody likes ________.”
All of us who work on Web sites have one thing in common—we’re also Web
users. And like all Web users, we tend to have strong feelings about what we like
and don’t like about Web sites.
As individuals, we love Flash animations because they’re cool; or we hate them
because they take a long time to download. We love menus down the left side of
each page because they’re familiar and easy to use, or we hate them because they’re
so boring. We really enjoy using sites with ______, or we find ______ to be a royal pain.
And when we’re working on a Web team, it turns out to be very hard to check
those feelings at the door.
The result is usually a room full of individuals with strong personal convictions
about what makes for a good Web site.
And given the strength of
these convictions—and
human nature—there’s a
natural tendency to project
these likes and dislikes onto
Web users in general: to think
that most Web users like the
same things we like. We tend
to think that most Web users

are like us.

He’s right.
They stink.

What’s so bad
about them?
I like pulldowns. What’s
his problem?
People don’t
like pulldowns.

[ 125 ]


c h a pt e r 8

It’s not that we think that everyone is like us. We know there are some people out
there who hate the things we love—after all, there are even some of them on our
own Web team. But not sensible people. And there aren’t many of them.

Farmers vs. cowmen
On top of this layer of personal passion, there’s another layer: professional
passion. Like the farmers and the cowmen in Oklahoma!, the players on a Web
team have very different perspectives on what constitutes good Web design based
on what they do for a living.1

PIZZAZZ!
The ideal Web
page as seen

by someone
whose job is…

CEO

Developer

Designer

Business development

Take designers and developers, for instance. Designers tend to think that most
people like sites that are visually interesting because they like sites that are
visually interesting. In fact, they probably became designers because they enjoy
good design; they find that it makes things more interesting and easier to
understand.2
Developers, on the other hand, tend to think people like sites with lots of cool
features because they like sites with lots of cool features.
The result is that designers want to build sites that look great, and developers
want to build sites with interesting, original, elegant features. I’m not sure who’s
the farmer and who’s the cowman in this picture, but I do know that their
differences in perspective often lead to conflict—and hard feelings—when it
comes time to establish design priorities.

1

In the play, the thrifty, God-fearing, family-oriented farmers are always at odds with the
freewheeling, loose-living cowmen. Farmers love fences, cowmen love the open range.

2

Yes, I’m dealing in stereotypes here. But I think they’re useful stereotypes.

[ 126 ]


t h e fa r m e r a n d t h e cow m a n

At the same time, designers and programmers find themselves siding together in
another, larger clash between what Art Kleiner describes as the cultures of hype
and craft.3
While the hype culture (upper management, marketing, and business
development) is focused on making whatever promises are necessary to attract
venture capital, users, strategic partners, and revenue-generating deals to the
site, the burden of delivering on those promises lands on the shoulders of the
craft culture artisans like the designers and programmers.
This Internet version of the perennial struggle between art and commerce (or
perhaps farmers and cowmen vs. the railroad barons) adds another level of
complexity to any discussions of usability issues—often in the form of apparently
arbitrary edicts handed down from the hype side of the fence.4
The CEO likes the site, but
he wants everything to be
twice as large as it is…

…in time for the trade
show next week.

3

See “Corporate Culture in Internet Time” in strategy+business magazine

(www.strategy-business.com/press/article/10374, free registration required).

4

I once saw a particularly puzzling feature on the Home page of a prominent—and otherwise
sensibly designed—site. When I asked about it, I was told, “Oh, that. It came to our CEO in a
dream, so we had to add it.” True story.

[ 127 ]


c h a pt e r 8

The myth of the Average User
The belief that most Web users are like us is enough to produce gridlock in the
average Web design meeting. But behind that belief lies another one, even more
insidious: the belief that most Web users are like anything.
As soon as the clash of personal and professional opinions results in a stalemate,
the conversation usually turns to finding some way (whether it’s an expert
opinion, research, focus groups, or user tests) to determine what most users like
or don’t like—to figure out what the Average Web User is really like. The only
problem is, there is no Average User.
In fact, all of the time I’ve spent watching people use the Web has led me to
the opposite conclusion: all Web users are unique, and all Web use is
basically idiosyncratic.
The more you watch users carefully and listen to them articulate their intentions,
motivations, and thought processes, the more you realize that their individual
reactions to Web pages are based on so many variables that attempts to describe
users in terms of one-dimensional likes and dislikes are futile and counterproductive. Good design, on the other hand, takes this complexity into account.
And the worst thing about the myth of the Average User is that it reinforces the

idea that good Web design is largely a matter of figuring out what people like. It’s
an attractive notion: either pulldowns are good (because most people like them),
or they’re bad (because most people don’t). You should have links to everything in
the site on the Home page, or you shouldn’t. Menus on the top work better than
menus down the side. Frames, pages that scroll, etc. are either good or bad, black
or white.
The problem is there are no simple “right” answers for most Web design
questions (at least not for the important ones). What works is good, integrated
design that fills a need—carefully thought out, well executed, and tested.
Take the use of Flash, for example.5 If asked, some percent of users will say they
really like Flash, and an equal percent will probably say they hate it. But what
5

Flash, Macromedia’s tool for creating animated and interactive user interfaces, not flash
(lowercase), the arbitrary use of whiz-bang features to make a site more interesting.

[ 128 ]


t h e fa r m e r a n d t h e cow m a n

they really hate is Flash used badly: large, complicated animations that take a
long time to download and don’t add any value. If you observe them carefully and
ask the right questions, you’ll likely find that these same people will appreciate
sites that use small, hardworking, well-thought-out bits of Flash to add a
pleasant bit of sizzle or useful functionality without getting in the way.
That’s not to say that there aren’t some things you should never do, and some things
you should rarely do. There are some ways to design Web pages that are clearly
wrong. It’s just that they aren’t the things that Web teams usually argue about.

The antidote for religious debates
The point is, it’s not productive to ask questions like “Do most people like
pulldown menus?” The right kind of question to ask is “Does this pulldown, with
these items and this wording in this context on this page create a good experience
for most people who are likely to use this site?”
And there’s really only one way to answer that kind of question: testing. You have
to use the collective skill, experience, creativity, and common sense of the team to
build some version of the thing (even a crude version), then watch ordinary
people carefully as they try to figure out what it is and how to use it.
There’s no substitute for it.
Where debates about what people like waste time and drain the team’s energy,
testing tends to defuse arguments and break impasses by moving the discussion
away from the realm of what’s right or wrong and into the realm of what works
or doesn’t work. And by opening our eyes to just how varied users’ motivations,
perceptions, and responses are, testing makes it hard to keep thinking that all
users are like us.
Can you tell that I think testing is a good thing?
The next chapter explains how to test your own site.

[ 129 ]


789
9
c h a pt e r

Usability testing
on 10 cents a day
keeping testing simple—so you do enough of it

Why didn’t we do this sooner?
—what everyone says at some point during the
first usability test of their web site

A

bout once a month, I get one of these phone calls:

Ed Grimley at XYZ Corp
gave me your name.

…two weeks?

We’re launching our site
in two weeks and we want to do
some usability testing.

As soon as I hear “launching in two weeks” (or even “two months”) and “usability
testing” in the same sentence, I start to get that old fireman-headed-into-theburning-chemical-factory feeling, because I have a pretty good idea of what’s
going on.
If it’s two weeks, then it’s almost certainly a request for a disaster check. The
launch is fast approaching and everyone’s getting nervous, and someone finally
says, “Maybe we better do some usability testing.”
If it’s two months, then odds are that what they want is to settle some ongoing
internal debates—usually about something very specific like color schemes.
Opinion around the office is split between two different designs; some people
like the sexy one, some like the elegant one. Finally someone with enough clout
to authorize the expense gets tired of the arguing and says, “All right, let’s get
some testing done to settle this.”

[ 131 ]


c h a pt e r 9

And while usability testing will sometimes settle these arguments, the main
thing it usually ends up doing is revealing that the things they were arguing
about aren’t all that important. People often test to decide which color drapes are
best, only to learn that they forgot to put windows in the room. For instance, they
might discover that it doesn’t make much difference whether you go with the
horizontal navigation bar or the vertical menus if nobody understands the value
proposition of your site.
Sadly, this is how most usability testing gets done: too little, too late, and for all
the wrong reasons.

Repeat after me:
Focus groups are not usability tests.
Sometimes that initial phone call is even scarier:
…we’re launching our site in
two weeks and we want to do
some focus group testing.

Focus group
testing?

When the last-minute request is for a focus group, it’s usually a sign that the
request originated in Marketing. When Web sites are being designed, the folks in
Marketing often feel like they don’t have much clout. Even though they’re the
ones who spend the most time trying to figure out who the site’s audience is and

what they want, the designers and developers are the ones with most of the
hands-on control over how the site actually gets put together.

[ 132 ]


u s a b i l i t y t e s t i n g o n 1 0 c e n t s a day

As the launch date approaches, the Marketing people may feel that their only hope
of sanity prevailing is to appeal to a higher authority: research. And the kind of
research they know is focus groups.
I often have to work very hard to make clients understand that what they need is
usability testing, not focus groups. Here’s the difference in a nutshell:
> In a focus group, a small group of people (usually 5 to 8) sit around a table and
react to ideas and designs that are shown to them. It’s a group process, and much
of its value comes from participants reacting to each other’s opinions. Focus
groups are good for quickly getting a sampling of users’ opinions and feelings
about things.
> In a usability test, one user at a time is shown something (whether it’s a Web
site, a prototype of a site, or some sketches of individual pages) and asked to
either (a) figure out what it is, or (b) try to use it to do a typical task.
Focus groups can be great for determining what your audience wants, needs, and
likes—in the abstract. They’re good for testing whether the idea behind the site
makes sense and your value proposition is attractive. And they can be a good way
to test the names you’re using for features of your site, and to find out how people
feel about your competitors.
But they’re not good for learning about whether your site works and how to improve it.
The kinds of things you can learn from focus groups are the things you need to
learn early on, before you begin designing the site. Focus groups are for EARLY in
the process. You can even run them late in the process if you want to do a reality

check and fine-tune your message, but don’t mistake them for usability testing.
They won’t tell you whether people can actually use your site.

Several true things about testing
Here are the main things I know about testing:
> If you want a great site, you’ve got to test. After you’ve worked on a site for
even a few weeks, you can’t see it freshly anymore. You know too much. The
only way to find out if it really works is to test it.

[ 133 ]


c h a pt e r 9

Testing reminds you that not everyone thinks the way you do, knows what you
know, uses the Web the way you do.
I used to say that the best way to think about testing was that it was like travel:
a broadening experience. It reminds you how different—and the same—people
are, and gives you a fresh perspective on things.
But I finally realized that testing is really more like having friends visiting from
out of town. Inevitably, as you make the tourist rounds with them, you see
things about your home town that you usually don’t notice because you’re so
used to them. And at the same time, you realize that a lot of things that you
take for granted aren’t obvious to everybody.
> Testing one user is 100 percent better than testing none. Testing always
works, and even the worst test with the wrong user will show you important
things you can do to improve your site. I make a point of always doing a live
user test at my workshops so that people can see that it’s very easy to do and it
always produces an abundance of valuable insights. I ask for a volunteer and
have him try to perform a task on a site belonging to one of the other attendees.

These tests last less than ten minutes, but the person whose site is being tested
usually scribbbles several pages of notes. And they always ask if they can have
the recording of the test to show to their team back home. (One person told me
that after his team saw the recording, they made one change to their site which
they later calculated had resulted in $100,000 in savings.)
> Testing one user early in the project is better than testing 50 near the
end. Most people assume that testing needs to be a big deal. But if you make it
into a big deal, you won’t do it early enough or often enough to get the most out
of it. A simple test early—while you still have time to use what you learn from
it—is almost always more valuable than a sophisticated test later.
Part of the conventional wisdom about Web development is that it’s very easy
to go in and make changes. The truth is, it turns out that it’s not that easy to
make changes to a site once it’s in use. Some percentage of users will resist
almost any kind of change, and even apparently simple changes often turn out
to have far-reaching effects, so anything you can keep from building wrong in
the first place is gravy.

[ 134 ]


u s a b i l i t y t e s t i n g o n 1 0 c e n t s a day

> The importance of recruiting representative users is overrated. It’s good
to do your testing with people who are like the people who will use your site,
but it’s much more important to test early and often. My motto—as you’ll see—
is “Recruit loosely, and grade on a curve.”
> The point of testing is not to prove or disprove something. It’s to
inform your judgment. People like to think, for instance, that they can use
testing to prove whether navigation system “a” is better than navigation system
“b”, but you can’t. No one has the resources to set up the kind of controlled

experiment you’d need. What testing can do is provide you with invaluable input
which, taken together with your experience, professional judgment, and
common sense, will make it easier for you to choose wisely—and with greater
confidence—between “a” and “b.”
> Testing is an iterative process. Testing isn’t something you do once.
You make something, test it, fix it, and test it again.
> Nothing beats a live audience reaction. One reason why the Marx
Brothers’ movies are so wonderful is that before they started filming
they would go on tour on the vaudeville circuit and perform scenes
from the movie, doing five shows a day, improvising constantly and
noting which lines got the best laughs. Even
Mrs. Teasdale (Margaret
after they’d settled on a line, Groucho
Dumont) and Rufus T. Firefly
would insist on trying slight variations to
eavesdrop in Duck Soup.
see if it could be improved.

Lost our lease, going-out-of-businesssale usability testing
Usability testing has been around for a long time, and the basic idea is pretty
simple: If you want to know whether your software or your Web site or your
VCR remote control is easy enough to use, watch some people while they try to
use it and note where they run into trouble. Then fix it, and test it again.
In the beginning, though, usability testing was a very expensive proposition. You
had to have a usability lab with an observation room behind a one-way mirror,
and at least two video cameras so you could record the users’ reactions and the
thing they were using. You had to recruit a lot of people so you could get results

[ 135 ]

c h a pt e r 9

THE TOP FIVE PLAUSIBLE EXCUSES FOR NOT TESTING WEB SITES
We don’t have
the time.

We don’t have
the money.

We don’t have
the expertise.

We don’t have a
usability lab.

We wouldn’t know
how to interpret
the results.

It’s true that most Web development schedules seem to be based
on the punchline from a Dilbert cartoon. If testing is going to add
to everybody’s to-do list, if you have to adjust development
schedules around tests and involve key people in preparing for
them, then it won’t get done. That’s why you have to make testing
as small a deal as possible. Done right, it will save time, because you
won’t have to (a) argue endlessly, and (b) redo things at the end.
Forget $5,000 to 15,000. If you can convince someone to bring in
a camcorder from home, you’ll only need to spend about $300 for
each round of tests.

The least-known fact about usability testing is that it’s incredibly
easy to do. Yes, some people will be better at it than others, but
I’ve never seen a usability test fail to produce useful results, no
matter how poorly it was conducted.

You don’t need one. All you really need is a room with a desk, a
computer, and two chairs where you won’t be interrupted.

One of the nicest things about usability testing is that the
important lessons tend to be obvious to everyone who’s watching.
The serious problems are hard to miss.

that were statistically significant. It was Science. It cost $20,000 to $50,000 a shot.
It didn’t happen very often.
But in 1989 Jakob Nielsen wrote a paper titled “Usability Engineering at a
Discount”1 and pointed out that it didn’t have to be that way. You didn’t need a
1

Proceedings of the Third International Conference on Human-Computer Interaction, Boston,
MA, Sept. 1989.

[ 136 ]


u s a b i l i t y t e s t i n g o n 1 0 c e n t s a day

usability lab, and you could achieve the same results with a lot fewer users.
The idea of discount usability testing was a huge step forward. The only problem
is that a decade later most people still perceive testing as a big deal, hiring

someone to conduct a test still costs $5,000 to $15,000, and as a result it doesn’t
happen nearly often enough.
What I’m going to commend to you in this chapter is something even more
drastic: Lost our lease, going-out-of-business-sale usability testing.
I’m going to try to explain how to do your own testing when you have no money
and no time. Don’t get me wrong: If you can afford to hire a professional to do your
testing, by all means do it! But don’t do it if it means you’ll do less testing.

TRADITIONAL TESTING

LOST-OUR-LEASE TESTING

NUMBER OF
USERS PER TEST

Usually eight or more to justify the
set-up costs

Three or four

RECRUITING
EFFORT

Select carefully to match target
audience

Grab some people. Almost anybody who
uses the Web will do.

WHERE TO TEST

A usability lab, with an observation
room and a one-way mirror

Any office or conference room

WHO DOES
THE TESTING

An experienced usability professional

Any reasonably patient human being

ADVANCE
PLANNING

Tests have to be scheduled weeks in
advance to reserve a usability lab and
allow time for recruiting

Tests can be done almost any time, with
little advance scheduling

PREPARATION

Draft, discuss, and revise a test
protocol

Decide what you’re going to show

WHAT/WHEN
DO YOU TEST?

Unless you have a huge budget, put all
your eggs in one basket and test once
when the site is nearly complete

Run small tests continually throughout
the development process

COST

$5,000 to $15,000 (or more)

$300 (a $50 to $100 stipend
for each user) or less

WHAT HAPPENS
AFTERWARDS

A 20-page written report appears a
week later, then the development team
meets to decide what changes to make

The development team (and interested
stakeholders) debrief over lunch the
same day

[ 137 ]

c h a pt e r 9

How many users should you test?
In most cases, I tend to think the ideal number of users for each round of testing is
three, or at most four.
The first three users are very likely to encounter nearly all of the most significant
problems,2 and it’s much more important to do more rounds of testing than to
wring everything you can out of each round. Testing only three users helps
ensure that you will do another round soon.3
Also, since you will have fixed the problems you uncovered in the first round, in
the next round it’s likely that all three users will uncover a new set of problems,
since they won’t be getting stuck on the first set of problems.
Testing only three or four users also makes it possible to test and debrief in the
same day, so you can take advantage of what you’ve learned right away. Also,
when you test more than four at a time, you usually end up with more notes than
anyone has time to process—many of them about things that are really “nits,”
which can actually make it harder to see the forest for the trees.
In fact this is one of the reasons why I’ve almost completely stopped generating
written reports (what I refer to as the “big honking report”) for my expert
reviews and for usability tests. I finally realized that for most Web teams their
ability to find problems greatly exceeds the resources they have available to fix
them, so it’s important to stay focused on the most serious problems. Instead of
written reports, nowadays I report my findings in a conference call with the
entire Web team, which may last for an hour or two. By the end of the call, we’ve
all agreed which problems are most important to fix, and how they’re going to fix
them.

2

See Jakob Nielsen’s March 2000 Alertbox column “Why You Only Need to Test with 5 Users”
at www.useit.com for a good discussion of the topic.

3

If you’re hiring someone to do the testing for you and money is no object, you might as well
test six or eight users since the additional cost per user will be comparatively low. But only if
it won’t mean you’ll do fewer rounds of testing.

[ 138 ]


u s a b i l i t y t e s t i n g o n 1 0 c e n t s a day

ONE TEST WITH 8 USERS
8 users

TOTAL PROBLEMS
FOUND: 5

Eight users may
find more problems
in a single test.
But the worst problems will usually
keep them from
getting far enough
to encounter
some others.

TOTAL PROBLEMS

FOUND: 9

TWO TESTS WITH 3 USERS
Second test: 3 users

First test: 3 users

But in the
second test,
with the first
set of problems
fixed, they’ll
find problems
they couldn’t
have seen in
the first test.

Three users may
not find as many
problems in a
single test.

Recruit loosely and grade on a curve
When people decide to test, they often spend a lot of time trying to recruit users
who they think will precisely reflect their target audience—for instance, male
accountants between the ages of 25 and 30 with one to three years of computer
experience who have recently purchased expensive shoes.
The best-kept secret of usability testing is the extent to which it doesn’t much
matter who you test.
For most sites, all you really need are people who have used the Web enough to

know the basics.

[ 139 ]


c h a pt e r 9

If you can afford to hire someone to recruit the participants for you and it won’t
reduce the number of rounds of testing that you do, then by all means be as
specific as you want. But if finding the ideal user means you’re going to do fewer
tests, I recommend a different approach:
Take anyone you can get (within limits) and grade on a curve.
In other words, try to find users who reflect your audience, but don’t get hung up
about it. Instead, try to make allowances for the differences between the people
you test and your audience. I favor this approach for three reasons:
> We’re all beginners under the skin. Scratch an expert and you’ll often find
someone who’s muddling through—just at a higher level.
> It’s usually not a good idea to design a site so that only your target
audience can use it. If you design a site for accountants using terminology
that you think all accountants will understand, what you’ll probably discover
is that a small but not insignificant number of accountants won’t know what
you’re talking about. And in most cases, you need to be addressing novices as
well as experts anyway, and if your grandmother can use it, an expert can.
> Experts are rarely insulted by something that is clear enough for
beginners. Everybody appreciates clarity. (True clarity, that is, and not just
something that’s been “dumbed down.”)
The exceptions:
> If your site is going to be used almost exclusively by one type of user and
it’s no harder to recruit from that group, then do it. For instance, if your
audience will be almost entirely women, then by all means test just women.

> If your audience is split between clearly defined groups with very
divergent interests and needs, then you need to test users from each group
at least once. For instance, if you’re building a university site, for at least one
round of testing you want to recruit two students, two professors, two high
school seniors, and two administrators. But for the other rounds, you can
choose any mix.

[ 140 ]


u s a b i l i t y t e s t i n g o n 1 0 c e n t s a day

> If using your site requires specific domain knowledge (e.g., a currency
exchange site for money management professionals), then you need to recruit
people with that domain knowledge for at least one round of tests. But don’t do
it for every round if it will reduce the number of tests you do.
When you’re recruiting:
> Offer a reasonable incentive. Typical stipends for a one-hour test session
range from $50 for “average” Web users to several hundred dollars for
professionals from a specific domain, like cardiologists for instance. I like to
offer people a little more than the going rate, since (a) it makes it clear that I
value their opinion, and (b) people tend to show up on time, eager to
participate. Remember, even if the session is only 30 minutes, people usually
have to block out another hour for travel time. Also, I’d rather have people who
are curious about the process than people who are desperate for the money.
> Keep the invitation simple. “We need to have a few people look at our Web
site and give us some feedback. It’s very easy, and would take about forty-five
minutes to an hour. And you’ll be paid $___ for your time.”
> Avoid discussing the site (or the organization behind the site)
beforehand. You want their first look to tell you whether they can figure out

what it is from a standing start. (Of course, if they’re coming to your office,
they’ll have a pretty good idea whose site it is.)
> Don’t be embarrassed to ask friends and neighbors. You don’t have to feel
like you’re imposing if you ask friends or neighbors to participate. Most people
enjoy the experience. It’s fun to have someone take your opinion seriously and
get paid for it, and they often learn something useful that they didn’t know
about the Web or computers in general.

[ 141 ]