Usability testing your website with real users

Mon, Jun 1, 2009

Testing / Optimization

Why you need to test your website with live users

Usability of a website is ridiculously important. Anyone who has run an e-commerce website for more than a minute will be able to relate any number of grim tales of abandoned shopping carts and aborted transactions. Though it’s easy to fall prey to hubris and complain that all such failures to close are the result of “user error”, the bottom line is the bottom line—making the website difficult to use means lost sales: sometimes huge numbers of lost sales. A recent test of shopping sites showed that almost half the tasks set to the test shoppers weren’t completed successfully!

The best way of catching design flaws is through live usability testing, and yet the common perception is that such testing is a very expensive and difficult proposition. Once upon a time, that may have been true, but now anyone can do their own usability testing on a very modest budget.

How to test your website with real users

There are a few different ways to set up your own usability testing. We’ll focus on a streamlined process on a modest budget.

Begin by identifying your precise and quantifiable goals for both the website and the usability testing. Next, identify the different user groups/demographics using your site and source a minimum of 5 people from each identified group (5 is a minimum number, but more than 5 and you start to get into diminishing returns territory quickly – see the graph below.)

What’s really interesting about the graph above is that it demonstrates that even only testing 3 users will show up more than 60% of your issues, and testing ANYONE is better than testing no one at all.

Now, you need to design and prepare a list of realistic/typical tasks for the participants to complete on your website. These will comprise the basis of your usability testing.

For the setup of the test itself, you have a few options. If you have a space in which you can run it, all you need are two chairs, a PC or Mac (with an Internet connection, if you’re testing a live site), a camcorder, and a tripod. If you’d rather do it remotely, then you could use something like WebEx to allow users to use their own computers. You could also use software like Camtasia or AutoScreenrecorder to record the users’ sessions for later discussion and review.

Do the test! Begin by asking the participant what his first impressions of the site are. Instruct participants to think aloud at all times. What captures their attention? What would they do first? Then assign the participants specific tasks to carry out. Monitor them as they attempt to complete these tasks. Watch, record, and listen. Note what they find easy, what they find confusing, and what they find impossible. Try to keep the testing of each participant between 45 minutes and an hour long.

This can be frustrating! Things that seem “obvious” to you will prove not to be for other people. Expect to be shocked to see how unfamiliar users are with navigating your site, or even with basic browser functions. Don’t fall prey to the temptation to touch the screen, the keyboard or the mouse; don’t explain how to do anything – by all means provide encouragement and reassurance, or ask questions about why they did something, but no more than that. Expect to be surprised.

Once the testing is complete, conduct and record a brief interview with each person, discussing their experience. When you’ve finished collecting all the data, review it with your team and any other stakeholders, fix any usability issues you find, and then come back and test again. Another test will discover whether the “fixes” worked or not. You have to be wary of the possibility of the fixes introducing new usability issues.


In its simplest form, web site usability testing requires only three ingredients: a Web site (or functional prototype), a usability tester, and an observer. The usability tester navigates the application or Web site as a real user would, and the observer identifies usability problems by observing the tester’s efforts. Add a few ingredients — more testers, more observers, a facilitator, a set of well-crafted test scenarios, and some video equipment — and you’ve got a recipe for success.

For a great overview of usability testing on a budget, we recommend reading this chapter from Krug’s excellent book.

Thoughts? Comments? Your experiences? Let us have it!

, , , ,

23 Responses to “Usability testing your website with real users”

  1. Grace Says:

    I agree– usability is extremely important. I think cross-browser usability also goes into that category. If you search it up on Google, you’ll come up with websites that let you view your website in other browsers and the loading times, etc.

    Grace’s last blog post..111 Places for Awesome Free and Premium Templates


  2. NetSurfer Says:

    Usability of the website is one of the greatest factors to consider when designing. Internet surfers these days tend to be not so patient when it comes to seeing websites cluttered.


  3. Getitfromme Says:

    True that. Usablity is the key. Once surfer arrives, he has to be able to actually use the site and understand the content. Otherwise, your site is a waste of one’s time.


  4. J Bode Says:

    I totally agree usability is very important…

    I design wordpress blog themes and I always try to create blogs that use space wisely as well as work in all browsers…

    unfortunately, some internet browsers are in a hurry and want to quickly find what they are looking for, make it easy for them…

    this is especially important for selling products and building a mailing list

    Thanks for the great post,

    J Bode

    J Bode’s last blog post..5 Cool Twitter Tricks You Should Be Using


    • Orange Says:

      I also design sites using word press and yes, this post is very useful. Usability must be in fact a top priority when designing websites.


  5. Sharif Says:

    Nice Post .

    Sharif’s last blog post..Sacrifice-Elton John (Fav song)


  6. keysxtacy Says:

    Key is Usability. Knowing how to give your users what they want. Great post!


  7. Saosin Says:

    I happened to have this book by Krug’s and it’s relly useful. Highly recommended.


  8. Bj Says:

    Krug is the man. Can’t wait for more of his inputs.


  9. Guy Says:

    I have learned from this post. Great job!


  10. Counter Says:

    Ineresting entry. This is very useful. Thanks.


  11. Drew Says:

    For among many designers, this should be addressed well. Usabilty. Enough said.


  12. Beyond Says:

    Usability in all aspect of webdesign is very important. It shouldn’t be overlooked.


  13. Coldplay Says:

    Haha. It is relly frustrating! I tried what you post here and I really should be patient when doing this.


  14. Fotta Says:

    Very useful post! Now I need to get people whom I can work with the testing.


  15. Dota Says:

    Goodluck Fotta. Gotta do a lot of working there…


  16. Rob Says:

    I always thought 8 test subjects were enough, and by looking at the graph it looks about right. Jakob recently released some data that found a designer working solo had a 50/50 chance of getting a design right, and that by using just 2 subjects could make a world of difference.

    I ran a poll a while back asking webmasters if they have watched at least five people complete a task on a website they developed. 61% said no. Pretty sad.
    .-= Rob´s last blog ..Fry’s Tanks My Day =-.


  17. Inbox Says:

    Testing is really frustrating so gotta have a lot of time and patience for doing it.


  18. Likod Says:

    Check with other browsers as well if your site works.


  19. MyDailyBlog.Info Says:

    thanks,good info
    .-= MyDailyBlog.Info´s last blog ..Dapatkan Maklumat Yang Detail Dengan QuarkBase! =-.


  20. ThinkLikeaUser Says:

    Someone just tweeted this link :)

    I like the correlation you are making between making money and making a website effective for the users.

    Prospects only become customers when they “know like and trust” the seller – and you can’t “know like and trust” a website that is difficult, inflexible, boring or confusing – all the things a usability study would cover.

    Lets hope it’s not just the big guys who focus on usability – SMEs need to do it to – especially as Google Local Search takes off.
    .-= ThinkLikeaUser´s last blog ..World Usability Day 2009 – What I’m Doing. =-.


  21. Mohammed Alaa Says:

    finally someone talks about usability test for live users. it’s not recommended it’s a must since your main goal is simple -> SUCCESS and generate more money.
    .-= Mohammed Alaa´s last blog ..About Senior Web Designer =-.



  1. Twitted by HCInews - 08. Jun, 2009

    [...] This post was Twitted by HCInews – [...]

Leave a Reply