Thursday, July 12, 2012

Applying Some Nielsen Principles

Oh boy, it's been about a month since the last post!  Yet again, a lot has happened.

I went to another Design Research Support Group meeting - always fun and enlightening.  I brought homemade baked goods (cookies) again and have apparently set a precedent/developed a reputation for myself.  Must think of more things to bake to keep this going.  I also volunteered to host the next meeting.  I sent out the invitation with a warning about our dog MAYBE barking/growling as people enter.  Hopefully no one is scared of a 78 pound pit lab mix.

As per usual, I got some really useful tips out of the meeting.  One of the women had just recently started working at a startup and she posed a problem to the group.  She had done user research with 6 participants and wanted advice on how to present her findings to the engineers and not have them come away thinking "she only talked to six people?" and thinking that she probably just spoke to 6 friends.  The advice started right away from multiple people in the group:
  • Show the engineers the videos from the participants.  One guy from the group said that what he does is create a private Vimeo account and has the engineers log in and watch the usability tests.  Some issues become so readily apparent by just watching someone use the product.
  • Let the engineers know how the participants were recruited and that it's 6 participants, but that's 60 minutes times 6 people, which equals a lot of time and information.  
  • Do some mockups and wireframes to offer possible solutions so that you're not just telling the engineers all that is wrong.  Offer potential next steps.
  • Research shows that after 5 participants, the usefulness of getting more feedback tapers off.  Show Jakob Nielsen's graph (shown below) of why you only need to test with 5 users.  
We (the group) got an e-mail from the Researcher about a week later saying that she had presented her interviews/usability testing to the engineers and received rave reviews.  She showed them Jakob Nielsen's graph and edited a highlight reel of usability problems uncovered during testing.  The front-end developer asked her to add her recommendations to their project management tool.  It was her first meeting she had attended and she got some immediately useful/applicable advice!

Other things - the online Stanford Human Computer Interaction class has wrapped up, but I still have to catch up on a few video lectures.  The 5-week class went over a very wide range of topics!  Some were covered briefly, like typography, which I'm sure you could spend A LOT more time learning about.  I'm glad that I got a taste of so many subjects involved in HCI and overall, I think it was a great class.

Just to give you a taste of what else I've been learning, I recently watched a lecture about Heuristic Evaluation.  This was developed by Jakob Nielsen about 20 years ago.  I guess the theme of this entry might be Jakob Nielsen.  Heuristic Evaluation is less expensive than User Testings, since you use a small set of evaluators (3-5).
  • Evaluators examine the UI independently and check for compliance with usability principles ("heuristics").
  • Different evaluators will find different problems.  A single evaluator achieves poor results and only finds 35% of usability problems.  Five evaluators find 75% of problems. 
  • Evaluators only communicate afterwards.  Findings are then aggregated.
  • This can be performed on working UI or sketches. 
  • The evaluators step through the designs several times.  They examine the details, flow, and architecture, and they consult a list of usability principles. 
From what I can tell, Heuristic Evaluation is very similar to "Expert Review" except that Expert Review may be more holistic and qualitative and takes into account the expert's own personal experience.

In other news, I'm still off and running with Shoefitr.  We've been conducting user interviews, concept validation, and we have even intercepted people on the street and in shops to ask them to look at our designs and give feedback.  Learning a lot!


5 comments:

  1. Here's some math nerdiness: Coming from a psych research background, I too thought, "N=6? Pshawwww." But then I considered that for most lab experiments there are multiple conditions being tested, and that if you have, for example a 2X2X2 design (relatively simple), and you counterbalance with an N of 96, that's 12 people in a specific condition. Since it sounds like for the usability experiment testers were looking at a single version of the product, 6 is actually a good sample. Nielsen's graph is pretty on point also in terms of pointing out that people will find errors pretty efficiently. I think that when presenting findings from a small sample it definitely helps to do all of the items mentioned above, and also show engineers what does work and what people find intuitive and good, not just errors.

    ReplyDelete
  2. Hey just bumped into your blog. Fun reading! What meetup are you referring to? Sounds fun! I'd love to be a part of it. check out designing for the digital age - great book! Best, m

    ReplyDelete
    Replies
    1. Hey! Sorry to reply after SO LONG! I need to check this email address more often. It's not a formal meetup, it's a google group - it's actually become more of a distribution list at this point where people crowd source advice. If you email me your email address, I can tell you more! betagraceblog@gmail.com

      Delete
  3. I was surfing the Internet for information and came across your blog. I am impressed by the information you have on this blog. It shows how well you understand this subject. NIW

    ReplyDelete