[Talk-us] UX Review

Peter Batty peter.batty at gmail.com
Tue Mar 9 14:32:48 GMT 2010


I very highly recommend this book on web usability for ideas on how to do
usability testing: 'Don't Make Me Think: A Common Sense Approach to Web
Usability' by Steve Krug http://bit.ly/bHJ8ni. A friend of mine in Denver is
a great usability person and we've used this approach on several projects
and found it very useful.

A couple of key things I took from my experience in this area: usability
testing really isn't complicated - the main thing is that you just need to
do it, not assume that you know what will be obvious to someone else. You
sit someone down in front of the system, give them a scenario or a task to
do, and watch them. It is humbling and frustrating to watch someone else use
your software with no instructions. You have to resist the temptation to
jump in to tell them what to do. You also ask them to "think out loud" about
what they are trying to do. That's pretty much it. I think that doing a few
of these in person tests will tell you much more, more easily, than larger
scale automated tests. Things that seem obvious to you but not to others
come out really quickly.

The second surprising thing is that you need far fewer tests than you think
to provide useful insights. The times I've done it, the glaring issues were
obvious after testing 2 or 3 people. We would fix those issues and then move
on to another small set of tests.

I'd be happy to work with you to find a few volunteers to test, and record
and publish the results. As you say, we need to figure out the right
scenarios to test.

Cheers,
    Peter.

On Tue, Mar 9, 2010 at 7:02 AM, SteveC <steve at asklater.com> wrote:

> Dear all
>
> One of the clear pieces of feedback from all the talk about improving the
> OSM UX was "show us these users who really find it difficult to use OSM".
> So, we're going to do that. We have a rough plan of action below
>
> Nate Bolt of the splendiferous boltpeters.com has volunteered to help OSM
> with a user interaction review. Nate, as part of Bolt | Peters, did exactly
> this work with wikipedia:
>
>        http://usability.wikimedia.org/wiki/Usability_and_Experience_Study
>
> Where they basically screen recorded people trying to edit in wikipedia
> with the following goals:
>
> • identify obstacles that novice users encounter in editing a Wikipedia
> article—including, but not limited to—adding personal content, fixing a
> typo, adding a reference, and contributing to discussion pages
> • identify obstacles in creating a new article
> • evaluate the self-sufficiency and legibility of help materials and
> documents found on Wikipedia.org
> • evaluate how novice users interact with templates
> • discover user experience patterns and issues that have not been
> previously identified.
>
> Together with Mike Migurski, famed geohead, creator of walking papers and
> graphical butterfly at Stamen.com, we've come up with a rough plan to get
> feedback from new users to OSM. It's a very similar approach to that taken
> by Bolt | Peters with wikipedia. We want to find out from the real newbies
> what the issues are and lay them out clearly. Nate is independent of OSM and
> will be able to present a cold hard look at what's good and what we need to
> work on to improve things so we get more newbies contributing.
>
> * A small piece of javascript from ethnio.com is put on the new user page
> in openstreetmap
> * Once in some very small sample size (perhaps between 1 in 1,000 and 1 in
> 10,000 signups) a popup appears
> * The popup says something like "Hi! We'd really like to know why you came
> to OSM" and they say simply why. This is open ended on purpose so we catch
> as many things as we can, not just what we're looking for, but things we
> won't expect.
> * They're offered to record a short (10 minute max) screencast of them
> trying to achieve whatever it is (like look at a map, find OSMers, add a PoI
> and so on)
> * That screencast is analyzed in aggregate with many others by Bolt |
> Peters with all their expertise in doing this stuff, and they come back with
> a  set of findings.
>
> We're looking at both http://www.usertesting.com/ and
> http://www.openhallway.com/ to do the recording. Both Mike and I will pay
> for it, and might solicit donations and stuff if it looks beyond our budget.
>
> I can't say this enough, and I always get responses from people who think
> that I just set something out in stone - so I'll be super super super clear:
> NONE OF THIS IS SET IN STONE. We need your feedback on everything before we
> go ahead. I mean _everything_. So, please tell us what you think about it
> all. Here are some questions:
>
> What should our goals be? (General UX? How good/bad signup is? How good/bad
> editing is? How is it finding info?)
> How often should we ask a signup for feedback? (the more the better but we
> can only look at so many)
> How can we include more crowd source feedback? (I think of asking random
> signups for feedback as crowdsourcing it)
> What else should we think about?
>
> Yours &c.
>
> Steve
>
> _______________________________________________
> Talk-us mailing list
> Talk-us at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk-us
>
:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20100309/901b44b7/attachment.html>


More information about the Talk-us mailing list