<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">whoops - this was meant for talk!<br><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>From: </b></span><span style="font-family:'Helvetica'; font-size:medium;">Peter Batty <<a href="mailto:peter.batty@gmail.com">peter.batty@gmail.com</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Date: </b></span><span style="font-family:'Helvetica'; font-size:medium;">March 9, 2010 9:32:48 AM EST<br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>To: </b></span><span style="font-family:'Helvetica'; font-size:medium;">SteveC <<a href="mailto:steve@asklater.com">steve@asklater.com</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Cc: </b></span><span style="font-family:'Helvetica'; font-size:medium;">Talk Openstreetmap <<a href="mailto:talk-us@openstreetmap.org">talk-us@openstreetmap.org</a>><br></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;"><span style="font-family:'Helvetica'; font-size:medium; color:rgba(0, 0, 0, 1);"><b>Subject: </b></span><span style="font-family:'Helvetica'; font-size:medium;"><b>Re: [Talk-us] UX Review</b><br></span></div><br>I very highly recommend this book on web usability for ideas on how to do usability testing: 'Don't Make Me Think: A Common Sense Approach to Web Usability' by Steve Krug <a href="http://bit.ly/bHJ8ni">http://bit.ly/bHJ8ni</a>. A friend of mine in Denver is a great usability person and we've used this approach on several projects and found it very useful.<div>
<br></div><div>A couple of key things I took from my experience in this area: usability testing really isn't complicated - the main thing is that you just need to do it, not assume that you know what will be obvious to someone else. You sit someone down in front of the system, give them a scenario or a task to do, and watch them. It is humbling and frustrating to watch someone else use your software with no instructions. You have to resist the temptation to jump in to tell them what to do. You also ask them to "think out loud" about what they are trying to do. That's pretty much it. I think that doing a few of these in person tests will tell you much more, more easily, than larger scale automated tests. Things that seem obvious to you but not to others come out really quickly.</div>
<div><br></div><div>The second surprising thing is that you need far fewer tests than you think to provide useful insights. The times I've done it, the glaring issues were obvious after testing 2 or 3 people. We would fix those issues and then move on to another small set of tests.</div>
<div><br></div><div>I'd be happy to work with you to find a few volunteers to test, and record and publish the results. As you say, we need to figure out the right scenarios to test.</div><div><br></div><div>Cheers,</div>
<div> Peter.<br><br><div class="gmail_quote">On Tue, Mar 9, 2010 at 7:02 AM, SteveC <span dir="ltr"><<a href="mailto:steve@asklater.com">steve@asklater.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; ">
Dear all<br><br>One of the clear pieces of feedback from all the talk about improving the OSM UX was "show us these users who really find it difficult to use OSM". So, we're going to do that. We have a rough plan of action below<br>
<br>Nate Bolt of the splendiferous <a href="http://boltpeters.com/" target="_blank">boltpeters.com</a> has volunteered to help OSM with a user interaction review. Nate, as part of Bolt | Peters, did exactly this work with wikipedia:<br>
<br> <a href="http://usability.wikimedia.org/wiki/Usability_and_Experience_Study" target="_blank">http://usability.wikimedia.org/wiki/Usability_and_Experience_Study</a><br><br>Where they basically screen recorded people trying to edit in wikipedia with the following goals:<br>
<br>• identify obstacles that novice users encounter in editing a Wikipedia article—including, but not limited to—adding personal content, fixing a typo, adding a reference, and contributing to discussion pages<br>• identify obstacles in creating a new article<br>
• evaluate the self-sufficiency and legibility of help materials and documents found on <a href="http://Wikipedia.org">Wikipedia.org</a><br>• evaluate how novice users interact with templates<br>• discover user experience patterns and issues that have not been previously identified.<br>
<br>Together with Mike Migurski, famed geohead, creator of walking papers and graphical butterfly at <a href="http://Stamen.com">Stamen.com</a>, we've come up with a rough plan to get feedback from new users to OSM. It's a very similar approach to that taken by Bolt | Peters with wikipedia. We want to find out from the real newbies what the issues are and lay them out clearly. Nate is independent of OSM and will be able to present a cold hard look at what's good and what we need to work on to improve things so we get more newbies contributing.<br>
<br>* A small piece of javascript from <a href="http://ethnio.com/" target="_blank">ethnio.com</a> is put on the new user page in openstreetmap<br>* Once in some very small sample size (perhaps between 1 in 1,000 and 1 in 10,000 signups) a popup appears<br>
* The popup says something like "Hi! We'd really like to know why you came to OSM" and they say simply why. This is open ended on purpose so we catch as many things as we can, not just what we're looking for, but things we won't expect.<br>
* They're offered to record a short (10 minute max) screencast of them trying to achieve whatever it is (like look at a map, find OSMers, add a PoI and so on)<br>* That screencast is analyzed in aggregate with many others by Bolt | Peters with all their expertise in doing this stuff, and they come back with a set of findings.<br>
<br>We're looking at both <a href="http://www.usertesting.com/" target="_blank">http://www.usertesting.com/</a> and <a href="http://www.openhallway.com/" target="_blank">http://www.openhallway.com/</a> to do the recording. Both Mike and I will pay for it, and might solicit donations and stuff if it looks beyond our budget.<br>
<br>I can't say this enough, and I always get responses from people who think that I just set something out in stone - so I'll be super super super clear: NONE OF THIS IS SET IN STONE. We need your feedback on everything before we go ahead. I mean _everything_. So, please tell us what you think about it all. Here are some questions:<br>
<br>What should our goals be? (General UX? How good/bad signup is? How good/bad editing is? How is it finding info?)<br>How often should we ask a signup for feedback? (the more the better but we can only look at so many)<br>
How can we include more crowd source feedback? (I think of asking random signups for feedback as crowdsourcing it)<br>What else should we think about?<br><br>Yours &c.<br><br>Steve<br><br>_______________________________________________<br>
Talk-us mailing list<br><a href="mailto:Talk-us@openstreetmap.org">Talk-us@openstreetmap.org</a><br><a href="http://lists.openstreetmap.org/listinfo/talk-us" target="_blank">http://lists.openstreetmap.org/listinfo/talk-us</a><br>
</blockquote></div>:<br></div>
</blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Yours &c.</div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br></div><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Steve</div></span></span>
</div>
<br></body></html>