All posts tagged as "Usability testing"

I am thrilled to announce that we have announced the first of the speakers, more details, and the registration page for the first North American conference focused on mobile design, Design for Mobile.

Speakers include researchers, visual designers, interaction designers, UI developers, and design strategists. They work in operators, device manufacturers, open source, academia, content companies, technology companies. They work on web, applications, services, devices, and best practices.

We've more in the works, so stay tuned.

Register now!

Usability pundit Jakob Nielsen has done much to promote usability, especially on the (desktop) web. He and the folks at the Nielsen Norman Group do a lot of meta-analysis on usability methods.

So when he measures usability ROI measures, be sure he is doing it across a large number of sites. He reports the average ROI for usability on web sites has declined from 135% (2002) to 83% (2008). What I found particularly interesting about this article is the first of his two explanations for the decline in ROI:

We have now harvested most of the low-hanging fruit from the truly horrible websites that dominated the lost decade of Web usability (approximately 1993–2003). In those early years, Web design was abominable — think splash screens, search that didn’t find anything, bloated graphics everywhere. The only good thing about these early designs was that they were so bad that it was easy for usability people to be heroes: even the smallest study would inevitably reveal several immense opportunities for improvement.

Now, over in the mobile space, the “low hanging fruit” is usually still present – even on Nokia devices and Google applications. I thus expect the mobile usability ROI metrics to be closer to 135%.

Certainly the usability testing we do for our clients has delivered significant strategic and design understanding, resulting in products that better serve user needs with less friction. And more revenue and lower costs.

If you are afraid that your site or application might have some of that “low hanging fruit”, invest a little money in an expert review. We’ll do that for you, and it’s even cheaper than a small usability test.

I’ve been getting an increasing number of requests for information on “what do I need to learn to get into mobile design?”. I always take this opportunity to mention my book, Designing the Mobile User Experience, which was written to help user experience, marketing, and product professionals translate their skills into mobile design and business.

Why? We think:

• iPhone awareness – in recent user research, one participant said of mobile Internet, “No, not on my phone. That’s what the iPhone is for.”


• one phone for every two people – 50% penetration is getting lots of business attention


• unlimited data plans – finally! For me, it’s one of the biggest draws of the iPhone.

Designing the Mobile User Experience has information about mobile users, differences in mobile use worldwide, design principles, some design patterns, sources for further design recommendations, industry structure, visual design for mobile media, and information on mobile user research including usability testing.

Usability testing of mobile web sites and applications has extra complexities over testing on full-sized computers. Chief among these is neglecting to test on a device with which the user is accustomed.

If the participant is accustomed to using a Motorola RAZR (Openwave browser), the iPhone or even Opera Mini on the RAZR will have a learning curve. If a Sprint UI user is asked to use a Series 60 device, all their learned behaviors will differ.

A person accustomed to the Opera Mini rendering engine will be less able to predict behavior of the iPhone browser. A person accustomed to an iPhone will feel crippled with most mass-market phone interfaces.

Is this any way to understand user behavior with your web site? Or worse, your application?

In the desktop world, browser controls are fairly consistent across operating systems and browsers. A Windows Internet Explorer user will be startled but have a shallow learning curve moving to Safari on a Mac. In contrast, a user with heavy use of a Back softkey will do poorly if testing with a device with a hardware back button.

The images above show a web site rendered on three different devices or emulators. The rendering and browser controls differ wildly, as do the native user interfaces.

Our practice is to recruit participants who use one of a target set of devices. For the usability test, the participant uses a matching device. In this way, the participant evaluates our client’s site or application, not the device being used.

One counter-practice is to use the “best available” device out there. This isn’t a great solution. First, you’ll not understand the true context of your application. Second, just because the device is “best” (which is arguable) does not mean it is familiar. That Nokia Series 40 user may not understand the iPhone. Finally, there are so many rendering differences between devices that a great application on one device could conceivably be utterly unusable on another. Capturing an array of devices somewhat guards against that problem.

Whenever reviewing the results of a usability test, be sure to consider whether issues were introduced by the tested application, the device the application was on, or the interaction between the two. We do everything we can to reduce device-created issues.

Alltel, a regional operator in the US, partnered with Frog Design to launch Celltop widgets. I mentioned this briefly in January.

This past week, Scott Weiss of Usable Products announced a usability study comparing Celltop to WAP and native UI for a variety of tasks.

What I found so interesting about the Usable Products study was that they compared the usability of a new interaction (CellTop) to that of well-learned applications (e.g., native UI for looking at call logs). As the entire session was only one hour, which included several tasks attempted both with and without CellTop. Thus the users had a year or two of experience with the native UI and were comparing to a UI with a new interaction paradigm. It's no wonder Celltop fared poorly.

Some tasks did not have quite as steep an experience difference. Checking weather is likely something many users only do occasionally, but the limitations of the mobile browser on the tested device means that the experience was reasonably familiar.

Celltop does indeed break many of the user expectations a bit. Some of this was done with intent. At the Austin Mobile Monday event in April, a key Celltop designer actually mentioned that she did not see the point in softkeys, and she pretty much ignored them. I disagree with her, but perhaps if everybody ignored softkeys the Celltop learning curve would have been more shallow.

In short, before believing that the Celltop widgets were indeed "less usable" than the native and WAP counterparts, I would like to see the test repeated with users who had been using Celltop for a few weeks. Alternately, give the users the opportunity to explore Celltop for half an hour, setting it up as they wish, on their own phones. Then compare task usability.

If you are lucky enough to live in Europe, I'm pleased to announce that Designing the Mobile User Experience is now available!

• a comprehensive guide to the mobile user experience, offering guidance to help make appropriate product development and design decisions.
• the tools necessary for product development professionals to understand development in the mobile environment.
• an overview of the components affecting the user experience and principles uniquely applicable to the mobile application field.
• the industry structure and power dynamics, providing insight into how mobile technologies and platforms become available on current and future phones.
• user research and design methodologies accounting for device proliferation
• user interface design patterns, design resources, and user research methods for mobile user interface design.

Designing the Mobile User Experience is an invaluable resource for information architects, user experience planners and designers, interaction designers, human factors specialists, ergonomists, product marketing specialists, and brand managers. Managers and directors within organizations entering the mobile space, advanced students, partnership managers, software architects, solution architects, development managers, graphic designers, visual designers, and interface designers will also find this to be an excellent guide to the topic.

Most of the rest of us will have to wait until April to get our hands on the book.