gestures in mobile UI

Tags: Design,  Design Tips,  Gestures
October 16, 2009 by Barbara

Over on the very nice UX Exchange, user mattti asked the question, Touch gesture usability in mobile devices – which ones work?.

As we’ve been doing a bit of work in gestures this year, I tossed off this response.

Touch gestures are only one type of gestures for mobile. Technologies such as accelerometers, light sensors, other cameras, pressure sensors, RFID, NFC, Bluetooth, and various location technologies are all possible.

Then there are the “display” mechanisms: vibration, rumble, electrical fields, force feedback, sound, and so forth.

In general, this space is in its infancy.

Phones should always be designed with one-hand use and two-hand use in mind, but it is up to the product team to decide the relative emphasis. Clearly making and receiving calls should be one-handed. Should web? I don’t think that’s as necessary. But usability can be achieved either way.

When designing gestures, you must keep in mind that they lack any innate visual affordance. You must rely on other methods to enable discoverability. There are a few approaches to this. The iPhone’s “flick” (scroll fast) gesture is a very simple extension to an existing behavior; it uses something people already do and just responds more naturally to it.

A second approach is training. The Palm Pre requires users to go through a training video to get the “back” gesture, as you really can’t use the device without it. There is no corresponding button.

Apple has used advertising to teach people about pinch to zoom. It’s less effective: many users complain that they just can’t read the browser text. They’ve not discovered how to zoom in.

Like I said, this space is very much in its infancy. But principles we’ve observed include:

  1. Carefully train any gestures critical to the use of the device. Swipe left for back, pinch/spread to zoom out/in, double-tap to zoom to fit.

  2. Avoid making many gestures critical to the use of the device. Provide menu or icon alternatives for most gestures. Gestures (and voice control) provide shortcuts, not navigation through menus.

  3. Use natural finger or thumb paths rather than requiring strict rectilinear paths. In other words, use good biomechanics and be forgiving of error. Example: The iPhone unlock screen requires too much precision in start and end points to unlock. It is too easy to stop at the wrong place.

  4. Test your gestures with real users. Redesign, test again.

  5. Consider a device mode in which everything is done through the menu. This is the “share with somebody else” mode. Verbally communicating gestures while driving is frustrating for everybody.

  6. Be smart about automatically doing things. Sometimes, for example, a person might want to lay down while reading a web page. Indeed, “in bed” is a common context for use. The iPhone makes this challenging, as it automatically rotates sideways. Some applications have a “lock” mode in which they won’t rotate.

  7. Where possible, re-use gestures. Pinch to zoom should be preserved. Flick to scroll should be preserved.

  8. Think outside the box. Or off the screen. I want a function that intelligently locks and unlocks my screen based on device position and light input (i.e., it’s upside down in a pocket or purse then the keys are locked; I pull it out and they are unlocked). This is far more important than an extra on-screen gesture.

Also, be sure to read some of Kevin Arthur’s work. I keep coming back to Evaluating Gesture Usability – it includes a method to actually develop and test good gestures.


«« choosing simple design isn’t so simple  |  the page fold myth »»

3 Comments »

  1. Very good. Glad you are thinking about this.

    Someone should take the lead for:

    1. Defining guidelines — you have started above
    2. “Standardize” on common gesture patterns — not sure if you have this in your design wiki but if not, you should; would be useful.

    enrique

    Comment by C. Enrique Ortiz — October 19, 2009 @ 11:26 am

  2. first of all i want to congratulate nokia for this revolutionary step taken in the field of technology to develop gestures mobiles but i want to ask that will these gestures mobile be able to understand the gestures of different country people because different country people have different sense of gestures but this gestures technology will be able to understand only some specified gestures so plese cllarify my question by sending the answer on my email id i.e “groverpriyanka.2@gmail.com”

    Comment by priyanka — December 22, 2009 @ 9:25 am

  3. @priyanka – While Nokia has done some nice ethnographic style research across cultures, the academic research on gestures and language is fascinating. It turns out that the differences in gestures associated with language production is not between cultures, but between individuals. As best as I can tell, there is no biological similarity of gestures.

    That leaves only learned gestures. Yes, there are differences between cultures; user research should try to uncover these. But we have to use caution when adopting existing gestures.

    For anybody who doesn’t know about the Nokia research, you can find it at Nokia Conversations. And it talks about cultural differences.

    We actually did some similar research. For “focus the camera”, participants had a few major approaches. Younger people tended towards moving the device itself closer and further from the subject. Older people tended towards turning a virtual SLR lens in front of the device. Either is learnable; they have different design implications. Which to choose? Good question. We don’t really know yet.

    Comment by Barbara — December 24, 2009 @ 7:43 pm

RSS feed for comments on this post.

Leave a comment

*
To prove you're a person (not a spam script), type the security word shown in the picture.
Anti-Spam Image