- Now that you mention it, I admit I glossed over remote inputs vs co-located input devices for my argument. I have several touchscreen devices (didn’t have the laptop there when I did the video, so didn’t use it) and don’t find myself using them in different ways. Parallax bugs me, so seems to negate some of the advantages now.

- Visual feedback will always be inherently slow and require too much attention; my future world has you using touch devices in the car, while walking, etc. If only for navigation and simple supporting tasks, touch feedback (good term) seems a lot more useful than visual-only feedback.

- I certainly am slightly mocking the NUI trend. I hope someday it becomes like the way we say “intuitive” in quotes. No GUI is intuitive to anyone.

- He’s stopped publishing it, but I think Bill DeRouchey’s History of the Button blog is still out there. Read it for a history of the, well, button. Indeed, only so old, but I think most of the very fine sensing fingers can do is one way; you cannot gouge, write, etc. with fingers directly…

- …though I didn’t explicitly say it, the finger sensing is a good thing and means that haptic systems should let you feel interfaces. I also didn’t quite say it, but I want stylii (and maybe several of them; love how there are a variety of them for the Wacom tablet, like an airbrush shaped one) but robust finger support is also important.

- I haven’t gotten to use a haptic responding device for more than trying-it-out time, but it is clearly the way to go. Exciting versions of haptics should someday be coming, where you can feel stuff before you touch it even, and that will be great.

- I have personally never been able to use gesturing (in the sense of make a delete swoop with the mouse to delete things)… actually, my innate pinch-sense for Apple is the opposite of their programming). This makes me worry others are not getting it, and adoption rates before iPhone seem to indicate this. So, I tend to worry about some of those, without enough data to back it up specifically.

- I am not sure (though clearly it’s open to discussion) that there is that much different now between touch and point. I am not sure mouse pointing is that different from any other type. Or, my pen tablet detects pointing above the surface; you touch the surface to select a button or lay down a line (a nice system, really, for the current technology). Perhaps future haptics will use the acoustical system that lays the feelable-interface slightly above the surface, so you feel something that isn’t quite real. Not sure what that will mean just yet.

- Not sure I totally buy the difference between typing on a touchscreen or a remote keyboard. In both cases the finger occludes the input, so something else has to say what you pressed (or for word entry, very often it is more useful to focus on the result, whether the word is getting spelled right, instead of the individual keypress). So maybe again I am overly generalizing, but my impression is that typing on a computer, on a keyboard/keypad mobile phone and on a touch device are very similar: input below, watch above for results, glance as needed at the keypad (more for hapticless touch, but still).
I think I personally spend more time looking at input when I am using stylii and can see the direct results of the action, but I am not sure if this means anything in the long run.

But, with a touch screen, the basis of the interaction occurs when “touching” onscreen objects, such as icons. The underlying metaphor is that I am touching an icon or object in order to select it—and, in response the object lights up, or (in some cases) the phone vibrates. The basis of the interaction itself comes when I “touch” an object that appears at a set location, and the feedback that I receive oftentimes originates from that same location (e.g., in the form of the icon changing color). Why is this type of interaction unnatural? And, this whole system of interaction doesn’t work if I simply “point” at the screen—I have to actually touch it. With gesture recognition systems, input methods such as pointing can work—not so with my mobile’s touch screen, however.

As an interaction designer who got my start in cognitive psychology, I am weary of the talk “natural interaction”. There isn’t much about the act of talking on a mobile phone to begin with that makes it natural, is there? What is natural, anyway? As interaction designers, shouldn’t we be more worried about other qualities of interaction that might enhance the user’s experience? Simply arguing that some interaction is natural or not has little bearing on the quality of an experience.

That said, I liked this post—clearly, I found it thought provoking.

I completely agree with you that the one-direction communication we have currently in touch interfaces is lacking, errr, the other direction.

“Which brings me to the second point. Fingers are pretty good at pressing buttons, but I am not sure they are much good at anything else.”

‘Buttons’ are a recent invention, which I have had trouble tracking down the history. But it can’t be more than some hundreds of years. Before then, hands were mostly used for much more tactile-complex tasks. If you do some reading on our tactile skills, you’ll find them very impressive. Human skin has much more resolution than any current technology can meet the requirements of.

There are a few *haptic* commercially available interfaces around, like the Phantom, which apparently do a decent enough job — but the challenge there is that a lot of force (ie electrical power) required to provide reasonable haptic feedback.

I suspect we’ll need the haptic version of ‘the icon’ — an abstraction of haptics/tactile information which takes a lot less power + resolution.

The output part is really important, but don’t forget about other modes of feedback. Even though visual feedback without touch feedback is still not exactly natural, I think people adapt to it well and it goes a long way to having the same benefit.

I agree with you also that fingertip input is only a tiny subset of what people can do with their hands, so calling these interfaces “natural” as if they’re the ultimate is a bit silly.

A side note that isn’t directly relevant but I think helps in clarifying differences between input devices: I found your use of “pointing” here a little confusing because you’re combining indirect pointing (or indirect touch) like with a mouse, tablet, or touchpad, and with direct pointing or direct touch on a touchscreen with a finger or pen. I’m not sure if this makes a difference for your argument, but I think it’s a useful distinction. It would be that people are better at doing indirect input without feedback than direct input without feedback.