Give your computer the finger: Touch-screen tech comes of age

Tiny devices and the 'fat finger' problem

Some researchers say that a logical extension of touch technology is gesture recognition, by which a system recognizes hand or finger movements across a screen or close to it without requiring an actual touch.

"Our technology is halfway there," IBM's Pinhanez says, "because we recognize the gesture of touching rather than the occlusion of a particular area. You can go over buttons without triggering them."

The occlusion problem, where a finger or hand blocks a user's view of what he's doing and leads him to make mistakes, is being attacked in some novel ways. Microsoft and MERL collaborated on development of a research prototype called LucidTouch, a two-sided, "pseudo-translucent" mobile device that allows users to issue commands with their fingers on either the front or the back of the device.

"The problem we are addressing is what some people have called the 'fat finger problem,'" says Patrick Baudisch at Microsoft Research. When a user touches the back of the device, he sees an image of his fingers behind instead of in front of the things to be touched on-screen. LucidTouch can accept input from as many as 10 fingers at once.

LucidTouch is particularly useful in two situations, Baudisch says: when multitouch interaction is desired, and when a touch screen is very small, perhaps as small as a watch face. He declined to say when or if LucidTouch might become a product, saying researchers would continue to perfect it while looking for applications such as mobile gaming, art and spreadsheets.

Asked about an extension of LucidTouch from touch to gesture recognition, Baudisch says the Microsoft prototypes already can act on finger gestures, with the system recognizing finger motions as well as positions and understanding the meaning of different numbers of fingers. For example, the motion of one finger is seen as equivalent to a mouse movement, a finger touch is interpreted as a click, and two fingers touching and moving is seen as a scroll command.

Touch technology in its many variations is an idea whose time has come, Baudisch says. "It's been around a long time, but traditionally in niche markets. The technology was more expensive, and there were ergonomic problems," he says. "But it's all kind of coming together right now."

The rise of mobile devices is a big catalyst, he says, with the devices getting smaller and their screens bigger. When a screen covers the entire device, there is no room for conventional buttons, he points out. And that will give impetus to other types of interaction, such as voice, he says.

Touch on a grander scale

But not all the advances in touch technology are going into tiny mobile screens. Microsoft's Surface computer uses a two-way touch screen that is 30 inches across, big enough for several people to sit around and use simultaneously. It is intended to lie flat and present a 360-degree user interface.

Cameras embedded inside Surface sense user input in the form of touch and gestures (finger movements across the screen) and can capture the information needed to identify objects laid on it. This information is shipped to a garden-variety Windows Vista PC for processing and the results returned to Surface by a Digital Light Processing projector. It is a vision-based system, not capacitive or resistive as are many conventional touch devices. (See videos of Surface in action.)