Gestural Interaction

The future of how humans interact with digital technology is evolving at a rapid pace. Since the revolution of the personal computer users have been restricted by the limitations of computer prosthetics (controller, remote, mouse, etc...). Digital technology has enriched some our interactions in life while arguably reducing others, but the limitations of computer prosthetics are showing signs of waning with emerging technology.

The people at MIT's Media Lab seem to have some new ideas of their own to expand the possibilities. Gestural interaction is one area that is having success. In their own words, gestural interaction is meant to "create interactions that merge body, object, space, screen, and surface." A group of Ph.D. students at MIT have been working to perfect a seamless interface that is based off of hand gestures. Different hand gestures become the tool in which to control a digital interface. They are creating a digital sign language that computers can understand.  We have seen other examples of gestural interaction with the Nintendo Wii, which still requires a prosthetic. Xbox 360's Project Natal (click to learn more) shows a lot of promise, but there is no release date at this point. The future of user interface design and interaction design will continue to be exciting as gestural interaction sits on the verge of reaching mainstream consumer culture. Watch the video below of the MIT students demonstrating their prototype.

MIT's Media Lab also developed a portable gestural interaction device that can be used anywhere. This "sixth sense" as they have coined it, has been highly published. Go to TED if you would like to see the product demonstration.

Another individual has taken gestural interaction in a slightly different direction. Instead of using the human hand and motion only, Tom Gerhardt has thrown mud into the equation. His "Mud Tub", uses shape-recognition, a gyroscope, gestural interaction, and mud. In this instance mud is the prosthetic, though a rather unorthodox one.  

The Mud Tub is meant to enhance how we experience human interactions with digital technology. Gerhardt built a game that is played by pushing mud around. There are a variety of games that are motion and pressure sensitive based upon the force produced by one's hand. I do not mean to suggest that mud should be a primary medium within new technologies, but that it is important to keep an open mind and ask, "What could possibly be used to enhance our interaction with technology?" The Mud Tub is a great example of experimenting with technology to merge the flat screen with a more immersive and multi-sensory interaction. I could see a lot of potential with organic interfaces like the Mud Tub being used for teaching kids along with a variety of other applications. Gerhardt's playfully says, "see, a computer can be squishy." This alone expands the framework of opportunities within interaction design.

Watch the video below to see how the Mud Tub works.

User interfaces have always been restricted due to the need for optimization across multiple platforms. What is exciting about gestural interaction and organic interfaces (such as the Mud Tub) are the capabilities of introducing more intuitive and multi-sensory interactions into our everyday lives. The real challenge is thinking through all the possibilities, barriers, and applications of that technology.

Imagine no longer needing that over-complicated TV remote that has become a victim of feature creep, or being able to reduce the impact on the environment by not having to produce as many prosthetics to consumer electronics.  The evolution of gestural interfaces will transform and hopefully improve our experiences within everyday life.

Further Reading

Project Natal, MIT Media Lab, Sixth Sense, TED - Demo
 

Posted by Brent Loosli