Multi-Touch Sensing through LED Matrix Displays

Multi touch led display, (Photo from Jeff Han's website)

Day after day we confront new Human-Machine-Interfaces (HMI for short) such as multitouch displays. Multitouch displays seem to be the most interesting HMI today (as it has been proved by announcing the iPhone by Apple). The project presented here demonstrates a novel idea of making a multitouch display using LED matrixes and nothing more as a sensor. A picture tells a million words and movies tell even more! So take a look and guess how it operates :

Video hosted on you tube, originally from Jeff Han’s website

The theory behind the operation is to use the LEDs both to emit light and sense. While the neighbor LED is emitting light the other ones behind it are acting like a sensor to determine the amount of the reflected light which depends on the distance of your hand to the display. This way an array of sensors are fromed. These arrays are scaned to fully cover the display.

The novelty here is the act of using LEDs as sensors to sense the amount of the light that is being directed toward them. Non-ideal component such as LEDs embed characteristics like capacitive capacity and reverse leak currents in real world. These undesirable characteristics, with a little intelligence, broadens the functionality of every LED to the sensor world, unbeleivably. (read Make blog , Mitsubishi Electric Research Laboratories , and justDIY for more information on how to use LEDs as sensors)

This idea has been initiated originally by Jeff Han who is a consulting research scientist for NYU’s Department of Computer Science . On his website he says “My research interests have historically been real-time computer graphics and real-time computer vision, but I’ve taken on a more recent focus on human-computer interfaces and machine learning.”

Suggested projects of him : MediaMirror,Holodust, Perceptive Pixel

Report by Farshid Jafari Harandi

Tags: ATMEGA8 AVR, LED sensor, Multi touch LED sensing, Multitouch distance sensor