Real-Time Markerless Hand Computer Interaction

Abstract

The objective: In present day, the computer mouse is the prime method of interacting with a computer. Though effective, the mouse's single point of interaction limits the user from obtaining complete control. One should have the freedom to control applications as if one were using one's own hands. This type of freedom is presented with the concept of multi-touch; multi-touch users have the freedom to manipulate and interact with the applications with the aid of multiple points of contact. The downside of multi-touch is that the software, as well as the hardware limits the user. This type of ease should be available for users without the need to worry about the expense. Touch-less interactivity is considered to be the next-generation form of communication for computer users and therefore new approaches are being introduced into the market constantly. Multi-touch will enable computer programs to be more complex function-wise without taking away from the simplicity the user experiences. Our aim is to take that functionality that multi-touch provides, without the "touch," and make it affordable for everyone.

Methods/Materials

We started with a very simple approach that only required a camera (proliferating on all laptops and desktops) and computer software coded in Visual C++. Since the software was the driving force behind the project, we had to heavily research algorithms since we had no previous knowledge of their functionality. After analyzing over 20 algorithms and trying each one using a brute-force method, we discovered over time that the best way to approach this was to combine multiple algorithms.

Results

It is necessary to have a good camera (high Frames Per Second) for this program to work. The program tracks a contrast in color and uses the tracked contrast as a binary 1 (positive) and everything else as 0 (negative). If the code were to track a shape rather than a color, it may be more effective in helping the camera understand the contours of the hand and it will not delay as much. The results indicate an optimal position, lighting, and color for the current program.

Conclusions/Discussion

There is a huge impact that the environment has on the program, it won#t work as well with constantly varying background colors and will not work at distances over five feet meaning a background independent code would be preferable. In order for our project to have more potential, the method for tracking must change.

The design and construction of a computer program to aid a user to control and manipulate applications on their personal computer by using the movements of their hands in 3-D space.

Science Fair Project done By Tushar Alladi