Researchers at MIT are developing a system that will enable aircraft-carrier crew to guide autonomous planes on the carrier deck by using certain standard hand gestures. Due to the growing implementation of robot planes for routine air missions, the MIT researchers are developing a system based on similar gestures.
Interpreting hand signals has two challenges: to detect the position of the signaler’s hands from a digital image and to determine the representation of a particular gesture from a series of images.
The software used by MIT researchers, based on few variables, depicted the contents of each video frame. The variables include 3-D data of the positions of wrists and elbows, other positions of hands such as the thumbs up or down, and closed or open hands. The sequences of such abstract representations were stored in a database, which helped them prepare their gesture-classification algorithm.
As the crewmembers on the aircraft carrier’s deck are constantly under motion, classification of signals is the main challenge in developing the algorithm. The algorithm is performed in a chain of short body-pose sequences, each measuring 60 frames in length. The sequences can also overlap, and a single sequence may not have adequate information to determine a gesture, while a new gesture can also start in the middle of a frame.
The algorithm is created for each frame in a sequence based on the probability of being included in each of the 24 gestures, followed by evaluating the weighted average of the probabilities. Gesture identification relies on the weighted averages of sequences that represent relativity. However, the tests showed only 76% precision when the algorithm identified the gestures in the training database.
Source: http://web.mit.edu/