Research published in the International Journal of the Digital Human reveals a new model for computerised human gesture recognition. The tool, VirSen1.0, developed by Kana Matsuo, Chengshuo Xia, and Yuta Sugiura of the Department of Information and Computer Science at Keio University in Yokohama, Japan uses cutting-edge sensor simulation and user-centric design to overcome various obstacles and open up new ways for people to interact with technology.
Gesture recognition technology has long been plagued by the conundrum of efficiently configuring and placing the requisite sensors, interpreting the results, and ensuring the accuracy of the machine learning system that will recognise those gestures. The team has developed VirSen1.0, as a virtual environment paired with a tailor-made user interface to underpin the development of gesture recognition systems. In training, the system generates sensor data and synchronizes this with an avatar’s movements to train the model’s classifier. Thus, it can assign and configure sensors that will be able to quickly and accurately provide the necessary data for gesture recognition and the assignment of meaning to those gestures.
The team explains that the breakthrough with VirSen1.0 lies in its ability to overcome the intricacies of sensor influence. It does this by using a permutation feature importance (PFI) technique, a tool that illuminates the impact of individual sensors on the classifier’s performance. A user study demonstrated efficacy of the PFI approach but also highlighted how version 1.0 might be improved. Indeed, whereas trial-and-error in sensor placement will remain important, the next version of the system will build on those kinds of successes and failures and allow users to configure a system optimally based on previous experimental setups.
As the gap between human gestures and machine comprehension narrows, the impact of this kind of development could find wide application in the development of gesture recognition, which in turn will have use across a range of industries, in healthcare, and in recreation.
Matsuo, K., Xia, C. and Sugiura, Y. (2023) ‘VirSen1.0: toward sensor configuration recommendation in an interactive optical sensor simulator for human gesture recognition’, Int. J. Digital Human, Vol. 2, No. 3, pp.223–241.