Silent Speech Interface Enabled by Soft Magnetic Skin

In this work, we present a wireless, unobtrusive, and robust silent speech interface for tracking and decoding speech-relevant movements of the temporomandibular joint. Our solution employs a single soft magnetic skin placed behind the ear for wireless and socially acceptable silent speech recognition. The developed system alleviates several concerns associated with existing interfaces based on face-worn sensors, including a large number of sensors, highly visible interfaces on the face, and obtrusive interconnections between sensors and data acquisition components. With machine learning-based signal processing techniques, good speech recognition accuracy is achieved (93.2% accuracy for phonemes, and 87.3% for a list of words from same viseme groups). Moreover, the reported silent speech interface demonstrates robustness against noises from both ambient environments and users’ daily motions. Finally, its potential in assistive technology and human-machine interactions are illustrated through two demonstrations – silent speech enabled smartphone assistant and drone control.

This videos shows the magnetic skin enabled silent speech interface for the real-time drone control.

This videos shows the magnetic skin enabled silent speech interface for the real-time smartphone control.