Speech Recognition and Eye Tracking Combined
It makes sense that speech is one of the faster ways to enter text. It is not as remote or removed from our thought process as, say, typing or writing is. Speech comes at the pace of our thoughts, and we can say what we think in real time, without having to transcribe and keep up with our stream of consciousness. Humans have been measured dictating to a computer at 102 words a minute – far more efficient than writing and typing. At Eye Tracking Update, we often touch on new devices that use eye tracking and biometrics in noteworthy ways.
We recently found a study out of the University of Cambridge (the same institution which employs physicist and ALS-patient Stephen Hawking) introducing a new program called Speech Dasher. Speech Dasher appears to allow writing utilizing a combination of speech and a zooming interface controlled with gaze using an eye tracker.
Many speech recognition programs seem slow and archaic, unable to recognize difficult and poorly pronounced words or words in quick succession. Speech Dasher supplements information from a speech recognizer with a letter-based language model as well as the user themselves, and they say that it allows fast writing of anything predicted by the recognizer while simultaneously providing “seamless fallback to letter-by-letter spelling” for unrecognized words.
In studies, the program seems to make up for word errors, and expert users were able to write at 40 words per minute despite a recognition word error rate of 22%. The system is built off a previous program called Dasher, a text entry interface in which users write by “zooming through a world of nested letter boxes.” Similar to typing/texting on many touchscreens (iPhone in particular), the size of the box expands proportionally to that letter’s probability in the language model. As more letters are written, the program makes predictions, allowing common words to be written quickly.
Previously, any type of pointing device controlled Dasher, zooming in on letters onscreen. The Cambridge researchers equipped Dasher with a Tobii eye tracker and supplemented information from a speech recognizer, which allowed users to quickly navigate through entire words as they were predicted onscreen. This allowed users to recite a sentence and then go back and correct it as needed in far less time than it took to regularly correct for errors.
Speech Dasher: Fast Writing using Speech and Gaze
Related articles:
- Improving Speech Recognition Accuracy Via Crowd Sourcing
- Tracking Bias, Biometrically
- Eye Controlled Video Games? Better Late Than Never
- Eye-Com Eye Tracking Drive Simulator Studies Driver Inattention
- Old News But Relevant News: Apple and Eye Tracking
- Iris Recognition is Advancing. Could It Help Eye Tracking?
- Are Limits of Eye Tracking Real?
- Eye Tracking Technology as Memory Aid
- The Price of Eye Tracking
- Eye Tracking: Viewing the World Through Someone Else’s Eyes