A visual speech recognition system for an ultrasound-based silent speech interface
Jun Cai
(1)
,
Thomas Hueber
(2)
,
Bruce Denby
(1)
,
Elie-Laurent Benaroya
(1)
,
Gérard Chollet
(3)
,
Pierre Roussel
(1)
,
Gérard Dreyfus
(1)
,
Lise Crevier-Buchman
(4, 5)
1
SIGMA -
Laboratoire Signaux, Modèles et Apprentissage Statistique
2 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée
3 LTCI - Laboratoire Traitement et Communication de l'Information
4 LPP - LPP - Laboratoire de Phonétique et Phonologie - UMR 7018
5 Voice & Speech lab
2 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée
3 LTCI - Laboratoire Traitement et Communication de l'Information
4 LPP - LPP - Laboratoire de Phonétique et Phonologie - UMR 7018
5 Voice & Speech lab
Thomas Hueber
- Fonction : Auteur
- PersonId : 5965
- IdHAL : thomas-hueber
- ORCID : 0000-0002-8296-5177
- IdRef : 143151568
Gérard Chollet
- Fonction : Auteur
- PersonId : 176991
- IdHAL : gerard-chollet
- ORCID : 0000-0003-4245-146X
- IdRef : 078020824
Lise Crevier-Buchman
- Fonction : Auteur
- PersonId : 175723
- IdHAL : lise-crevier-buchman
- ORCID : 0000-0002-2900-0528
- IdRef : 035788739
Résumé
The development of a continuous visual speech recognizer for a silent speech interface has been investigated using a visual speech corpus of ultrasound and video images of the tongue and lips. By using high-speed visual data and tied-state cross-word triphone HMMs, and including syntactic information via domain-specific language models, word-level recognition accuracy as high as 72% was achieved on visual speech. Using the Julius system, it was also found that the recognition should be possible in nearly real-time.
Domaines
LinguistiqueFormat du dépôt | Notice |
---|---|
Type de dépôt | Communication dans un congrès |
Titre |
en
A visual speech recognition system for an ultrasound-based silent speech interface
|
Résumé |
en
The development of a continuous visual speech recognizer for a silent speech interface has been investigated using a visual speech corpus of ultrasound and video images of the tongue and lips. By using high-speed visual data and tied-state cross-word triphone HMMs, and including syntactic information via domain-specific language models, word-level recognition accuracy as high as 72% was achieved on visual speech. Using the Julius system, it was also found that the recognition should be possible in nearly real-time.
|
Auteur(s) |
Jun Cai
1
, Thomas Hueber
2
, Bruce Denby
1
, Elie-Laurent Benaroya
1
, Gérard Chollet
3
, Pierre Roussel
1
, Gérard Dreyfus
1
, Lise Crevier-Buchman
4, 5
1
SIGMA -
Laboratoire Signaux, Modèles et Apprentissage Statistique
( 133759 )
- ESPCI, 10 rue Vauquelin, 75231 Paris cedex 05
- France
2
GIPSA-MAGIC -
GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée
( 388728 )
- GIPSA-lab, 11 rue des Mathématiques, Grenoble Campus BP46, F-38402 SAINT MARTIN D'HERES CEDEX
- France
3
LTCI -
Laboratoire Traitement et Communication de l'Information
( 162010 )
- 46 rue Barrault F-75634 Paris Cedex 13
- France
4
LPP -
LPP - Laboratoire de Phonétique et Phonologie - UMR 7018
( 986 )
- Université Sorbonne Nouvelle
Maison de la Recherche
4, rue des Irlandais
75005 PARIS
- France
5
Voice & Speech lab
( 188575 )
- France
|
Source |
Proceedings of the 17th International Congress of Phonetic Sciences
|
Vulgarisation |
Non
|
Actes |
Oui
|
Comité de lecture |
Oui
|
Invité |
Non
|
Audience |
Internationale
|
Date de publication |
2011-03-01
|
Page/Identifiant |
384-387
|
Date début congrès |
2011-08-17
|
Date fin congrès |
2011-08-21
|
Ville |
Hong Kong
|
Pays |
Chine
|
Titre du congrès |
ICPhS 2011 - 17th International Congress of Phonetic Sciences
|
Langue du document |
Anglais
|
Domaine(s) |
|
Projet(s) ANR |
|
Mots-clés |
en
Silent speech interface, visual speech recognition, vocal tract ultrasound imaging
|
Loading...