Establishing degrees of closeness between audio recordings along different dimensions using large-scale cross-lingual models
Maxime Fily
(1)
,
Guillaume Wisniewski
,
Séverine Guillaume
(1)
,
Gilles Adda
(2, 3)
,
Alexis Michaud
(1)
Maxime Fily
- Fonction : Auteur
- PersonId : 754685
- IdHAL : maxime-fily
- ORCID : 0009-0006-8292-5358
- IdRef : 270285113
Guillaume Wisniewski
- Fonction : Auteur
- PersonId : 748468
- IdHAL : guillaume-wisniewski
- ORCID : 0000-0002-4445-080X
- IdRef : 128062290
Séverine Guillaume
- Fonction : Auteur
- PersonId : 12704
- IdHAL : severine-guillaume
- ORCID : 0000-0003-1772-2600
Gilles Adda
- Fonction : Auteur
- PersonId : 176074
- IdHAL : gilles-adda
- ORCID : 0000-0001-8544-5874
- IdRef : 031887392
Alexis Michaud
- Fonction : Auteur
- PersonId : 419
- IdHAL : alexis-michaud
- ORCID : 0000-0003-1165-2680
- IdRef : 095131507
Résumé
In the highly constrained context of low-resource language studies, we explore vector representations of speech from a pretrained model to determine their level of abstraction with regard to the audio signal. We propose a new unsupervised method using ABX tests on audio recordings with carefully curated metadata to shed light on the type of information present in the representations. ABX tests determine whether the representations computed by a multilingual speech model encode a given characteristic. Three experiments are devised: one on room acoustics aspects, one on linguistic genre, and one on phonetic aspects. The results confirm that the representations extracted from recordings with different linguistic/extra-linguistic characteristics differ along the same lines. Embedding more audio signal in one vector better discriminates extra-linguistic characteristics, whereas shorter snippets are better to distinguish segmental information. The method is fully unsupervised, potentially opening new research avenues for comparative work on under-documented languages.
Format du dépôt | Fichier |
---|---|
Type de dépôt | Communication dans un congrès |
Titre |
en
Establishing degrees of closeness between audio recordings along different dimensions using large-scale cross-lingual models
|
Résumé |
en
In the highly constrained context of low-resource language studies, we explore vector representations of speech from a pretrained model to determine their level of abstraction with regard to the audio signal. We propose a new unsupervised method using ABX tests on audio recordings with carefully curated metadata to shed light on the type of information present in the representations. ABX tests determine whether the representations computed by a multilingual speech model encode a given characteristic. Three experiments are devised: one on room acoustics aspects, one on linguistic genre, and one on phonetic aspects. The results confirm that the representations extracted from recordings with different linguistic/extra-linguistic characteristics differ along the same lines. Embedding more audio signal in one vector better discriminates extra-linguistic characteristics, whereas shorter snippets are better to distinguish segmental information. The method is fully unsupervised, potentially opening new research avenues for comparative work on under-documented languages.
|
Auteur(s) |
Maxime Fily
1
, Guillaume Wisniewski
, Séverine Guillaume
1
, Gilles Adda
2, 3
, Alexis Michaud
1
1
LACITO -
Langues et civilisations à tradition orale
( 406905 )
- 7, rue Guy Môquet, 94800, VILLEJUIF
- France
2
STL -
Sciences et Technologies des Langues - LISN
( 1065372 )
- France
3
LISN -
Laboratoire Interdisciplinaire des Sciences du Numérique
( 1061259 )
- Campus Universitaire bât 507
Rue du Belvédère
91405 Orsay cedex
et
Campus Universitaire bât 640
1 rue Raimond Castaing
91190 Gif-sur-Yvette
- France
|
Licence |
Paternité
|
Vulgarisation |
Non
|
Comité de lecture |
Oui
|
Invité |
Non
|
Audience |
Internationale
|
Langue du document |
Anglais
|
Titre du congrès |
Findings of the Association for Computational Linguistics: EACL 2024
|
Date début congrès |
2024-03
|
Ville |
St. Julian’s
|
Pays |
Malte
|
Actes |
Oui
|
Données associées |
|
Financement |
|
Domaine(s) |
|
Organisateur du congrès |
|
Projet(s) ANR |
|
arXiv Id | 2402.05581 |
Loading...