CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages - HAL-SHS - Sciences de l'Homme et de la Société Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages

CoSwID, une méthode d'identification de l'alternance codique appropriée aux langues peu dotées

Laurent Kevers

Résumé

We propose a method for identifying monolingual textual segments in multilingual documents. It requires only a minimal number of linguistic resources-word lists and monolingual corpora-and can therefore be adapted to many under-resourced languages. Taking these languages into account when processing multilingual documents in NLP tools is important as it can contribute to the creation of essential textual resources. This language identification task-code switching detection being its most complex form-can also provide added value to various existing data or tools. Our research demonstrates that a language identification module performing well on short texts can be used to efficiently analyse a document through a sliding window. The results obtained for code switching identification-between 87.29% and 97.97% accuracy-are state-of-the-art, which is confirmed by the benchmarks performed on the few available systems that have been used on our test data.
Fichier principal
Vignette du fichier
KeversLaurent_sigul-2022.pdf (181.37 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03722821 , version 1 (13-07-2022)

Identifiants

  • HAL Id : hal-03722821 , version 1

Citer

Laurent Kevers. CoSwID, a Code Switching Identification Method Suitable for Under-Resourced Languages. 1st Annual Meeting of the ELRA/ISCA Special Interest Group on Under-Resourced Languages (SIGUL2022 @LREC2022), European Language Resources Association, Jun 2022, Marseille, France. pp.112-121. ⟨hal-03722821⟩
62 Consultations
113 Téléchargements

Partager

Gmail Facebook X LinkedIn More