Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement - HAL-SHS - Sciences de l'Homme et de la Société
Conference Papers Year : 2021

Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement

Bingzhi Li
  • Function : Author
  • PersonId : 1095862
Benoît Crabbé

Abstract

Many recent works have demonstrated that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. We take a critical look at this line of research by showing that it is possible to achieve high accuracy on this agreement task with simple surface heuristics, indicating a possible flaw in our assessment of neural networks' syntactic ability. Our fine-grained analyses of results on the long-range French objectverb agreement show that contrary to LSTMs, Transformers are able to capture a non-trivial amount of grammatical structure.
Fichier principal
Vignette du fichier
main.pdf (196.75 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

halshs-03755089 , version 1 (21-08-2022)

Identifiers

Cite

Bingzhi Li, Guillaume Wisniewski, Benoît Crabbé. Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement. 2021 Conference on Empirical Methods in Natural Language Processing, Nov 2021, Punta Cana, Dominican Republic. ⟨10.18653/v1/2021.emnlp-main.377⟩. ⟨halshs-03755089⟩
50 View
107 Download

Altmetric

Share

More