How Distributed are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks - HAL-SHS - Sciences de l'Homme et de la Société Access content directly
Conference Papers Year : 2022

How Distributed are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks

Abstract

This work addresses the question of the localization of the syntactic information that is encoded in the Transformers representations. We tackle this question from two perspectives, considering the object-past participle agreement in French, by identifying, first, in which part of the sentence and, second, in which part of the representation syntactic information is encoded. The results of our experiments using probing, causal analysis and feature selection method, show that syntactic information is encoded locally in a way consistent with the French grammar.
Fichier principal
Vignette du fichier
main.pdf (353.47 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

halshs-03755082 , version 1 (21-08-2022)

Identifiers

Cite

Bingzhi Li, Guillaume Wisniewski, Benoit Crabbé. How Distributed are Distributed Representations? An Observation on the Locality of Syntactic Information in Verb Agreement Tasks. 60th Annual Meeting of the Association for Computational Linguistics, May 2022, Dublin, France. pp.501-507, ⟨10.18653/v1/2022.acl-short.54⟩. ⟨halshs-03755082⟩
47 View
27 Download

Altmetric

Share

Gmail Facebook X LinkedIn More