Evaluating the Language Abilities of Large Language Models vs. Humans : Three Caveats
Leivada, Evelina (Universitat Autònoma de Barcelona. Departament de Filologia Catalana)
Dentella, Vittoria (Universitat Rovira i Virgili. Departament d'Estudis Anglesos i Alemanys)
Günther, Fritz (Humboldt-Universitat zu Berlin. Institut für Psychologie)

Data:	2024
Descripció:	12 pàg.
Resum:	We identify and analyze three caveats that may arise when analyzing the linguistic abilities of Large Language Models. The problem of unlicensed generalizations refers to the danger of interpreting performance in one task as predictive of the models' overall capabilities, based on the assumption that because a specific task performance is indicative of certain underlying capabilities in humans, the same association holds for models. The human-like paradox refers to the problem of lacking human comparisons, while at the same time attributing human-like abilities to the models. Last, the problem of double standards refers to the use of tasks and methodologies that either cannot be applied to humans or they are evaluated differently in models vs. humans. While we recognize the impressive linguistic abilities of LLMs, we conclude that specific claims about the models' human-likeness in the grammatical domain are premature.
Drets:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, la comunicació pública de l'obra i la creació d'obres derivades, fins i tot amb finalitats comercials, sempre i quan es reconegui l'autoria de l'obra original.
Llengua:	Anglès
Document:	Article ; recerca ; Versió publicada
Publicat a:	Biolinguistics, Vol. 18 (2024) , p. e14391, ISSN 1450-3417

DOI: 10.5964/bioling.14391

12 p, 299.8 KB

El registre apareix a les col·leccions:
Articles > Articles de recerca
Articles > Articles publicats