XClose

UCL Psychology and Language Sciences

Home
Menu

Linguistics Seminar Talk - Tal Linzen

07 October 2024, 2:00 pm–3:30 pm

Linguistics seminar

Language model predictions do not explain human syntactic processing

Event Information

Open to

All

Organiser

Alina Konradt

Location

Room B07
Chandler House
2 Wakefield Street
London
WC1N 1PF
United Kingdom

Title:  Event polysemy and the illocutionary/descriptive distinction

Abstract  

Prediction of upcoming events has emerged as a unifying framework for understanding human cognition. Concurrently, deep learning models trained to predict upcoming words have proved to be an effective foundation for artificial intelligence. The combination of these two developments presents a prospect for a unified framework for human sentence comprehension. We present an evaluation of this hypothesis using reading times from 2000 participants, who read a diverse set of syntactically complex English sentences. We find that standard LSTM and transformer language models drastically underpredicted human processing difficulty and left much of the item-wise variability unaccounted for. The discrepancy was reduced, but not eliminated, when we considered a model that assigns a higher weight to syntax than is necessarily for the word prediction objective. We conclude that humans’ next word predictions differ from those of standard language models, and that prediction error (surprisal) at the word level is unlikely to be a sufficient account of human sentence reading patterns.

About the Speaker

Tal Linzen

at New York University & Google

More about Tal Linzen