Syntactic headedness aligns with visual narrative structure.

Abstract

Recent research has highlighted how visual narrative sequences involve similar structures and processing as syntax in language, yet no work has asked whether these structures may interact with each other. Here we examine whether the syntactic word orders and/or headedness of languages influence the orders of particular reversible constructions in visual narrative sequencing (POV, refiners). We first compared data from two corpora of annotated comics (TINTIN Corpus: 761 comics, 144 countries, 51 languages; Visual Language Research Corpus: 376 comics, 10 countries, 9 languages) to data about syntactic structure from multiple different language corpora (Grambank, WALS, Universal Dependencies). Across analyses, syntactic headedness of the languages of the comics aligned with ordering of constructions in visual narratives. These differences extended beyond the systematic diversity in visual narrative constructions across comics' styles. Next, reanalysis of behavioral data from Cohn, van Middelaar, et al. (2024) showed that the syntactic structure of participants' langauges affected their preferences for the ordering of visual narrative constructions. Altogether, this combination of corpus and experimental findings suggest that the headedness of syntactic structures in language can permeate the sequencing of visual narratives.