Early Organization of Simultaneous Multimodal Behaviors: Perception- and Production-Based Contingencies as a Scaffold for Language Development.
Abstract
Aim
This study investigated how infants' simultaneous multimodal behaviors (SMBs)-the concurrent coordination of vocal, gaze, facial, and manual signals-are organized during natural caregiver-infant interactions at 3 and 6 months. Within a contingency-based framework, we examined how perception-based and production-based SMBs reflect early multisensory and sensorimotor coordination and how these patterns relate to later language development with the aim of characterizing SMBs as functional units of early communicative organization.
Method
Twenty typically developing Hebrew-learning infants were recorded in naturalistic mother-infant interactions at 3 and 6 months and were assessed for language outcomes at 15 months using the MacArthur-Bates Communicative Development Inventories. Frame-by-frame multimodal coding was conducted using Mangold INTERACT software. Each frame was annotated across four channels (verbal, gaze, facial, manual), yielding 180 theoretically possible SMBs. Frequency, total duration, mean duration, and prevalence were derived. Age-related differences were tested, and principal component analysis (PCA) was applied to identify latent clusters of SMBs.
Results
Infants produced roughly one third of the theoretically possible combinations, indicating constraints on early multimodal integration. A small subset of SMBs (≈20%) accounted for the majority of occurrences, following a Pareto-like distribution. Between 3 and 6 months, patterns shifted from face gaze and tactile configurations toward gaze shift and exploratory coordination. PCA revealed five factors at each age; several integrated perception-based and production-based SMBs. Notably, the first factor at 3 months-comprising mixed SMBs with neutral facial expression, gaze shift, and phasic hand movements-was positively correlated with productive vocabulary at 15 months.
Conclusions
Infants' multimodal communication is selectively organized around high-probability behavioral anchors that integrate perception-based and production-based contingencies. These early configurations structure infants' interactional experience, support contingency learning, and are prospectively associated with later expressive language development, providing a systems-level account of early communication that bridges dynamic, social-pragmatic, and emergentist learning approaches. SUPPLEMENTAL MATERIAL https://doi.org/10.23641/asha.33307341.