Integrating the BERT-Deformer Architecture to Optimize Multimodal Translation of Japanese Literature

Abstract

Accurate multimodal translation of Japanese literature requires effective integration of linguistic semantics and visual context while maintaining long-range dependency modeling and cultural fidelity. This study presents a BERT-Deformer architecture that combines a pre-trained Japanese BERT encoder with deformable attention to achieve hierarchical fusion of textual and visual information. The proposed framework employs deep contextual representations for literary texts and dynamically focuses on salient cross-modal features through adaptive reference-point sampling, enabling efficient processing of long sequences and culturally sensitive semantic alignment. Experimental results demonstrate superior performance over conventional Transformer-based approaches, achieving improvements in BLEU-4, METEOR, and BERTScore while exhibiting enhanced robustness for long-text translation and more accurate handling of culturally loaded expressions. Attention analysis further confirms that the model effectively exploits visual cues to strengthen semantic disambiguation and cross-modal consistency. Beyond literary translation, the proposed architecture provides a scalable multimodal information processing paradigm with potential relevance to semantic communication and intelligent electromagnetic information systems, where reliable cross-modal representation and context-aware information transmission are essential for high-fidelity knowledge delivery.