Skip to content
Open access

The explain-on-blackboard paradigm for interleaved visual-text reasoning

Nov 2026 · Multimedia Systems · Vol 32 · 0 citations · 39 references

TL;DR

A pedagogical distillation pipeline building on an Explain-on-Blackboard (EoB) formulation, inspired by dual-coding and multimedia-learning theories, that generates temporally and semantically aligned teaching trajectories that comprise a discourse channel for logical explanation and a canvas channel for incremental HTML/SVG construction.

Abstract

Interleaved visual-text chain-of-thought reasoning, which generates intermediate visual artifacts to support deduction, is a promising direction for complex multimodal tasks. However, in our benchmark evaluation we observe a consistent representation tax: models perform systematically worse when translating abstract reasoning into structured spatial visualizations than when producing textual explanations alone. We conjecture that this gap is correlated with (rather than solely caused by) an alignment bias in conventional supervised fine-tuning (SFT), which optimizes outcome labels while providing little supervision on synchronization between visual semantics and textual reasoning. To empirically probe this hypothesis, we propose a pedagogical distillation pipeline building on an Explain-on-Blackboard (EoB) formulation. Inspired by dual-coding and multimedia-learning theories, teacher models generate temporally and semantically aligned teaching trajectories that comprise a discourse channel for logical explanation and a canvas channel for incremental HTML/SVG construction. A dynamic-rubric quality assurance mechanism with multi-stage filtering retains only high-fidelity trajectories for distillation. To measure progress on this problem, we further construct Cas-Bench, a 1,000-problem multidisciplinary evaluation protocol with process-level rubrics. In our experiments an 8B-parameter model fine-tuned on 4,000 distilled trajectories obtains a 69.76% relative improvement on Cas-Bench and 2–4 absolute-point gains on MMMU, MMMU-Pro, MathVision, and MathVista over its base model.

Read PDF

Similar papers

Preprint Nov 2026

Emergent Vision-Shaped Thinking in Text-Only Reasoning through Multimodal Training

Does multimodal training shape how a language model thinks, even when the input is text only? We test this with targeted causal ablations across 10 multimodal configurations from 6 model families, with matched text-only backbones and cross-family transfer controls. We contrast spatial-imagery tasks that invite mental s...

Qi Zhang, Bo-Han Wang, Qian-Ru Sun et al. · 0 citations
#artificial intelligence Preprint Nov 2026

Seeing Isn't Enough: Unlocking Visual Reasoning by Bridging the Reasoning-Access Gap with TIGER

Multimodal Large Language Models (MLLMs) inherit strong textual reasoning capabilities from their language backbones and text pretraining, but often fail to apply the same level of competency to visual inputs. We show that this degradation cannot be attributed to only perception failures, by evaluating models on semant...

G. Lau, Minh Huynh Nguyen, B. Low · 0 citations
Book Open access Nov 2026

ADROIT: Adaptive Decision for Reasoning-Optimized Interleaved Thought in Multimodal Reasoning

Interleaved-modal Chain-of-Thought (I-MCoT) has enhanced the capabilities of Vision-Language Models (VLMs) in Visual Question Answering (VQA) by grounding the reasoning process in inserted visual evidence. However, existing approaches often operate under a largely unexamined assumption that visual insertions are advant...

Xi-Ping Li, Jiang-Hong Ma · 0 citations
Sep 2026

Distilling Visual Reasoning into Text Space

V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning.

Wen-Han Yang, Nilay Naharas, Ali Payani et al. · 0 citations
Sep 2026

WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning

WM-VLM is introduced, which equips a pretrained VLM with a lightweight world model branch for generating intermediate visual states and suggests that internal world models offer a promising path toward VLMs that reason in both language and visual space.

Yuheng Zha, Yi-Lei Wang, Qiyue Gao et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Cycle-of-Thought: Self-Evaluative Iterative Reasoning for Table Visual Question Answering in Document Images

Recent advances in Vision-Language Models (VLMs) have significantly improved Visual Question Answering (VQA) performance, yet their reasoning over structured content such as tables remains inconsistent, often plagued by spatial misalignment and logical hallucinations. Existing approaches for improving table VQA typical...

Somraj Gautam, Gaurav Harit · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.