FMFT: A Memory-Augmented Multi-Feature Fusion Transformer Enhanced With Retrieval Augmented Graph-Based Few-Shot Learning
TL;DR
This work proposes a Few-Shot Memory Fusion Transformer (FMFT), a memory-augmented, multi-feature-fusion transformer enhanced with the Retrieval-Augmented Graph (RAG) and Few-Shot learning that significantly outperforms competitive baselines in both full-resource and few-shot scenarios.
Abstract
Indic text is syntactically compact, semantically ambiguous, and highly context-dependent, making English translation particularly challenging. Traditional neural machine translation (NMT) methods, even when enhanced with pre-trained language models (PLMs) and linguistic features such as part-of-speech (POS) tags, syntactic dependencies, and semantic role labels (SRLs), struggle to generalise in few-shot settings and often lack consistency across language structures. To address the above limitations, this work proposes a Few-Shot Memory Fusion Transformer (FMFT). The FMFT is a memory-augmented, multi-feature-fusion transformer enhanced with the Retrieval-Augmented Graph (RAG) and Few-Shot learning. FMFT combines multi-granular linguistic signals with a graph-structured semantic memory network, in which nodes represent key tokens (e.g., phrases, named entities) and edges represent purely syntactic, semantic, and contextual relationships. A graph-based dynamic retrieval mechanism that uses Personalised PageRank and a minimum-cost spanning tree enables on-the-fly retrieval of context for each translation instance. This is fused with PLM features via a gated, attention-based multi-feature fusion module that adaptively balances linguistic signals. A multi-head memory attention mechanism incorporates this information into the decoder to enhance semantic clarity and long-range dependency handling. FMFT also learns through non-parametric continual learning with a scalable long-context memory buffer. It adapts to various evolving language inputs without requiring retraining. Evaluation of five Indic-English corpora reveals that FMFT significantly outperforms competitive baselines in both full-resource and few-shot scenarios, achieving improvements in BLEU, ROUGE, METEOR, and Chrf scores. Its handling of rare or ambiguous constructs makes it particularly effective for translating low-resource historical languages. At the same time, its modular graph-based design supports scalability, interpretability, and reuse for domain-specific translation tasks.