Design of an English cross-scene model for enhanced mobile multimedia based on federated learning

Abstract

As mobile multimedia technology in education develops quickly, cross-scenario English teaching has created new challenges for efficient and accurate speech translation and recognition systems. Traditional machine learning methods have obvious deficiencies in privacy protection, data storage, and cross-scenario data utilization. In response to the above problems, this work proposes a Federated Enhanced Multi-modal Translation and Recognition (FE-MTR) model. This model aims to improve the accuracy of translation and recognition in mobile multimedia English teaching while ensuring data privacy and security. The proposed model constructs a high-performance global model without sharing the original data by combining speech and text features in the multimodal feature learning framework, introducing a local training strategy and an improved attention mechanism. The experimental results show that the translation and recognition effects of FE-MTR on the training, validation, and test sets are remarkably better than those of the comparison models. Among them, the Bilingual Evaluation Understudy (BLEU) value reaches 43.5, and the Word Error Rate (WER) and Character Error Rate (CER) decrease significantly. The difference in cross-scenario transfer accuracy is the smallest, indicating that FE-MTR has good robustness and generalization ability. In conclusion, the proposed FE-MTR model can accurately implement the translation and recognition tasks in mobile multimedia English teaching. At the same time, it can protect user privacy and adapt to different teaching scenarios and it is an effective, intelligent solution for mobile education and online classrooms.