Modern Continual Learning with Foundation Models, Evaluation Challenges, and Future Directions

Abstract

Continual learning (CL) aims to develop intelligent systems capable of learning continuously from sequential data while retaining previously acquired knowledge. As AI systems are increasingly deployed in dynamic real-world environments, CL has become essential for enabling long-term adaptation without catastrophic forgetting. This review provides a structured overview of major CL paradigms, including task-incremental, domain-incremental, class-incremental, online, multimodal, and federated CL. We examine the theoretical foundations of CL, particularly the stability–plasticity dilemma, catastrophic forgetting, transfer dynamics, and representation learning. In addition, we analyze major methodological categories, including regularization-based, replay-based, architecture-based, optimization-based, representation-learning, and parameter-efficient approaches. Recent developments involving transformers, prompt learning, foundation models, and multimodal adaptation are also discussed as emerging directions in modern CL research. Furthermore, this review highlights important issues related to benchmark fragmentation, evaluation inconsistency, memory constraints, computational efficiency, scalability, and privacy-aware learning. We also summarize key application domains, including computer vision, natural language processing, robotics, healthcare, and medical imaging. Finally, we identify open research challenges and future directions toward scalable, reliable, and deployment-oriented lifelong learning systems capable of operating effectively in continuously evolving environments.