This survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields, and points out the main limitations of current models and prospects the future research directions.
Abstract
Representation learning has become a cornerstone of artificial intelligence, designed to automatically extract low‐dimensional, meaningful features from high‐dimensional, sparse raw data. By drastically reducing the reliance on manual feature engineering, representation learning enhances model performance across a wide range of tasks. The field has evolved significantly over the past decades, transitioning from early linear methods, such as Principal Component Analysis (PCA), to modern deep learning paradigms powered by neural networks, generative adversarial networks (GANs), and pre‐trained models. Although the rapid development of representation learning has significantly promoted the progress of natural language processing (NLP), computer vision, and recommender systems, the general practitioners still have a poor understanding of its historical background, core principles, and wide range of applications. To some extent, this limits the full development of its potential. To this end, this survey aims to provide a comprehensive and easily understandable overview for a wider audience. This survey conducts a systematic literature review to tease out the evolution of representation learning and analyse its core drivers. At the same time, this survey deeply explains the basic principles of representation learning, and introduces its practical application cases in various fields. This survey also points out the main limitations of current models and prospects the future research directions.
Generative Adversarial Networks (GANs) have radically transformed the field of artificial intelligence by offering a highly effective model for understanding complex data distributions and creating new, high-quality content. The evolution of the sector has been incredibly rapid since their invention, resulting in advanced architectural frameworks, highly efficient training techniques, and numerous applications. This review delivers an in-depth and up- to-date overview of the GANs environment, principally highlighting the progress made over 2020 and beyond. We kick off by presenting the basic adversarial concept and the problems that come along with it. Next, through a systematic approach, we depict and discuss the progress of the GANs structures, starting with the superior output of StyleGAN, going through the amalgamation of transformers and 3D data studies, and so on. An enormous part of this paper is invested in presenting a compact, yet extensive, panorama of the most advanced applications found across computer vision, healthcare, data science, and other novel areas. Moreover, we take a deep and critical dive into persistent issues like evaluation, mode collapse, and theoretical understanding, their challenges, and the ways in which recent works have tried to solve them. To wrap up, we consider the onward path of GANs in light of the impressive proliferation of diffusion models and propose the idea of hybrid generative systems in the future while emphasizing the oppositional approach's extraordinary and enduring features.
Zahraa Salah Dhaif, Hind Jumaa Serteep· International Journal of Adv...· 0 citations
Image processing has become one of the most consequential frontiers of artificial intelligence (AI). The convergence of deep learning [1], large-scale data, and accelerated computing has transformed how machines perceive, interpret, and reconstruct visual information, enabling capabilities that were unattainable with classical, hand-engineered pipelines. Convolutional neural networks first demonstrated that hierarchical feature learning could surpass manually designed descriptors on large-scale recognition tasks [2], deep residual learning subsequently enabled the stable optimization of substantially deeper networks [3], and the emergence of attention-based and transformer architectures has further reshaped the field by modeling long-range spatial dependencies that earlier models struggled to capture [4,5]. In parallel, generative modeling—propelled by generative adversarial networks (GANs) [6] and, more recently, diffusion-based frameworks [7]—has redefined the boundaries of image synthesis, restoration, and cross-modal translation. Despite this rapid progress, translating AI-driven models into dependable, deployable systems remains challenging. Real-world imagery is rarely clean: it is degraded by noise, rain, blur, low illumination, and domain shift; annotated data are often scarce, particularly in medical and remote-sensing settings, despite the success of label-efficient architectures such as U-Net [8] and the recent emergence of promptable segmentation foundation models [9]; and high-stakes applications increasingly demand not only accuracy but also robustness, calibrated uncertainty, computational efficiency, and interpretability
S. Easwaramoorthy· Computer Modeling in Enginee...· 0 citations
In the last decade, we have witnessed the immense development and impact of the computer vision domain and how it has affected various aspects of life. The trigger for this evolution came in September 2012, when AlexNet, a neural network architecture, achieved unprecedented results on the ImageNet Large Scale Visual Recognition Challenge, marking a turning point for deep learning–based visual recognition. This led to significant progress in deep learning over the next few decades, spurring advances in vision tasks including image classification, object detection, segmentation, and generative modeling. This systematic literature review provides a chronological analysis of the developments that have shaped modern computer vision. It reviews the early days of computer vision and its advances in model architectures from convolutional networks to residual, attention, transformer, and hybrid architectures, and it closely analyzes important design patterns about connectivity and efficiency. It also considers the spectrum of learning strategies, ranging from supervised to self-supervised, weakly supervised, and open-vocabulary learning, as well as transfer and multi-task methodologies. The review further highlights optimization and scheduling methods that facilitate the training of large-scale models, and analyzes how performance commonly goes beyond accuracy to include localization quality and efficiency metrics. In short, this is a critical review of computer vision, illustrating how architectural design, learning paradigms, and evaluation practices have co-evolved over time to facilitate more flexible and scalable systems, and outlining new research directions.
The rapid growth of artificial intelligence and deep learning has revolutionized diverse domains, including computer vision where transfer learning through pre-trained models has become a fundamental technique for enhancing performance of models and reducing computational costs. Although numerous pre-trained deep learning models have been widely adopted, their effectiveness varies significantly across application domains and datasets. The lack of comprehensive comparative studies makes it challenging for researchers to identify the most suitable models for achieving optimal performance in specific tasks. This study systematically compares seven pre-trained feature extractors across three architectural families, convolutional neural networks (CNNs), Vision Transformers (ViTs), and self-supervised models to provide practical guidance on model selection for downstream deep learning tasks. These models were evaluated on five benchmark datasets. Features were extracted with frozen weights and evaluated using linear probing, k-nearest neighbor retrieval, and 5-shot classification. ConvNeXt-B achieved the highest mean linear probe accuracy (90.40%), while DINOv2-S produced the best feature geometry for retrieval tasks (87.64%). CLIP-ViT-B/32 demonstrated the strongest cross-domain transfer, leading on texture recognition and satellite imagery few-shot classification. Older CNN architectures lagged significantly behind modern models by approximately 7%. Overall, this study simultaneously evaluates multiple feature extractors across multiple visual domains and evaluation protocols.
Rafeek Sibrikhan, M. Mufassirin· Sri Lankan Journal of Techno...· 0 citations
In the deep learning era, computer vision has advanced quickly, radically changing how machines interpret and evaluate visual data. Data-driven deep neural networks that can immediately learn complicated representations from images have essentially replaced traditional computer vision algorithms that depended on manually created feature extraction. With an emphasis on important architectures including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and new hybrid models, this review article offers a thorough overview of contemporary deep learning-based computer vision. It also looks at popular benchmark datasets, such as ImageNet, COCO, and CIFAR, emphasizing how important they are for model building and assessment. The study also examines a variety of practical uses, from autonomous systems and facial recognition technology to object identification and medical picture analysis. Despite impressive advancements, a number of issues still exist, including as high processing demands, reliance on data, interpretability of models, and ethical issues with prejudice and privacy. The report concludes by outlining future research possibilities, with a focus on real-time deployment, multimodal learning, and efficient model construction. The purpose of this review is to offer an organized and perceptive overview of computer vision's present situation and promise in the deep learning age.
Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.
Kwame Nkosi· International Journal of App...· 0 citations