A Correlative Microscopy Dataset for Multimodal Data Fusion and Image Matching in Materials Science

Acquisition Geometry-Assisted Whole-Group Localization of X-ray Fluorescence Maps in Optical Microscopy Images

X-ray fluorescence (XRF) microscopy maps elemental distributions, while optical microscopy can provide complementary morphological context. Localizing XRF fields of view (FOVs) in optical images is difficult because the two modalities differ in contrast mechanism and resolution. Most current workflows place each XRF tile independently, even when acquisition metadata already record the tiles'relative scan positions. This study formalizes XRF tile-group localization, in which one optical-frame placement is estimated for the whole group, constrained by acquisition geometry and quantified using group intersection-over-union (GroupIoU). In a controlled case study, independent localization failed with GroupIoU 0.000, whereas group localization achieved 0.931. Replacing the normalized cross-correlation (NCC) metric with mutual information (MI) gave nearly identical results, showing that the outcome is not specific to one local similarity metric. In another multiscale case study, using a coarse XRF survey scan to connect the fine-scale tile group to the optical image increased mean GroupIoU from 0.694 to 0.856. These case studies support using acquisition geometry as an explicit constraint when localizing related XRF tiles.

A Fourier-based global denoising model for the smart removal of artifacts in microscopy images

Microscopy techniques, such as scanning tunneling microscopy (STM), atomic force microscopy (AFM), and scanning electron microscopy (SEM), are essential tools in material imaging at micro- and nanoscale resolutions to extract physical knowledge and material structure–property relationships. However, tuning microscopy controls (e.g. scanning speed, current setpoint, tip bias, etc) to obtain high-quality images is a nontrivial and time-consuming effort. In contrast, with substandard images, the key features are not accurately discovered due to noise and artifacts, leading to erroneous analysis. Existing denoising models mostly build on generalizing weak signals as noises, whereas the strong signals are enhanced as key features, which is not always the case in microscopy images; hence, a significant amount of hidden physical information can be completely erased. To address these limitations, we propose a global denoising model (GDM) to smartly remove artifacts from microscopy images while preserving weaker but physically important features. The proposed model is developed based on 1) first designing a two-imaging input channel of nonpair and goal-specific preprocessed images with user-defined trade-off information between the two channels and 2) then integrating a loss function of pixel- and fast Fourier-transformed (FFT) based on training the U-Net model. We compare the proposed GDM with a non-FFT denoising model over STM-generated images of copper (Cu) and silicon (Si) materials and AFM-generated Pantoea sp. YR343 bio-film images and SEM-generated plastic degradation images. Next, we showcase the tuning effect between two imaging input channels on the trade-off performance between artifact removal and feature preservation. On STM, AFM, and SEM datasets, the proposed GDM improves the FFT-based peak-to-noise ratio by up to 4.5 dB on average relative to noisy inputs on simulated datasets and by up to 3.5 dB on experimental datasets. We believe that the proposed workflow can be extended to improve other microscopy image quality and will benefit the experimentalists with the proposed design flexibility to smartly tune according to domain experts’ preferences.

QuantEM: An optimized platform of vision transformer-based models for segmentation and analysis of electron microscopy data

Electron microscopy (EM) is essential for resolving cellular ultrastructure, yet quantitative analysis remains limited by labor-intensive segmentation and the scarcity of generalizable models. Here we present QuantEM, an open-source platform for segmentation and analysis of EM data across imaging modalities, tissues, and species. We assembled the largest curated collection of intracellular EM datasets to date, comprising over 15,000 two-dimensional images and 1,700 three-dimensional acquisitions from more than 600 datasets, including nearly 4,000 newly released acquisitions. Using this resource, we trained an EM-specific vision transformer foundation model and systematically optimized adaptation strategies for organelle segmentation. QuantEM provides pretrained models for mitochondria, endoplasmic reticulum, nuclei, and lipid droplets, integrated with interactive proofreading and downstream quantitative analyses through standalone and napari interfaces. Across diverse naive datasets, QuantEM consistently matches or exceeds existing models on zero-shot segmentation while requiring less data for finetuning. We further demonstrate its utility by revealing previously unrecognized subcellular compartmentation of hepatic glucokinase using immuno-electron microscopy.

XRF-to-Optical Field-of-View Localization with Vision Language Models

This paper evaluates training-free vision language model (VLM) localization on two datasets representing same-section high-correspondence and adjacent-section low-correspondence imaging and tests unconstrained and metadata-constrained search and VLMs with geometric controls, classical template matching, and two alternative training-free approaches.

Whole-organ surface mapping using multiview projection reconstruction

Deep learning for nanoscience scanning electron microscope image classification

Materials science investigates the relationships between a material’s structure, properties, and fabrication processes. Recently, artificial intelligence (AI) and deep learning (DL) techniques have significantly enhanced the ability to analyze complex visual data, particularly in scanning electron microscopy (SEM) images. These approaches enable the detection of subtle morphological patterns that may not be easily identified through manual inspection, facilitating more accurate characterization of nanomaterials. In this study, a multi-class classification framework is proposed for SEM images of nanostructures, categorized into nanowires, fibers, and tips (NFT). To address the issue of class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) was incorporated using two complementary strategies: pixel-level and feature-level representations. In the pixel-level approach, synthetic samples were generated directly from flattened image data, while in the feature-level approach, deep features were first extracted using a pre-trained ResNet50 model before applying SMOTE. For classification, transfer learning was employed using three convolutional neural network architectures: SqueezeNet, ShuffleNet, and GoogLeNet. In addition, a Multi-Layer Perceptron (MLP) classifier was used for feature-level representations. The experimental results demonstrate that both SMOTE strategies effectively address class imbalance, while the pixel-level approach achieved the highest classification performance. The pixel-level approach achieved a classification accuracy of up to 98.35% using GoogLeNet and ShuffleNet, whereas the feature-level approach achieved an accuracy of 97.44% while offering a computationally efficient alternative. These findings highlight the effectiveness of combining SMOTE with transfer learning for handling imbalanced SEM datasets and illustrate the trade-off between classification performance and computational efficiency.