From predicting the weather to simulating disease transmission, scientific software plays an increasingly pivotal role in developing scientific understanding that informs our everyday lives. However, they are also some of the most difficult software systems to properly test. Because the developers are often the scientists themselves, they may not be familiar with established testing techniques and are likely to focus on scientific results rather than software quality. From a technical standpoint, scientific software systems tend to exhibit several characteristics that make traditional testing techniques impractical and often insufficient. They have large, complex input spaces, are computationally expensive to run, often rely on stochastic black-box components, and are applied in exploratory contexts where the expected outcomes are not known. Causality-driven testing has recently emerged as a potential solution to these challenges. Building on the concept of metamorphic testing — investigating whether a change to the input configuration yields the expected corresponding change in output — causal testing employs domain knowledge and existing runtime data to efficiently use the limited test runs that can be performed. In this walkthrough, we will use real examples to demonstrate the limitations of traditional unit testing. We will then show how metamorphic testing can go beyond these limitations, how causal reasoning can be employed to increase the efficiency of the testing process, and how the Causal Testing Framework can automate much of this. The slides can be viewed by opening `slides.html` in your browser or `slides.pdf` in any PDF viewer. If you want to view the interactive jupyter notebook, uncompress the source zip file (or clone the repository) and follow the instructions in the README file.
Michael S. Foster, Sylvia Whittle· Zenodo (CERN European Organi...· 0 citations
Disinformation and false content produced by generative AI often moves faster than journalists, fact-checkers and human rights defenders can verify. Across a decade of EU-funded projects - InVID (2016–2018), WeVerify (2018–2021), and vera.ai (2022–2025) - research groups have produced a steady stream of natural language processing (NLP) classifiers for persuasion techniques, subjectivity, AI-generated text, and more. Individually, each is a valuable research output. Yet together, they were unusable in a journalist’s daily workflow, being spread across papers, demos, and APIs that couldn’t talk to each other. The solution is the Verification Assistant, a component of the Verification Plugin published by Agence France-Presse (AFP), which bridges the “research-to-practice” gap. Maintained by RSEs at the University of Sheffield since 2023, it accepts a URL or media file, extracts content, routes it to NLP classifiers contributed by researchers across the project consortia, and returns one cohesive set of credibility indicators. Despite the vera.ai project ending in 2025, the Assistant and plugin remain in active use and continued maintenance. This is a real-world demonstration of research software outlasting the project that funded it. This talk uses the Assistant as a case study for what RSEs contribute to the research journey on a multi-institutional project. Without RSE input, journalists would not have access to these cutting edge research outputs to fight the spread of disinformation. I’ll cover the technical side (API services, architecture, software surviving handovers between teams) and the coordination side (working with researchers across institutions, the human side of building something collaboratively). Slides: https://docs.google.com/presentation/d/1a8jwge1i7Ww0G_sb0vOuyWmO6fiQdjc_CRRNMLxq4Ww/edit?usp=sharing Video: https://drive.google.com/file/d/1hfM9GC1vPKNCVDdvJfS5Xau_dapZYdt3/view?usp=sharing
Rosanna Milner, Michael S. Foster, Twin Karmakharm et al.· Zenodo (CERN European Organi...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
