Master of Science Birk Sebastian Frostelid Torpmann-Hagen will Friday February 20th, 2026, at 12:15 hold his Thesis Defense for the PhD degree in Science. The title of the thesis is:
« Runtime Verification and Evaluation of Visual Deep Learning Systems under Distributional Shift »
Deep Neural Networks form the basis of most modern approaches to Artificial Intelligence, and are often reported to achieve excellent performance on benchmarks. However, recent work has revealed that this performance does not necessarily generalize to real-world scenarios. This is attributable to the fact that neural networks are sensitive to minor and even imperceptible deviations in the properties of the input data, referred to as distributional shifts. In computer vision tasks, distributional shifts encompass sensor-faults such as image-noise or distortions, and generally any increase in the prevalence of properties for which the network has not learned to account. In addition to the concerns this raises in terms of reliability, the incidence of distributional shift is also typically neglected as part of conventional approaches to evaluation, resulting in inflated performance metrics. To address these issues, this thesis targets visual deep-learning systems and argues that reliability should be verified and evaluated at runtime. In particular, we introduce a suite of online methods that (i) detect and gate potentially incorrect predictions (runtime verification), (ii) provide several continuous measures of performance for a given set of inputs (runtime evaluation), and (iii) translate performance into decision-relevant quantities (runtime risk assessment). Central to our approach is the conceptualization of inductive support — in simple terms, the degree to which the network can interpret properties in the data — as a noisy surrogate for correctness. We operationalize support with out-of-distribution detectors, which form the basis for each of the aforementioned methods. Empirical validation across multiple domain adaptation benchmarks and a polyp segmentation case study demonstrates that these methods enhance both output reliability and evaluation validity relative to conventional approaches. Beyond these purely technical benefits, our methods also align with emerging regulatory expectations for post-market monitoring, transparency, and robustness.
1st Opponent: Professor Jenny Benois-Pineau, School of Sciences and Technologies, University of Bordeaux
2nd Opponent: Associate Professor Massimiliano Ruocco, IDI, NTNU
Internal member and leader of the committee: Professor John Markus Bjørndalen, IFI, UiT