Published in ICCV, 2019
We present Drive&Act, a domain-specific benchmark with 12 hours and 9.6M frames of driver behaviors across 83 categories. Captured from six views using color, infrared, depth, and 3D pose data, it challenges fine-grained, multi-modal, and cross-view activity recognition. We also provide benchmarks using state-of-the-art video and pose-based methods.
Published in CVPR Workshops, 2020
We introduce ZS-Drive&Act, the first zero-shot benchmark for recognizing unseen fine-grained driver behaviors using activity-driven attributes. Evaluating several zero-shot methods, we improve recognition by 2.79% through a fusion of semantic attributes and word vectors in a Wasserstein GAN-based approach.
Published in ITSC, 2020
We present a diagnostic framework to interpret deep CNNs for driver monitoring by visualizing predictions, analyzing learned features, and conducting failure analysis. Our study reveals common errors stem from object/movement biases, class similarity, and data imbalance. The Inflated 3D Net shows the best performance with more distinct feature clusters and higher recognition rates.
Published in IV, 2020
We tackle unsupervised domain adaptation for driver activity recognition, where models trained on labeled color images adapt to unlabeled infrared images. Using enhanced Variational Auto-Encoders for image translation with classification-driven optimization, our approach learns a shared latent space, boosting cross-domain recognition by 13.75% over conventional methods.
Published in CVPR, 2021
We propose a semi-weakly supervised segmentation method using a student-teacher model and multi-label deep supervision. Our approach integrates various annotation types - full, bounding boxes, global labels, or none - reducing expensive labels by 94.22% while achieving segmentation close to fully supervised models. Validated on retinal fluid segmentation, it narrows the gap to the best baseline to just 5% mean IoU.
Published in CVPR, 2021
We introduce Efficient Concurrent Attention Networks (ECANets) to improve semantic segmentation on 360° omnidirectional images, addressing the large performance drop from narrow FoV models. ECANets capture long-range dependencies and leverage multi-source, omni-supervised training with both labeled and unlabeled data. Evaluated on the new WildPASS dataset and the PASS benchmark, our approach sets new state-of-the-art mIoU scores of 69.0% and 60.2%, respectively.
Published in IV, 2021
This paper proposes an automated method to predict driver intentions from YouTube mock road test videos and dialogs, avoiding costly labels. By detecting and filtering out casual smalltalk, our approach improves prediction accuracy using cleaner conversation data.
Published in IROS, 2021
This work introduces Sims4Action, a dataset created by capturing Activities of Daily Living (ADL) from the game THE SIMS 4 to address challenges in collecting real-world annotated data. It enables flexible data generation with varied environments and views. The GamingToReal benchmark tests how well models trained on game data transfer to real videos. Results show promise but highlight challenges in mixing simulated and real data, opening new research directions. The dataset will be publicly available.
Published in CVPR, 2022
This work tackles the challenge of training panoramic segmentation models using abundant pinhole camera annotations. The proposed Trans4PASS model uses Deformable Patch Embedding and MLP modules to handle panoramic distortions and learns shared semantics via Mutual Prototypical Adaptation for domain alignment. It matches fully supervised results indoors and sets a new state-of-the-art outdoors, reducing the need for many labeled panoramas.
Published in AAAI, 2022
This paper introduces a semi-supervised segmentation method that matches unlabeled pixels to labeled reference pixels, avoiding confirmation bias and needing no model changes. It achieves full supervision performance with 95% fewer labels and improves retinal fluid segmentation by up to 15% IoU.
Published in MICCAI, 2022
This paper tackles the limitation of fixed-category supervision in radiology image classification by using direct text supervision from unstructured medical reports. It introduces a contrastive global-local dual-encoder to learn concepts without relying on predefined labels, enabling open set recognition and better use of weakly annotated data. Evaluated on large chest X-ray datasets, the approach matches performance of traditional label-supervised methods while allowing free-form classification.
Published in ECCV, 2022
This paper tackles semantic volume segmentation with scarce annotations, training on sparsely labeled volumes where only some slices are annotated. It introduces Contrastive Constrained Regularization (Con2R), enabling 3D models to achieve up to 88% of fully supervised accuracy using <4% 2D labels. Experiments on retinal fluid and brain tumor segmentation show Con2R’s effectiveness in extreme low-label settings.
Published in BMVC, 2022
This paper addresses the gap in medical image processing where models lack explicit anatomical knowledge needed for understanding radiology reports. It introduces PAXRay, a dataset integrating detailed anatomical segmentations from CT scans into X-ray images to improve medical phrase grounding linking report content to image regions. Using anatomical info, models achieve up to 50% better grounding on the OpenI dataset compared to standard methods. The dataset is publicly available.
Published in arXiv, technical report, 2023
This paper proposes using 3D CT-based pseudo-labeling to train fine-grained anatomical segmentation models for chest X-rays without manual annotations. From 10,021 thoracic CTs with 157 labels, 3D segmentations are projected to 2D, enabling high-accuracy CXR models with mIoU up to 0.93 and strong agreement with radiologists. The method supports explainable clinical measures like cardio-thoracic ratio and could aid thoracic pathology analysis.
Published in CVPR, 2023
This paper addresses segmentation in expert-driven domains with scarce pixel-wise annotations, evaluating existing methods for handling diverse annotation types. Finding current approaches limited, it proposes Decoupled Semantic Prototypes (DSP), a method that learns from image-level, point, box, and pixel annotations, achieving notable accuracy gains in semi-weakly supervised organelle segmentation.
Published in CVPR, 2023
This paper presents DeLiVER, a benchmark for arbitrary-modal semantic segmentation across Depth, LiDAR, multiple Views, Events, and RGB, with variations for severe weather and sensor failures. It introduces CMNeXt, a flexible fusion model with a Self-Query Hub and Parallel Pooling Mixer, enabling scaling from 1 to 81 modalities with minimal extra parameters. CMNeXt achieves state-of-the-art results on six datasets, with up to +9.10% mIoU gain on DeLiVER over the RGB-only baseline.
Published in ICCV Workshops, 2023
This paper introduces Mirror U-Net, a PET/CT tumor segmentation model that replaces traditional fusion with multimodal fission, splitting features into modality-specific and shared decoders. This setup combines fission and multi-task learning to strengthen unimodal features while preserving multimodal ones. Evaluated on AutoPET and MSD BrainTumor, Mirror U-Net achieves state-of-the-art performance. Code is publicly available.
Published in WACV, 2024
This paper introduces the 360BEV task-mapping panoramic images with depth to bird’s-eye-view semantic maps for holistic indoor scene representation. It releases two datasets (360BEV-Matterport and 360BEV-Stanford) and proposes 360Mapper, a method that outperforms prior approaches by +7.60% and +9.70% mIoU on the respective datasets.
Published in ISBI, 2024
The paper studies domain adaptation for cell organelle segmentation, where models lose over 60% accuracy on new imaging data. It evaluates unsupervised and weakly supervised methods, finding that image-level labels help. The proposed StyleFilter method leverages these labels, improving performance by 19.2% absolute Dice over naive transfer in electron microscopy adaptation.
Published in TPAMI, 2024
The paper presents Trans4PASS+, a transformer-based model for panoramic semantic segmentation, addressing image distortions and scarce annotations via Deformable Patch Embedding and Deformable MLP modules. It improves Mutual Prototypical Adaptation with pseudo-label rectification and introduces SynPASS, a 9,080-panoramic image dataset for Synthetic-to-Real adaptation. The method achieves state-of-the-art results on four domain-adaptive benchmarks.
Published in EMNLP, 2024
This paper introduces SciEx, a multilingual, multimodal benchmark of university computer science exam questions for testing LLMs on scientific tasks. It includes freeform, image-based questions in English and German, with human expert grading revealing the best model scores only 59.4%. The authors compare LLM and student performance and propose LLM-as-a-judge grading, achieving high correlation with experts.
Published in arXiv, MICCAI autoPET III Challenge, 2024
This paper details a data selection strategy for the autoPET3 datacentric track, showing that removing the easiest training samples (based on model loss) reduces false positives—especially in PSMA-PET—and improves both false negative volume and Dice score over the baseline.
Published in MICCAI, 2024
The paper presents APEx, a segmentation model that integrates anatomical and pathological features to improve pathology segmentation. Using a query-based transformer and a mixing strategy, APEx enables anatomy-informed pathology predictions, achieving state-of-the-art results on FDG-PET-CT and Chest X-ray tasks. Code and models are publicly available.
Published in NeurIPS, 2024
The paper introduces Muscles in Time (MinT), a large-scale synthetic dataset of muscle activation generated by enriching motion capture data with OpenSim biomechanical simulations. Covering 227 subjects and 402 muscles over nine hours, MinT enables detailed study of muscle activation timing and supports training models for muscle activation estimation from pose sequences.
Published in arXiv, technical report, 2025
The paper addresses instance segmentation of foreign objects in chest X-rays, where data scarcity hinders training. It generates synthetic data by inserting shapes or cut-pasted labels with anatomically guided placement. This method matches fully supervised performance while using 93% fewer manual annotations.
Published in AAAI, 2025
The paper proposes Connected-Component (CC)-Metrics, a segmentation evaluation protocol for multi-instance detection where each connected component is equally important. Designed for cases like metastases segmentation in PET/CT, it removes bias toward large tumors by evaluating metrics per component using proximity-based matching. This approach corrects issues in overlap-, distance-, and counting-based metrics, providing fairer clinical assessment.
Published in arXiv, technical report, 2025
The paper introduces CHAOS, a robustness benchmark for testing MLLMs on charts with textual and visual perturbations at varying severity. It evaluates 13 models across ChartQA and Chart-to-Text tasks, revealing weaknesses and guiding future research in chart understanding. Data and code are publicly released.
Published in arXiv, technical report, 2025
Improving label quality in medical image segmentation is costly, but its benefits remain unclear. We systematically evaluate its impact using multiple pseudo-labeled versions of CT datasets, generated by models like nnU-Net, TotalSegmentator, and MedSAM. Our results show that while higher-quality labels improve in-domain performance, gains remain unclear if below a small threshold. For pre-training, label quality has minimal impact, suggesting that models rather transfer general concepts than detailed annotations. These findings provide guidance on when improving label quality is worth the effort.
Published in to appear at MICCAI Workshop on Efficient Medical AI, 2025
The paper studies visual in-context learning (VICL) for training generalist models in retinal OCT, enabling task definition on the fly without task-specific models. It proposes an evaluation protocol for VICL in OCT, establishes a baseline using multiple datasets, and releases code to support further research.
Published in to appear at ICCV, 2025
This paper explores visual in-context learning for adapting a single model to multi-step vision tasks without retraining. It introduces a synthetic task generation engine from segmentation data, studies codebooks and masking-based training.
Published in to appear at MICCAI Workshop on Statistical Atlases and Computational Modeling of the Heart, 2025
This work supports preoperative planning for transcatheter aortic valve replacement (TAVR) by using AI-based semantic segmentation to measure relevant anatomical structures in CT scans. Fine-grained TAVR-specific pseudo-labels are derived from coarse anatomical data to train segmentation models, with a loss function adaptation improving Dice score performance by 1.27%. The pseudo-labeled dataset is publicly available at https://doi.org/10.5281/zenodo.16274176.
Published:
 | At the CVPR 2020 workshop on Visual Learning With Limited Labels, I presented our paper on Zero-Shot Learning for Driver Behavior Recognition in a virtual poster session. |
Published:
 | At the Intelligent Vehicles Symposium in 2020, I presented our paper on Deep Classification-driven Domain Adaptation for Cross-Modal Driver Behavior Recognition in a virtual talk in the session on Driver State Recognition. |
Published:
 | At the CVPR 2021, I presented our paper on Multi-label Deep Supervision for Medical Image Segmentation in a virtual poster session. |
Published:
 | I had the pleasure to present and discuss my doctoral research on semi-weakly supervised learning to the team at the Zeiss Innovation Hub@KIT. |
Published:
 | I was invited to give a talk on resource-effective semantic segmentation to the AI Team at Web Computing GmbH, where I presented the research areas of semi- and weakly supervised learning. |
Published:
 | In Tel Aviv at ECCV 2022 I was fortunate to presented our paper on Graph-Constrained Contrastive Regularization for Semi-Weakly Volumetric Segmentation in person, with a lot of interested researchers stopping by. |
Published:
 | In Vancouver at CVPR 2023 I presented two papers, first, the paper Delivering arbitrary-modal semantic segmentation on multi-modal street-scene segmentation by lead authors Jiaming Zhang and Ruiping Liu, and later I presented the paper Decoupled semantic prototypes enable learning from diverse annotation types for semi-weakly segmentation in expert-driven domains which explores techniques for training segmentation models with diverse annotation-types to ease the annotation process. I was also fortunate to attend the CVPR Doctoral Consortium, a wonderful event where I could present my work to Prof. René Vidal and discuss computer vision in clinical settings with him. |
Published:
 | I gave a presentation at SRH Holding, talking to the representative body for disabled employees about fundamentals of AI, current advances and challenges as well as a brief overview of some applications of AI for accessibility. |
Published:
 | In Daejeon, at the MICCAI Workshops 2025, together with my former student Alessio Negrini I presented our paper Conquering the Retina: Bringing Visual in-Context Learning to OCT in the poster session of the Efficient Medical AI 2025 Workshop and later I presented our poster on Semantic Segmentation for Preoperative Planning in Transcatheter Aortic Valve Replacement in the context of the STACOM workshop. I further had the pleasure to present the paper GRASPing Anatomy to Improve Pathology Segmentation for my colleague Alexander Jaus and his student Keyi Li in the the MLMI workshop’s poster session. |
Published:
 | In Honolulu, at the ICCV 2025, apart from attending exceptional workshops on a diverse set of topics such as world models, data-efficient learning and foundation models, I presented our paper Is Visual in-Context Learning for Compositional Medical Tasks within Reach? in one of the poster sessions. Engaging with many researchers at the conference has been a joy and I gathered a lot of inspiration through the phenomenal work happening in the community. |
3 ETCS, 2SWS Practical course, Karlsruhe Institute of Technology, 2024
I contributed as a supervisor to the installments: SS2022, SS2024
3 ETCS, 2SWS Seminar, Karlsruhe Institute of Technology, 2024
I contributed as a supervisor to the installments: WS2021, WS2022, WS2023, WS2024
3 ETCS, 2SWS Seminar, Karlsruhe Institute of Technology, 2025
I contributed as lecturer and supervisor to the installments: SS2024, SS2025
3 ETCS, 2SWS Lecture, Karlsruhe Institute of Technology, 2025
Lecture shared among multiple lecturers. I contributed to the installments: SS2020, SS2021, SS2024, SS2025
3 ETCS, 2SWS Lecture, Karlsruhe Institute of Technology, 2025
Main coordinator of the lecture since 2022, shared conception and lecturing since 2021. I contributed to the installments: WS2021, WS2022, WS2023, WS2024, WS2025
Day-to-day thesis supervision, Karlsruhe Institute of Technology, 2026
Frequent supervisor in master’s and bachelor’s thesis projects