Reproducible Multimodal Affordance Prediction

Tommaso Apicella1, Alessio Xompero2, Andrea Cavallaro3
Istituto Italiano di Tecnologia1
Independent Researcher2
EPFL3
arXiv (coming soon) Repository (coming soon) Affordance Sheet tool (coming soon)

Abstract

Affordance prediction is the identification of potential actions an agent can perform on a target object from multimodal inputs. Affordance prediction methods are difficult to evaluate and compare due to heterogeneous problem formulations, inconsistent dataset annotations, incomplete reporting of experimental protocols, and limited information about deployment conditions. These limitations challenge fair benchmarking and performance comparison. To promote transparency, we propose the Affordance Sheet, a documentation detailing task formulation with its input modalities, model architectures and training information, datasets, and experimental protocols. Affordance Sheets enable reproducible benchmarking and reliable evaluation of affordance models for real-world scenarios, including generalisation to novel conditions and human safety.

Motivation

Evaluation and reproducibility gaps: Affordance prediction models are difficult to compare fairly due to fragmented problem formulations, inconsistent dataset annotations, incomplete reporting of experimental setups, and limited access to code and model weights.

Limited real-world validation: Existing methods are mostly tested in controlled laboratory environments, leaving their generalization to novel conditions, robustness against occlusions, and human safety in physical interactions largely unvalidated.

Importance of affordance reproducibility: Non-reproducible results compromise robustness to occlusions, safety, and generalization to diverse real-world scenarios. Ensuring reproducibility therefore supports both the credibility of research findings and the development of reliable systems capable of consistent and safe interaction with humans in real-world environments.

Reproducibility in visual affordance

Reproducibility challenges (RCs) in different redefinitions of visual affordance prediction include: data availability for benchmarking (RC1); availability of a method's implementation (RC2); availability of the trained model (RC3); details of the experimental setups (RC4); and details of the performance measures used for the evaluation (RC5).


Research challenges

Affordance sheet

To promote reproducibility in affordance prediction and overcome reproducibility challenges, we propose the Affordance Sheet, an organised collection of good practices that can facilitate fair comparisons and the development of new solutions (see the Table below). Our Affordance Sheet includes Model Cards and adds sections that complement the released information.


Affordance sheet

Reference

If you use the information in the paper please cite the following reference
@inproceedings{apicella2026reproducible,
title={Reproducible Multimodal Affordance Prediction},
author={Apicella, Tommaso and Xompero, Alessio and Cavallaro, Andrea},
booktitle={European Conference on Computer Vision Workshops},
year={2026},
}

If you have any further enquiries, question, or comments, please open an issue on the Github repository page.