WUR-AV-Bench
A Benchmark for Active Vision in Complex Greenhouse Environments

A real-world benchmark, dataset, and evaluation suite for active viewpoint planning under severe occlusion.

Thomas A. W. Versmissen1,2,  Xin Wang1,  Gert Kootstra1,  Ricardo da Silva Torres2
1 Agricultural Biosystems Engineering, Wageningen University & Research
2 Artificial Intelligence Group, Wageningen University & Research
📄 Paper soon 💾 Dataset soon 💻 Code soon

Overview

Active vision is the process of moving a camera to resolve occlusions and gather more informative views, which is important for robots operating in cluttered greenhouses. WUR-AV-Bench provides a densely sampled real-world dataset and a four-task evaluation protocol for developing and comparing active viewpoint-planning methods, without needing a physical robot.

Overview of the four-task active vision benchmark pipeline
Data are collected from 25 tomato plants with a robot-mounted RGB-D camera. A stratified RGB subset drives the image-level tasks (I and II). The full dataset, including RGB-D images, point clouds, and camera poses, drives the active-vision reconstruction tasks (III and IV).

Abstract

We present WUR-AV-Bench, a benchmark consisting of a dataset and evaluation tools for developing, evaluating, and comparing active viewpoint planning methods in complex greenhouse environments with tomato plants subject to severe occlusions. The dataset comprises densely sampled viewpoints containing RGB-D images, 3D point clouds, and camera poses, together with object-pose annotations and per-object occlusion and complexity scores. The benchmark defines four progressively more challenging tasks, requiring methods to detect leaf and tomato truss nodes in single images, estimate their 2D orientation, and actively select informative viewpoints for 3D localization and 5D pose estimation under severe occlusion. These tasks enable systematic analysis of perception accuracy, viewpoint planning efficiency, and robustness across different levels of scene complexity. To demonstrate the benchmark, we evaluate two baseline methods: a random viewpoint selection method and an active viewpoint selection method. Results are reported in terms of precision, recall, and viewpoint efficiency across varying occlusion levels. The presented benchmark provides a systematic and reproducible framework for evaluating active viewpoint planning methods in complex agricultural environments.

The Dataset

The data were recorded in a commercial-style greenhouse with cherry-tomato plants, across different times of day, weather conditions, and lighting.

25
tomato plants
25,709
RGB-D viewpoints
4
progressive tasks
5-DoF
node pose labels

What each viewpoint contains

  • An RGB image from a RealSense L515 camera
  • A structured 3D point cloud (RGB-D)
  • A camera pose from a calibrated 6-DoF robot arm

Every plant is annotated at its peduncle and petiole nodes, the junctions where trusses and leaves attach to the stem, with full 5-DoF poses. Each node also has an occlusion score and each plant a complexity score, which are used to stratify the evaluation.

Reconstructed 3D point cloud of a tomato plant
A reconstructed plant point cloud showing the dense, self-occluding canopy.

Detector-training dataset

Alongside the benchmark, we release a separate dataset with image-level annotations, namely segmentation masks and keypoints for peduncles and petioles. It can be used to train the object detectors that the active-vision tasks depend on.

Four Progressive Tasks

Complexity increases from a single 2D image to full 5D pose estimation across many actively selected viewpoints.

Task I

2D Object Detection

Detect peduncles and petioles in single images, using a background-aware protocol that does not penalize correct but task-irrelevant background objects.

Task II

2D Pose Estimation

Localize each node and predict its 2D orientation from two keypoints, scored with a strict on-node distance criterion.

Task III

3D Reconstruction

Actively select viewpoints to localize nodes in 3D with as few observations as possible, rewarding both accuracy and viewpoint efficiency.

Task IV

5D Pose Estimation

Extend 3D localization with orientation, giving full 5-DoF node poses relevant to de-leafing and harvesting.

Occlusion and Scene Complexity

In dense canopies, target nodes are visible from only a small fraction of viewpoints. This is where informed viewpoint planning is expected to help, and where less challenging benchmarks do not separate strong methods from weak ones.

Stratified evaluation

Each annotated node has an occlusion score, defined as the fraction of surrounding viewpoints from which it is hidden. Each plant has a complexity score. Results are reported separately for the low, medium, and high terciles, which shows how a method holds up as scenes become harder.

Node annotations on a tomato plant image with zoomed detail
Peduncle and petiole node annotations, with keypoints defining position and orientation.

Baselines and Results

Two reference baselines share one perception pipeline: a random viewpoint selector, and an active planner that samples candidate views around a region of interest and picks the one with the highest expected information gain.

Random versus active planning

The active planner improves node recall over the random baseline at every occlusion level, with the largest gains on the least-occluded nodes. Both baselines still remain far from complete reconstruction, which indicates the benchmark is challenging and leaves clear room for improvement.

Node recall by occlusion level for active versus random planners
Task III node recall by occlusion level, for active and random viewpoint planning.

Evaluation Code and Tools

The benchmark ships as a single Python package, wur-av-bench. Its main component is the evaluation code; the remaining commands support scoring, annotation, and preprocessing.

Evaluation wur-av-bench eval

The core of the benchmark. It matches predicted node reconstructions to the ground truth and reports precision, recall, and F1 per viewpoint, stratified by plant complexity, together with 3D visualizations of matches and errors.

Occlusion scoring wur-av-bench visibility

Computes the per-node occlusion score by rendering each node from many sampled viewpoints. This score is the basis for the occlusion-stratified evaluation.

3D annotation tool wur-av-bench annotate

An Open3D tool for annotating 5-DoF node poses directly on point clouds.

Pose utilities convert-pose / convert-axes / fit-stem-lines

Convert keypoints to poses, switch between Open3D and ROS conventions, and fit per-plant detection filters.

The dataset, code repository, and paper links above are placeholders and will be activated once each resource is publicly released.

Citation

Citation details will be added once the paper is published.