SBM-RGBD Challenge @ RGBD2017

SBM-RGBD Challenge @ RGBD2017

The aim of the SBM-RGBD Challenge, organized in conjunction with the Workshop on Background Learning for Detection and Tracking from RGBD Videos (RGBD2017, http://rgbd2017.na.icar.cnr.it/), is to evaluate and compare scene background modelling methods for moving object detection on RGBD videos of the SBM-RGBD dataset (updated on July 13th, 2017).
Researchers from both academia and industry are invited to test their moving object detection algorithms on the SBM-RGBD dataset and to report their results in order to participate to the Challenge (see Instructions below). Results from all submissions that meet certain minimum quality standards will be reported and maintained in the Challenge Results webpage.

Winners

The winners are the authors of the SCAD algorithm, described in the paper: T. Minematsu, A. Shimada, H. Uchiyama and R.-i. Taniguchi, Simple Combination of Appearance and Depth for Foreground Segmentation, in S. Battiato, G. Gallo, G.M. Farinella, M. Leo (Eds), New Trends in Image Analysis and Processing-ICIAP 2017 Workshops, Lecture Notes in Computer Science, Springer, 2017.

Deadline to submit results

The Challenge is now closed, but you can still submit your results with the below procedure to be added into the Challenge Results webpage.

Instructions for prospective participants

Results can be reported for one, multiple, or all Video categories of the SBM-RGBD dataset. However, in any category, results must be reported for all sequences in that category.
Only one set of tuning parameters should be used for all videos.
Numerical scores can be computed using the provided MatlabScripts.zip. These scripts take the output produced by an algorithm, the ground truth, and a region-of-interest mask (see SBM-RGBD dataset - download) and compute the performance metrics described in SBM-RGBD dataset - Performance evaluation.
In order for method results to be reported among the SBM-RGBD Challenge results, please follow these steps:

Download videos to your computer from SBM-RGBD dataset - Video categories (Please, make sure you download the latest version).
Run your algorithm on each video of the downloaded categories.

The resulting images must be black-and-white (Black = 0, White = 255) images in .png format.
There must be 1 binary image for each frame included in the temporal range specified in the file temporalROI.txt (see SBM-RGBD dataset - download).
The name of the files containing the resulting binary images must comply to the following standard: "bin" + 6 digit numbers + ".png", ex: bin000023.png.

For each video, put the resulting binary images in the corresponding directory of "results" folder
- The "results" folder must have the structure provided in results.zip. Please don't put other folders in the "results" folder.
- The "results" folder must contain the results for every video sequence of one (or more) category.
When ready to submit results, zip the "results" folder and send a WeTransfer link to it to sbmrgbdchallenge@gmail.com, also providing the following information:
Contact information:
1. First Name (ex. John)
2. Last Name (ex. Smith)
3. E-Mail address (ex. jsmith@hotmail.com)
4. University or Company (ex. University of Oxford)
Information on the tested method:
1. Short name/Acronym (ex. MotionTech)
2. Project web site, if any (ex. www.motionTech.org)
3. Method's parameters (ex. alpha = 0.001, beta=10, W=400)
4. Processing time (ex. ~15 fps for a 720x480 video with Matlab code running on a core i5 laptop)
5. Reference to paper, if any (ex. J. Smith "A new motion detection method called motionTech". In IEEE Transactions on Image Processing, 15(1):1168-1177, 2006)

Challenge organizers
Massimo Camplani, University of Bristol, UK
Lucia Maddalena, National Research Council, Italy
Gabriel Moya' Alcover, Universitat de les Illes Balears, Spain
Alfredo Petrosino, University of Naples Parthenope, Italy
Luis Salgado, Universidad Politecnica de Madrid & Universidad Autonoma de Madrid, Spain