Results of the SBM-RGBD Challenge @ RGBD2017

Methods are ordered by date of arrival. Click on the method name for details.

A) Average results on the whole dataset

Method Name Recall Specificity FPR FNR PWC Precision F-Measure
[1]0.8391 0.9958 0.0042 0.0895 1.0828 0.8796 0.8557
[2]0.7707 0.9708 0.0292 0.1578 5.4010 0.7247 0.7068
[3] 0.7786 0.9739 0.0261 0.1499 3.1911 0.7474 0.7472
[4] 0.7065 0.9869 0.0131 0.2221 2.8848 0.7498 0.7157
[5] 0.8493 0.9947 0.0053 0.0793 1.0292 0.8764 0.8606
[6] 0.8847 0.9932 0.0068 0.0439 0.9088 0.8698 0.8757
[7] 0.7622 0.9817 0.0183 0.1664 2.8806 0.7556 0.7470
[8] 0.9186 0.9984 0.0016 0.0100 0.2373 0.9103 0.9143
[9] 0.8211 0.9955 0.0045 0.1075 1.0854 0.8795 0.8477

B) Average results for each category

Method Name Recall Specificity FPR FNR PWC Precision F-Measure
Bootstrapping
RGBD-SOBS 0.8842 0.9925 0.0075 0.1158 2.3270 0.9080 0.8917
RGB-SOBS 0.8023 0.9814 0.0186 0.1977 4.4221 0.8165 0.8007
SRPCA 0.7284 0.9914 0.0086 0.2716 3.7409 0.9164 0.8098
AvgM-D 0.4587 0.9861 0.0139 0.5413 7.1960 0.6941 0.5350
Kim 0.8805 0.9965 0.0035 0.1195 1.5227 0.9566 0.9169
SCAD 0.8997 0.9940 0.0060 0.1003 1.8015 0.9319 0.9134
cwisardH+ 0.5727 0.9616 0.0384 0.4273 8.1381 0.5787 0.5669
MFCN 0.9866 0.9985 0.0015 0.0134 0.2286 0.9885 0.9876
BSABU 0.8070 0.9954 0.0046 0.1930 1.8163 0.9166 0.8562
ColorCamouflage
RGBD-SOBS 0.9563 0.9927 0.0073 0.0437 1.2161 0.9434 0.9488
RGB-SOBS 0.4310 0.9767 0.0233 0.5690 16.0404 0.8018 0.4864
SRPCA 0.8476 0.9389 0.0611 0.1524 4.3124 0.8367 0.8329
AvgM-D 0.9001 0.9793 0.0207 0.0999 2.0719 0.8096 0.8508
Kim 0.9737 0.9927 0.0073 0.0263 0.7389 0.9754 0.9745
SCAD 0.9875 0.9904 0.0096 0.0125 0.7037 0.9677 0.9775
cwisardH+ 0.9533 0.9849 0.0151 0.0467 1.1931 0.9502 0.9510
MFCN 0.9859 0.9977 0.0023 0.0141 0.4272 0.9893 0.9876
BSABU 0.8980 0.9923 0.0077 0.1020 1.2333 0.9489 0.9219
DepthCamouflage
RGBD-SOBS 0.8401 0.9985 0.0015 0.1599 0.9778 0.9682 0.8936
RGB-SOBS 0.9725 0.9856 0.0144 0.0275 1.5809 0.8354 0.8935
SRPCA 0.8679 0.9778 0.0222 0.1321 2.9944 0.7850 0.8083
AvgM-D 0.8368 0.9922 0.0078 0.1632 1.6943 0.8860 0.8538
Kim 0.8702 0.9968 0.0032 0.1298 0.9820 0.9433 0.9009
SCAD 0.9841 0.9963 0.0037 0.0159 0.4432 0.9447 0.9638
cwisardH+ 0.6821 0.9949 0.0051 0.3179 2.4049 0.9016 0.7648
MFCN 0.9870 0.9986 0.0014 0.0130 0.2134 0.9741 0.9804
BSABU 0.8975 0.9985 0.0015 0.1025 0.8171 0.9717 0.9325
IlluminationChanges
RGBD-SOBS 0.4514 0.9955 0.0045 0.0486 0.9321 0.4737 0.4597
RGB-SOBS 0.4366 0.9715 0.0285 0.0634 3.5022 0.4759 0.4527
SRPCA 0.4795 0.9816 0.0184 0.0205 1.9171 0.4159 0.4454
AvgM-D 0.3392 0.9858 0.0142 0.1608 3.0717 0.4188 0.3569
Kim 0.4479 0.9935 0.0065 0.0521 1.1395 0.4587 0.4499
SCAD 0.4699 0.9927 0.0073 0.0301 0.9715 0.4567 0.4610
cwisardH+ 0.4707 0.9914 0.0086 0.0293 1.0754 0.4504 0.4581
MFCN 0.4986 0.9987 0.0013 0.0014 0.1255 0.4912 0.4949
BSABU 0.4628 0.9905 0.0095 0.0372 1.3139 0.4748 0.4673
IntermittentMotion
RGBD-SOBS 0.8921 0.9970 0.0030 0.1079 0.8648 0.9544 0.9202
RGB-SOBS 0.9265 0.9028 0.0972 0.0735 9.3877 0.4054 0.5397
SRPCA 0.8893 0.9629 0.0371 0.1107 3.7026 0.7208 0.7735
AvgM-D 0.8976 0.9912 0.0088 0.1024 1.4603 0.9115 0.9027
Kim 0.9418 0.9938 0.0062 0.0582 0.9213 0.9385 0.9390
SCAD 0.9563 0.9914 0.0086 0.0437 0.8616 0.9243 0.9375
cwisardH+ 0.8086 0.9558 0.0442 0.1914 5.0851 0.5984 0.6633
MFCN 0.9906 0.9987 0.0013 0.0094 0.2466 0.9836 0.9870
BSABU 0.8999 0.9973 0.0027 0.1001 0.6916 0.9543 0.9236
OutOfRange
RGBD-SOBS 0.9170 0.9975 0.0025 0.0830 0.5613 0.9362 0.9260
RGB-SOBS 0.8902 0.9896 0.0104 0.1098 1.3610 0.8237 0.8527
SRPCA 0.8785 0.9878 0.0122 0.1215 1.6100 0.7443 0.8011
AvgM-D 0.6319 0.9860 0.0140 0.3681 2.7663 0.6360 0.6325
Kim 0.9040 0.9961 0.0039 0.0960 0.8228 0.9216 0.9120
SCAD 0.9286 0.9965 0.0035 0.0714 0.5711 0.9357 0.9309
cwisardH+ 0.8959 0.9956 0.0044 0.1041 0.8731 0.9038 0.8987
MFCN 0.9917 0.9982 0.0018 0.0083 0.2018 0.9613 0.9763
BSABU 0.8532 0.9967 0.0033 0.1468 1.0470 0.9110 0.8806
Shadows
RGBD-SOBS 0.9323 0.9970 0.0030 0.0677 0.7001 0.9733 0.9500
RGB-SOBS 0.9359 0.9881 0.0119 0.0641 1.5128 0.9140 0.9218
SRPCA 0.7592 0.9768 0.0232 0.2408 4.0602 0.8128 0.7591
AvgM-D 0.8812 0.9876 0.0124 0.1188 1.9330 0.8927 0.8784
Kim 0.9270 0.9934 0.0066 0.0730 1.0771 0.9404 0.9314
SCAD 0.9665 0.9910 0.0090 0.0335 1.0093 0.9276 0.9458
cwisardH+ 0.9518 0.9877 0.0123 0.0482 1.3942 0.9062 0.9264
MFCN 0.9893 0.9983 0.0017 0.0107 0.2178 0.9842 0.9867
BSABU 0.9290 0.9979 0.0021 0.0710 0.6788 0.9794 0.9515

References:

[1]L. Maddalena and A. Petrosino, Exploiting Color and Depth for Background Subtraction, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 254--265, 2017.

[2] L. Maddalena, A. Petrosino, The SOBS algorithm: What are the limits?, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.21-26, 16-21 June 2012.

[3]S. Javed, T. Bouwmans, M. Sultana and S. K. Jung, Moving Object Detection on RGB-D Videos Using Graph Regularized Spatiotemporal RPCA, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 230--241, 2017.

[4] Unpublished.

[5] Unpublished.

[6] T. Minematsu, A. Shimada, H. Uchiyama and R.-i. Taniguchi, Simple Combination of Appearance and Depth for Foreground Segmentation,in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 266--277, 2017.

[7] M. De Gregorio and M. Giordano, CwisarDH+: Background Detection in RGBD Videos by Learning of Weightless Neural Networks, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 245--253, 2017.

[8] D. Zeng and M. Zhu, Background Subtraction Using Multiscale Fully Convolutional Network, IEEE Access 2018.

[9] N. Dorudian et al., Background subtraction using adaptive blind update, to be submitted.