Results of the SBM-RGBD Challenge @ RGBD2017

Methods are ordered by date of arrival. Click on the method name for details.

A) Average results on the whole dataset

Method Name Recall Specificity FPR FNR PWC Precision F-Measure
[1]0.8391 0.9958 0.0042 0.0895 1.0828 0.8796 0.8557
[2]0.7707 0.9708 0.0292 0.1578 5.4010 0.7247 0.7068
[3] 0.7786 0.9739 0.0261 0.1499 3.1911 0.7474 0.7472
[4] 0.7065 0.9869 0.0131 0.2221 2.8848 0.7498 0.7157
[5] 0.8493 0.9947 0.0053 0.0793 1.0292 0.8764 0.8606
[6] 0.8847 0.9932 0.0068 0.0439 0.9088 0.8698 0.8757
[7] 0.7622 0.9817 0.0183 0.1664 2.8806 0.7556 0.7470
[8] 0.9186 0.9984 0.0016 0.0100 0.2373 0.9103 0.9143
[9] 0.8211 0.9955 0.0045 0.1075 1.0854 0.8795 0.8477
[10] 0.8936 0.9994 0.0006 0.0350 0.4896 0.9209 0.9067
[10] 0.6605 0.9918 0.0082 0.2681 4.7041 0.8116 0.6913

B) Average results for each category

Method Name Recall Specificity FPR FNR PWC Precision F-Measure
Bootstrapping
RGBD-SOBS 0.8842 0.9925 0.0075 0.1158 2.3270 0.9080 0.8917
RGB-SOBS 0.8023 0.9814 0.0186 0.1977 4.4221 0.8165 0.8007
SRPCA 0.7284 0.9914 0.0086 0.2716 3.7409 0.9164 0.8098
AvgM-D 0.4587 0.9861 0.0139 0.5413 7.1960 0.6941 0.5350
Kim 0.8805 0.9965 0.0035 0.1195 1.5227 0.9566 0.9169
SCAD 0.8997 0.9940 0.0060 0.1003 1.8015 0.9319 0.9134
cwisardH+ 0.5727 0.9616 0.0384 0.4273 8.1381 0.5787 0.5669
MFCN 0.9866 0.9985 0.0015 0.0134 0.2286 0.9885 0.9876
BSABU 0.8070 0.9954 0.0046 0.1930 1.8163 0.9166 0.8562
DMSN (SDE) 0.9306 0.9991 0.0009 0.0694 1.1234 0.9902 0.9589
DMSN (SIE) 0.7318 0.9748 0.0252 0.2682 6.0063 0.8094 0.7510
ColorCamouflage
RGBD-SOBS 0.9563 0.9927 0.0073 0.0437 1.2161 0.9434 0.9488
RGB-SOBS 0.4310 0.9767 0.0233 0.5690 16.0404 0.8018 0.4864
SRPCA 0.8476 0.9389 0.0611 0.1524 4.3124 0.8367 0.8329
AvgM-D 0.9001 0.9793 0.0207 0.0999 2.0719 0.8096 0.8508
Kim 0.9737 0.9927 0.0073 0.0263 0.7389 0.9754 0.9745
SCAD 0.9875 0.9904 0.0096 0.0125 0.7037 0.9677 0.9775
cwisardH+ 0.9533 0.9849 0.0151 0.0467 1.1931 0.9502 0.9510
MFCN 0.9859 0.9977 0.0023 0.0141 0.4272 0.9893 0.9876
BSABU 0.8980 0.9923 0.0077 0.1020 1.2333 0.9489 0.9219
DMSN (SDE) 0.9594 0.9995 0.0005 0.0406 1.0580 0.9953 0.9768
DMSN (SIE) 0.2389 0.9982 0.0018 0.7611 17.4555 0.7712 0.3199
DepthCamouflage
RGBD-SOBS 0.8401 0.9985 0.0015 0.1599 0.9778 0.9682 0.8936
RGB-SOBS 0.9725 0.9856 0.0144 0.0275 1.5809 0.8354 0.8935
SRPCA 0.8679 0.9778 0.0222 0.1321 2.9944 0.7850 0.8083
AvgM-D 0.8368 0.9922 0.0078 0.1632 1.6943 0.8860 0.8538
Kim 0.8702 0.9968 0.0032 0.1298 0.9820 0.9433 0.9009
SCAD 0.9841 0.9963 0.0037 0.0159 0.4432 0.9447 0.9638
cwisardH+ 0.6821 0.9949 0.0051 0.3179 2.4049 0.9016 0.7648
MFCN 0.9870 0.9986 0.0014 0.0130 0.2134 0.9741 0.9804
BSABU 0.8975 0.9985 0.0015 0.1025 0.8171 0.9717 0.9325
DMSN (SDE) 0.9698 0.9996 0.0004 0.0302 0.2262 0.9938 0.9816
DMSN (SIE) 0.6901 0.9954 0.0046 0.3099 2.8522 0.9300 0.7360
IlluminationChanges
RGBD-SOBS 0.4514 0.9955 0.0045 0.0486 0.9321 0.4737 0.4597
RGB-SOBS 0.4366 0.9715 0.0285 0.0634 3.5022 0.4759 0.4527
SRPCA 0.4795 0.9816 0.0184 0.0205 1.9171 0.4159 0.4454
AvgM-D 0.3392 0.9858 0.0142 0.1608 3.0717 0.4188 0.3569
Kim 0.4479 0.9935 0.0065 0.0521 1.1395 0.4587 0.4499
SCAD 0.4699 0.9927 0.0073 0.0301 0.9715 0.4567 0.4610
cwisardH+ 0.4707 0.9914 0.0086 0.0293 1.0754 0.4504 0.4581
MFCN 0.4986 0.9987 0.0013 0.0014 0.1255 0.4912 0.4949
BSABU 0.4628 0.9905 0.0095 0.0372 1.3139 0.4748 0.4673
DMSN (SDE) 0.4929 0.9992 0.0008 0.0071 0.1598 0.4962 0.4945
DMSN (SIE) 0.4107 0.9948 0.0052 0.0893 1.4433 0.4529 0.4291
IntermittentMotion
RGBD-SOBS 0.8921 0.9970 0.0030 0.1079 0.8648 0.9544 0.9202
RGB-SOBS 0.9265 0.9028 0.0972 0.0735 9.3877 0.4054 0.5397
SRPCA 0.8893 0.9629 0.0371 0.1107 3.7026 0.7208 0.7735
AvgM-D 0.8976 0.9912 0.0088 0.1024 1.4603 0.9115 0.9027
Kim 0.9418 0.9938 0.0062 0.0582 0.9213 0.9385 0.9390
SCAD 0.9563 0.9914 0.0086 0.0437 0.8616 0.9243 0.9375
cwisardH+ 0.8086 0.9558 0.0442 0.1914 5.0851 0.5984 0.6633
MFCN 0.9906 0.9987 0.0013 0.0094 0.2466 0.9836 0.9870
BSABU 0.8999 0.9973 0.0027 0.1001 0.6916 0.9543 0.9236
DMSN (SDE) 0.9690 0.9993 0.0007 0.0310 0.3548 0.9916 0.9801
DMSN (SIE) 0.8182 0.9972 0.0028 0.1818 1.1730 0.9025 0.8539
OutOfRange
RGBD-SOBS 0.9170 0.9975 0.0025 0.0830 0.5613 0.9362 0.9260
RGB-SOBS 0.8902 0.9896 0.0104 0.1098 1.3610 0.8237 0.8527
SRPCA 0.8785 0.9878 0.0122 0.1215 1.6100 0.7443 0.8011
AvgM-D 0.6319 0.9860 0.0140 0.3681 2.7663 0.6360 0.6325
Kim 0.9040 0.9961 0.0039 0.0960 0.8228 0.9216 0.9120
SCAD 0.9286 0.9965 0.0035 0.0714 0.5711 0.9357 0.9309
cwisardH+ 0.8959 0.9956 0.0044 0.1041 0.8731 0.9038 0.8987
MFCN 0.9917 0.9982 0.0018 0.0083 0.2018 0.9613 0.9763
BSABU 0.8532 0.9967 0.0033 0.1468 1.0470 0.9110 0.8806
DMSN (SDE) 0.9717 0.9996 0.0004 0.0283 0.1738 0.9883 0.9799
DMSN (SIE) 0.9250 0.9954 0.0046 0.0750 0.7642 0.9319 0.9275
Shadows
RGBD-SOBS 0.9323 0.9970 0.0030 0.0677 0.7001 0.9733 0.9500
RGB-SOBS 0.9359 0.9881 0.0119 0.0641 1.5128 0.9140 0.9218
SRPCA 0.7592 0.9768 0.0232 0.2408 4.0602 0.8128 0.7591
AvgM-D 0.8812 0.9876 0.0124 0.1188 1.9330 0.8927 0.8784
Kim 0.9270 0.9934 0.0066 0.0730 1.0771 0.9404 0.9314
SCAD 0.9665 0.9910 0.0090 0.0335 1.0093 0.9276 0.9458
cwisardH+ 0.9518 0.9877 0.0123 0.0482 1.3942 0.9062 0.9264
MFCN 0.9893 0.9983 0.0017 0.0107 0.2178 0.9842 0.9867
BSABU 0.9290 0.9979 0.0021 0.0710 0.6788 0.9794 0.9515
DMSN (SDE) 0.9618 0.9993 0.0007 0.0382 0.3310 0.9910 0.9754
DMSN (SIE) 0.8087 0.9868 0.0132 0.1913 3.2344 0.8835 0.8217

References:

[1]L. Maddalena and A. Petrosino, Exploiting Color and Depth for Background Subtraction, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 254--265, 2017.

[2] L. Maddalena, A. Petrosino, The SOBS algorithm: What are the limits?, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp.21-26, 16-21 June 2012.

[3]S. Javed, T. Bouwmans, M. Sultana and S. K. Jung, Moving Object Detection on RGB-D Videos Using Graph Regularized Spatiotemporal RPCA, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 230--241, 2017.

[4] Unpublished.

[5] Unpublished.

[6] T. Minematsu, A. Shimada, H. Uchiyama and R.-i. Taniguchi, Simple Combination of Appearance and Depth for Foreground Segmentation,in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 266--277, 2017.

[7] M. De Gregorio and M. Giordano, CwisarDH+: Background Detection in RGBD Videos by Learning of Weightless Neural Networks, in Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing - ICIAP 2017. Lecture Notes in Computer Science, vol 10590, Springer, pp. 245--253, 2017.

[8] D. Zeng and M. Zhu, Background Subtraction Using Multiscale Fully Convolutional Network, IEEE Access 2018.

[9] N. Dorudian, S. Lauria, S. Swift, Moving Object Detection Using Adaptive Blind Update and RGB-D Camera, IEEE Sensors Journal 19(18), 2019.

[10] I. Houhou, A. Zitouni, Y. Ruichek, SE. Bekhouche, M. Kas, A. Taleb-Ahmed, RGBD deep multi-scale network for background subtraction, International Journal of Multimedia Information Retrieval, vol.11, pp.395-407, May, 2022.