Background of the project

    Shadow detection has always been a basic and challenging problem in computer vision —— For an input image , We mark shadow areas by generating binary images , The detection of shadow area is to further obtain the illumination in the image 、 The shape and position of the object , And the parameters of the camera . meanwhile , The existence of shadows is also an algorithm for further understanding images in computer vision , For example, object detection and tracking , Bring obstacles .

      Under natural conditions , Due to the presence of shelter , The light emitted by the light source cannot all incident on the object surface , Thus producing shadows . One side , Shaded areas provide enough information , Used to estimate ambient brightness 、 Infer the geometry of the shelter and determine the position of the light source [1]. On the other hand , The existence of shadow pixels in the image will lead to the uncertainty of image content , Target recognition 、 Target tracking 、 Video surveillance 、 It is harmful for tasks such as autonomous driving . therefore , Research on automatic shadow detection algorithm , And use it as a preprocessing step of computer vision task , Mark the position of the shaded area in advance , Avoid confusion with the target to be dealt with , It can improve the robustness of computer vision algorithm .

    The existing shadow detection algorithms can be divided into 4 class ： Based on Eigen image （intrinsic image） Methods [2-4]、 Area based approach [5-8]、 Method based on feature extraction [9-13]、 Methods based on deep learning [14-18]. Finlayson etc. [2-4] Propose a complete Eigen image theory （intrinsic image theory）, The theory is based on the assumption of uniform illumination , Use entropy to minimize （entropy minimization） Method to obtain the gray invariant image （greyscale invariant image）, Then complete shadow detection and shadow removal . however , Because the assumption of uniform illumination is not always true , Image quality varies , The robustness of this automatic algorithm is not strong when it is used to process ordinary network pictures . Region based methods usually use clustering （clustering） The algorithm divides the image pixels into different small blocks , According to color 、 brightness 、 Texture and other features classify pixel blocks into shadow areas or non shadow areas . Hoiem etc. [5-6] Based on support vector machine （support vector machine,SVM） Methods single region classifier and double region classifier are constructed respectively , Use graph cut （graph cut） Achieve the final prediction . Vincente etc. [7] and Yuan etc. [8] with Hoiem etc. [5-6] Similar algorithm flow , However, there are differences in classifiers and classification basis . Vincente etc. [7] Train individually for each different characteristic SVM classifier , Finally get multi-core SVM classifier . Yuan etc. [8] Use logical regression （logistic regression） Decision tree and decision tree （decision trees） Build a single region classifier . Region based methods include clustering 、 Multi feature extraction 、 classification 、 Random field optimization and other algorithm processes , High time complexity , Not suitable for real-time shadow detection . The method based on feature extraction focuses on the different characteristics of shaded areas and non shaded areas . Generally speaking , The brightness and saturation of the shadow area are low , The hue is basically unchanged . Use this feature to set the threshold , Preliminary shadow detection results can be obtained . But considering the scenario 、 Light conditions 、 The diversity of reflectance of object surface , Use only 1 It is not enough to segment two kinds of features by threshold . Some algorithms [9-10] Pay attention to the color information of the shadow area , use HSI、YCbCr Wait for color space to assist in the detection task . Other algorithms [10-13] Pay attention to the geometric features and edge information of the object , Regular use sobel Operator or canny operator , Such algorithms often focus on extracting low-order information of images , Ignore the relationship between pixel blocks and the semantic information of objects in the image , The processing effect of complex scenes is poor . Due to the rise of deep learning methods , In recent years, scholars have proposed a series of Convolutional Neural Networks （convolutional neural network,CNN） A new method of shadow detection based on model . Khan etc. [14] First, we use deep learning method to detect shadows , Training 2 Three networks are used to detect shadow areas and shadow edges , Input the test results into the conditional random field （conditional random field,CRF） Get the results of classification , Compared with the traditional algorithm, the effect is significantly improved . Vincente etc. [15] Earliest use 2 A cascaded network realizes shadow detection , The first 1 A network is used to preliminarily extract shadow marks , And input it as a shadow probability map together with the original map to the 2 A network , Finally, the refined shadow marking results are obtained . Hosseinzadeh etc. [16] Use according to color and texture features SVM The classifier gets the shadow a priori graph （shadow prior map）, Combine it with the original picture and send it to the trained CNN The Internet , Output shadow detection results . Nguyen etc. [17] Introducing conditions to generate confrontation network （conditional generative adversarial networks,cGAN）, generator （generator） Output shadow marks , Judging device （discriminator） Distinguish the true and false shadow marks , The confrontation between the two makes the generator have the ability to detect the shadow area of the image , This method is compared with Vincente[15] The method has further improved the effect . Le etc. [18] Also learn from the idea of generating confrontation Networks , The training generator generates an image after the shadow area is attenuated , The training discriminator outputs shadow marks , Strengthen the ability of the network to distinguish shadow areas from the image source . The existing CNN Shadow detection methods often use cascaded Networks [14-16] Or generate confrontation network [17-18] In the form of , It increases the difficulty of model training , At the same time, it is not conducive to the realization of fast detection algorithm . Compared with , Single network structure can realize end-to-end training , Reduce the difficulty of model training , It can also reduce the amount of calculation to a certain extent , Speed up the algorithm , Obvious advantages .

Related papers ：

1.Mask-ShadowGAN: Learning to Remove Shadows from Unpaired Data

The effect is as follows ：

2. DSC： Direction-aware Spatial Context Features for Shadow Detection

code：https://github.com/xw-hu/DSC

          https://github.com/stevewongv/DSC-PyTorch 

3.Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection 

github:https://github.com/zijundeng/BDRAR

reference ：

1.Attention Res-Unet: An efficient shadow detection algorithm

2. Hu Xiaowei, Chinese University of Hong Kong ： For shadow detection DSC features

3. Research on shadow detection and removal algorithm based on single image