当前位置:网站首页>Kindling the Darkness: A Practical Low-light Image Enhancer
Kindling the Darkness: A Practical Low-light Image Enhancer
2022-07-22 19:03:00 【yijun009】
Kindling the Darkness: A Practical Low-light Image Enhancer
Motivation
In the problem of image enhancement , There is no real ground truth There is , Because people's favorite light levels are different . therefore , It is not appropriate to map images only to images with specific light levels .
• How to effectively estimate the illumination component from a single image , And flexibly adjust the light level ?
• After improving the brightness of dark areas , How to eliminate the degradation of noise and color distortion previously hidden in the dark ?
• How in the absence of clear ground-truth Training model for low light image enhancement ( Just refer to a few images )?
abstract
The network is subject to Retinex Theoretical inspiration , Decompose the image into two parts . An ingredient (illumination) Responsible for dimming , And another kind. (reflectance) Responsible for removing color decay . In this way , The original space is decoupled into two smaller subspaces , Expect to regularize better / Study . The network uses paired images taken under different exposure conditions for training , Instead of using any real reflection and lighting information of the ground .
main contributions
The main contributions of this work can be summarized in the following aspects.
- The network is subject to Retinex Theoretical inspiration , Decompose the image into illumination and reflectance Two parts , Thus, the original space is decoupled into smaller parts
- The network is used in different light / The paired images captured under exposure conditions are trained , Instead of using any ground-truth Of illumination and reflectance Information .( This is very good )
- The model we designed provides a mapping function , The brightness can be flexibly adjusted according to the different needs of users .( This is good )
- The network also includes a module , It can effectively remove visual defects magnified by illuminating dark areas . ( What defects ? Pay attention later )
- Extensive experiments were carried out , An advanced scheme to prove the effectiveness and superiority of our design .( Is this sentence for rounding up words ? To learn )
specifically
A good weak light image intensifier should be able to effectively eliminate the degradation hidden in the dark , And flexibly adjust the light / Exposure conditions .
functionally , Can be divided into layer decomposition, reflectance restoration, and illumination adjustment Three modules .
Layer decomposition
according to Retinex Theory can decompose the input ,illumination map It is flexible to adjust the light / The key to exposure conditions .
Data Usage & Priors
No lighting conditions are determined ground-truth. No, / Very few ground-truth Of reflectance and illumination .layer Decomposition is inherently uncertain , therefore , other priors/regularizers Very important . Assume that the image is not degraded , Then different shots of a specific scene should have the same reflectance . Even though illumination map It may change a lot , But they have simple and consistent structures . In practice , The degradation reflected in weak light images is usually more serious than that in bright images , This will be reflectance Partial repair .
It inspired us to , stay relative Image in strong light reflectance It can be used as a reference for reflection from degraded low light images (ground-truth). Someone will ask why synthetic data is not used ? Because it is difficult to synthesize . Different sensors cause different forms of decay .
Illumination Guided Reflectance Restoration
On the mathematical model , The fading low light image can be expressed as I = R ∘ L + E \mathbf{I}=\mathbf{R} \circ \mathbf{L}+\mathbf{E} I=R∘L+E, E \mathbf{E} E It's the recession component , So we have :
I = R ∘ L + E = R ~ ∘ L = ( R + E ~ ) ∘ L = R ∘ L + E ~ ∘ L \mathbf{I}=\mathbf{R} \circ \mathbf{L}+\mathbf{E}=\tilde{\mathbf{R}} \circ \mathbf{L}=(\mathbf{R}+\tilde{\mathbf{E}}) \circ \mathbf{L}=\mathbf{R} \circ \mathbf{L}+\tilde{\mathbf{E}} \circ \mathbf{L} I=R∘L+E=R~∘L=(R+E~)∘L=R∘L+E~∘L
R ~ \tilde{\mathbf{R}} R~ For recession Reflectance .Reflectance It is impossible for restoration to uniformly process the whole image ,illumination Figure can be a good guide .
Arbitrary Illumination Manipulation
Different people / The lighting intensity that applications like can be very different . therefore , The actual system needs to provide an interface for any lighting manipulation . The actual system needs to provide an interface for any lighting operation . In the literature , The three main ways to enhance light conditions are fusion 、 Light level reservation and gamma correction (fusion, light level appointment, and gamma correction.). be based on fusion Methods , Due to the fixed fusion mode , Lack of light regulation function . If you use the second , The training data set must contain images at the target level , This limits its flexibility . For gamma correction , Although it can achieve its goal by setting different values , But it may not reflect different light ( Exposure ) The relationship between levels . This paper advocates learning a flexible mapping function from real data , It allows the user to specify any light / Exposure level .
Method
Layer Decomposition Net
stay retinex In theory , The input picture is decomposed into reflectdance And illumination map. At the same time, suppose , Regardless of image degradation ( noise 、 Color distortion, etc ) Under the circumstances , Different exposure pictures in the same scene reflectdance It should be the same ,illumination maps It should be smooth . So pass decomposition network Output low reflectance As restoration network The input of , Used to generate enhanced pictures ; and high reflectance maps Used as a ground truth, Supervised with restoration network Output image calculation of loss.
decomposition network Divided into two branches, One is class U-Net structure , add sigmoid layer, Used to generate reflectance maps; The other is composed of a simple convolution layer and sigmoid layer form , Used to generate illumination maps.
total loss:
L L D : = L r e c L D + 0.01 L r s L D + 0.08 L i s L D + 0.1 L m c L D \mathcal{L}^{L D}:=\mathcal{L}_{r e c}^{L D}+0.01 \mathcal{L}_{r s}^{L D}+0.08 \mathcal{L}_{i s}^{L D}+0.1 \mathcal{L}_{m c}^{L D} LLD:=LrecLD+0.01LrsLD+0.08LisLD+0.1LmcLD
reflectance similarity loss: L r s L D : = ∥ R l − R h ∥ 2 2 \mathcal{L}_{r s}^{L D}:=\left\|\mathbf{R}_{l}-\mathbf{R}_{h}\right\|_{2}^{2} LrsLD:=∥Rl−Rh∥22
illumination smoothness loss: L i s L D : = ∥ ∇ L l max ( ∣ ∇ I l ∣ , ϵ ) ∥ 1 + ∥ ∇ L h max ( ∣ ∇ I h ∣ , ϵ ) ∥ 1 \mathcal{L}_{i s}^{L D}:=\left\|\frac{\nabla \mathbf{L}_{l}}{\max \left(\left|\nabla \mathbf{I}_{l}\right|, \epsilon\right)}\right\|_{1}+\left\|\frac{\nabla \mathbf{L}_{h}}{\max \left(\left|\nabla \mathbf{I}_{h}\right|, \epsilon\right)}\right\|_{1} LisLD:=∥∥∥max(∣∇Il∣,ϵ)∇Ll∥∥∥1+∥∥∥max(∣∇Ih∣,ϵ)∇Lh∥∥∥1
L \mathbf{L} L Express illumination, I \mathbf{I} I Indicates input . This loss Where the gradient of the input graph is smaller , The network's influence on it illumination Values are more sensitive ( It can't be big , Better not change ).
mutual consistency loss( This loss What do you mean , Ask people ): L m c L D : = ∥ M ∘ exp ( − c ⋅ M ) ∥ 1 \mathcal{L}_{m c}^{L D}:=\|\mathbf{M} \circ \exp (-c \cdot \mathbf{M})\|_{1} LmcLD:=∥M∘exp(−c⋅M)∥1, M : = ∣ ∇ L l ∣ + ∣ ∇ L h ∣ \mathbf{M}:=\left|\nabla \mathbf{L}_{l}\right|+\left|\nabla \mathbf{L}_{h}\right| M:=∣∇Ll∣+∣∇Lh∣
reconstruction error loss: L r e c L D : = ∥ I l − R l ∘ L l ∥ 1 + ∥ I h − R h ∘ L h ∥ 1 \mathcal{L}_{r e c}^{L D}:=\left\|\mathbf{I}_{l}-\mathbf{R}_{l} \circ \mathbf{L}_{l}\right\|_{1}+\left\|\mathbf{I}_{h}-\mathbf{R}_{h} \circ \mathbf{L}_{h}\right\|_{1} LrecLD:=∥Il−Rl∘Ll∥1+∥Ih−Rh∘Lh∥1
See the big picture on the Internet .
Reflectance Restoration Net
Dark light image reflectance maps It will be better than the normal exposure pictures reflectance maps The situation is much worse , So we need to use normal exposure reflectance maps As a supervisor , Calculation loss To restrain , At the same time illumination maps Provide guidance , Finally enhance the picture .
L R R : = ∥ R ^ − R h ∥ 2 2 − SSIM ( R ^ , R h ) + ∥ ∇ R ^ − ∇ R h ∥ 2 2 \mathcal{L}^{R R}:=\left\|\hat{\mathbf{R}}-\mathbf{R}_{h}\right\|_{2}^{2}-\operatorname{SSIM}\left(\hat{\mathbf{R}}, \mathbf{R}_{h}\right)+\left\|\nabla \hat{\mathbf{R}}-\nabla \mathbf{R}_{h}\right\|_{2}^{2} LRR:=∥∥∥R^−Rh∥∥∥22−SSIM(R^,Rh)+∥∥∥∇R^−∇Rh∥∥∥22
Illumination Adjustment Net
Parameters α Used to adjust illumination maps The brightness of , adopt (Lt / Ls) Then calculate the average , obtain .t by target,s by source; When α>1 when , Brightening ,α Less than 1 Time darkening . The key is , Parameters here α You can set it yourself , That is, as bright as you want , Very flexible . actually α In the network, it is actually a feature map, Weighted to the original illumination maps On .
L I A : = ∥ L ^ − L t ∥ 2 2 + ∥ ∣ ∇ L ^ ∣ − ∣ ∇ L t ∣ ∥ 2 2 \mathcal{L}^{I A}:=\left\|\hat{\mathbf{L}}-\mathbf{L}_{t}\right\|_{2}^{2}+\left\||\nabla \hat{\mathbf{L}}|-\left|\nabla \mathbf{L}_{t}\right|\right\|_{2}^{2} LIA:=∥∥∥L^−Lt∥∥∥22+∥∥∥∣∇L^∣−∣∇Lt∣∥∥∥22
Experienment
边栏推荐
- [summary of linked list skills] 141. Circular linked list (simple)
- Go language learning: go language journey (III)
- Summary of all usage of join in SQL syntax (simple example)
- 用LaTeX写论文时如何加资助信息
- PCV, PIL, pilot installation
- Pat class B 1010 univariate polynomial derivation (problem meaning understanding)
- 二、IDEA搭建JFinal项目+代码自动生成+数据库操作测试(三种方式)
- Leetcode 653. sum of two IV - input BST
- Domestic ngrok achieves intranet penetration
- [QT source code reuse] simulate the pop-up mode of qcompleter
猜你喜欢
二、IDEA搭建JFinal项目+代码自动生成+数据库操作测试(三种方式)
用LaTeX写论文时如何加资助信息
Strncpy() copy string (limited by length)
Leetcode 105. constructing binary trees from preorder and inorder traversal sequences
OSI七层网络模型
strncpy() 复制字符串(受长度限制)
国内 Ngrok 实现内网穿透
fucking-algorithm
NRF24L01 wireless module setting transmit receive mode method
Summary of all usage of join in SQL syntax (simple example)
随机推荐
Exercise 7-4 find out the elements that are not common to two arrays (C language)
paper - A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising
Latex如何写引用时将作者名字缩写为et al
JUC-同步器
Tcpdump 简单用法
Parameter index out of range (1 > number of parameters, which is 0).
用LaTeX写论文时如何加资助信息
There is no session in the tensorflow module
pytorch自定义dataloder的时候,返回参数
JVM-系统优化
leetCode笔记
1. Where is the date for qdate (), 2. QT_ Usage Summary of version
Leetcode 2028. find out the missing observation data
Leetcode: 184. the highest paid employee in the Department
PTA basic question 7-23 currency conversion (20 points) (true)
1. Lei Dian: transfer MySQL database to Oracle, 2.qt5.12.3 connect Oracle 12C database
Tcpdump simple usage
Wechat scans the QR code of the website, but only displays the link address, and cannot jump to the web page
JVM调优实战-从零开始 | 项目有关JVM调优总结
Go concurrency mode: pipeline and cancellation