Understand the original Gan loss and hinge Gan loss

original Gan Loss

$$\underset{G}{\min }\underset{D}{\max }V(D,G)=E_{x\sim P_{data}}[logD(x)]+E_{z\sim P_z(Z)}[log(1-D(G(Z)))]$$

The Loss Our goal is to optimize two opposing goals at the same time , namely maximize V(D) and minimize V(G).

First , Maximize V(D) when , The function images are

therefore , If you want to Maximize V(D), It only needs D(x) → 1,D(G(Z)) → 0 that will do , Even for real images Discriminator The output probability of D(x) Tend to be 1, And for the generated image Discriminator The output probability of is close to 0, To maximize V(D) So as to optimize the purpose of the discriminator .

secondly , To optimize Generator, Minimize V(G), Because the first term in the formula $E_{x\sim P_{data}}[logD(x)]$ Not included G, So just minimize $E_{z\sim P_z(Z)}[log(1-D(G(Z)))$ that will do , From the function image , To minimize the V(G) Just need to make D(G(Z)) →1 that will do . Train the generator and the discriminator respectively for a period of time , Then train together , That is to say Gan Network optimization process .

Hinge Gan Loss

Hinge Gan Loss yes Hinge Loss And traditional Gan Loss A combination of , First understand Hinge Loss.

$HingeLoss=max(0,1-ty)$ The image of is shown in the above figure ,$ty\geqq 1$ The parts of all become 0 , among $t$ Indicates the desired output tag $\pm 1$ , and y Express SVM Direct input of, such as $y=w\ast x+b$. It can also be expressed by the following formula :
$$HingeLoss=\{\begin{array}{l}1-ty,ifty<1\\ 0,otherwise\end{array}$$
The meaning of the above formula is , If the predicted label is correct ( namely t And y Same number ), And $|y|$ >1 when ,loss by 0

If the predicted label is wrong ( namely t And y Different sign ), be loss With y Linear growth . Allied , When $|y|$ <1 when , Even if t And y Same number ( Correct classification ), But there will still be losses due to insufficient spacing .

Hinge Loss variant

Hinge Loss There are also the following variants :
$$L(y,\hat{y})=max(0,m-y+\hat{y})$$
among : $y$ Express positive ( real ) Sample scores ,$\hat{y}$ Express negative ( forecast ) Sample scores ,m Represents the minimum spacing between positive and negative samples (margin).

Hinge Loss Our goal is to try our best to widen the score gap between positive and negative samples , In the above variants, the minimum score spacing of positive and negative samples should meet margin Conditions ( Suppose in a classification problem , The machine learned nothing , Give the same score for each class , This is the time margin The existence of has a role , bring loss At least m, Instead of being 0).

Hinge Gan Loss

$$V(D,G)=L_D+L_G$$

$$L_D=E[max(0,1-D(x))]+E[max(0,1+D(G(z)))]$$
Optimization objectives ： D(x) → 1,D(G(z)) → 0

For discriminators , Only $D(x)<1$( The probability of real samples is less than 1) and $D(G(z))>-1$( The probability of forging samples is greater than 0) These two situations will produce loss, Need to be optimized , Other cases loss by 0, Thus, the training of the discriminator is stabilized to a certain extent .
$$L_G=-E[D(G(z))]$$
Optimization objectives : D(G(z)) → 1

Reference resources

https://zh.wikipedia.org/zh-cn/Hinge_loss
https://zhuanlan.zhihu.com/p/72195907