[resource record] what is reversible neural networks and its relationship with VAE and Gan; What are bits per pixel, bits per dim

INN

Two good explanation videos ：

https://www.youtube.com/watch?v=WqP45Iyvd3o  The speaker here is Analyzing Inverse Problems with Invertible Neural Networks The author of . This article is very suitable for getting started INN The concept of .

https://www.youtube.com/watch?v=IpbeIwSr7r0  This is a review , It involves many articles . among , I think it's worth looking at  NICE: NON-LINEAR INDEPENDENT COMPONENTS ESTIMATION,DENSITY ESTIMATION USING REAL NVP.

BPP

in addition , stay NICE In the article , I didn't know when I looked bits per pixel（bpp） What is it? . After checking, I found that this is an evaluation generative model Very important indicators . Describe in detail in this article 3.1: A NOTE ON THE EVALUATION OF GENERATIVE MODELS https://arxiv.org/pdf/1511.01844.pdf（ There are many good concepts ）. In short , Just use one training set Training generative model after , use model Ask for one test set in X Of P(X), And then take log obtain log likelihood. Then according to Shannon information theory Amount of information in = -log(p(x)), We can define bits per pixel This is the amount of information divided by a total pixels/channel Of Number （ A constant ）. Specifically, this constant , The definition seems a little complicated , Some discussions can be seen ：

https://www.reddit.com/r/MachineLearning/comments/56m5o2/discussion_calculation_of_bitsdims/

https://stats.stackexchange.com/questions/423120/what-is-bits-per-dimension-bits-dim-exactly-in-pixel-cnn-papers

https://www.tutorialspoint.com/dip/concept_of_bits_per_pixel.htm

To make a long story short , For one generative model,log-likelihood higher value is better and bits-per-dimension (BPD) lower value is better.