A Very Short Introduction to Inception Score(IS)

Generative Adversarial Networks or GANs for short were successful in generating high-quality images, videos, and audio. We have seen various use cases of GANs. Some of the popular GAN networks are BigGAN, StyleGAN, GameGAN, and PGGAN. They have widespread adoption in the industry and academia. But, evaluation of generated samples can be very tricky and prone to errors if done subjectively by humans. IS was introduced to overcome this problem.

What is the Inception Score?

The Inception Score(IS) is an objective performance metric, used to evaluate the quality of generated images or synthetic images, generated by Generative Adversarial Networks(GANs). It measures how realistic and diverse the output images are. It can be used instead of subjective evaluation by humans. After FID(Frechlet Inception Distance), it is the second most important evaluation performance metric. I have written an article explaining FID and how to calculate it.

It was introduced in 2016 by Tim Salimans et al., in the paper titled “Improved Techniques for Training GANs”. It was named after Inception Network(A pre-trained deep learning model on the Imagenet dataset) for image classification by Google. The authors used the Inception Network to extract the feature vectors of images.

It measures two things:

1. Diversity (Variety) — How diverse the generated images are —The entropy of the overall distribution should be high.
2. Quality (Goodness)— How good the generated images are — Low entropy with high predictability is required.

The lowest IS can be zero and the highest IS can be infinity. Higher IS is always better.

Formula

The formula to calculate the Inception Score(IS) is

There are three main terms in the above formula:

1. Conditional Probability Distribution — p(y/x).
   It should be highly predictable and with low entropy. Here y is the set of labels and x is the image.
2. Marginal Probability Distribution — p(y)

Here, G(z) is the generated image by the generator model when provided with a latent vector of random numbers. If the data distribution for y is uniform with high entropy, then the synthetic images will be diverse

3. KL-Divergence — KL-Divergence between conditional probability(p(y/x)) and marginal probability(p(y))

How to calculate the Inception Score?

1. Pass the generated images through the Inception model to get the conditional label distribution p(y|x)
2. Calculate the marginal probability distribution p(y)
3. Calculate the KL Divergence between p(y) and p(y|x)
4. Calculate the sum over classes and take the average of outputs over images
5. Finally, take the exponential of the averaged value.

Now you have the Inception Score!

Limitations of the Inception Score

1. IS will be high if the network generates one image per class only, which is not a good representation of the classes.
2. The Inception Network is trained on the Imagenet dataset consists of 1000 classes only. The Inception score will be low if you train your GAN on a class outside of these 1000 classes.
3. It works on the square and relatively small size images —ex. 300x300

Thank you for reading.

Want to contact me — ahikailash1@gmail.com

About Me:

I am a Co-Founder and CTO of MateLabs. At Mate Labs, we are making demand forecasting easy for enterprises using automation in machine learning. I am also a Co-founder of Raven Protocol. At Raven Protocol, we are building the world’s first decentralized and distributed Artificial Intelligence Platform. And I represent Snapy, which is instant object detection and discovery tool for consumers.

Note: I had published a book(In 2019) on GANs titled “Generative Adversarial Networks Projects”, in which I have covered most of the widely popular GAN architectures and their implementations. DCGAN, StackGAN, CycleGAN, Pix2pix, Age-cGAN, and 3D-GAN have been covered in details at the implementation level. Each architecture has a chapter dedicated to it. I have explained these networks in a very simple and descriptive language using Keras + Tensorflow(Backend). If you are working on GANs or planning to use GANs, give it a read and share your valuable feedback with me at ahikailash1@gmail.com

Generative Adversarial Networks Projects: Build next-generation generative models using TensorFlow…

You can grab your copy from:

http://www.amazon.com/Generative-Adversarial-Networks-Projects-next-generation/dp/1789136679

https://www.amazon.in/Generative-Adversarial-Networks-Projects-next-generation/dp/1789136679

https://www.packtpub.com/big-data-and-business-intelligence/generative-adversarial-networks-projects