15.1 Introduction to GANs

A generator and a discriminator playing a two-player game.

These notes walk through the lecture slides. The original deck is unchanged; use (slides) on the course hub if you want that PDF.

Figures and notes follow Géron, Hands-On Machine Learning; Deisenroth, Faisal, and Ong, Mathematics for Machine Learning; and Theodoridis, Machine Learning: A Bayesian and Optimization Perspective.

1. Generative adversarial networks

Generative adversarial networks (GANs) are from Goodfellow et al., 2014. The idea caught on immediately; stable training took years.

A GAN is two neural nets.

Generator from noise; discriminator guesses fake versus real

2. Applications

Use	Sketch
Image generation and editing	Faces, objects, landscapes; inpainting missing patches.
Data augmentation	Extra synthetic rows when labels are expensive.
Text-to-image	A sentence in, a matching scene out.
Style transfer	Copy the look of one image onto another.
Molecular generation	New drug-like or materials structures.
Anomaly detection	Train on normal data; flag points far from that law.
Domain adaptation	Map a source domain onto a target domain and keep the meaning.

3. Generator

The generator takes a random draw (usually Gaussian) and emits a sample, often an image.

Treat that draw as a latent coding. Functionally this is the decoder of a VAE: Gaussian noise in, a new image out. The training rule is different.

Write $z$ for the noise and

x = G (z; θ_{g})

for a fake sample with the same dimension as a real observation $x_{n}$ , $n = 1, \dots, N$ . $θ_{g}$ are the generator weights. Training aims to make the fakes statistically indistinguishable from the reals.

4. Discriminator

The discriminator sees either a fake from $G$ or a real training image, and guesses which. It is a binary classifier. On input $x$ it returns

y = D (x; θ_{d}),

the probability that $x$ is real. Then $1 - D (x; θ_{d})$ is the probability it is fake. $θ_{d}$ are the discriminator weights.

5. Training

Opposite goals. The discriminator wants to tell fakes from reals. The generator wants fakes that fool the discriminator. You cannot train this as one ordinary net. Each iteration has two phases.

Train the discriminator. A batch of reals, plus as many fakes from the current generator. Labels: 0 fake, 1 real. One binary-cross-entropy step. Backprop updates only $θ_{d}$ .
Train the generator. A fresh batch of fakes. No reals. Every label is 1 (real): you want the discriminator to be wrong. Freeze $θ_{d}$ ; backprop updates only $θ_{g}$ .

6. The cost function

A two-player min-max game:

{min}_{θ_{g}} {max}_{θ_{d}} J (θ_{g}, θ_{d}),

J (θ_{g}, θ_{d}) = 𝔼_{x ~ p_{r} (x)} [\ln D (x; θ_{d})] + 𝔼_{z ~ p_{z} (z)} [\ln (1 - D (G (z; θ_{g}); θ_{d}))] .

$p_{r} (x)$ is the real-data law. With $θ_{g}$ held fixed, the discriminator is trained so that $D (x; θ_{d})$ is large on reals and $1 - D (G (z; θ_{g}); θ_{d})$ is large on fakes.

7. Algorithm

Initialize $θ_{d}^{(0)}$ and $θ_{g}^{(0)}$ . Minibatch size $K$ . Let $m$ be the number of discriminator steps per generator step ( $m = 1$ in the original paper).

While $θ_{d}$ and $θ_{g}$ have not converged:

For $t = 1, \dots, m$ :
Sample $z^{(i)}$ , $i = 1, \dots, K$ , from $p_{z} (z)$ .
Sample $x^{(i)}$ , $i = 1, \dots, K$ , from $p_{r} (x)$ .
Ascend on $θ_{d}$ using

\nabla_{θ_{d}} {\frac{1}{K} \sum_{i = 1}^{K} (\ln D (x^{(i)}; θ_{d}) + \ln (1 - D (G (z^{(i)}; θ_{g}); θ_{d})))} .

Then sample a new $z^{(i)}$ , $i = 1, \dots, K$ , from $p_{z} (z)$ and descend on $θ_{g}$ using

\nabla_{θ_{g}} {\frac{1}{K} \sum_{i = 1}^{K} \ln (1 - D (G (z^{(i)}; θ_{g}); θ_{d}))} .

Practice

What two networks are playing the GAN game, and what does each want?
In the generator step, why are all labels set to 1 (real), and which weights stay frozen?
With $θ_{g}$ held fixed, what is the discriminator maximizing on real $x$ and on fake $G (z)$ ?

Ahmad Mousavi