A framework for learning a data distribution and generating new samples via a game between two neural networks

Two models compete: one generates samples, the other critiques them

Training drives the generator to produce increasingly realistic outputs

Discriminator provides a learned, adaptive loss signal instead of fixed distance metrics

Adversarial pressure approximates matching the real data distribution

Role

Inputs

Output

Typical Architecture (Images)

Role

Inputs

Output

Typical Architecture (Images)

Meaning

Properties

D tries to maximize its ability to distinguish real vs fake

G tries to minimize D’s ability to detect fakes

Value function

Practical Generator Loss

Ideally, G reproduces the real data distribution

D cannot do better than random guessing (≈ 0.5) in the classic setup

Collect and preprocess real dataset (normalization, resizing, augmentation)

Step 1: Update Discriminator

Step 2: Update Generator

Balance matters

Common heuristics

Adds conditioning variable y (class label, text embedding, etc.)

Enables controlled generation (e.g., “generate digit 7”)

Convolutional architecture guidelines for stable image GANs

Common baseline for image synthesis

Uses Wasserstein distance approximation

Replaces discriminator with a critic (no sigmoid probability)

Improves stability and correlates loss with sample quality

Adds gradient penalty to enforce Lipschitz constraint

More stable than weight clipping

Uses least squares loss to reduce vanishing gradients

Often improves visual quality

Paired image-to-image translation (input-output pairs)

Uses conditional adversarial loss + L1 reconstruction loss

Unpaired image-to-image translation

Cycle-consistency loss enforces invertible mapping

Style-based generator with mapping network and style modulation

Strong control over attributes and high-fidelity synthesis

Encourages disentanglement by maximizing mutual information with latent codes

G: upsampling blocks → convolution → normalization → activation

D: convolution → downsampling → activation (often LeakyReLU)

G often uses BatchNorm/InstanceNorm

D often uses spectral normalization; BatchNorm can sometimes hurt D

G output often uses Tanh (if data scaled to [-1, 1])

D uses LeakyReLU; output depends on loss type

Spectral normalization for D

Gradient penalty (WGAN-GP, R1/R2 regularization)

Concatenate labels to z or feature maps

Conditional batch normalization / FiLM / adaptive instance normalization

Oscillations, divergence, sensitive hyperparameters

Generator produces limited variety (few modes)

Partial collapse: reduced diversity; full collapse: near-identical outputs

Especially when D quickly dominates early training

High visual quality can hide low diversity

No single metric fully captures realism + diversity + usefulness

Non-saturating loss

WGAN / WGAN-GP

LSGAN, hinge loss (common in modern GANs)

Spectral normalization (D)

Gradient penalty (WGAN-GP, R1)

Label smoothing (sometimes) and instance noise

Use convolutional inductive bias for images

Avoid checkerboard artifacts (prefer resize-conv over naive deconv)

Tune D:G update ratio (e.g., 1:1 or more D steps)

Use TTUR (two time-scale update rule): different learning rates for G and D

Maintain moving average of G weights (EMA) for better samples

Differentiable augmentation (DiffAugment) to help with limited data

Balanced batches and consistent preprocessing

FID (Fréchet Inception Distance)

IS (Inception Score)

Precision/Recall for Generative Models

Visual inspection for artifacts, diversity, and fidelity

Latent interpolations to assess smoothness and mode coverage

Downstream performance (augmentation benefit, detection performance, etc.)

Photorealistic face generation, object generation, texture synthesis

Style transfer (e.g., summer↔winter), domain adaptation, photo↔painting

Upscaling images with sharper details (e.g., SRGAN-like approaches)

Filling missing regions, removing objects, guided edits

Generate additional training samples for rare classes or limited datasets

Learn normal distribution; anomalies produce higher reconstruction/critic mismatch

Frame synthesis and temporal consistency constraints

Waveform or spectrogram generation, voice conversion

Denoising, modality translation (e.g., MRI↔CT), privacy-preserving synthesis

Face swapping, synthetic media manipulation risks

Memorization and potential leakage of training examples

Generated outputs reflect dataset bias; can amplify stereotypes

Content creation vs fraud

Need for watermarking, provenance, and detection methods

GANs: sharper samples, harder training, implicit likelihood

VAEs: stable training, explicit likelihood lower bound, often blurrier samples

Diffusion: typically more stable and high quality; slower sampling (though improving)

GANs: fast sampling once trained; training can be unstable

Combine adversarial losses with reconstruction/perceptual losses for fidelity and realism

Train DCGAN on MNIST/CIFAR-10/celeba subset

Normalize images to match output activation (e.g., Tanh ↔ [-1,1])

Monitor losses, sample grids, and diversity over time

Save checkpoints and keep an EMA generator for evaluation

D loss near 0 with G loss exploding: D too strong

Repeated identical samples: mode collapse

Checkerboard artifacts: upsampling method issue