Module 9: Advanced Deep LearningBy Ashok Kumar Yadav4 min read

Autoencoders and GANs: Image Generation with PyTorch

Autoencoders learn compact representations by reconstructing their input. GANs learn to generate new samples by pitting two networks against each other. Both start from the idea of a latent space.

Autoencoder encoder-decoder diagram next to a GAN generator and discriminator loop

Quick answer: An autoencoder compresses data to a latent vector and reconstructs it, which is useful for denoising and anomaly detection. A GAN trains a generator to fool a discriminator, which produces new realistic samples.

Autoencoder versus GAN

An autoencoder minimises reconstruction error between its input and output. A GAN has no reconstruction target: the generator maps random noise to a sample and the discriminator scores real versus fake. The generator improves only because the discriminator keeps getting better.

Aspect	Autoencoder	GAN
Goal	Reconstruct input	Generate new samples
Loss	MSE or BCE	Adversarial
Training	Stable	Needs careful balancing
Typical use	Denoising, anomaly detection	Image synthesis

Build a convolutional autoencoder

This model compresses 28x28 grayscale images and reconstructs them. After training, large reconstruction errors flag unusual inputs.

Encoder and decoder with a 16-dimensional latent space

import torch, torch.nn as nn

class AE(nn.Module):
    def __init__(self):
        super().__init__()
        self.enc = nn.Sequential(
            nn.Conv2d(1, 16, 3, 2, 1), nn.ReLU(),     # 14x14
            nn.Conv2d(16, 32, 3, 2, 1), nn.ReLU(),    # 7x7
            nn.Flatten(), nn.Linear(32*7*7, 16))      # latent size 16
        self.dec = nn.Sequential(
            nn.Linear(16, 32*7*7), nn.ReLU(),
            nn.Unflatten(1, (32, 7, 7)),
            nn.ConvTranspose2d(32, 16, 3, 2, 1, output_padding=1), nn.ReLU(),
            nn.ConvTranspose2d(16, 1, 3, 2, 1, output_padding=1), nn.Sigmoid())
    def forward(self, x):
        return self.dec(self.enc(x))

x = torch.rand(8, 1, 28, 28)
print(AE()(x).shape)   # torch.Size([8, 1, 28, 28])

Build a simple GAN

The generator turns a 100-dimensional noise vector into an image. The discriminator outputs the probability that an image is real.

One GAN training step

z_dim = 100
G = nn.Sequential(nn.Linear(z_dim, 256), nn.LeakyReLU(0.2),
                  nn.Linear(256, 784), nn.Tanh())
D = nn.Sequential(nn.Linear(784, 256), nn.LeakyReLU(0.2),
                  nn.Linear(256, 1), nn.Sigmoid())
optG = torch.optim.Adam(G.parameters(), 2e-4, betas=(0.5, 0.999))
optD = torch.optim.Adam(D.parameters(), 2e-4, betas=(0.5, 0.999))
bce = nn.BCELoss()

def train_step(real):                      # real: (B, 784) scaled to [-1, 1]
    B = real.size(0)
    ones, zeros = torch.ones(B, 1), torch.zeros(B, 1)
    # 1) train discriminator
    fake = G(torch.randn(B, z_dim)).detach()
    lossD = bce(D(real), ones) + bce(D(fake), zeros)
    optD.zero_grad(); lossD.backward(); optD.step()
    # 2) train generator to fool D
    lossG = bce(D(G(torch.randn(B, z_dim))), ones)
    optG.zero_grad(); lossG.backward(); optG.step()
    return lossD.item(), lossG.item()

Fix unstable GAN training

If the discriminator wins too easily the generator gets no useful gradient. If the generator produces only a few similar images, that is mode collapse. Common fixes: use convolutional (DCGAN) layers with batch norm, add label smoothing, lower the discriminator learning rate, or switch to a Wasserstein loss.

Practical tipJudge GANs by looking at generated samples every few epochs. The loss values alone are misleading.

Hands-on practice

Run the examples in the downloadable Jupyter notebook. The extra checks below focus on mistakes that appear in real projects, not just syntax.

Measure reconstruction error

Practice example

import torch
x=torch.tensor([[0.,1.],[1.,0.]])
x_hat=torch.tensor([[.1,.8],[.9,.2]])
print(torch.mean((x-x_hat)**2).item())

0.025

Reconstruction loss is useful for autoencoders but does not directly measure perceptual quality.

Track generator and discriminator losses together

Practice example

history={'g':[1.2,1.0,.9],'d':[.7,.65,.62]}
print('g_last',history['g'][-1],'d_last',history['d'][-1])

g_last 0.9 d_last 0.62

GAN losses can look stable while samples collapse, so inspect generated outputs too.

Self-check

Can you explain why the first practice result is trustworthy rather than just plausible?
What would you monitor or test before using this Autoencoders & GANs workflow in production?
Which assumption in the example would break first on a larger or messier dataset?

Frequently asked questions

What is a latent space?

A lower-dimensional space where similar inputs sit close together. Autoencoders learn it by compression, and GAN generators sample from it.

What is mode collapse?

The generator produces only a narrow set of outputs that fool the discriminator instead of covering the full variety of the data.

Are GANs still used?

Yes, especially for fast image synthesis and data augmentation, although diffusion models now lead many image-generation benchmarks.

Primary references

Use these official sources when exact syntax, defaults or definitions matter beyond the examples here.

Written by Ashok Kumar Yadav
Ashok Kumar Yadav writes Xenors tutorials on applied AI, programming and data workflows. Every lesson is built around code you can run and check yourself.
GitHub · LinkedIn