Layers: input, hidden, output

Parameters: weights, biases

Activations: ReLU, sigmoid, tanh, softmax

Training: forward pass, loss, backpropagation, optimization (SGD/Adam)

Depth vs width

Regularization: dropout, weight decay, batch normalization

Data requirements and compute constraints

Image classification, object detection, segmentation

Any grid-like data (images, spectrograms)

Local receptive fields and spatial feature learning

Weight sharing via convolution kernels

Translation equivariance

Convolution layers (1D/2D/3D)

Pooling (max/average) or strided convolutions

Normalization (BatchNorm/LayerNorm)

Nonlinearities (ReLU/GELU)

Classifier head (fully connected or global average pooling)

LeNet-style: Conv → Pool → FC

VGG-style: stacked small convolutions

ResNet-style: residual/skip connections

U-Net-style: encoder–decoder with skip connections (segmentation)

Sequential data: text, time series, speech

Language modeling, sequence labeling

Hidden state carries information across time steps

Shared parameters across sequence positions

Input at time t → hidden state update → output at time t

Many-to-one, one-to-many, many-to-many configurations

Difficulty with long-term dependencies (vanishing/exploding gradients)

Mitigations: gradient clipping, careful initialization, normalization

Designed to capture long-range dependencies better than vanilla RNNs

Cell state (long-term memory)

Hidden state (short-term representation)

Gates

Stacked/bidirectional LSTMs

Sequence-to-sequence with encoder–decoder

More parameters and compute than vanilla RNN

Often strong on moderate-length sequences and smaller datasets

CNNs: spatial/local patterns in grids

RNNs/LSTMs: temporal dependencies in sequences

CNNs: highly parallelizable

RNNs/LSTMs: more sequential, slower on long sequences

RNN: short-term bias

LSTM: improved long-term retention via gates

CNN feature extractor → RNN/LSTM for sequence modeling (e.g., video, OCR)

Uses past and future context for sequence labeling

Focuses on relevant time steps/features for better context handling

Spatial locality and translational patterns dominate

Order and temporal dependencies are central

Need explicit sequence modeling

Very long sequences or need more parallelism (e.g., attention-based models)