Stride in CNN
Stride refers to how many positions the convolutional filter moves between successive applications. A stride of one means the filter shifts by one pixel at...
Defining Stride and Its Effect on Output Size
Stride refers to how many positions the convolutional filter moves between successive applications. A stride of one means the filter shifts by one pixel at a time, producing a dense output. A stride of two means the filter jumps two pixels between applications, producing an output roughly half the spatial dimensions of the input in each direction. Stride therefore directly controls spatial downsampling without requiring a separate pooling layer.
As an example, applying a 3-by-3 filter with stride one to a 32-by-32 input with no padding produces a 30-by-30 output. The same filter with stride two produces a 15-by-15 output. The computation is cheaper because the filter is applied fewer times, and the resulting feature map contains information summarized over larger input regions.
Exam tip: Memorize the output size formula: output equals the floor of the quantity input minus kernel size divided by stride, plus one. Practice applying it to both spatial dimensions independently, since rectangular kernels and non-square inputs require separate calculations for height and width.
Strided Convolution as a Downsampling Tool
Early convolutional network architectures such as AlexNet used max pooling after convolutional layers to reduce spatial resolution. Later work showed that replacing pooling with strided convolutions often performs equally well or better because the network can learn the optimal downsampling weights rather than using a fixed max or average operation.
In practice, architectures like ResNet use a stride of two in specific convolutional layers to halve the feature map dimensions while simultaneously increasing the number of channels to preserve representational capacity. This pattern of doubling channels with each halving of spatial resolution is a common design heuristic that balances parameter count and expressiveness.
When designing a custom architecture, placing strided convolutions too early can destroy fine-grained spatial detail that later layers need. A common approach is to use stride one for early layers that build low-level features and introduce stride two only once the network has already extracted sufficient detail.
Fractional Stride and Transposed Convolutions
Stride values less than one are implemented through transposed convolutions, sometimes called deconvolutions or upsampling convolutions. Rather than skipping positions during the forward pass, a transposed convolution inserts zeros between input values and applies the filter, producing an output larger than the input. This makes transposed convolutions the standard tool for spatial upsampling in generative models and semantic segmentation networks.
In a U-Net style architecture for medical image segmentation, the encoder path uses strided convolutions to reduce spatial resolution while increasing channel depth, and the decoder path uses transposed convolutions to restore spatial resolution at each stage, combining features from the encoder via skip connections.
A common interview question asks candidates to describe the difference between upsampling with nearest-neighbor interpolation followed by a convolution versus using a transposed convolution directly. The key distinction is that the transposed convolution learns its upsampling weights from data, while nearest-neighbor interpolation is a fixed function.