Back to Study Material Python

PyTorch

PyTorch is Facebook's open-source deep learning framework and is the preferred tool in academic research. Its define-by-run (dynamic) computation graph mak...

PyTorch

Deep Learning Research with PyTorch

PyTorch is Facebook's open-source deep learning framework and is the preferred tool in academic research. Its define-by-run (dynamic) computation graph makes debugging intuitive and supports advanced architectures like recurrent nets with variable-length input and meta-learning algorithms.

Install with pip install torch torchvision. Check for GPU availability: device = 'cuda' if torch.cuda.is_available() else 'cpu'.

Tensors are PyTorch's core data structure: import torch x = torch.tensor([[1.0, 2.0], [3.0, 4.0]]) x = x.to(device) # move to GPU if available

Autograd tracks operations on tensors and computes gradients automatically: x = torch.tensor(2.0, requires_grad=True) y = x ** 3 y.backward() print(x.grad) # dy/dx = 3x^2 = 12.0

Building a neural network with nn.Module: import torch.nn as nn

class MLP(nn.Module): def __init__(self): super().__init__() self.net = nn.Sequential( nn.Linear(784, 256), nn.ReLU(), nn.Dropout(0.3), nn.Linear(256, 10) ) def forward(self, x): return self.net(x)

model = MLP().to(device) optimizer = torch.optim.Adam(model.parameters(), lr=1e-3) loss_fn = nn.CrossEntropyLoss()

Training loop: for epoch in range(num_epochs): model.train() for X_batch, y_batch in train_loader: X_batch, y_batch = X_batch.to(device), y_batch.to(device) optimizer.zero_grad() preds = model(X_batch) loss = loss_fn(preds, y_batch) loss.backward() optimizer.step()

DataLoader handles batching and shuffling: from torch.utils.data import DataLoader, TensorDataset dataset = TensorDataset(torch.tensor(X_train), torch.tensor(y_train)) loader = DataLoader(dataset, batch_size=64, shuffle=True)

Practical tip: Always call optimizer.zero_grad() before loss.backward(). PyTorch accumulates gradients by default, so forgetting this will cause incorrect weight updates that are hard to diagnose.

Exam tip: PyTorch's model.train() and model.eval() toggle dropout and batch normalization behavior. Forgetting model.eval() during inference causes different results each run, which is a common source of subtle bugs.