MiniTorch

A deep-learning framework built from scratch, verified against PyTorch

Problem

Autodiff is easy to use and easy to treat as magic. I wanted to understand it by rebuilding it — no deep-learning dependencies.

Approach

Reverse-mode automatic differentiation over NumPy: a small tensor/graph engine, a numerically stable LogSoftmax, and an Adam optimizer.

Outcome

Gradients and losses match PyTorch numerically. Enough of a framework to train real models, with every line accounted for.