MiniTorch
A deep-learning framework built from scratch, verified against PyTorch
- Problem
- Autodiff is easy to use and easy to treat as magic. I wanted to understand it by rebuilding it — no deep-learning dependencies.
- Approach
- Reverse-mode automatic differentiation over NumPy: a small tensor/graph engine, a numerically stable LogSoftmax, and an Adam optimizer.
- Outcome
- Gradients and losses match PyTorch numerically. Enough of a framework to train real models, with every line accounted for.
Python NumPy