level: technical
this paper walks through pytorch's automatic differentiation engine as it computes gradients for a physics-informed neural network. the example uses a small 1-3-3-1 multilayer perceptron to solve the initial value problem y'(t)+y(t)=0, y(0)=1. the training requires two levels of differentiation: first, the network computes the physics derivative ŷ'(t) with respect to the input t, and second, the optimizer needs gradients of the loss with respect to all parameters, where the loss itself depends on ŷ'(t).
the authors trace the entire pipeline node by node. they show the computational graph built during the forward pass, then follow the reverse-mode backward traversal that computes all 22 parameter gradients in a single pass. a key part is the graph-on-graph mechanism enabled by setting create_graph=true. this allows pytorch to differentiate through the physics-informed residual correctly, building a second computational graph on top of the first to handle the nested derivatives.
every adjoint value in the trace is verified against hand derivations from tahimi (2026), linking the p and q notation used in that work. the explicit numerical values make it possible to see exactly how gradients flow through the network, from the loss back to each weight and bias. this detailed walkthrough helps clarify what happens inside the autograd engine when training pinns, a setting that pushes automatic differentiation beyond standard deep learning use cases.
why it matters: understanding the exact gradient flow in pinns helps debug and design more reliable physics-informed models, which are used in scientific simulations and engineering.