source: arxiv statistics ml: expressivity and statistical trade-offs in diffusion policy learning
level: research
diffusion policies represent action distributions by running a diffusion process backward in time, with a learned drift term. researchers have now identified the drift lipschitz budget k as the key factor controlling expressivity. a larger k allows the policy to concentrate more sharply near optimal deterministic actions, reducing value approximation error at a rate of 1/k. this means that by tuning k, one can directly trade off how precisely the policy can match the best possible behavior.
the study also examines statistical trade-offs when learning from finite data. with a fixed k, the policy's performance gap from optimal scales with the number of samples and the action dimension. a larger k improves approximation but increases statistical error, creating a tension. the analysis provides explicit bounds that show how to balance these effects, guiding practitioners in choosing k based on available data and problem complexity.
the findings apply to both vanilla score-matching and a new flow-matching variant. the flow-matching approach uses a deterministic path, which can simplify training. the theoretical results hold for both methods, offering a unified view of diffusion policy learning. this work gives clear mathematical insight into why diffusion policies work well in practice and how to set their key parameter for reliable performance.
why it matters: it gives a principled way to tune diffusion policies in reinforcement learning, helping practitioners balance accuracy and data efficiency when modeling complex action distributions.
source: arxiv statistics ml: expressivity and statistical trade-offs in diffusion policy learning