level: research
multi-objective learning trains models on several goals at once. the multi-gradient descent algorithm (mgda) finds a shared update direction that does not hurt any objective. in stochastic settings, mini-batch noise makes this direction jumpy. the vanilla stochastic mgda (smg) converges slowly because the conflict-avoidant direction is only 1/2-hölder continuous with respect to the jacobian. this means small gradient changes can cause large direction shifts, and the exponent 1/2 is the best possible in worst cases.
the authors show that under extra regularity assumptions, the conflict-avoidant direction becomes lipschitz continuous. lipschitz continuity bounds how fast the direction can change, which reduces bias from noisy gradients. this smoother behavior allows faster convergence. they build on this insight to design a new stochastic multi-objective optimizer that adapts to the problem's regularity. the method adjusts update steps to avoid conflicts more reliably than standard smg.
the proposed algorithm uses an adaptive control mechanism for the update direction. it detects when the problem is regular enough to trust the direction estimate. experiments on multi-task learning and fairness benchmarks show faster convergence and better pareto efficiency. the approach narrows the gap between deterministic mgda and its stochastic variants. it works with standard deep learning setups without heavy tuning.
why it matters: faster stochastic multi-objective optimization helps train models that balance accuracy, fairness, and efficiency with fewer data samples.