source: pytorch blog: how linkedin uses pytorch to solve extreme-scale optimization problems

level: technical

linkedin faces optimization problems with hundreds of millions of users and trillions of decision variables, such as matching jobs to seekers or balancing email volume. these are framed as linear programs that maximize business value under constraints like budgets or fairness. traditional solvers using simplex or interior-point methods fail at this scale due to expensive matrix factorizations. first-order methods, which rely on gradient updates and matrix-vector multiplications, offer a practical alternative. linkedin's original dualip solver used a scala/spark stack but was cpu-bound and hard to extend.

the new dualip-pytorch version expresses the solver as a dataflow of sparse matrix-vector operations and blockwise projections, similar to neural network training. it uses pytorch's gpu acceleration, sparse tensor support, and collective communication for distributed execution. variables are partitioned across gpus while dual variables are synchronized via all-reduce and broadcast, enabling near-linear scaling. techniques like row normalization, regularization continuation, and accelerated gradient methods improve convergence speed while maintaining accuracy.

compared to the cpu-based scala implementation, dualip-pytorch on 8 gpus achieves a 75x speedup in per-iteration wall clock time. the system reduces engineering overhead and supports flexible problem formulations. by unifying optimization and machine learning stacks, linkedin can solve production-grade linear programs at previously infeasible scales. the dominant computations map naturally to high-throughput gpu execution, making the solver both faster and easier to maintain.

why it matters: this approach shows how gpu-accelerated first-order methods can solve massive constrained optimization problems in production, enabling faster and more scalable decision systems for ai-driven platforms.

source: pytorch blog: how linkedin uses pytorch to solve extreme-scale optimization problems