source: pytorch blog: driving the future of open source ai: an update from pytorch foundation projects
level: technical
pytorch 2.13 brings flexattention to apple silicon with up to 12x speedup over sdpa, a new cutedsl inductor backend, and memory-saving nn.linearcrossentropyloss. executorch expands on-device llm capabilities with weekly examples and a hugging face partnership. vllm achieves a stable bi-weekly release cycle and completes model runner v2 redesign, improving gptq performance. it supports new models like kimi k3 and qwen 3.8 and publishes a q3 2026 roadmap focused on agentic workloads and premium tokens. the first vllm conference is set for ray summit in august.
deepspeed integrates ulysses sequence parallelism into hugging face libraries and replaces intel gpu support with direct torch.xpu integration. it earns a best-paper honorable mention at asplos 2026 for superoffload and has three papers accepted, covering automatic sequence parallelism, energy analysis on grace hopper, and efficient offloading. ray improves gb200/gb300 support, actor scheduling, and rdma. it powers frontier models like mai-thinking-1 and nemotron 3 ultra. ray data 2.57 will debut a new high-performance engine, and weight-syncing integrations with vllm and skyrl are in progress.
helion delivers cross-hardware attention kernels with cutedsl for nvidia and pallas for tpus, outperforming flashattention-4 on blackwell and hand-written kernels on tpu. it introduces llm-guided autotuning, achieving 10x tuning efficiency. safetensors releases gil-free serialization for better multi-threaded performance, adds python 3.14 support, and implements mps fast-loading for apple silicon. future work includes cuda fast-path optimization and exploring io_uring and gpu direct storage integrations.
why it matters: these updates show how open-source ai tools are improving performance, hardware compatibility, and developer workflows, directly impacting model training and inference efficiency.
source: pytorch blog: driving the future of open source ai: an update from pytorch foundation projects