source: pytorch blog: helion on tpu: towards hardware heterogeneous kernel authoring
level: technical
helion is pytorch's high-level domain-specific language for writing machine learning kernels that run efficiently across different hardware. a new collaboration with google adds a tpu backend that translates helion code into pallas, the low-level language for google's tpus. this lets developers write familiar pytorch-style code and get optimized tpu kernels without needing to master pallas or tpu architecture details.
tpus differ from gpus by using a sequential execution model with wide vector units and explicit memory management. to get good performance, kernels must overlap memory transfers with computation. helion's compiler generates pipelined code that does this automatically. for a flash attention kernel, it autotunes between two strategies: one that streams key-value tiles from high-bandwidth memory for long sequences, and another that preloads all key-value data into on-chip memory to eliminate compute bubbles for shorter sequences. on tpu v7, the generated kernel reaches 838 tflops, about 79% of peak tensor core utilization.
across a range of kernels, helion on tpu averages 1.55x speedup over pytorch eager mode and 1.12x over torch.compile with xla. the biggest gains appear on fused or complex operations like flash attention, where automatic compilers struggle. for standard ops like matrix multiply, performance is comparable. the tpu backend depends on torchtpu, expected to release publicly later this year. helion is open source and available on github.
why it matters: it lowers the barrier to writing fast tpu kernels, helping teams target tpus without specialized knowledge and potentially reducing cloud costs.
source: pytorch blog: helion on tpu: towards hardware heterogeneous kernel authoring