Accelerating Sparse Data Orchestration via Dynamic Reflexive Tiling

Scope

Sparse tensors are difficult to tile efficiently because useful values are distributed irregularly. This work makes tiling respond to the data that is present at runtime.

Abstract

Tensor operations with multiple sparse inputs are often limited by memory traffic. Conventional tiling divides tensors into fixed coordinate-space regions, which can leave accelerator buffers mostly empty. Dynamic reflexive tiling continuously re-tiles active regions using the sparsity of all operands. The method improves buffer occupancy and data reuse, applies across several sparse accelerators and higher-order tensor kernels, and can also be implemented in software.