pytorch
419b82e0 - [ao][sparsity] L1 norm based block data sparsifier

Commit
2 years ago
[ao][sparsity] L1 norm based block data sparsifier L1-Norm Sparsifier This sparsifier computes the *L1-norm* of every sparse block and "zeroes-out" the ones with the lowest norm. The level of sparsity defines how many of the blocks is removed. This sparsifier is controlled by three variables: 1. `sparsity_level` defines the number of *sparse blocks* that are zeroed-out 2. `sparse_block_shape` defines the shape of the sparse blocks. Note that the sparse blocks originate at the zero-index of the tensor. 3. `zeros_per_block` is the number of zeros that we are expecting in each sparse block. By default we assume that all elements within a block are zeroed-out. However, setting this variable sets the target number of zeros per block. The zeros within each block are chosen as the *smallest absolute values*. Test Plan: ```python test/test_ao_sparsity.py TestNormDataSparsifiers``` Pull Request resolved: https://github.com/pytorch/pytorch/pull/79534 Approved by: https://github.com/z-a-f
Author
Committer
Parents
Loading