pytorch
eca21fbd - [ao][sparsity] Data Sparsifier Benchmarking: Model quality evaluation of the sparsified DLRM model (#81779)

Commit
2 years ago
[ao][sparsity] Data Sparsifier Benchmarking: Model quality evaluation of the sparsified DLRM model (#81779) The objective is to perform evaluation of the model quality after sparsifying the embeddings of the dlrm model. The ```evaluation_model_metrics.py``` makes use of the ```sparse_model_metadata.csv``` file dumped by the ```evaluate_disk_savings.py```. The model metrics such as accuracy, auc, f1 etc are calculated on the test-dataset for various sparsity levels, block shapes and norms available on the metadata csv file. **Results**: The model accuracy decreases slowly with sparsity levels. Even at 90% sparsity levels, the model accuracy decreases only by 2%. After running `evaluate_memory_savings.py`, run: `python evaluate_model_metrics.py --raw_data_file=<path_to_raw_data_txt_file> --processed_data_file=<path_to_kaggleAdDisplayChallenge_processed.npz> --sparse_model_metadata=<path_to_sparse_model_metadata_csv>` Dependencies: DLRM Repository (https://github.com/facebookresearch/dlrm) Test Plan: None Pull Request resolved: https://github.com/pytorch/pytorch/pull/81779 Approved by: https://github.com/z-a-f
Author
Committer
Parents
Loading