pytorch
9b6ccde0 - fix precision error in constraint solver (#101307)

Commit
1 year ago
fix precision error in constraint solver (#101307) When adding guards to the constraint solver, we check that they are consistent, i.e., they do not simplify to false when their free symbols are substituted with the corresponding concrete values. However this check may "spuriously" fail because it doesn't take into account precision errors when comparing floats. Since the symbols involved are all positive integers, we try to approximate floats in the guards with rationals, providing concrete values as hints: `sympy.nsimplify` does the job. As an alternative approach, we considered using `sympy.evalf` to compare with reduced precision. But we did not pursue it because * the choice of what is a good reduced precision feels arbitrary (`sympy` uses `1e15` by default); * more importantly, there is no guarantee that we will not encounter the same problem when solving downstream. Differential Revision: [D45826951](https://our.internmc.facebook.com/intern/diff/D45826951/) Pull Request resolved: https://github.com/pytorch/pytorch/pull/101307 Approved by: https://github.com/ezyang
Author
Committer
Parents
Loading