Add sympy to binary linux test - fix conda nightly (#97281)
Try to fix following nightly conda linux failure:
https://github.com/pytorch/pytorch/actions/runs/4476375944/jobs/7868006749
We will have to revert builder sympy install:
https://github.com/pytorch/builder/commit/ce427de8a88675c584f63008f4bb9164ab9eef79
Pull Request resolved: https://github.com/pytorch/pytorch/pull/97281
Approved by: https://github.com/malfet
