Do not add special `CUDNN` search path rules for `torch_python` (#37349)
Summary:
Those rules never worked until https://github.com/pytorch/pytorch/pull/37275 and afterwards they are causing crashes in manywheels builds, because getting `cudnn` linked into `libtorch_python` and `libtorch_cuda` causes double-free exceptions, see: https://app.circleci.com/pipelines/github/pytorch/pytorch/160350/workflows/85696e1c-1e67-4780-8ceb-18bc0a614507/jobs/5254443
Pull Request resolved: https://github.com/pytorch/pytorch/pull/37349
Test Plan: Enable `manywheels` build by temporarily enabling `manywheels` build on this PR and validate that it fixes the issue, see https://app.circleci.com/pipelines/github/pytorch/pytorch/160796/workflows/13227fbc-97c0-47f6-9a87-e840e1a4b5de/jobs/5267315/steps
Differential Revision: D21264484
Pulled By: malfet
fbshipit-source-id: 01f1082cdb0c078cb0fdd7da381c037df7c89b6f
