Revert D27001339: Use type-erased union for Buffer.
Test Plan: revert-hammer
Differential Revision:
D27001339 (https://github.com/pytorch/pytorch/commit/c618dc13d2aa23625cb0d7ada694137532a4fa33)
Original commit changeset: 26d7dc19d69d
fbshipit-source-id: 6e036ed7e1f71c9cf20e3361607c4fe4fa2d3d02
