Cleanup MemoryRef tests (#54681)
aka revert revert https://github.com/JuliaLang/julia/pull/54647. Sorry
@Keno for merging with broken tests. Between unrelated failures on
windows CI upload and the fact that the errors were all truncated, I
missed it.
