inference: avoid world age assertion (#24449)
This was over-aggressive. It was only correct in the case where we where
depending on the `.inferred` field or `.rettype` contents exactly.
However, sometimes inference of the method instead depends on `.def.source`.
fix #23981
fix #23768
