bpart: Don't invalidate for changes that inference can't see
This implements the optimizations I promised in #57602 by checking
in invalidation whether or not the information that inference sees
changes. The primary situation in which this is useful is avoiding
an invalidation for `export` flag changes or changes of which module
a binding is being imported from that do not change what the actual
binding being imported is. Base itself uses these patterns sparingly,
so the bootstrap impact over #57615 is minimal (though non-zero).
