fix(category_theory/elements): speed up `groupoid_of_elements` (#13372)
from 14.5s to 6s
It's [reported](https://leanprover.zulipchat.com/#narrow/stream/116395-maths/topic/Quadratic.20Hilbert.20symbol.20over.20.E2.84.9A/near/278592167) that this is causing timeouts in recent bors batches.