gh-38841: Fix `transformation` parameter for LLL on matrices over QQ
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes #12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes #12345". -->
There was an oversight when interpreting the result of LLL() on an
integer matrix which lead to a type error. It assumed a matrix would
always be returned but if `transformation=True` was passed it would
instead be a tuple. This is now fixed and a test has been added.
If a `transformation` parameter is added for BKZ in the future this will
need to be done for it as well.
### :memo: Checklist
<!-- Put an `x` in all the boxes that apply. -->
- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [x] I have created tests covering the changes.
- [x] I have updated the documentation and checked the documentation
preview.
URL: https://github.com/sagemath/sage/pull/38841
Reported by: TheBlupper
Reviewer(s):
