DRILL-7278: Refactor result set loader projection mechanism
Drill 1.16 added a enhanced scan framework based on the row set
mechanisms, and a "provisioned schema" feature build on top
of that framework. Conversion of the log reader plugin to use
the framework identified additional features we wish to add,
such as marking a column as "special" (not expanded in a wildcard
query.)
This work identified that the code added for provisioned schemas in
Drill 1.16 worked, but is a bit overly complex, making it hard to add
the desired new feature.
This patch refactors the "reader" projection code:
* Create a "projection set" mechanism that the reader can query to ask,
"the caller just added a column. Should it be projected or not?"
* Unifies the type conversion mechanism added as part of provisioned
schemas.
* Added the "special column" property for both "reader" and "provided"
schemas.
* Verified that provisioned schemas work with maps (at least on the scan
framework side.)
* Replaced the previous "schema transformer" mechanism with a new "type
conversion" mechanism that unifies type conversion, provided schemas
and an optional custom type conversion mechanism.
* Column writers can report if they are projected. Moved this query
from metadata to the column writer itself.
* Extended and clarified documentation of the feature.
* Revised and/or added unit tests.
closes #1797