drill
0dbd501c - DRILL-7278: Refactor result set loader projection mechanism

Commit
5 years ago
DRILL-7278: Refactor result set loader projection mechanism Drill 1.16 added a enhanced scan framework based on the row set mechanisms, and a "provisioned schema" feature build on top of that framework. Conversion of the log reader plugin to use the framework identified additional features we wish to add, such as marking a column as "special" (not expanded in a wildcard query.) This work identified that the code added for provisioned schemas in Drill 1.16 worked, but is a bit overly complex, making it hard to add the desired new feature. This patch refactors the "reader" projection code: * Create a "projection set" mechanism that the reader can query to ask, "the caller just added a column. Should it be projected or not?" * Unifies the type conversion mechanism added as part of provisioned schemas. * Added the "special column" property for both "reader" and "provided" schemas. * Verified that provisioned schemas work with maps (at least on the scan framework side.) * Replaced the previous "schema transformer" mechanism with a new "type conversion" mechanism that unifies type conversion, provided schemas and an optional custom type conversion mechanism. * Column writers can report if they are projected. Moved this query from metadata to the column writer itself. * Extended and clarified documentation of the feature. * Revised and/or added unit tests. closes #1797
Author
Committer
Parents
Loading