Adds data source properties to fsspec-based connectors #1279
ryannikolaidis
merged
36 commits into
main
from
data-source-properties-fsspec
Adds data source properties to fsspec-based connectors
d32ff419
Fixes issue when properties are not implemented
30ee75bd
Adds data source properties to fsspec-based connectors <- Ingest test…
4dd36186
Merge branch 'main' into data-source-properties-fsspec
1436a0b2
linting
caafc1e0
ryannikolaidis
commented
on 2023-09-04
ryannikolaidis
commented
on 2023-09-04
ryannikolaidis
commented
on 2023-09-04
For fsspec-based connectors: sets file_metadata as a functools.cached…
4e050d67
ryannikolaidis
commented
on 2023-09-05
ryannikolaidis
commented
on 2023-09-05
ryannikolaidis
commented
on 2023-09-05
ryannikolaidis
commented
on 2023-09-05
Adds contextlib.suppress to catch NotImplementedError(s) from fsspec …
a4fe4cb1
Adds data source properties to fsspec-based connectors <- Ingest test…
1c8652a1
Sets remote filesystem url as the source_url
4c720da9
Merge branch 'data-source-properties-fsspec' of github.com:Unstructur…
34b55db1
ryannikolaidis
commented
on 2023-09-06
ryannikolaidis
commented
on 2023-09-06
ryannikolaidis
commented
on 2023-09-06
Adds data source properties to fsspec-based connectors <- Ingest test…
e053d21d
Single line format for properties set within suppress context.
bfacb5ce
Merge branch 'data-source-properties-fsspec' of github.com:Unstructur…
c7a5ce59
linting
9ebd0fa1
Merge branch 'main' into data-source-properties-fsspec
8c5dd7ea
Updates changelog, version bumped to 0.10.13-dev3
1c3d9ca8
ryannikolaidis
approved these changes
on 2023-09-07
Removes str casting from checksum result
dad943eb
Merge branch 'main' into data-source-properties-fsspec
905bf158
Adds data source properties to fsspec-based connectors <- Ingest test…
134e9023
ryannikolaidis
requested changes
on 2023-09-08
sets etag as version for gcs connector
82db4bcf
Merge branch 'data-source-properties-fsspec' of github.com:Unstructur…
cafc392f
Merge branch 'main' into data-source-properties-fsspec
349ed925
Adds data source properties to fsspec-based connectors <- Ingest test…
e3b0c547
Merge branch 'main' into data-source-properties-fsspec
6ade6735
Linting, bump version
e8a15427
ryannikolaidis
approved these changes
on 2023-09-11
Merge branch 'main' into data-source-properties-fsspec
e9296974
Adds data source properties to fsspec-based connectors <- Ingest test…
f29644f5
Changes logic of setting data source properties
11a6e083
Merge branch 'data-source-properties-fsspec' of github.com:Unstructur…
ed0780f3
Merge branch 'main' into data-source-properties-fsspec
3cef401f
Merge branch 'main' into data-source-properties-fsspec
04ed98a7
renames `set_source_metadata` to `update_source_metadata`
a530299b
sets default `source_metadata`
182858ad
Adds data source properties to fsspec-based connectors <- Ingest test…
83cf6111
Merge branch 'main' into data-source-properties-fsspec
01ba8fee
update changelog note
0792b86c
Login to write a write a comment.
Login via GitHub