daft.io.IOConfig

class IOConfig(s3=None, azure=None, gcs=None, http=None)

Create configurations to be used when accessing storage

Parameters:

• s3 – Configuration to use when accessing URLs with the s3:// scheme

• azure – Configuration to use when accessing URLs with the az:// or abfs:// scheme

• gcs – Configuration to use when accessing URLs with the gs:// or gcs:// scheme

Example

>>> io_config = IOConfig(s3=S3Config(key_id="xxx", access_key="xxx", num_tries=10), azure=AzureConfig(anonymous=True), gcs=GCSConfig(...))
>>> daft.read_parquet(["s3://some-path", "az://some-other-path", "gs://path3"], io_config=io_config)

__init__()

Methods

__init__()
replace([s3, azure, gcs, http])

Attributes

azure	Configuration to be used when accessing Azure URLs
gcs	Configuration to be used when accessing Azure URLs
http	Configuration to be used when accessing Azure URLs
s3	Configuration to be used when accessing s3 URLs

• Get Started
• Documentation
• Github
• Blog
• Community