daft.io.IOConfig

class daft.io.IOConfig(s3=None, azure=None, gcs=None)

Create configurations to be used when accessing storage

Parameters:

• s3 – Configuration to use when accessing URLs with the s3:// scheme

• azure – Configuration to use when accessing URLs with the az:// or abfs:// scheme

• gcs – Configuration to use when accessing URLs with the gs:// or gcs:// scheme

Example

>>> io_config = IOConfig(s3=S3Config(key_id="xxx", access_key="xxx", num_tries=10), azure=AzureConfig(anonymous=True), gcs=GCSConfig(...))
>>> daft.read_parquet(["s3://some-path", "az://some-other-path", "gs://path3"], io_config=io_config)

__init__()

Methods

__init__()
from_json(input)
replace([s3, azure, gcs])

Attributes

azure	Configuration to be used when accessing Azure URLs
gcs	Configuration to be used when accessing Azure URLs
s3	Configuration to be used when accessing s3 URLs

• Get Started
• Documentation
• Github
• Blog
• Community