daft.io.IOConfig#

class daft.io.IOConfig(s3=None, azure=None, gcs=None)#

Create configurations to be used when accessing storage

Parameters

  • s3 – Configuration to use when accessing URLs with the s3:// scheme

  • azure – Configuration to use when accessing URLs with the az:// or abfs:// scheme

  • gcs – Configuration to use when accessing URLs with the gs:// or gcs:// scheme

Example

>>> io_config = IOConfig(s3=S3Config(key_id="xxx", access_key="xxx", num_tries=10), azure=AzureConfig(anonymous=True), gcs=GCSConfig(...))
>>> daft.read_parquet(["s3://some-path", "az://some-other-path", "gs://path3"], io_config=io_config)
__init__()#

Methods

__init__()

from_json(input)

Attributes

azure

Configuration to be used when accessing Azure URLs

gcs

Configuration to be used when accessing Azure URLs

s3

Configuration to be used when accessing s3 URLs