daft.context.set_runner_ray

Contents

daft.context.set_runner_ray#

set_runner_ray(address: Optional[str] = None, noop_if_initialized: bool = False, max_task_backlog: Optional[int] = None, force_client_mode: bool = False) DaftContext[source]#

Configure Daft to execute dataframes using the Ray distributed computing framework.

Parameters:
  • address – Ray cluster address to connect to. If None, connects to or starts a local Ray instance.

  • noop_if_initialized – If True, skip initialization if Ray is already running.

  • max_task_backlog – Maximum number of tasks that can be queued. None means Daft will automatically determine a good default.

  • force_client_mode – If True, forces Ray to run in client mode.

Returns:

Updated Daft execution context configured for Ray.

Return type:

DaftContext

Note

Can also be configured via environment variable: DAFT_RUNNER=ray