GroupBy#

When performing aggregations such as sum, mean and count, you may often want to group data by certain keys and aggregate within those keys.

Calling df.groupby() returns a GroupedDataFrame object which is a view of the original DataFrame but with additional context on which keys to group on. You can then call various aggregation methods to run the aggregation within each group, returning a new DataFrame.

class daft.dataframe.GroupedDataFrame(df: daft.dataframe.dataframe.DataFrame, group_by: daft.expressions.expressions.ExpressionsProjection)[source]#
agg(to_agg: List[Tuple[Union[Expression, str], str]]) DataFrame[source]#

Perform aggregations on this GroupedDataFrame. Allows for mixed aggregations.

Example

>>> df = df.groupby('x').agg([
>>>     ('x', 'sum'),
>>>     ('x', 'mean'),
>>>     ('y', 'min'),
>>>     ('y', 'max'),
>>> ])
Parameters:

to_agg (List[Tuple[ColumnInputType, str]]) – list of (column, agg_type)

Returns:

DataFrame with grouped aggregations

Return type:

DataFrame

agg_concat() DataFrame[source]#

Performs grouped concat on this GroupedDataFrame.

Returns:

DataFrame with grouped concatenated list per column.

Return type:

DataFrame

agg_list() DataFrame[source]#

Performs grouped list on this GroupedDataFrame.

Returns:

DataFrame with grouped list per column.

Return type:

DataFrame

count() DataFrame[source]#

Performs grouped count on this GroupedDataFrame.

Returns:

DataFrame with grouped count per column.

Return type:

DataFrame

map_groups(udf: Expression) DataFrame[source]#

Apply a user-defined function to each group. The name of the resultant column will default to the name of the first input column.

Example

>>> df = daft.from_pydict({"group": ["a", "a", "a", "b", "b", "b"], "data": [1, 20, 30, 4, 50, 600]})
>>>
>>> @daft.udf(return_dtype=daft.DataType.float64())
... def std_dev(data):
...     return [statistics.stdev(data.to_pylist())]
>>>
>>> df = df.groupby("group").map_groups(std_dev(df["data"]))
>>> df.show()
╭───────┬────────────────────╮
│ group ┆ data               │
│ ---   ┆ ---                │
│ Utf8  ┆ Float64            │
╞═══════╪════════════════════╡
│ a     ┆ 14.730919862656235 │
├╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ b     ┆ 331.62026476076517 │
╰───────┴────────────────────╯
(Showing first 2 of 2 rows)
Parameters:

udf (Expression) – User-defined function to apply to each group.

Returns:

DataFrame with grouped aggregations

Return type:

DataFrame

max(*cols: Union[Expression, str]) DataFrame[source]#

Performs grouped max on this GroupedDataFrame.

Parameters:

*cols (Union[str, Expression]) – columns to max

Returns:

DataFrame with grouped max.

Return type:

DataFrame

mean(*cols: Union[Expression, str]) DataFrame[source]#

Performs grouped mean on this GroupedDataFrame.

Parameters:

*cols (Union[str, Expression]) – columns to mean

Returns:

DataFrame with grouped mean.

Return type:

DataFrame

min(*cols: Union[Expression, str]) DataFrame[source]#

Perform grouped min on this GroupedDataFrame.

Parameters:

*cols (Union[str, Expression]) – columns to min

Returns:

DataFrame with grouped min.

Return type:

DataFrame

sum(*cols: Union[Expression, str]) DataFrame[source]#

Perform grouped sum on this GroupedDataFrame.

Parameters:

*cols (Union[str, Expression]) – columns to sum

Returns:

DataFrame with grouped sums.

Return type:

DataFrame