org.apache.spark.sql.expressions

Class UserDefinedAggregateFunction

Object

org.apache.spark.sql.expressions.UserDefinedAggregateFunction

All Implemented Interfaces:

java.io.Serializable

public abstract class UserDefinedAggregateFunction
extends Object
implements scala.Serializable

:: Experimental :: The base class for implementing user-defined aggregate functions (UDAF).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
UserDefinedAggregateFunction()

Method Summary

Methods

Modifier and Type	Method and Description
Column	apply(Column... exprs) Creates a Column for this UDAF using given Columns as input arguments.
Column	apply(scala.collection.Seq<Column> exprs) Creates a Column for this UDAF using given Columns as input arguments.
abstract StructType	bufferSchema() A StructType represents data types of values in the aggregation buffer.
abstract DataType	dataType() The DataType of the returned value of this UserDefinedAggregateFunction.
abstract boolean	deterministic() Returns true iff this function is deterministic, i.e.
Column	distinct(Column... exprs) Creates a Column for this UDAF using the distinct values of the given Columns as input arguments.
Column	distinct(scala.collection.Seq<Column> exprs) Creates a Column for this UDAF using the distinct values of the given Columns as input arguments.
abstract Object	evaluate(Row buffer) Calculates the final result of this UserDefinedAggregateFunction based on the given aggregation buffer.
abstract void	initialize(MutableAggregationBuffer buffer) Initializes the given aggregation buffer, i.e.
abstract StructType	inputSchema() A StructType represents data types of input arguments of this aggregate function.
abstract void	merge(MutableAggregationBuffer buffer1, Row buffer2) Merges two aggregation buffers and stores the updated buffer values back to buffer1.
abstract void	update(MutableAggregationBuffer buffer, Row input) Updates the given aggregation buffer buffer with new input data from input.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UserDefinedAggregateFunction

public UserDefinedAggregateFunction()

Method Detail

apply

public Column apply(Column... exprs)

Creates a Column for this UDAF using given Columns as input arguments.

Parameters:

exprs - (undocumented)

Returns:

(undocumented)

distinct

public Column distinct(Column... exprs)

Creates a Column for this UDAF using the distinct values of the given Columns as input arguments.

Parameters:

exprs - (undocumented)

Returns:

(undocumented)

inputSchema

public abstract StructType inputSchema()

A StructType represents data types of input arguments of this aggregate function. For example, if a UserDefinedAggregateFunction expects two input arguments with type of DoubleType and LongType, the returned StructType will look like

new StructType() .add("doubleInput", DoubleType) .add("longInput", LongType)

The name of a field of this StructType is only used to identify the corresponding input argument. Users can choose names to identify the input arguments.

Returns:

(undocumented)

bufferSchema

public abstract StructType bufferSchema()

A StructType represents data types of values in the aggregation buffer. For example, if a UserDefinedAggregateFunction's buffer has two values (i.e. two intermediate values) with type of DoubleType and LongType, the returned StructType will look like

new StructType() .add("doubleInput", DoubleType) .add("longInput", LongType)

The name of a field of this StructType is only used to identify the corresponding buffer value. Users can choose names to identify the input arguments.

Returns:

(undocumented)

dataType

public abstract DataType dataType()

The DataType of the returned value of this UserDefinedAggregateFunction.

Returns:

(undocumented)

deterministic

public abstract boolean deterministic()

Returns true iff this function is deterministic, i.e. given the same input, always return the same output.

Returns:

(undocumented)

initialize

public abstract void initialize(MutableAggregationBuffer buffer)

Initializes the given aggregation buffer, i.e. the zero value of the aggregation buffer.

The contract should be that applying the merge function on two initial buffers should just return the initial buffer itself, i.e. merge(initialBuffer, initialBuffer) should equal initialBuffer.

Parameters:

buffer - (undocumented)

update

public abstract void update(MutableAggregationBuffer buffer,
          Row input)

Updates the given aggregation buffer buffer with new input data from input.

This is called once per input row.

Parameters:

buffer - (undocumented)

input - (undocumented)

merge

public abstract void merge(MutableAggregationBuffer buffer1,
         Row buffer2)

Merges two aggregation buffers and stores the updated buffer values back to buffer1.

This is called when we merge two partially aggregated data together.

Parameters:

buffer1 - (undocumented)

buffer2 - (undocumented)

evaluate

public abstract Object evaluate(Row buffer)

Calculates the final result of this UserDefinedAggregateFunction based on the given aggregation buffer.

Parameters:

buffer - (undocumented)

Returns:

(undocumented)

apply

public Column apply(scala.collection.Seq<Column> exprs)

Creates a Column for this UDAF using given Columns as input arguments.

Parameters:

exprs - (undocumented)

Returns:

(undocumented)

distinct

public Column distinct(scala.collection.Seq<Column> exprs)

Creates a Column for this UDAF using the distinct values of the given Columns as input arguments.

Parameters:

exprs - (undocumented)

Returns:

(undocumented)