org.apache.spark.ml.clustering

Class ExpectationAggregator

Object

org.apache.spark.ml.clustering.ExpectationAggregator

All Implemented Interfaces:

java.io.Serializable

public class ExpectationAggregator
extends Object
implements scala.Serializable

ExpectationAggregator computes the partial expectation results.

param: numFeatures The number of features. param: bcWeights The broadcast weights for each Gaussian distribution in the mixture. param: bcGaussians The broadcast array of Multivariate Gaussian (Normal) Distribution in the mixture. Note only upper triangular part of the covariance matrix of each distribution is stored as dense vector (column major) in order to reduce shuffled data size.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ExpectationAggregator(int numFeatures, Broadcast<double[]> bcWeights, Broadcast<scala.Tuple2<DenseVector,DenseVector>[]> bcGaussians)

Method Summary

Modifier and Type	Method and Description
ExpectationAggregator	add(Vector instance, double weight) Add a new training instance to this ExpectationAggregator, update the weights, means and covariances for each distributions, and update the log likelihood.
long	count()
DenseVector[]	covs()
double	logLikelihood()
DenseVector[]	means()
ExpectationAggregator	merge(ExpectationAggregator other) Merge another ExpectationAggregator, update the weights, means and covariances for each distributions, and update the log likelihood.
double[]	weights()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExpectationAggregator

public ExpectationAggregator(int numFeatures,
                             Broadcast<double[]> bcWeights,
                             Broadcast<scala.Tuple2<DenseVector,DenseVector>[]> bcGaussians)

Method Detail

add

public ExpectationAggregator add(Vector instance,
                                 double weight)

Add a new training instance to this ExpectationAggregator, update the weights, means and covariances for each distributions, and update the log likelihood.

Parameters:

instance - The instance of data point to be added.

weight - The instance weight.

Returns:

This ExpectationAggregator object.

count

public long count()

covs

public DenseVector[] covs()

logLikelihood

public double logLikelihood()

means

public DenseVector[] means()

merge

public ExpectationAggregator merge(ExpectationAggregator other)

Merge another ExpectationAggregator, update the weights, means and covariances for each distributions, and update the log likelihood. (Note that it's in place merging; as a result, this object will be modified.)

Parameters:

other - The other ExpectationAggregator to be merged.

Returns:

This ExpectationAggregator object.

weights

public double[] weights()