org.apache.spark.ml.evaluation

Class ClusteringEvaluator

Object

org.apache.spark.ml.evaluation.Evaluator

org.apache.spark.ml.evaluation.ClusteringEvaluator

All Implemented Interfaces:

java.io.Serializable, Params, HasFeaturesCol, HasPredictionCol, DefaultParamsWritable, Identifiable, MLWritable

public class ClusteringEvaluator
extends Evaluator
implements HasPredictionCol, HasFeaturesCol, DefaultParamsWritable

:: Experimental ::

Evaluator for clustering results. The metric computes the Silhouette measure using the specified distance measure.

The Silhouette is a measure for the validation of the consistency within clusters. It ranges between 1 and -1, where a value close to 1 means that the points in a cluster are close to the other points in the same cluster and far from the points of the other clusters.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ClusteringEvaluator()
ClusteringEvaluator(String uid)

Method Summary

Modifier and Type	Method and Description
static Params	clear(Param<?> param)
ClusteringEvaluator	copy(ParamMap pMap) Creates a copy of this instance with the same UID and some extra params.
Param<String>	distanceMeasure() param for distance measure to be used in evaluation (supports "squaredEuclidean" (default), "cosine")
double	evaluate(Dataset<?> dataset) Evaluates model output and returns a scalar metric.
static String	explainParam(Param<?> param)
static String	explainParams()
static ParamMap	extractParamMap()
static ParamMap	extractParamMap(ParamMap extra)
static Param<String>	featuresCol()
static <T> scala.Option<T>	get(Param<T> param)
static <T> scala.Option<T>	getDefault(Param<T> param)
String	getDistanceMeasure()
static String	getFeaturesCol()
String	getMetricName()
static <T> T	getOrDefault(Param<T> param)
static Param<Object>	getParam(String paramName)
static String	getPredictionCol()
static <T> boolean	hasDefault(Param<T> param)
static boolean	hasParam(String paramName)
static boolean	isDefined(Param<?> param)
boolean	isLargerBetter() Indicates whether the metric returned by evaluate should be maximized (true, default) or minimized (false).
static boolean	isSet(Param<?> param)
static ClusteringEvaluator	load(String path)
Param<String>	metricName() param for metric name in evaluation (supports "silhouette" (default))
static Param<?>[]	params()
static Param<String>	predictionCol()
static void	save(String path)
static <T> Params	set(Param<T> param, T value)
ClusteringEvaluator	setDistanceMeasure(String value)
ClusteringEvaluator	setFeaturesCol(String value)
ClusteringEvaluator	setMetricName(String value)
ClusteringEvaluator	setPredictionCol(String value)
static String	toString()
String	uid() An immutable unique ID for the object and its derivatives.
static MLWriter	write()

Methods inherited from class org.apache.spark.ml.evaluation.Evaluator

evaluate

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Constructor Detail

ClusteringEvaluator

public ClusteringEvaluator(String uid)

ClusteringEvaluator

public ClusteringEvaluator()

Method Detail

load

public static ClusteringEvaluator load(String path)

toString

public static String toString()

params

public static Param<?>[] params()

explainParam

public static String explainParam(Param<?> param)

explainParams

public static String explainParams()

isSet

public static final boolean isSet(Param<?> param)

isDefined

public static final boolean isDefined(Param<?> param)

hasParam

public static boolean hasParam(String paramName)

getParam

public static Param<Object> getParam(String paramName)

set

public static final <T> Params set(Param<T> param,
                                   T value)

get

public static final <T> scala.Option<T> get(Param<T> param)

clear

public static final Params clear(Param<?> param)

getOrDefault

public static final <T> T getOrDefault(Param<T> param)

getDefault

public static final <T> scala.Option<T> getDefault(Param<T> param)

hasDefault

public static final <T> boolean hasDefault(Param<T> param)

extractParamMap

public static final ParamMap extractParamMap(ParamMap extra)

extractParamMap

public static final ParamMap extractParamMap()

predictionCol

public static final Param<String> predictionCol()

getPredictionCol

public static final String getPredictionCol()

featuresCol

public static final Param<String> featuresCol()

getFeaturesCol

public static final String getFeaturesCol()

save

public static void save(String path)
                 throws java.io.IOException

Throws:

java.io.IOException

write

public static MLWriter write()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

copy

public ClusteringEvaluator copy(ParamMap pMap)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Evaluator

Parameters:

pMap - (undocumented)

Returns:

(undocumented)

isLargerBetter

public boolean isLargerBetter()

Description copied from class: Evaluator

Indicates whether the metric returned by evaluate should be maximized (true, default) or minimized (false). A given evaluator may support multiple metrics which may be maximized or minimized.

Overrides:

isLargerBetter in class Evaluator

Returns:

(undocumented)

setPredictionCol

public ClusteringEvaluator setPredictionCol(String value)

setFeaturesCol

public ClusteringEvaluator setFeaturesCol(String value)

metricName

public Param<String> metricName()

param for metric name in evaluation (supports "silhouette" (default))

Returns:

(undocumented)

getMetricName

public String getMetricName()

setMetricName

public ClusteringEvaluator setMetricName(String value)

distanceMeasure

public Param<String> distanceMeasure()

param for distance measure to be used in evaluation (supports "squaredEuclidean" (default), "cosine")

Returns:

(undocumented)

getDistanceMeasure

public String getDistanceMeasure()

setDistanceMeasure

public ClusteringEvaluator setDistanceMeasure(String value)

evaluate

public double evaluate(Dataset<?> dataset)

Description copied from class: Evaluator

Evaluates model output and returns a scalar metric. The value of isLargerBetter specifies whether larger values are better.

Specified by:

evaluate in class Evaluator

Parameters:

dataset - a dataset that contains labels/observations and predictions.

Returns:

metric