org.apache.spark.ml.classification

Class GBTClassifier

Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Estimator<M>

org.apache.spark.ml.Predictor<FeaturesType,E,M>

org.apache.spark.ml.classification.Classifier<FeaturesType,E,M>

org.apache.spark.ml.classification.ProbabilisticClassifier<Vector,GBTClassifier,GBTClassificationModel>

org.apache.spark.ml.classification.GBTClassifier

All Implemented Interfaces:

java.io.Serializable, Logging, ClassifierParams, ProbabilisticClassifierParams, Params, HasCheckpointInterval, HasFeaturesCol, HasLabelCol, HasMaxIter, HasPredictionCol, HasProbabilityCol, HasRawPredictionCol, HasSeed, HasStepSize, HasThresholds, HasValidationIndicatorCol, PredictorParams, DecisionTreeParams, GBTClassifierParams, GBTParams, TreeClassifierParams, TreeEnsembleParams, DefaultParamsWritable, Identifiable, MLWritable

public class GBTClassifier
extends ProbabilisticClassifier<Vector,GBTClassifier,GBTClassificationModel>
implements GBTClassifierParams, DefaultParamsWritable, Logging

Gradient-Boosted Trees (GBTs) (http://en.wikipedia.org/wiki/Gradient_boosting) learning algorithm for classification. It supports binary labels, as well as both continuous and categorical features.

The implementation is based upon: J.H. Friedman. "Stochastic Gradient Boosting." 1999.

Notes on Gradient Boosting vs. TreeBoost: - This implementation is for Stochastic Gradient Boosting, not for TreeBoost. - Both algorithms learn tree ensembles by minimizing loss functions. - TreeBoost (Friedman, 1999) additionally modifies the outputs at tree leaf nodes based on the loss function, whereas the original gradient boosting method does not. - We expect to implement TreeBoost in the future: [https://issues.apache.org/jira/browse/SPARK-4240]

See Also:

Serialized Form

Note:

Multiclass labels are not currently supported.

Constructor Summary

Constructors

Constructor and Description
GBTClassifier()
GBTClassifier(String uid)

Method Summary

Modifier and Type	Method and Description
GBTClassifier	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
static GBTClassifier	load(String path)
static MLReader<T>	read()
GBTClassifier	setCacheNodeIds(boolean value)
GBTClassifier	setCheckpointInterval(int value) Specifies how often to checkpoint the cached node IDs.
GBTClassifier	setFeatureSubsetStrategy(String value)
GBTClassifier	setImpurity(String value) The impurity setting is ignored for GBT models.
GBTClassifier	setLossType(String value)
GBTClassifier	setMaxBins(int value)
GBTClassifier	setMaxDepth(int value)
GBTClassifier	setMaxIter(int value)
GBTClassifier	setMaxMemoryInMB(int value)
GBTClassifier	setMinInfoGain(double value)
GBTClassifier	setMinInstancesPerNode(int value)
GBTClassifier	setSeed(long value)
GBTClassifier	setStepSize(double value)
GBTClassifier	setSubsamplingRate(double value)
GBTClassifier	setValidationIndicatorCol(String value)
static String[]	supportedLossTypes() Accessor for supported loss settings: logistic
String	uid() An immutable unique ID for the object and its derivatives.

Methods inherited from class org.apache.spark.ml.classification.ProbabilisticClassifier

setProbabilityCol, setThresholds

Methods inherited from class org.apache.spark.ml.classification.Classifier

setRawPredictionCol

Methods inherited from class org.apache.spark.ml.Predictor

fit, setFeaturesCol, setLabelCol, setPredictionCol, transformSchema

Methods inherited from class org.apache.spark.ml.Estimator

fit, fit, fit, fit

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.tree.GBTClassifierParams

getLossType, getOldLossType, lossType

Methods inherited from interface org.apache.spark.ml.tree.GBTParams

getOldBoostingStrategy, getValidationTol, stepSize, validationTol

Methods inherited from interface org.apache.spark.ml.tree.TreeEnsembleParams

featureSubsetStrategy, getFeatureSubsetStrategy, getOldStrategy, getSubsamplingRate, subsamplingRate

Methods inherited from interface org.apache.spark.ml.tree.DecisionTreeParams

cacheNodeIds, getCacheNodeIds, getMaxBins, getMaxDepth, getMaxMemoryInMB, getMinInfoGain, getMinInstancesPerNode, getOldStrategy, maxBins, maxDepth, maxMemoryInMB, minInfoGain, minInstancesPerNode

Methods inherited from interface org.apache.spark.ml.PredictorParams

validateAndTransformSchema

Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

getLabelCol, labelCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.param.shared.HasCheckpointInterval

checkpointInterval, getCheckpointInterval

Methods inherited from interface org.apache.spark.ml.param.shared.HasSeed

getSeed, seed

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter, maxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasStepSize

getStepSize

Methods inherited from interface org.apache.spark.ml.param.shared.HasValidationIndicatorCol

getValidationIndicatorCol, validationIndicatorCol

Methods inherited from interface org.apache.spark.ml.tree.TreeClassifierParams

getImpurity, getOldImpurity, impurity

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Methods inherited from interface org.apache.spark.ml.classification.ProbabilisticClassifierParams

validateAndTransformSchema

Methods inherited from interface org.apache.spark.ml.param.shared.HasRawPredictionCol

getRawPredictionCol, rawPredictionCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasProbabilityCol

getProbabilityCol, probabilityCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasThresholds

getThresholds, thresholds

Constructor Detail

GBTClassifier

public GBTClassifier(String uid)

GBTClassifier

public GBTClassifier()

Method Detail

supportedLossTypes

public static final String[] supportedLossTypes()

Accessor for supported loss settings: logistic

load

public static GBTClassifier load(String path)

read

public static MLReader<T> read()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

setMaxDepth

public GBTClassifier setMaxDepth(int value)

Specified by:

setMaxDepth in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxBins

public GBTClassifier setMaxBins(int value)

Specified by:

setMaxBins in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMinInstancesPerNode

public GBTClassifier setMinInstancesPerNode(int value)

Specified by:

setMinInstancesPerNode in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMinInfoGain

public GBTClassifier setMinInfoGain(double value)

Specified by:

setMinInfoGain in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxMemoryInMB

public GBTClassifier setMaxMemoryInMB(int value)

Specified by:

setMaxMemoryInMB in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setCacheNodeIds

public GBTClassifier setCacheNodeIds(boolean value)

Specified by:

setCacheNodeIds in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setCheckpointInterval

public GBTClassifier setCheckpointInterval(int value)

Specifies how often to checkpoint the cached node IDs. E.g. 10 means that the cache will get checkpointed every 10 iterations. This is only used if cacheNodeIds is true and if the checkpoint directory is set in SparkContext. Must be at least 1. (default = 10)

Specified by:

setCheckpointInterval in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setImpurity

public GBTClassifier setImpurity(String value)

The impurity setting is ignored for GBT models. Individual trees are built using impurity "Variance."

Specified by:

setImpurity in interface TreeClassifierParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setSubsamplingRate

public GBTClassifier setSubsamplingRate(double value)

Specified by:

setSubsamplingRate in interface TreeEnsembleParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setSeed

public GBTClassifier setSeed(long value)

Specified by:

setSeed in interface DecisionTreeParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxIter

public GBTClassifier setMaxIter(int value)

Specified by:

setMaxIter in interface GBTParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setStepSize

public GBTClassifier setStepSize(double value)

Specified by:

setStepSize in interface GBTParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setFeatureSubsetStrategy

public GBTClassifier setFeatureSubsetStrategy(String value)

Specified by:

setFeatureSubsetStrategy in interface TreeEnsembleParams

Parameters:

value - (undocumented)

Returns:

(undocumented)

setLossType

public GBTClassifier setLossType(String value)

setValidationIndicatorCol

public GBTClassifier setValidationIndicatorCol(String value)

copy

public GBTClassifier copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Predictor<Vector,GBTClassifier,GBTClassificationModel>

Parameters:

extra - (undocumented)

Returns:

(undocumented)