org.apache.spark.ml.regression

Class GeneralizedLinearRegression

Object

org.apache.spark.ml.PipelineStage

org.apache.spark.ml.Estimator<M>

org.apache.spark.ml.Predictor<FeaturesType,Learner,M>

org.apache.spark.ml.regression.GeneralizedLinearRegression

All Implemented Interfaces:

java.io.Serializable, Logging, Params, HasFeaturesCol, HasFitIntercept, HasLabelCol, HasMaxIter, HasPredictionCol, HasRegParam, HasSolver, HasTol, HasWeightCol, PredictorParams, GeneralizedLinearRegressionBase, DefaultParamsWritable, Identifiable, MLWritable

public class GeneralizedLinearRegression
extends Predictor<FeaturesType,Learner,M>
implements GeneralizedLinearRegressionBase, DefaultParamsWritable, Logging

:: Experimental ::

Fit a Generalized Linear Model (see Generalized linear model (Wikipedia)) specified by giving a symbolic description of the linear predictor (link function) and a description of the error distribution (family). It supports "gaussian", "binomial", "poisson", "gamma" and "tweedie" as family. Valid link functions for each family is listed below. The first link function of each family is the default one. - "gaussian" : "identity", "log", "inverse" - "binomial" : "logit", "probit", "cloglog" - "poisson" : "log", "identity", "sqrt" - "gamma" : "inverse", "identity", "log" - "tweedie" : power link function specified through "linkPower". The default link power in the tweedie family is 1 - variancePower.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GeneralizedLinearRegression.Binomial$ Binomial exponential family distribution.
static class	GeneralizedLinearRegression.CLogLog$
static class	GeneralizedLinearRegression.Family$
static class	GeneralizedLinearRegression.FamilyAndLink$
static class	GeneralizedLinearRegression.Gamma$ Gamma exponential family distribution.
static class	GeneralizedLinearRegression.Gaussian$ Gaussian exponential family distribution.
static class	GeneralizedLinearRegression.Identity$
static class	GeneralizedLinearRegression.Inverse$
static class	GeneralizedLinearRegression.Link$
static class	GeneralizedLinearRegression.Log$
static class	GeneralizedLinearRegression.Logit$
static class	GeneralizedLinearRegression.Poisson$ Poisson exponential family distribution.
static class	GeneralizedLinearRegression.Probit$
static class	GeneralizedLinearRegression.Sqrt$
static class	GeneralizedLinearRegression.Tweedie$

Constructor Summary

Constructors

Constructor and Description
GeneralizedLinearRegression()
GeneralizedLinearRegression(String uid)

Method Summary

Modifier and Type	Method and Description
GeneralizedLinearRegression	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
static GeneralizedLinearRegression	load(String path)
static MLReader<T>	read()
GeneralizedLinearRegression	setFamily(String value) Sets the value of param family.
GeneralizedLinearRegression	setFitIntercept(boolean value) Sets if we should fit the intercept.
GeneralizedLinearRegression	setLink(String value) Sets the value of param link.
GeneralizedLinearRegression	setLinkPower(double value) Sets the value of param linkPower.
GeneralizedLinearRegression	setLinkPredictionCol(String value) Sets the link prediction (linear predictor) column name.
GeneralizedLinearRegression	setMaxIter(int value) Sets the maximum number of iterations (applicable for solver "irls").
GeneralizedLinearRegression	setOffsetCol(String value) Sets the value of param offsetCol.
GeneralizedLinearRegression	setRegParam(double value) Sets the regularization parameter for L2 regularization.
GeneralizedLinearRegression	setSolver(String value) Sets the solver algorithm used for optimization.
GeneralizedLinearRegression	setTol(double value) Sets the convergence tolerance of iterations.
GeneralizedLinearRegression	setVariancePower(double value) Sets the value of param variancePower.
GeneralizedLinearRegression	setWeightCol(String value) Sets the value of param weightCol.
String	uid() An immutable unique ID for the object and its derivatives.

Methods inherited from class org.apache.spark.ml.Predictor

fit, setFeaturesCol, setLabelCol, setPredictionCol, transformSchema

Methods inherited from class org.apache.spark.ml.Estimator

fit, fit, fit, fit

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.regression.GeneralizedLinearRegressionBase

family, getFamily, getLink, getLinkPower, getLinkPredictionCol, getOffsetCol, getVariancePower, hasLinkPredictionCol, hasOffsetCol, hasWeightCol, link, linkPower, linkPredictionCol, offsetCol, solver, validateAndTransformSchema, variancePower

Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

getLabelCol, labelCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

featuresCol, getFeaturesCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasPredictionCol

getPredictionCol, predictionCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.param.shared.HasFitIntercept

fitIntercept, getFitIntercept

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter, maxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasTol

getTol, tol

Methods inherited from interface org.apache.spark.ml.param.shared.HasRegParam

getRegParam, regParam

Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol

getWeightCol, weightCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasSolver

getSolver

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Constructor Detail

GeneralizedLinearRegression

public GeneralizedLinearRegression(String uid)

GeneralizedLinearRegression

public GeneralizedLinearRegression()

Method Detail

load

public static GeneralizedLinearRegression load(String path)

read

public static MLReader<T> read()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

setFamily

public GeneralizedLinearRegression setFamily(String value)

Sets the value of param family. Default is "gaussian".

Parameters:

value - (undocumented)

Returns:

(undocumented)

setVariancePower

public GeneralizedLinearRegression setVariancePower(double value)

Sets the value of param variancePower. Used only when family is "tweedie". Default is 0.0, which corresponds to the "gaussian" family.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setLinkPower

public GeneralizedLinearRegression setLinkPower(double value)

Sets the value of param linkPower. Used only when family is "tweedie".

Parameters:

value - (undocumented)

Returns:

(undocumented)

setLink

public GeneralizedLinearRegression setLink(String value)

Sets the value of param link. Used only when family is not "tweedie".

Parameters:

value - (undocumented)

Returns:

(undocumented)

setFitIntercept

public GeneralizedLinearRegression setFitIntercept(boolean value)

Sets if we should fit the intercept. Default is true.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setMaxIter

public GeneralizedLinearRegression setMaxIter(int value)

Sets the maximum number of iterations (applicable for solver "irls"). Default is 25.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setTol

public GeneralizedLinearRegression setTol(double value)

Sets the convergence tolerance of iterations. Smaller value will lead to higher accuracy with the cost of more iterations. Default is 1E-6.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setRegParam

public GeneralizedLinearRegression setRegParam(double value)

Sets the regularization parameter for L2 regularization. The regularization term is

$$ 0.5 * regParam * L2norm(coefficients)^2 $$

Default is 0.0.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setWeightCol

public GeneralizedLinearRegression setWeightCol(String value)

Sets the value of param weightCol. If this is not set or empty, we treat all instance weights as 1.0. Default is not set, so all instances have weight one. In the Binomial family, weights correspond to number of trials and should be integer. Non-integer weights are rounded to integer in AIC calculation.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setOffsetCol

public GeneralizedLinearRegression setOffsetCol(String value)

Sets the value of param offsetCol. If this is not set or empty, we treat all instance offsets as 0.0. Default is not set, so all instances have offset 0.0.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setSolver

public GeneralizedLinearRegression setSolver(String value)

Sets the solver algorithm used for optimization. Currently only supports "irls" which is also the default solver.

Parameters:

value - (undocumented)

Returns:

(undocumented)

setLinkPredictionCol

public GeneralizedLinearRegression setLinkPredictionCol(String value)

Sets the link prediction (linear predictor) column name.

Parameters:

value - (undocumented)

Returns:

(undocumented)

copy

public GeneralizedLinearRegression copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Specified by:

copy in class Predictor<Vector,GeneralizedLinearRegression,GeneralizedLinearRegressionModel>

Parameters:

extra - (undocumented)

Returns:

(undocumented)