org.apache.spark

Class Accumulator<T>

Object

org.apache.spark.Accumulable<T,T>

org.apache.spark.Accumulator<T>

All Implemented Interfaces:

java.io.Serializable

Deprecated.

use AccumulatorV2. Since 2.0.0.

public class Accumulator<T>
extends Accumulable<T,T>

A simpler value of Accumulable where the result type being accumulated is the same as the types of elements being merged, i.e. variables that are only "added" to through an associative and commutative operation and can therefore be efficiently supported in parallel. They can be used to implement counters (as in MapReduce) or sums. Spark natively supports accumulators of numeric value types, and programmers can add support for new types.

An accumulator is created from an initial value v by calling SparkContext.accumulator. Tasks running on the cluster can then add to it using the += operator. However, they cannot read its value. Only the driver program can read the accumulator's value, using its value() method.

The interpreter session below shows an accumulator being used to add up the elements of an array:

 scala> val accum = sc.accumulator(0)
 accum: org.apache.spark.Accumulator[Int] = 0

 scala> sc.parallelize(Array(1, 2, 3, 4)).foreach(x => accum += x)
 ...
 10/09/29 18:41:08 INFO SparkContext: Tasks finished in 0.317106 s

 scala> accum.value
 res2: Int = 10
 

param: initialValue initial value of accumulator param: param helper object defining how to add elements of type T param: name human-readable name associated with this accumulator param: countFailedValues whether to accumulate values from failed tasks

See Also:

Serialized Form

Method Summary

Methods

Modifier and Type	Method and Description
static void	$plus$eq(T term) Deprecated.
static void	$plus$plus$eq(R term) Deprecated.
static void	add(T term) Deprecated.
static long	id() Deprecated.
static R	localValue() Deprecated.
static void	merge(R term) Deprecated.
static scala.Option<String>	name() Deprecated.
static void	setValue(R newValue) Deprecated.
static String	toString() Deprecated.
static void	value_$eq(R newValue) Deprecated.
static R	value() Deprecated.
static R	zero() Deprecated.

Methods inherited from class org.apache.spark.Accumulable

add, id, localValue, merge, name, setValue, toString, value, zero

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

id

public static long id()

Deprecated.

name

public static scala.Option<String> name()

Deprecated.

zero

public static R zero()

Deprecated.

$plus$eq

public static void $plus$eq(T term)

Deprecated.

add

public static void add(T term)

Deprecated.

$plus$plus$eq

public static void $plus$plus$eq(R term)

Deprecated.

merge

public static void merge(R term)

Deprecated.

value

public static R value()

Deprecated.

localValue

public static R localValue()

Deprecated.

value_$eq

public static void value_$eq(R newValue)

Deprecated.

setValue

public static void setValue(R newValue)

Deprecated.

toString

public static String toString()

Deprecated.