org.apache.spark.streaming.dstream

Class MapWithStateDStream<KeyType,ValueType,StateType,MappedType>

Object

org.apache.spark.streaming.dstream.DStream<MappedType>

org.apache.spark.streaming.dstream.MapWithStateDStream<KeyType,ValueType,StateType,MappedType>

All Implemented Interfaces:

java.io.Serializable

public abstract class MapWithStateDStream<KeyType,ValueType,StateType,MappedType>
extends DStream<MappedType>

:: Experimental :: DStream representing the stream of data generated by mapWithState operation on a pair DStream. Additionally, it also gives access to the stream of state snapshots, that is, the state data of all keys after a batch has updated them.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
MapWithStateDStream(StreamingContext ssc, scala.reflect.ClassTag<MappedType> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
abstract DStream<scala.Tuple2<KeyType,StateType>>	stateSnapshots() Return a pair DStream where each RDD is the snapshot of the state of all the keys.

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, compute, context, count, countByValue, countByValueAndWindow, countByWindow, dependencies, filter, flatMap, foreachRDD, foreachRDD, glom, map, mapPartitions, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, repartition, saveAsObjectFiles, saveAsTextFiles, slice, slice, slideDuration, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, window, window

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapWithStateDStream

public MapWithStateDStream(StreamingContext ssc,
                   scala.reflect.ClassTag<MappedType> evidence$1)

Method Detail

stateSnapshots

public abstract DStream<scala.Tuple2<KeyType,StateType>> stateSnapshots()

Return a pair DStream where each RDD is the snapshot of the state of all the keys.