org.apache.spark.streaming.dstream

Class ForEachDStream<T>

Object

org.apache.spark.streaming.dstream.DStream<scala.runtime.BoxedUnit>

org.apache.spark.streaming.dstream.ForEachDStream<T>

All Implemented Interfaces:

java.io.Serializable, Logging

public class ForEachDStream<T>
extends DStream<scala.runtime.BoxedUnit>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ForEachDStream(DStream<T> parent, scala.Function2<RDD<T>,Time,scala.runtime.BoxedUnit> foreachFunc, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
scala.Option<RDD<scala.runtime.BoxedUnit>>	compute(Time validTime) Method that generates a RDD for the given time
scala.collection.immutable.List<DStream<T>>	dependencies()
scala.Option<Job>	generateJob(Time time) Generate a SparkStreaming job for the given time.
Duration	slideDuration() Time interval after which the DStream generates a RDD

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, checkpointData, checkpointDuration, clearCheckpointData, clearMetadata, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, getCreationSite, getOrCompute, glom, graph, initialize, isInitialized, isTimeValid, map, mapPartitions, mustCheckpoint, parentRememberDuration, persist, persist, print, reduce, reduceByWindow, reduceByWindow, register, remember, rememberDuration, repartition, restoreCheckpointData, saveAsObjectFiles, saveAsTextFiles, setContext, setGraph, slice, slice, ssc, storageLevel, transform, transform, transformWith, transformWith, union, updateCheckpointData, validate, window, window, zeroTime

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

ForEachDStream

public ForEachDStream(DStream<T> parent,
              scala.Function2<RDD<T>,Time,scala.runtime.BoxedUnit> foreachFunc,
              scala.reflect.ClassTag<T> evidence$1)

Method Detail

dependencies

public scala.collection.immutable.List<DStream<T>> dependencies()

slideDuration

public Duration slideDuration()

Description copied from class: DStream

Time interval after which the DStream generates a RDD

Specified by:

slideDuration in class DStream<scala.runtime.BoxedUnit>

compute

public scala.Option<RDD<scala.runtime.BoxedUnit>> compute(Time validTime)

Description copied from class: DStream

Method that generates a RDD for the given time

Specified by:

compute in class DStream<scala.runtime.BoxedUnit>

generateJob

public scala.Option<Job> generateJob(Time time)

Description copied from class: DStream

Generate a SparkStreaming job for the given time. This is an internal method that should not be called directly. This default implementation creates a job that materializes the corresponding RDD. Subclasses of DStream may override this to generate their own jobs.

Overrides:

generateJob in class DStream<scala.runtime.BoxedUnit>