org.apache.spark.broadcast

Class HttpBroadcastFactory

Object

org.apache.spark.broadcast.HttpBroadcastFactory

All Implemented Interfaces:

BroadcastFactory

public class HttpBroadcastFactory
extends Object
implements BroadcastFactory

A BroadcastFactory implementation that uses a HTTP server as the broadcast mechanism. Refer to HttpBroadcast for more details about this mechanism.

Constructor Summary

Constructors

Constructor and Description
HttpBroadcastFactory()

Method Summary

Methods

Modifier and Type	Method and Description
void	initialize(boolean isDriver, SparkConf conf, SecurityManager securityMgr)
<T> HttpBroadcast<T>	newBroadcast(T value_, boolean isLocal, long id, scala.reflect.ClassTag<T> evidence$1) Creates a new broadcast variable.
void	stop()
void	unbroadcast(long id, boolean removeFromDriver, boolean blocking) Remove all persisted state associated with the HTTP broadcast with the given ID.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HttpBroadcastFactory

public HttpBroadcastFactory()

Method Detail

initialize

public void initialize(boolean isDriver,
              SparkConf conf,
              SecurityManager securityMgr)

Specified by:

initialize in interface BroadcastFactory

newBroadcast

public <T> HttpBroadcast<T> newBroadcast(T value_,
                                boolean isLocal,
                                long id,
                                scala.reflect.ClassTag<T> evidence$1)

Description copied from interface: BroadcastFactory

Creates a new broadcast variable.

Specified by:

newBroadcast in interface BroadcastFactory

Parameters:

value_ - value to broadcast

isLocal - whether we are in local mode (single JVM process)

id - unique id representing this broadcast variable

stop

public void stop()

Specified by:

stop in interface BroadcastFactory

unbroadcast

public void unbroadcast(long id,
               boolean removeFromDriver,
               boolean blocking)

Remove all persisted state associated with the HTTP broadcast with the given ID.

Specified by:

unbroadcast in interface BroadcastFactory

Parameters:

removeFromDriver - Whether to remove state from the driver

blocking - Whether to block until unbroadcasted