org.apache.spark.graphx

Class EdgeRDD<ED>

Object

org.apache.spark.rdd.RDD<Edge<ED>>

org.apache.spark.graphx.EdgeRDD<ED>

All Implemented Interfaces:

java.io.Serializable, Logging

Direct Known Subclasses:

EdgeRDDImpl

public abstract class EdgeRDD<ED>
extends RDD<Edge<ED>>

EdgeRDD[ED, VD] extends RDD[Edge[ED} by storing the edges in columnar format on each partition for performance. It may additionally store the vertex attributes associated with each edge to provide the triplet view. Shipping of the vertex attributes is managed by impl.ReplicatedVertexView.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
EdgeRDD(SparkContext sc, scala.collection.Seq<Dependency<?>> deps)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Iterator<Edge<ED>>	compute(Partition part, TaskContext context) :: DeveloperApi :: Implemented by subclasses to compute a given partition.
static <ED,VD> EdgeRDDImpl<ED,VD>	fromEdgePartitions(RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> edgePartitions, scala.reflect.ClassTag<ED> evidence$6, scala.reflect.ClassTag<VD> evidence$7) Creates an EdgeRDD from already-constructed edge partitions.
static <ED,VD> EdgeRDDImpl<ED,VD>	fromEdges(RDD<Edge<ED>> edges, scala.reflect.ClassTag<ED> evidence$4, scala.reflect.ClassTag<VD> evidence$5) Creates an EdgeRDD from a set of edges.
abstract <ED2,ED3> EdgeRDD<ED3>	innerJoin(EdgeRDD<ED2> other, scala.Function4<Object,Object,ED,ED2,ED3> f, scala.reflect.ClassTag<ED2> evidence$2, scala.reflect.ClassTag<ED3> evidence$3) Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.
abstract <ED2> EdgeRDD<ED2>	mapValues(scala.Function1<Edge<ED>,ED2> f, scala.reflect.ClassTag<ED2> evidence$1) Map the values in an edge partitioning preserving the structure but changing the values.
abstract RDD<scala.Tuple2<Object,EdgePartition<ED,Object>>>	partitionsRDD()
abstract EdgeRDD<ED>	reverse() Reverse all the edges in this RDD.
abstract EdgeRDD<ED>	withTargetStorageLevel(StorageLevel targetStorageLevel) Changes the target storage level while preserving all other properties of the EdgeRDD.

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, cache, cartesian, checkpoint, checkpointData, coalesce, collect, collect, collectPartitions, computeOrReadCheckpoint, conf, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doCheckpoint, elementClassTag, filter, filterWith, first, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCheckpointFile, getCreationSite, getNarrowAncestors, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, iterator, keyBy, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, markCheckpointed, max, min, name, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, reduce, repartition, retag, retag, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, union, unpersist, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

EdgeRDD

public EdgeRDD(SparkContext sc,
       scala.collection.Seq<Dependency<?>> deps)

Method Detail

fromEdges

public static <ED,VD> EdgeRDDImpl<ED,VD> fromEdges(RDD<Edge<ED>> edges,
                                   scala.reflect.ClassTag<ED> evidence$4,
                                   scala.reflect.ClassTag<VD> evidence$5)

Creates an EdgeRDD from a set of edges.

fromEdgePartitions

public static <ED,VD> EdgeRDDImpl<ED,VD> fromEdgePartitions(RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> edgePartitions,
                                            scala.reflect.ClassTag<ED> evidence$6,
                                            scala.reflect.ClassTag<VD> evidence$7)

Creates an EdgeRDD from already-constructed edge partitions.

partitionsRDD

public abstract RDD<scala.Tuple2<Object,EdgePartition<ED,Object>>> partitionsRDD()

compute

public scala.collection.Iterator<Edge<ED>> compute(Partition part,
                                          TaskContext context)

Description copied from class: RDD

:: DeveloperApi :: Implemented by subclasses to compute a given partition.

Specified by:

compute in class RDD<Edge<ED>>

mapValues

public abstract <ED2> EdgeRDD<ED2> mapValues(scala.Function1<Edge<ED>,ED2> f,
                           scala.reflect.ClassTag<ED2> evidence$1)

Map the values in an edge partitioning preserving the structure but changing the values.

Parameters:

f - the function from an edge to a new edge value

Returns:

a new EdgeRDD containing the new edge values

reverse

public abstract EdgeRDD<ED> reverse()

Reverse all the edges in this RDD.

Returns:

a new EdgeRDD containing all the edges reversed

innerJoin

public abstract <ED2,ED3> EdgeRDD<ED3> innerJoin(EdgeRDD<ED2> other,
                               scala.Function4<Object,Object,ED,ED2,ED3> f,
                               scala.reflect.ClassTag<ED2> evidence$2,
                               scala.reflect.ClassTag<ED3> evidence$3)

Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.

Parameters:

other - the EdgeRDD to join with

f - the join function applied to corresponding values of this and other

Returns:

a new EdgeRDD containing only edges that appear in both this and other, with values supplied by f

withTargetStorageLevel

public abstract EdgeRDD<ED> withTargetStorageLevel(StorageLevel targetStorageLevel)

Changes the target storage level while preserving all other properties of the EdgeRDD. Operations on the returned EdgeRDD will preserve this storage level.

This does not actually trigger a cache; to do this, call RDD.cache() on the returned EdgeRDD.