org.apache.spark.mllib.rdd

Class MLPairRDDFunctions<K,V>

java.lang.Object

org.apache.spark.mllib.rdd.MLPairRDDFunctions<K,V>

All Implemented Interfaces:

java.io.Serializable

public class MLPairRDDFunctions<K,V>
extends java.lang.Object
implements scala.Serializable

Machine learning specific Pair RDD functions.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
MLPairRDDFunctions(RDD<scala.Tuple2<K,V>> self, scala.reflect.ClassTag<K> evidence$1, scala.reflect.ClassTag<V> evidence$2)

Method Summary

Methods

Modifier and Type	Method and Description
static <K,V> MLPairRDDFunctions<K,V>	fromPairRDD(RDD<scala.Tuple2<K,V>> rdd, scala.reflect.ClassTag<K> evidence$3, scala.reflect.ClassTag<V> evidence$4) Implicit conversion from a pair RDD to MLPairRDDFunctions.
RDD<scala.Tuple2<K,java.lang.Object>>	topByKey(int num, scala.math.Ordering<V> ord) Returns the top k (largest) elements for each key from this RDD as defined by the specified implicit Ordering[T].

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MLPairRDDFunctions

public MLPairRDDFunctions(RDD<scala.Tuple2<K,V>> self,
                  scala.reflect.ClassTag<K> evidence$1,
                  scala.reflect.ClassTag<V> evidence$2)

Method Detail

fromPairRDD

public static <K,V> MLPairRDDFunctions<K,V> fromPairRDD(RDD<scala.Tuple2<K,V>> rdd,
                                        scala.reflect.ClassTag<K> evidence$3,
                                        scala.reflect.ClassTag<V> evidence$4)

Implicit conversion from a pair RDD to MLPairRDDFunctions.

topByKey

public RDD<scala.Tuple2<K,java.lang.Object>> topByKey(int num,
                                             scala.math.Ordering<V> ord)

Returns the top k (largest) elements for each key from this RDD as defined by the specified implicit Ordering[T]. If the number of elements for a certain key is less than k, all of them will be returned.

Parameters:

num - k, the number of top elements to return

ord - the implicit ordering for T

Returns:

an RDD that contains the top k values for each key