org.apache.spark.sql

Interface Row

All Superinterfaces:

java.io.Serializable

All Known Implementing Classes:

MutableAggregationBuffer

public interface Row
extends scala.Serializable

Represents one row of output from a relational operator. Allows both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access.

It is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null.

To create a new Row, use RowFactory.create() in Java or Row.apply() in Scala.

A Row object can be constructed by providing field values. Example:

 import org.apache.spark.sql._

 // Create a Row from values.
 Row(value1, value2, value3, ...)
 // Create a Row from a Seq of values.
 Row.fromSeq(Seq(value1, value2, ...))
 

A value of a row can be accessed through both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access. An example of generic access by ordinal:

 import org.apache.spark.sql._

 val row = Row(1, true, "a string", null)
 // row: Row = [1,true,a string,null]
 val firstValue = row(0)
 // firstValue: Any = 1
 val fourthValue = row(3)
 // fourthValue: Any = null
 

For native primitive access, it is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null. An example of native primitive access:

 // using the row from the previous example.
 val firstValue = row.getInt(0)
 // firstValue: Int = 1
 val isNull = row.isNullAt(3)
 // isNull: Boolean = true
 

In Scala, fields in a Row object can be extracted in a pattern match. Example:

 import org.apache.spark.sql._

 val pairs = sql("SELECT key, value FROM src").rdd.map {
   case Row(key: Int, value: String) =>
     key -> value
 }
 

Method Summary

Methods

Modifier and Type	Method and Description
boolean	anyNull() Returns true if there are any NULL values in this row.
java.lang.Object	apply(int i) Returns the value at position i.
Row	copy() Make a copy of the current Row object.
boolean	equals(java.lang.Object o)
int	fieldIndex(java.lang.String name) Returns the index of a given field name.
java.lang.Object	get(int i) Returns the value at position i.
<T> T	getAnyValAs(int i) Returns the value of a given fieldName.
<T> T	getAs(int i) Returns the value at position i.
<T> T	getAs(java.lang.String fieldName) Returns the value of a given fieldName.
boolean	getBoolean(int i) Returns the value at position i as a primitive boolean.
byte	getByte(int i) Returns the value at position i as a primitive byte.
java.sql.Date	getDate(int i) Returns the value at position i of date type as java.sql.Date.
java.math.BigDecimal	getDecimal(int i) Returns the value at position i of decimal type as java.math.BigDecimal.
double	getDouble(int i) Returns the value at position i as a primitive double.
float	getFloat(int i) Returns the value at position i as a primitive float.
int	getInt(int i) Returns the value at position i as a primitive int.
<K,V> java.util.Map<K,V>	getJavaMap(int i) Returns the value at position i of array type as a Map.
<T> java.util.List<T>	getList(int i) Returns the value at position i of array type as List.
long	getLong(int i) Returns the value at position i as a primitive long.
<K,V> scala.collection.Map<K,V>	getMap(int i) Returns the value at position i of map type as a Scala Map.
<T> scala.collection.Seq<T>	getSeq(int i) Returns the value at position i of array type as a Scala Seq.
short	getShort(int i) Returns the value at position i as a primitive short.
java.lang.String	getString(int i) Returns the value at position i as a String object.
Row	getStruct(int i) Returns the value at position i of struct type as an Row object.
java.sql.Timestamp	getTimestamp(int i) Returns the value at position i of date type as java.sql.Timestamp.
<T> scala.collection.immutable.Map<java.lang.String,T>	getValuesMap(scala.collection.Seq<java.lang.String> fieldNames) Returns a Map(name -> value) for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive ie.
int	hashCode()
boolean	isNullAt(int i) Checks whether the value at position i is null.
int	length() Number of elements in the Row.
java.lang.String	mkString() Displays all elements of this sequence in a string (without a separator).
java.lang.String	mkString(java.lang.String sep) Displays all elements of this sequence in a string using a separator string.
java.lang.String	mkString(java.lang.String start, java.lang.String sep, java.lang.String end) Displays all elements of this traversable or iterator in a string using start, end, and separator strings.
StructType	schema() Schema for the row.
int	size() Number of elements in the Row.
scala.collection.Seq<java.lang.Object>	toSeq() Return a Scala Seq representing the row.
java.lang.String	toString()

Method Detail

size

int size()

Number of elements in the Row.

length

int length()

Number of elements in the Row.

schema

StructType schema()

Schema for the row.

Returns:

(undocumented)

apply

java.lang.Object apply(int i)

Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types:

   BooleanType -> java.lang.Boolean
   ByteType -> java.lang.Byte
   ShortType -> java.lang.Short
   IntegerType -> java.lang.Integer
   FloatType -> java.lang.Float
   DoubleType -> java.lang.Double
   StringType -> String
   DecimalType -> java.math.BigDecimal

   DateType -> java.sql.Date
   TimestampType -> java.sql.Timestamp

   BinaryType -> byte array
   ArrayType -> scala.collection.Seq (use getList for java.util.List)
   MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
   StructType -> org.apache.spark.sql.Row
 

Parameters:

i - (undocumented)

Returns:

(undocumented)

get

java.lang.Object get(int i)

Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types:

   BooleanType -> java.lang.Boolean
   ByteType -> java.lang.Byte
   ShortType -> java.lang.Short
   IntegerType -> java.lang.Integer
   FloatType -> java.lang.Float
   DoubleType -> java.lang.Double
   StringType -> String
   DecimalType -> java.math.BigDecimal

   DateType -> java.sql.Date
   TimestampType -> java.sql.Timestamp

   BinaryType -> byte array
   ArrayType -> scala.collection.Seq (use getList for java.util.List)
   MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
   StructType -> org.apache.spark.sql.Row
 

Parameters:

i - (undocumented)

Returns:

(undocumented)

isNullAt

boolean isNullAt(int i)

Checks whether the value at position i is null.

getBoolean

boolean getBoolean(int i)

Returns the value at position i as a primitive boolean.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getByte

byte getByte(int i)

Returns the value at position i as a primitive byte.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getShort

short getShort(int i)

Returns the value at position i as a primitive short.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getInt

int getInt(int i)

Returns the value at position i as a primitive int.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getLong

long getLong(int i)

Returns the value at position i as a primitive long.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getFloat

float getFloat(int i)

Returns the value at position i as a primitive float. Throws an exception if the type mismatches or if the value is null.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getDouble

double getDouble(int i)

Returns the value at position i as a primitive double.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.

getString

java.lang.String getString(int i)

Returns the value at position i as a String object.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getDecimal

java.math.BigDecimal getDecimal(int i)

Returns the value at position i of decimal type as java.math.BigDecimal.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getDate

java.sql.Date getDate(int i)

Returns the value at position i of date type as java.sql.Date.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getTimestamp

java.sql.Timestamp getTimestamp(int i)

Returns the value at position i of date type as java.sql.Timestamp.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getSeq

<T> scala.collection.Seq<T> getSeq(int i)

Returns the value at position i of array type as a Scala Seq.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getList

<T> java.util.List<T> getList(int i)

Returns the value at position i of array type as List.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getMap

<K,V> scala.collection.Map<K,V> getMap(int i)

Returns the value at position i of map type as a Scala Map.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getJavaMap

<K,V> java.util.Map<K,V> getJavaMap(int i)

Returns the value at position i of array type as a Map.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getStruct

Row getStruct(int i)

Returns the value at position i of struct type as an Row object.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getAs

<T> T getAs(int i)

Returns the value at position i. For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.ClassCastException - when data type does not match.

getAs

<T> T getAs(java.lang.String fieldName)

Returns the value of a given fieldName. For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

Parameters:

fieldName - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.UnsupportedOperationException - when schema is not defined.

java.lang.IllegalArgumentException - when fieldName do not exist.

java.lang.ClassCastException - when data type does not match.

fieldIndex

int fieldIndex(java.lang.String name)

Returns the index of a given field name.

Parameters:

name - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.UnsupportedOperationException - when schema is not defined.

java.lang.IllegalArgumentException - when a field name does not exist.

getValuesMap

<T> scala.collection.immutable.Map<java.lang.String,T> getValuesMap(scala.collection.Seq<java.lang.String> fieldNames)

Returns a Map(name -> value) for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

Parameters:

fieldNames - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.UnsupportedOperationException - when schema is not defined.

java.lang.IllegalArgumentException - when fieldName do not exist.

java.lang.ClassCastException - when data type does not match.

toString

java.lang.String toString()

Overrides:

toString in class java.lang.Object

copy

Row copy()

Make a copy of the current Row object.

Returns:

(undocumented)

anyNull

boolean anyNull()

Returns true if there are any NULL values in this row.

equals

boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

int hashCode()

Overrides:

hashCode in class java.lang.Object

toSeq

scala.collection.Seq<java.lang.Object> toSeq()

Return a Scala Seq representing the row. Elements are placed in the same order in the Seq.

Returns:

(undocumented)

mkString

java.lang.String mkString()

Displays all elements of this sequence in a string (without a separator).

mkString

java.lang.String mkString(java.lang.String sep)

Displays all elements of this sequence in a string using a separator string.

mkString

java.lang.String mkString(java.lang.String start,
                        java.lang.String sep,
                        java.lang.String end)

Displays all elements of this traversable or iterator in a string using start, end, and separator strings.

Parameters:

start - (undocumented)

sep - (undocumented)

end - (undocumented)

Returns:

(undocumented)

getAnyValAs

<T> T getAnyValAs(int i)

Returns the value of a given fieldName.

Parameters:

i - (undocumented)

Returns:

(undocumented)

Throws:

java.lang.UnsupportedOperationException - when schema is not defined.

java.lang.ClassCastException - when data type does not match.

java.lang.NullPointerException - when value is null.