org.apache.spark.sql.parquet

Class CatalystArrayConverter

Object

parquet.io.api.Converter

parquet.io.api.GroupConverter

org.apache.spark.sql.parquet.CatalystConverter

org.apache.spark.sql.parquet.CatalystArrayConverter

public class CatalystArrayConverter
extends CatalystConverter

A parquet.io.api.GroupConverter that converts a single-element groups that match the characteristics of an array (see ParquetTypesConverter) into an ArrayType.

Constructor Summary

Constructors

Constructor and Description
CatalystArrayConverter(org.apache.spark.sql.types.DataType elementType, int index, CatalystConverter parent)
CatalystArrayConverter(org.apache.spark.sql.types.DataType elementType, int index, CatalystConverter parent, scala.collection.mutable.Buffer<Object> buffer)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.spark.sql.types.DataType	elementType()
void	end()
parquet.io.api.Converter	getConverter(int fieldIndex)
int	index() The index of this converter in the parent
static int	INITIAL_ARRAY_SIZE()
int	size() The number of fields this group has
void	start()

Methods inherited from class org.apache.spark.sql.parquet.CatalystConverter

ARRAY_CONTAINS_NULL_BAG_SCHEMA_NAME, ARRAY_ELEMENTS_SCHEMA_NAME, getCurrentRecord, MAP_KEY_SCHEMA_NAME, MAP_SCHEMA_NAME, MAP_VALUE_SCHEMA_NAME, THRIFT_ARRAY_ELEMENTS_SCHEMA_NAME_SUFFIX

Methods inherited from class parquet.io.api.GroupConverter

asGroupConverter, isPrimitive

Methods inherited from class parquet.io.api.Converter

asPrimitiveConverter

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CatalystArrayConverter

public CatalystArrayConverter(org.apache.spark.sql.types.DataType elementType,
                      int index,
                      CatalystConverter parent,
                      scala.collection.mutable.Buffer<Object> buffer)

CatalystArrayConverter

public CatalystArrayConverter(org.apache.spark.sql.types.DataType elementType,
                      int index,
                      CatalystConverter parent)

Method Detail

INITIAL_ARRAY_SIZE

public static int INITIAL_ARRAY_SIZE()

elementType

public org.apache.spark.sql.types.DataType elementType()

index

public int index()

Description copied from class: CatalystConverter

The index of this converter in the parent

getConverter

public parquet.io.api.Converter getConverter(int fieldIndex)

Specified by:

getConverter in class parquet.io.api.GroupConverter

size

public int size()

Description copied from class: CatalystConverter

The number of fields this group has

start

public void start()

Specified by:

start in class parquet.io.api.GroupConverter

end

public void end()

Specified by:

end in class parquet.io.api.GroupConverter