org.apache.spark.sql.hive.execution

Class HiveFileFormat

Object

org.apache.spark.sql.hive.execution.HiveFileFormat

All Implemented Interfaces:

Logging, org.apache.spark.sql.execution.datasources.FileFormat, DataSourceRegister

public class HiveFileFormat
extends Object
implements org.apache.spark.sql.execution.datasources.FileFormat, DataSourceRegister, Logging

FileFormat for writing Hive tables.

TODO: implement the read logic.

Constructor Summary

Constructors

Constructor and Description
HiveFileFormat()
HiveFileFormat(org.apache.spark.sql.hive.HiveShim.ShimFileSinkDesc fileSinkConf)

Method Summary

Modifier and Type	Method and Description
scala.Option<StructType>	inferSchema(SparkSession sparkSession, scala.collection.immutable.Map<String,String> options, scala.collection.Seq<org.apache.hadoop.fs.FileStatus> files)
org.apache.spark.sql.execution.datasources.OutputWriterFactory	prepareWrite(SparkSession sparkSession, org.apache.hadoop.mapreduce.Job job, scala.collection.immutable.Map<String,String> options, StructType dataSchema)
String	shortName() The string that represents the format that this data source provider uses.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.execution.datasources.FileFormat

buildReader, buildReaderWithPartitionValues, isSplitable, supportBatch

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

HiveFileFormat

public HiveFileFormat(org.apache.spark.sql.hive.HiveShim.ShimFileSinkDesc fileSinkConf)

HiveFileFormat

public HiveFileFormat()

Method Detail

shortName

public String shortName()

Description copied from interface: DataSourceRegister

The string that represents the format that this data source provider uses. This is overridden by children to provide a nice alias for the data source. For example:

   override def shortName(): String = "parquet"
 

Specified by:

shortName in interface DataSourceRegister

Returns:

(undocumented)

inferSchema

public scala.Option<StructType> inferSchema(SparkSession sparkSession,
                                            scala.collection.immutable.Map<String,String> options,
                                            scala.collection.Seq<org.apache.hadoop.fs.FileStatus> files)

Specified by:

inferSchema in interface org.apache.spark.sql.execution.datasources.FileFormat

prepareWrite

public org.apache.spark.sql.execution.datasources.OutputWriterFactory prepareWrite(SparkSession sparkSession,
                                                                                   org.apache.hadoop.mapreduce.Job job,
                                                                                   scala.collection.immutable.Map<String,String> options,
                                                                                   StructType dataSchema)

Specified by:

prepareWrite in interface org.apache.spark.sql.execution.datasources.FileFormat