org.apache.spark.internal.io

Class HadoopMapRedCommitProtocol

Object

org.apache.spark.internal.io.FileCommitProtocol

org.apache.spark.internal.io.HadoopMapReduceCommitProtocol

org.apache.spark.internal.io.HadoopMapRedCommitProtocol

All Implemented Interfaces:

java.io.Serializable, Logging

public class HadoopMapRedCommitProtocol
extends HadoopMapReduceCommitProtocol

An FileCommitProtocol implementation backed by an underlying Hadoop OutputCommitter (from the old mapred API).

Unlike Hadoop's OutputCommitter, this implementation is serializable.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.spark.internal.io.FileCommitProtocol

FileCommitProtocol.EmptyTaskCommitMessage$, FileCommitProtocol.TaskCommitMessage

Constructor Summary

Constructors

Constructor and Description
HadoopMapRedCommitProtocol(String jobId, String path)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapred.OutputCommitter	setupCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class org.apache.spark.internal.io.HadoopMapReduceCommitProtocol

abortJob, abortTask, commitJob, commitTask, deleteWithJob, newTaskTempFile, newTaskTempFileAbsPath, onTaskCommit, setupJob, setupTask

Methods inherited from class org.apache.spark.internal.io.FileCommitProtocol

deleteWithJob, instantiate, onTaskCommit

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

HadoopMapRedCommitProtocol

public HadoopMapRedCommitProtocol(String jobId,
                                  String path)

Method Detail

setupCommitter

public org.apache.hadoop.mapred.OutputCommitter setupCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)