org.apache.spark.sql.sources.v2

Interface StreamWriteSupport

All Superinterfaces:

org.apache.spark.sql.execution.streaming.BaseStreamingSink, DataSourceV2

@InterfaceStability.Evolving
public interface StreamWriteSupport
extends DataSourceV2, org.apache.spark.sql.execution.streaming.BaseStreamingSink

A mix-in interface for DataSourceV2. Data sources can implement this interface to provide data writing ability for structured streaming.

Method Summary

Modifier and Type	Method and Description
StreamWriter	createStreamWriter(String queryId, StructType schema, OutputMode mode, DataSourceOptions options) Creates an optional StreamWriter to save the data to this data source.

Method Detail

createStreamWriter

StreamWriter createStreamWriter(String queryId,
                                StructType schema,
                                OutputMode mode,
                                DataSourceOptions options)

Creates an optional StreamWriter to save the data to this data source. Data sources can return None if there is no writing needed to be done.

Parameters:

queryId - A unique string for the writing query. It's possible that there are many writing queries running at the same time, and the returned DataSourceWriter can use this id to distinguish itself from others.

schema - the schema of the data to be written.

mode - the output mode which determines what successive epoch output means to this sink, please refer to OutputMode for more details.

options - the options for the returned data source writer, which is an immutable case-insensitive string-to-string map.