org.apache.spark.sql.sources.v2

Interface ContinuousReadSupport

All Superinterfaces:

DataSourceV2

@InterfaceStability.Evolving
public interface ContinuousReadSupport
extends DataSourceV2

A mix-in interface for DataSourceV2. Data sources can implement this interface to provide data reading ability for continuous stream processing.

Method Summary

Modifier and Type	Method and Description
ContinuousReader	createContinuousReader(java.util.Optional<StructType> schema, String checkpointLocation, DataSourceOptions options) Creates a ContinuousReader to scan the data from this data source.

Method Detail

createContinuousReader

ContinuousReader createContinuousReader(java.util.Optional<StructType> schema,
                                        String checkpointLocation,
                                        DataSourceOptions options)

Creates a ContinuousReader to scan the data from this data source.

Parameters:

schema - the user provided schema, or empty() if none was provided

checkpointLocation - a path to Hadoop FS scratch space that can be used for failure recovery. Readers for the same logical source in the same query will be given the same checkpointLocation.

options - the options for the returned data source reader, which is an immutable case-insensitive string-to-string map.