Package

org.apache.spark

io

Permalink

package io

IO codecs used for compression. See org.apache.spark.io.CompressionCodec.

Source
package.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. io
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. trait CompressionCodec extends AnyRef

    Permalink

    :: DeveloperApi :: CompressionCodec allows the customization of choosing different compression implementations to be used in block storage.

    :: DeveloperApi :: CompressionCodec allows the customization of choosing different compression implementations to be used in block storage.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for a codec is not guaranteed compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  2. class LZ4CompressionCodec extends CompressionCodec

    Permalink

    :: DeveloperApi :: LZ4 implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: LZ4 implementation of org.apache.spark.io.CompressionCodec. Block size can be configured by spark.io.compression.lz4.blockSize.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  3. class LZFCompressionCodec extends CompressionCodec

    Permalink

    :: DeveloperApi :: LZF implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: LZF implementation of org.apache.spark.io.CompressionCodec.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  4. final class NioBufferedFileInputStream extends InputStream

    Permalink

  5. class ReadAheadInputStream extends InputStream

    Permalink

  6. class SnappyCompressionCodec extends CompressionCodec

    Permalink

    :: DeveloperApi :: Snappy implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: Snappy implementation of org.apache.spark.io.CompressionCodec. Block size can be configured by spark.io.compression.snappy.blockSize.

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

  7. class ZStdCompressionCodec extends CompressionCodec

    Permalink

    :: DeveloperApi :: ZStandard implementation of org.apache.spark.io.CompressionCodec.

    :: DeveloperApi :: ZStandard implementation of org.apache.spark.io.CompressionCodec. For more details see - http://facebook.github.io/zstd/

    Annotations
    @DeveloperApi()
    Note

    The wire protocol for this codec is not guaranteed to be compatible across versions of Spark. This is intended for use as an internal compression utility within a single Spark application.

Inherited from AnyRef

Inherited from Any

Ungrouped