pyspark.sql.streaming.DataStreamWriter.outputMode

DataStreamWriter.outputMode(outputMode: str) → pyspark.sql.streaming.DataStreamWriter

Specifies how data of a streaming DataFrame/Dataset is written to a streaming sink.

New in version 2.0.0.

Options include:

• append: Only the new rows in the streaming DataFrame/Dataset will be written to

  the sink

• complete: All the rows in the streaming DataFrame/Dataset will be written to the sink

  every time these are some updates

• update: only the rows that were updated in the streaming DataFrame/Dataset will be

  written to the sink every time there are some updates. If the query doesn’t contain aggregations, it will be equivalent to append mode.

Notes

This API is evolving.

Examples

>>> writer = sdf.writeStream.outputMode('append')

pyspark.sql.streaming.DataStreamWriter.options pyspark.sql.streaming.DataStreamWriter.partitionBy