pyspark.sql.functions.array_union

pyspark.sql.functions.array_union(col1, col2)

Collection function: returns an array of the elements in the union of col1 and col2, without duplicates.

New in version 2.4.0.

Parameters:

col1Column or str

name of column containing array

col2Column or str

name of column containing array

Examples

>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([Row(c1=["b", "a", "c"], c2=["c", "d", "a", "f"])])
>>> df.select(array_union(df.c1, df.c2)).collect()
[Row(array_union(c1, c2)=['b', 'a', 'c', 'd', 'f'])]

pyspark.sql.functions.array_sort pyspark.sql.functions.arrays_overlap