Spark数据帧计算按行最小值
问题描述：
我正在尝试将几列的最小值放入单独的列中. (创建min列).该操作非常简单，但我无法为此找到合适的功能:
A B分钟
1 2 1
2 1 1
3 1 1
1 4 1
I'm trying to put the minimum value of a few columns into a separate column. (Creating the min column). The operation is pretty straight forward but I wasn't able to find the right function for that:
A B min
1 2 1
2 1 1
3 1 1
1 4 1
非常感谢您的帮助！
答
您可以使用如果您具有列名列表:
cols = ['A', 'B']
df.withColumn('min', least(*cols))
类似地在 Scala 中:
import org.apache.spark.sql.functions.least
df.withColumn("min", least($"A", $"B")).show
+---+---+---+
| A| B|min|
+---+---+---+
| 1| 2| 1|
| 2| 1| 1|
| 3| 1| 1|
| 1| 4| 1|
+---+---+---+
如果列存储在Seq中:
If the columns are stored in a Seq:
val cols = Seq("A", "B")
df.withColumn("min", least(cols.head, cols.tail: _*))