如何解决Spark 数据帧值到 Scala 列表
+----------------------------+
|User | Color |
+----------------------------+
|User1 | [Green,Blue,Red] |
|User2 | [Blue,Red] |
+----------------------------+
我正在尝试过滤 User1
并将颜色列表放入 Scala 列表中:
val colorsList: List[String] = List("Green","Blue","Red")
尝试 1:
val dfTest1 = myDataframe.where("User=='User1'").select("Color").rdd.map(r => r(0)).collect()
println(dfTest1) //[Ljava.lang.Object;@44022255
for(EachColor<- dfTest1){
println(EachColor) //WrappedArray(Green,Red)
}
尝试 2:
val dfTest2 = myDataframe.where("User=='User1'").select("Color").collectAsList.get(0).getList(0)
println(dfTest2) //[Green,Red] but type is util.List[nothing]
尝试 3:
val dfTest32 = myDataframe.where("User=='User1'").select("Color").rdd.map(r => r(0)).collect.toList
println(dfTest32) //List(WrappedArray(Green,Red))
for(EachColor <- dfTest32){
println(EachColor) //WrappedArray(Green,Red)
}
尝试 4:
val dfTest31 = myDataframe.where("User=='User1'").select("Color").map(r => r.getString(0)).collect.toList
//Exception : scala.collection.mutable.WrappedArray$ofRef cannot be cast to java.lang.String
解决方法
您可以尝试获取为 Seq[String]
并转换为 toList
:
val colorsList = df.where("User=='User1'")
.select("Color")
.rdd.map(r => r.getAs[Seq[String]](0))
.collect()(0)
.toList
或等效
val colorsList = df.where("User=='User1'")
.select("Color")
.collect()(0)
.getAs[Seq[String]](0)
.toList
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。