SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
11.
Spark 3.0.0 で高階関数の Scala API が追加
Spark 3.0.0 で高階関数の Scala API が追加される。以下のように簡単に呼べる
12.
参考文献・投稿
● Introducing New Built-in and Higher-Order Functions for Complex Data Types
in Apache Spark 2.4
https://databricks.com/blog/2018/11/16/introducing-new-built-in-functions-and-higher-order-functions-for-complex-data-types-in-
apache-spark.html
● Working with Nested Data Using Higher Order Functions in SQL on Databricks
https://databricks.com/blog/2017/05/24/working-with-nested-data-using-higher-order-functions-in-sql-on-databricks.html
● SPARK-27297 Add higher order functions to Scala API
https://issues.apache.org/jira/browse/SPARK-27297
● SPARK-27297 PR
https://github.com/apache/spark/pull/24232