Gravy Analytics ingests ~17 billion records daily of data and improve and refine that data into many data products at various levels of aggregation. To meet the challenges of our product requirements and scale we constantly evaluate new technologies. Spark has become central to our ability to process ever increasing amounts of data through our data factory. In late 2017 and throughout 2018, we have improved our ability to work with Spark by migrating all Spark jobs to Scala. In this discussion, we’ll cover areas which were more difficult from a Spark perspective to develop in Java than Scala as well as some of the challenges we met along the way.