SlideShare a Scribd company logo
Functional programming in
Scala
https://github.com/ganeshayadiyala/functional-programming-in-scala
● Ganesha Yadiyala
● Big data consultant at
datamantra.io
● Consult in spark and scala
● ganeshayadiyala@gmail.com
Agenda
● Programming paradigms
● Functional Programming
● Advantages of FP
● History of Scala
● Basics of scala
● Functional Aspect of Scala
Programming Paradigms
Programming Paradigms
● Programming Paradigm is a approach to programming
● It will be based on set of principles or theory
● Different ways of thinking.
● Different types
○ Imperative
○ Functional
Imperative Programming
Imperative programming is a programming paradigm that uses
statements that change a program's state.
Example :
● C#
● C++
● Java
Functional Programming
A programming style that models computation as the evaluation of
expression.
Example:
● Haskell
● Erlang
● Lisp
● Clojure
Expression vs Statement
Expression : evaluated to produce a value without changing state.
Statement : executed to update state.
Example:
Expression -> val isEven = if(x%2 == 0) true else false
Example :
Statement -> boolean isEven = true
if(x%2 == 0)
isEven = true
else
isEven = false
Functional Programming
History of Functional Programming
● Mathematical abstraction Lambda Calculus was the foundation
for functional Programming
● Lambda calculus was developed in 1930s to be used for
functional definition, functional application and recursion.
● It states that any computation can be achieved by sending the
input to some black box, which produces its result.
● This black box is lambda functions, and in FP it’s functions.
● LISP was the first functional programming language. It was
defined by McCarthy in 1958
Imperative vs Functional Programming
Characteristics Imperative Functional
State changes Important Non-existent
Order of execution Important Low importance
Primary flow control
Loops, conditions and method
(function) calls
Function calls
Primary manipulation unit
Instances of structures or
classes
Functions
Characteristics of FP
● Immutable data
● Lazy evaluation
● No side effects
● Referential transparency
● Functions are first class citizens
Immutable data
Mutable Immutable
var collection = [1,2,3,4]
for ( i = 0; i<collection.length; i++) {
collection[i]+=1;
}
Uses loop for updating
collection is updated in place
val collection = [1,2,3,4]
val newCollection = collection.map(value
=> value +1)
Uses transformation for change
Creates a new copy of collection. Leaves
collection intact
Advantages of FP
● Better modularity
● Shorter code
● Increased developer productivity
● Less number of bugs
● Perfect for distributed computing because of immutability
● Easier debugging
Limitations of FP
● Real world programs have to do side effects
● Difficult to understand for the beginner
● Very difficult to debug the nature of lazy evaluation
● Memory intensive
History of Scala
Scala History
● Martin Odersky is the creator
● Curious about compilers from beginning
● Created first compiler in 1995 - Pizza
○ Generics,Function pointers, Case class and pattern matching
● Led to javac compiler and generics eventually
○ Java is a strict language and hard to implement these new ideas
Scala History cont..
● 1999 - Started working on a language
○ Combines Object orientation with functional programming
○ Created Funnel language
○ It was not a user friendly language
○ New languages with less libraries are hard to adopt
● In 2001, design for Scala started, released first in 2003
● Scala is not an extension of Java
● Scala translates into JAVA byte code
● One implementation of .Net compatible version, now out dated
Scala Philosophy
● Growable language
○ Embed new DSL/add new types
● Scalable language
○ Same concepts in small and large applications
● Deep rather than broad
○ Focus on abstraction and composition
Why scala
● Hybrid programming language
● Statically typed
● Runs on the jvm
● Can execute java code
Basics of Scala
Everything is an Object
● Supports object oriented programming
● Including primitive types, everything in scala is an object
Example:
● val x = 10
● x.toString()
Statically Typed Language
● var message = “hello”
● message = “bye”
● message = 100 //compilation error
● def abs(x:Int) : Int = { ??? }
● abs(“100”) //compilation error
Type Inference
● val employeeName : String = “Joy”
● val department = “HR” //Type inferred as String
● val age = 35 //Type inferred as Int
Collections
● List,Set,Map etc..
● Mutable
● Immutable
● Functional API
Example:
val integerList = List(1,2,3)
Functional Aspect of Scala
Operators are also a Function
For example
● val sum = 2 + 3
Is same as,
● val sum = 2.+(3) //because ‘+’ operator here is a function
Functions in Scala
● Higher order function
● Anonymous function
● Currying
● Tail recursion
Defining a Function in Scala
● Defining a function
In Scala -
def sum(x:Int,y:Int) : Int = x+y
In Java -
public int sum(int x,int y){
return x+y;
}
Assigning function to variable
Example :
def add = (x:Int,y:Int) => x+y
add(2,3) //should return 5
Higher Order Functions
Function that takes functions as a parameters or return function as
a result is called higher order functions
Example : com.ganesh.functions.HigherOrderFunctions.scala
Anonymous Function
Scala provides a relatively lightweight syntax for defining
anonymous functions.
Example:
(x: Int) => x + 1
Shorthand for above function is,
new Function1[Int, Int] {
def apply(x: Int): Int = x + 1
}
Currying
Currying allows to turn a function that expects two arguments into a
function that expects only one, and that function returns a function
that expects the second argument. Creating basically a chain of
functions.
Example : com.ganesh.functions.Currying.scala
Implicit Parameters
The final parameter list on a method can be marked implicit, which
means the values will be taken from the context in which they are
called.
● It means that if no value is supplied when called, the compiler
will look for an implicit value and pass it in for you.
● Implicits are useful in defining default implementations
available. When you need a custom implementation, you can
pass one in explicitly.
Example : com.ganesh.implicits.Implicits.scala
Tail Recursion
● Tail recursion is a special kind of recursion where the recursive
call is the very last thing in the function.
● tail-recursive function will only use a single stack frame as
opposed to hundreds or even thousands in case of normal
recursive function.
● Tail recursion is much faster because it executes using single
stack frame
Example : com.ganesh.functions.TailRecursion.scala
Transformation on Collections
We can perform different transformations on collection using
collection API.
Example:
com.ganesh.transformations.SimpleTransformations.scala
com.ganesh.transformations.AdvancedTransformations.scala
Error Handling
Scala does not have checked exceptions like Java, so you can't do
something like this to force a programmer to deal with an
exception.
Example :
com.ganesh.exceptionhandling.ExceptionThrowing.scala
com.ganesh.exceptionhandling.UsingOptions.scala
References
● https://www.coursera.org/learn/progfun1/ - Scala course by
Martin Odersky
● http://www.dipmat.unict.it/~barba/PROG-LANG/PROGRAMMI-
TESTI/READING-MATERIAL/ShortIntroFPprog-lang.htm -
Functional programming and lambda calculus
● Functional Programming in Scala

More Related Content

What's hot

Functional programming
Functional programmingFunctional programming
Functional programming
ijcd
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
Mohammad Hossein Rimaz
 
Scala: functional programming for the imperative mind
Scala: functional programming for the imperative mindScala: functional programming for the imperative mind
Scala: functional programming for the imperative mind
Sander Mak (@Sander_Mak)
 
Java 8 - Features Overview
Java 8 - Features OverviewJava 8 - Features Overview
Java 8 - Features Overview
Sergii Stets
 
Java 8 presentation
Java 8 presentationJava 8 presentation
Java 8 presentation
Van Huong
 
Java 8 lambda
Java 8 lambdaJava 8 lambda
Java 8 lambda
Manav Prasad
 
Introduction to Java Programming, Basic Structure, variables Data type, input...
Introduction to Java Programming, Basic Structure, variables Data type, input...Introduction to Java Programming, Basic Structure, variables Data type, input...
Introduction to Java Programming, Basic Structure, variables Data type, input...
Mr. Akaash
 
Core java
Core javaCore java
Core java
kasaragaddaslide
 
What is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | EdurekaWhat is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | Edureka
Edureka!
 
Capabilities for Resources and Effects
Capabilities for Resources and EffectsCapabilities for Resources and Effects
Capabilities for Resources and Effects
Martin Odersky
 
Spring boot
Spring bootSpring boot
Spring boot
sdeeg
 
Python Advanced – Building on the foundation
Python Advanced – Building on the foundationPython Advanced – Building on the foundation
Python Advanced – Building on the foundation
Kevlin Henney
 
Introducing Kafka's Streams API
Introducing Kafka's Streams APIIntroducing Kafka's Streams API
Introducing Kafka's Streams API
confluent
 
Java 8 features
Java 8 featuresJava 8 features
Java 8 features
NexThoughts Technologies
 
Groovy presentation
Groovy presentationGroovy presentation
Groovy presentation
Manav Prasad
 
Lambda Expressions in Java 8
Lambda Expressions in Java 8Lambda Expressions in Java 8
Lambda Expressions in Java 8
icarter09
 
Unified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache FlinkUnified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache Flink
DataWorks Summit/Hadoop Summit
 
Wrapper class
Wrapper classWrapper class
Wrapper class
kamal kotecha
 
Angular js PPT
Angular js PPTAngular js PPT
Angular js PPT
Imtiyaz Ahmad Khan
 

What's hot (20)

Functional programming
Functional programmingFunctional programming
Functional programming
 
Introduction to Scala
Introduction to ScalaIntroduction to Scala
Introduction to Scala
 
Scala: functional programming for the imperative mind
Scala: functional programming for the imperative mindScala: functional programming for the imperative mind
Scala: functional programming for the imperative mind
 
Java 8 - Features Overview
Java 8 - Features OverviewJava 8 - Features Overview
Java 8 - Features Overview
 
Java 8 presentation
Java 8 presentationJava 8 presentation
Java 8 presentation
 
Java 8 lambda
Java 8 lambdaJava 8 lambda
Java 8 lambda
 
Introduction to Java Programming, Basic Structure, variables Data type, input...
Introduction to Java Programming, Basic Structure, variables Data type, input...Introduction to Java Programming, Basic Structure, variables Data type, input...
Introduction to Java Programming, Basic Structure, variables Data type, input...
 
Core java
Core javaCore java
Core java
 
What is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | EdurekaWhat is Python Lambda Function? Python Tutorial | Edureka
What is Python Lambda Function? Python Tutorial | Edureka
 
Capabilities for Resources and Effects
Capabilities for Resources and EffectsCapabilities for Resources and Effects
Capabilities for Resources and Effects
 
Spring boot
Spring bootSpring boot
Spring boot
 
Python Advanced – Building on the foundation
Python Advanced – Building on the foundationPython Advanced – Building on the foundation
Python Advanced – Building on the foundation
 
Introducing Kafka's Streams API
Introducing Kafka's Streams APIIntroducing Kafka's Streams API
Introducing Kafka's Streams API
 
Java 8 features
Java 8 featuresJava 8 features
Java 8 features
 
Groovy presentation
Groovy presentationGroovy presentation
Groovy presentation
 
String, string builder, string buffer
String, string builder, string bufferString, string builder, string buffer
String, string builder, string buffer
 
Lambda Expressions in Java 8
Lambda Expressions in Java 8Lambda Expressions in Java 8
Lambda Expressions in Java 8
 
Unified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache FlinkUnified Stream and Batch Processing with Apache Flink
Unified Stream and Batch Processing with Apache Flink
 
Wrapper class
Wrapper classWrapper class
Wrapper class
 
Angular js PPT
Angular js PPTAngular js PPT
Angular js PPT
 

Viewers also liked

Interactive Data Analysis in Spark Streaming
Interactive Data Analysis in Spark StreamingInteractive Data Analysis in Spark Streaming
Interactive Data Analysis in Spark Streaming
datamantra
 
Real time ETL processing using Spark streaming
Real time ETL processing using Spark streamingReal time ETL processing using Spark streaming
Real time ETL processing using Spark streaming
datamantra
 
Introduction to Spark 2.0 Dataset API
Introduction to Spark 2.0 Dataset APIIntroduction to Spark 2.0 Dataset API
Introduction to Spark 2.0 Dataset API
datamantra
 
Productionalizing a spark application
Productionalizing a spark applicationProductionalizing a spark application
Productionalizing a spark application
datamantra
 
Introduction to spark 2.0
Introduction to spark 2.0Introduction to spark 2.0
Introduction to spark 2.0
datamantra
 
Building end to end streaming application on Spark
Building end to end streaming application on SparkBuilding end to end streaming application on Spark
Building end to end streaming application on Spark
datamantra
 
Anatomy of spark catalyst
Anatomy of spark catalystAnatomy of spark catalyst
Anatomy of spark catalyst
datamantra
 
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
Anatomy of Data Frame API :  A deep dive into Spark Data Frame APIAnatomy of Data Frame API :  A deep dive into Spark Data Frame API
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
datamantra
 
Introduction to concurrent programming with akka actors
Introduction to concurrent programming with akka actorsIntroduction to concurrent programming with akka actors
Introduction to concurrent programming with akka actors
datamantra
 
Introduction to Structured Streaming
Introduction to Structured StreamingIntroduction to Structured Streaming
Introduction to Structured Streaming
datamantra
 
Interactive workflow management using Azkaban
Interactive workflow management using AzkabanInteractive workflow management using Azkaban
Interactive workflow management using Azkaban
datamantra
 
Anatomy of Data Source API : A deep dive into Spark Data source API
Anatomy of Data Source API : A deep dive into Spark Data source APIAnatomy of Data Source API : A deep dive into Spark Data source API
Anatomy of Data Source API : A deep dive into Spark Data source API
datamantra
 
Building scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTPBuilding scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTP
datamantra
 
Introduction to Apache Spark
Introduction to Apache SparkIntroduction to Apache Spark
Introduction to Apache Spark
datamantra
 
Functional programming in Scala
Functional programming in ScalaFunctional programming in Scala
Functional programming in Scala
Damian Jureczko
 
Introduction to Spark Internals
Introduction to Spark InternalsIntroduction to Spark Internals
Introduction to Spark Internals
Pietro Michiardi
 
Platform for Data Scientists
Platform for Data ScientistsPlatform for Data Scientists
Platform for Data Scientists
datamantra
 
Telco analytics at scale
Telco analytics at scaleTelco analytics at scale
Telco analytics at scale
datamantra
 
Introduction to Spark Streaming
Introduction to Spark StreamingIntroduction to Spark Streaming
Introduction to Spark Streaming
datamantra
 
Evolution of apache spark
Evolution of apache sparkEvolution of apache spark
Evolution of apache spark
datamantra
 

Viewers also liked (20)

Interactive Data Analysis in Spark Streaming
Interactive Data Analysis in Spark StreamingInteractive Data Analysis in Spark Streaming
Interactive Data Analysis in Spark Streaming
 
Real time ETL processing using Spark streaming
Real time ETL processing using Spark streamingReal time ETL processing using Spark streaming
Real time ETL processing using Spark streaming
 
Introduction to Spark 2.0 Dataset API
Introduction to Spark 2.0 Dataset APIIntroduction to Spark 2.0 Dataset API
Introduction to Spark 2.0 Dataset API
 
Productionalizing a spark application
Productionalizing a spark applicationProductionalizing a spark application
Productionalizing a spark application
 
Introduction to spark 2.0
Introduction to spark 2.0Introduction to spark 2.0
Introduction to spark 2.0
 
Building end to end streaming application on Spark
Building end to end streaming application on SparkBuilding end to end streaming application on Spark
Building end to end streaming application on Spark
 
Anatomy of spark catalyst
Anatomy of spark catalystAnatomy of spark catalyst
Anatomy of spark catalyst
 
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
Anatomy of Data Frame API :  A deep dive into Spark Data Frame APIAnatomy of Data Frame API :  A deep dive into Spark Data Frame API
Anatomy of Data Frame API : A deep dive into Spark Data Frame API
 
Introduction to concurrent programming with akka actors
Introduction to concurrent programming with akka actorsIntroduction to concurrent programming with akka actors
Introduction to concurrent programming with akka actors
 
Introduction to Structured Streaming
Introduction to Structured StreamingIntroduction to Structured Streaming
Introduction to Structured Streaming
 
Interactive workflow management using Azkaban
Interactive workflow management using AzkabanInteractive workflow management using Azkaban
Interactive workflow management using Azkaban
 
Anatomy of Data Source API : A deep dive into Spark Data source API
Anatomy of Data Source API : A deep dive into Spark Data source APIAnatomy of Data Source API : A deep dive into Spark Data source API
Anatomy of Data Source API : A deep dive into Spark Data source API
 
Building scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTPBuilding scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTP
 
Introduction to Apache Spark
Introduction to Apache SparkIntroduction to Apache Spark
Introduction to Apache Spark
 
Functional programming in Scala
Functional programming in ScalaFunctional programming in Scala
Functional programming in Scala
 
Introduction to Spark Internals
Introduction to Spark InternalsIntroduction to Spark Internals
Introduction to Spark Internals
 
Platform for Data Scientists
Platform for Data ScientistsPlatform for Data Scientists
Platform for Data Scientists
 
Telco analytics at scale
Telco analytics at scaleTelco analytics at scale
Telco analytics at scale
 
Introduction to Spark Streaming
Introduction to Spark StreamingIntroduction to Spark Streaming
Introduction to Spark Streaming
 
Evolution of apache spark
Evolution of apache sparkEvolution of apache spark
Evolution of apache spark
 

Similar to Functional programming in Scala

Understanding Implicits in Scala
Understanding Implicits in ScalaUnderstanding Implicits in Scala
Understanding Implicits in Scala
datamantra
 
Functional Programming in Ruby
Functional Programming in RubyFunctional Programming in Ruby
Functional Programming in RubyAlex Teut
 
Functional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNextFunctional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNext
Unfold UI
 
Python functional programming
Python functional programmingPython functional programming
Python functional programming
Geison Goes
 
Yes scala can!
Yes scala can!Yes scala can!
Yes scala can!
amirmoulavi
 
Ruby Functional Programming
Ruby Functional ProgrammingRuby Functional Programming
Ruby Functional Programming
Geison Goes
 
Introduction to functional programming
Introduction to functional programmingIntroduction to functional programming
Introduction to functional programming
Konrad Szydlo
 
Functional Programming.pptx
Functional Programming.pptxFunctional Programming.pptx
Functional Programming.pptx
KarthickT28
 
Java 8
Java 8Java 8
Java 8
vilniusjug
 
(3) cpp procedural programming
(3) cpp procedural programming(3) cpp procedural programming
(3) cpp procedural programming
Nico Ludwig
 
An Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsAn Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsMiles Sabin
 
An introduction to Object Oriented JavaScript
An introduction to Object Oriented JavaScriptAn introduction to Object Oriented JavaScript
An introduction to Object Oriented JavaScript
TO THE NEW | Technology
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
Martin Odersky
 
Develop realtime web with Scala and Xitrum
Develop realtime web with Scala and XitrumDevelop realtime web with Scala and Xitrum
Develop realtime web with Scala and Xitrum
Ngoc Dao
 
Dart workshop
Dart workshopDart workshop
Dart workshop
Vishnu Suresh
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
Schwannden Kuo
 
Reactive Software Systems
Reactive Software SystemsReactive Software Systems
Reactive Software Systems
Behrad Zari
 
Introduction of Object Oriented JavaScript
Introduction of Object Oriented JavaScriptIntroduction of Object Oriented JavaScript
Introduction of Object Oriented JavaScript
NexThoughts Technologies
 
Scala functions
Scala functionsScala functions
Scala functions
Knoldus Inc.
 

Similar to Functional programming in Scala (20)

Understanding Implicits in Scala
Understanding Implicits in ScalaUnderstanding Implicits in Scala
Understanding Implicits in Scala
 
Functional Programming in Ruby
Functional Programming in RubyFunctional Programming in Ruby
Functional Programming in Ruby
 
Functional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNextFunctional Programming in JavaScript & ESNext
Functional Programming in JavaScript & ESNext
 
Python functional programming
Python functional programmingPython functional programming
Python functional programming
 
Yes scala can!
Yes scala can!Yes scala can!
Yes scala can!
 
Ruby Functional Programming
Ruby Functional ProgrammingRuby Functional Programming
Ruby Functional Programming
 
Introduction to functional programming
Introduction to functional programmingIntroduction to functional programming
Introduction to functional programming
 
Functional Programming.pptx
Functional Programming.pptxFunctional Programming.pptx
Functional Programming.pptx
 
Java 8
Java 8Java 8
Java 8
 
(3) cpp procedural programming
(3) cpp procedural programming(3) cpp procedural programming
(3) cpp procedural programming
 
An Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional ParadigmsAn Introduction to Scala - Blending OO and Functional Paradigms
An Introduction to Scala - Blending OO and Functional Paradigms
 
An introduction to Object Oriented JavaScript
An introduction to Object Oriented JavaScriptAn introduction to Object Oriented JavaScript
An introduction to Object Oriented JavaScript
 
Scala-Ls1
Scala-Ls1Scala-Ls1
Scala-Ls1
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
Develop realtime web with Scala and Xitrum
Develop realtime web with Scala and XitrumDevelop realtime web with Scala and Xitrum
Develop realtime web with Scala and Xitrum
 
Dart workshop
Dart workshopDart workshop
Dart workshop
 
PL Lecture 01 - preliminaries
PL Lecture 01 - preliminariesPL Lecture 01 - preliminaries
PL Lecture 01 - preliminaries
 
Reactive Software Systems
Reactive Software SystemsReactive Software Systems
Reactive Software Systems
 
Introduction of Object Oriented JavaScript
Introduction of Object Oriented JavaScriptIntroduction of Object Oriented JavaScript
Introduction of Object Oriented JavaScript
 
Scala functions
Scala functionsScala functions
Scala functions
 

More from datamantra

Multi Source Data Analysis using Spark and Tellius
Multi Source Data Analysis using Spark and TelliusMulti Source Data Analysis using Spark and Tellius
Multi Source Data Analysis using Spark and Tellius
datamantra
 
State management in Structured Streaming
State management in Structured StreamingState management in Structured Streaming
State management in Structured Streaming
datamantra
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetes
datamantra
 
Understanding transactional writes in datasource v2
Understanding transactional writes in  datasource v2Understanding transactional writes in  datasource v2
Understanding transactional writes in datasource v2
datamantra
 
Introduction to Datasource V2 API
Introduction to Datasource V2 APIIntroduction to Datasource V2 API
Introduction to Datasource V2 API
datamantra
 
Exploratory Data Analysis in Spark
Exploratory Data Analysis in SparkExploratory Data Analysis in Spark
Exploratory Data Analysis in Spark
datamantra
 
Core Services behind Spark Job Execution
Core Services behind Spark Job ExecutionCore Services behind Spark Job Execution
Core Services behind Spark Job Execution
datamantra
 
Optimizing S3 Write-heavy Spark workloads
Optimizing S3 Write-heavy Spark workloadsOptimizing S3 Write-heavy Spark workloads
Optimizing S3 Write-heavy Spark workloads
datamantra
 
Structured Streaming with Kafka
Structured Streaming with KafkaStructured Streaming with Kafka
Structured Streaming with Kafka
datamantra
 
Understanding time in structured streaming
Understanding time in structured streamingUnderstanding time in structured streaming
Understanding time in structured streaming
datamantra
 
Spark stack for Model life-cycle management
Spark stack for Model life-cycle managementSpark stack for Model life-cycle management
Spark stack for Model life-cycle management
datamantra
 
Productionalizing Spark ML
Productionalizing Spark MLProductionalizing Spark ML
Productionalizing Spark ML
datamantra
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streaming
datamantra
 
Building real time Data Pipeline using Spark Streaming
Building real time Data Pipeline using Spark StreamingBuilding real time Data Pipeline using Spark Streaming
Building real time Data Pipeline using Spark Streaming
datamantra
 
Testing Spark and Scala
Testing Spark and ScalaTesting Spark and Scala
Testing Spark and Scala
datamantra
 
Migrating to Spark 2.0 - Part 2
Migrating to Spark 2.0 - Part 2Migrating to Spark 2.0 - Part 2
Migrating to Spark 2.0 - Part 2
datamantra
 
Migrating to spark 2.0
Migrating to spark 2.0Migrating to spark 2.0
Migrating to spark 2.0
datamantra
 
Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetes
datamantra
 
Anatomy of Spark SQL Catalyst - Part 2
Anatomy of Spark SQL Catalyst - Part 2Anatomy of Spark SQL Catalyst - Part 2
Anatomy of Spark SQL Catalyst - Part 2
datamantra
 

More from datamantra (19)

Multi Source Data Analysis using Spark and Tellius
Multi Source Data Analysis using Spark and TelliusMulti Source Data Analysis using Spark and Tellius
Multi Source Data Analysis using Spark and Tellius
 
State management in Structured Streaming
State management in Structured StreamingState management in Structured Streaming
State management in Structured Streaming
 
Spark on Kubernetes
Spark on KubernetesSpark on Kubernetes
Spark on Kubernetes
 
Understanding transactional writes in datasource v2
Understanding transactional writes in  datasource v2Understanding transactional writes in  datasource v2
Understanding transactional writes in datasource v2
 
Introduction to Datasource V2 API
Introduction to Datasource V2 APIIntroduction to Datasource V2 API
Introduction to Datasource V2 API
 
Exploratory Data Analysis in Spark
Exploratory Data Analysis in SparkExploratory Data Analysis in Spark
Exploratory Data Analysis in Spark
 
Core Services behind Spark Job Execution
Core Services behind Spark Job ExecutionCore Services behind Spark Job Execution
Core Services behind Spark Job Execution
 
Optimizing S3 Write-heavy Spark workloads
Optimizing S3 Write-heavy Spark workloadsOptimizing S3 Write-heavy Spark workloads
Optimizing S3 Write-heavy Spark workloads
 
Structured Streaming with Kafka
Structured Streaming with KafkaStructured Streaming with Kafka
Structured Streaming with Kafka
 
Understanding time in structured streaming
Understanding time in structured streamingUnderstanding time in structured streaming
Understanding time in structured streaming
 
Spark stack for Model life-cycle management
Spark stack for Model life-cycle managementSpark stack for Model life-cycle management
Spark stack for Model life-cycle management
 
Productionalizing Spark ML
Productionalizing Spark MLProductionalizing Spark ML
Productionalizing Spark ML
 
Introduction to Structured streaming
Introduction to Structured streamingIntroduction to Structured streaming
Introduction to Structured streaming
 
Building real time Data Pipeline using Spark Streaming
Building real time Data Pipeline using Spark StreamingBuilding real time Data Pipeline using Spark Streaming
Building real time Data Pipeline using Spark Streaming
 
Testing Spark and Scala
Testing Spark and ScalaTesting Spark and Scala
Testing Spark and Scala
 
Migrating to Spark 2.0 - Part 2
Migrating to Spark 2.0 - Part 2Migrating to Spark 2.0 - Part 2
Migrating to Spark 2.0 - Part 2
 
Migrating to spark 2.0
Migrating to spark 2.0Migrating to spark 2.0
Migrating to spark 2.0
 
Scalable Spark deployment using Kubernetes
Scalable Spark deployment using KubernetesScalable Spark deployment using Kubernetes
Scalable Spark deployment using Kubernetes
 
Anatomy of Spark SQL Catalyst - Part 2
Anatomy of Spark SQL Catalyst - Part 2Anatomy of Spark SQL Catalyst - Part 2
Anatomy of Spark SQL Catalyst - Part 2
 

Recently uploaded

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
Tiktokethiodaily
 
一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单
enxupq
 
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
ahzuo
 
standardisation of garbhpala offhgfffghh
standardisation of garbhpala offhgfffghhstandardisation of garbhpala offhgfffghh
standardisation of garbhpala offhgfffghh
ArpitMalhotra16
 
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape ReportSOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar
 
Opendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptxOpendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptx
Opendatabay
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
slg6lamcq
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
TravisMalana
 
FP Growth Algorithm and its Applications
FP Growth Algorithm and its ApplicationsFP Growth Algorithm and its Applications
FP Growth Algorithm and its Applications
MaleehaSheikh2
 
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdfSample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Linda486226
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
axoqas
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
benishzehra469
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
jerlynmaetalle
 
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
ewymefz
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
AbhimanyuSinha9
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdfCh03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
haila53
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
pchutichetpong
 

Recently uploaded (20)

一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
1.Seydhcuxhxyxhccuuxuxyxyxmisolids 2019.pptx
 
一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单一比一原版(QU毕业证)皇后大学毕业证成绩单
一比一原版(QU毕业证)皇后大学毕业证成绩单
 
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
一比一原版(CBU毕业证)卡普顿大学毕业证如何办理
 
standardisation of garbhpala offhgfffghh
standardisation of garbhpala offhgfffghhstandardisation of garbhpala offhgfffghh
standardisation of garbhpala offhgfffghh
 
SOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape ReportSOCRadar Germany 2024 Threat Landscape Report
SOCRadar Germany 2024 Threat Landscape Report
 
Opendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptxOpendatabay - Open Data Marketplace.pptx
Opendatabay - Open Data Marketplace.pptx
 
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
一比一原版(Adelaide毕业证书)阿德莱德大学毕业证如何办理
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
 
FP Growth Algorithm and its Applications
FP Growth Algorithm and its ApplicationsFP Growth Algorithm and its Applications
FP Growth Algorithm and its Applications
 
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdfSample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
 
The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...The affect of service quality and online reviews on customer loyalty in the E...
The affect of service quality and online reviews on customer loyalty in the E...
 
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdfCh03-Managing the Object-Oriented Information Systems Project a.pdf
Ch03-Managing the Object-Oriented Information Systems Project a.pdf
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
 
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...
 

Functional programming in Scala

  • 2. ● Ganesha Yadiyala ● Big data consultant at datamantra.io ● Consult in spark and scala ● ganeshayadiyala@gmail.com
  • 3. Agenda ● Programming paradigms ● Functional Programming ● Advantages of FP ● History of Scala ● Basics of scala ● Functional Aspect of Scala
  • 5. Programming Paradigms ● Programming Paradigm is a approach to programming ● It will be based on set of principles or theory ● Different ways of thinking. ● Different types ○ Imperative ○ Functional
  • 6. Imperative Programming Imperative programming is a programming paradigm that uses statements that change a program's state. Example : ● C# ● C++ ● Java
  • 7. Functional Programming A programming style that models computation as the evaluation of expression. Example: ● Haskell ● Erlang ● Lisp ● Clojure
  • 8. Expression vs Statement Expression : evaluated to produce a value without changing state. Statement : executed to update state. Example: Expression -> val isEven = if(x%2 == 0) true else false
  • 9. Example : Statement -> boolean isEven = true if(x%2 == 0) isEven = true else isEven = false
  • 11. History of Functional Programming ● Mathematical abstraction Lambda Calculus was the foundation for functional Programming ● Lambda calculus was developed in 1930s to be used for functional definition, functional application and recursion. ● It states that any computation can be achieved by sending the input to some black box, which produces its result. ● This black box is lambda functions, and in FP it’s functions. ● LISP was the first functional programming language. It was defined by McCarthy in 1958
  • 12. Imperative vs Functional Programming Characteristics Imperative Functional State changes Important Non-existent Order of execution Important Low importance Primary flow control Loops, conditions and method (function) calls Function calls Primary manipulation unit Instances of structures or classes Functions
  • 13. Characteristics of FP ● Immutable data ● Lazy evaluation ● No side effects ● Referential transparency ● Functions are first class citizens
  • 14. Immutable data Mutable Immutable var collection = [1,2,3,4] for ( i = 0; i<collection.length; i++) { collection[i]+=1; } Uses loop for updating collection is updated in place val collection = [1,2,3,4] val newCollection = collection.map(value => value +1) Uses transformation for change Creates a new copy of collection. Leaves collection intact
  • 15. Advantages of FP ● Better modularity ● Shorter code ● Increased developer productivity ● Less number of bugs ● Perfect for distributed computing because of immutability ● Easier debugging
  • 16. Limitations of FP ● Real world programs have to do side effects ● Difficult to understand for the beginner ● Very difficult to debug the nature of lazy evaluation ● Memory intensive
  • 18. Scala History ● Martin Odersky is the creator ● Curious about compilers from beginning ● Created first compiler in 1995 - Pizza ○ Generics,Function pointers, Case class and pattern matching ● Led to javac compiler and generics eventually ○ Java is a strict language and hard to implement these new ideas
  • 19. Scala History cont.. ● 1999 - Started working on a language ○ Combines Object orientation with functional programming ○ Created Funnel language ○ It was not a user friendly language ○ New languages with less libraries are hard to adopt ● In 2001, design for Scala started, released first in 2003 ● Scala is not an extension of Java ● Scala translates into JAVA byte code ● One implementation of .Net compatible version, now out dated
  • 20. Scala Philosophy ● Growable language ○ Embed new DSL/add new types ● Scalable language ○ Same concepts in small and large applications ● Deep rather than broad ○ Focus on abstraction and composition
  • 21. Why scala ● Hybrid programming language ● Statically typed ● Runs on the jvm ● Can execute java code
  • 23. Everything is an Object ● Supports object oriented programming ● Including primitive types, everything in scala is an object Example: ● val x = 10 ● x.toString()
  • 24. Statically Typed Language ● var message = “hello” ● message = “bye” ● message = 100 //compilation error ● def abs(x:Int) : Int = { ??? } ● abs(“100”) //compilation error
  • 25. Type Inference ● val employeeName : String = “Joy” ● val department = “HR” //Type inferred as String ● val age = 35 //Type inferred as Int
  • 26. Collections ● List,Set,Map etc.. ● Mutable ● Immutable ● Functional API Example: val integerList = List(1,2,3)
  • 28. Operators are also a Function For example ● val sum = 2 + 3 Is same as, ● val sum = 2.+(3) //because ‘+’ operator here is a function
  • 29. Functions in Scala ● Higher order function ● Anonymous function ● Currying ● Tail recursion
  • 30. Defining a Function in Scala ● Defining a function In Scala - def sum(x:Int,y:Int) : Int = x+y In Java - public int sum(int x,int y){ return x+y; }
  • 31. Assigning function to variable Example : def add = (x:Int,y:Int) => x+y add(2,3) //should return 5
  • 32. Higher Order Functions Function that takes functions as a parameters or return function as a result is called higher order functions Example : com.ganesh.functions.HigherOrderFunctions.scala
  • 33. Anonymous Function Scala provides a relatively lightweight syntax for defining anonymous functions. Example: (x: Int) => x + 1 Shorthand for above function is, new Function1[Int, Int] { def apply(x: Int): Int = x + 1 }
  • 34. Currying Currying allows to turn a function that expects two arguments into a function that expects only one, and that function returns a function that expects the second argument. Creating basically a chain of functions. Example : com.ganesh.functions.Currying.scala
  • 35. Implicit Parameters The final parameter list on a method can be marked implicit, which means the values will be taken from the context in which they are called. ● It means that if no value is supplied when called, the compiler will look for an implicit value and pass it in for you. ● Implicits are useful in defining default implementations available. When you need a custom implementation, you can pass one in explicitly. Example : com.ganesh.implicits.Implicits.scala
  • 36. Tail Recursion ● Tail recursion is a special kind of recursion where the recursive call is the very last thing in the function. ● tail-recursive function will only use a single stack frame as opposed to hundreds or even thousands in case of normal recursive function. ● Tail recursion is much faster because it executes using single stack frame Example : com.ganesh.functions.TailRecursion.scala
  • 37. Transformation on Collections We can perform different transformations on collection using collection API. Example: com.ganesh.transformations.SimpleTransformations.scala com.ganesh.transformations.AdvancedTransformations.scala
  • 38. Error Handling Scala does not have checked exceptions like Java, so you can't do something like this to force a programmer to deal with an exception. Example : com.ganesh.exceptionhandling.ExceptionThrowing.scala com.ganesh.exceptionhandling.UsingOptions.scala
  • 39. References ● https://www.coursera.org/learn/progfun1/ - Scala course by Martin Odersky ● http://www.dipmat.unict.it/~barba/PROG-LANG/PROGRAMMI- TESTI/READING-MATERIAL/ShortIntroFPprog-lang.htm - Functional programming and lambda calculus ● Functional Programming in Scala