Clojure talk at Münster JUG

Clojure A small introduction Alex Ott http://alexott.net M¨nster JUG, July 2010 u Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 1 / 40

About me... Professional software developer & architect Working with diﬀerent programming languages – C/C++, Lisps, Haskell, Erlang, ... About 20 years of Lisp experience, including commercial projects Development on Unix/Linux platforms Author of articles on programming (including Clojure), Emacs, etc. Participate in many open source projects, including Incanter, labrepl, etc. Maintainer of “Planet Clojure” Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 2 / 40

Outline 1 What is Clojure? 2 Language basics 3 Interoperability with Java 4 Concurrent programming 5 Clojure in real life 6 Sources of information 7 Examples Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 3 / 40

What is Clojure? Lisp-like language, created by Rich Hickey and announced in 2007th Designed to work on existing platforms Functional programming language, with immutable data Special features for concurrent programming Open sourced with liberal license Already used in many projects, including commercial Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 4 / 40

Why new language? Lisp, but without compatibility with previous versions Immutable data and more support for pure functional programming comparing with other Lisps Support for concurrent programing inside language Better integration with target platforms: JVM & .Net Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 5 / 40

Main features Dynamically typed language Very simple syntax, like in other Lisps (code as data) Support for interactive development Designed in terms of abstractions Metaprogramming Multimethods and protocols (in version 1.2) Compiled into byte code of target platforms Access to big number of available libraries Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 6 / 40

Diﬀerences from other Lisps Changes in syntax, less parentheses Changes in naming More ﬁrst-class data structures – maps, sets, vectors Immutable, by default, data Ability to link meta-information with variables and functions “Lazy” collections There is no reader macros Case-sensitive names Lack of tail call optimization (JVM’s limitation) – explicit loops loop/recur Exceptions instead of signals and restarts Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 7 / 40

Base data types Integer numbers of any size – 14235344564564564564 Rational numbers – 26/7 Real numbers – 1.2345 and BigDecimals – 1.2345M Strings (String class from Java) – "hello world" Characters (Character from Java) – a, b, . . . Regular expressions – #"[abc]*" Boolean – true & false nil – the same as null in Java, not an empty list as in Lisp Symbols – test, var1, . . . Keywords – :test, :hello, . . . Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 8 / 40

Data structures Separate syntax for diﬀerent collections: Lists – (1 2 3 "abc") Vectors – [1 2 3 "abc"] Maps – {:key1 1234 :key2 "value"} Sets – #{:val1 "text" 1 2 10} Sequence – abstraction to work with all collections (including classes from Java/.Net) Common set of functions to work with sequences Lazy operations on sequences Vectors, maps, and sets are functions – to simplify access to data in them Persistent collections Transient collections – performance optimization defstruct – optimization of map to work with complex data structures Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 9 / 40

Syntax Very simple, homoiconic syntax – program is usual data structure Special procedure (reader) process text and produce data structures All objects represent themselves except symbols and lists Symbols link value with “variable” Lists represent forms, that could be: Special form – def, let, if, loop, . . . Macros Function call or expression, that could be used as function (maps, keywords, returned functional values, . . . ) To get list as-is you need to “quote” it (with quote or ’) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 10 / 40

Code example ( defn f i b o ( [ ] ( concat [1 1] ( f i b o 1 1))) ([ a b] ( l e t [ n (+ a b ) ] ( l a z y −s e q ( c o n s n ( f i b o b n ) ) ) ) ) ) ( t a k e 100000000 ( f i b o ) ) ( defn vrange2 [ n ] ( loop [ i 0 v ( transient [])] ( i f (< i n ) ( recur ( inc i ) ( conj ! v i )) ( persistent ! v )))) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 11 / 40

Life cycle of Clojure program Code is read from ﬁle, or REPL, or from other code Reader transforms program into data structures Macros are expanded Resulting code is evaluated/compiled Produced bytecode is executed by JVM File Evaluator/ bytecode JVM Code Data Compiler structures Reader Data Data structures structures Code Execution Macroexpand Developer (REPL) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 12 / 40

Functions Functions are first-class objects Function can have variable number of arguments Function’s definition: defn – top-level functions, fn – anonymous functions Simplified syntax for anonymous functions – #(code). For example: (map #(.toUpperCase %) ["test" "hello"]) (map #(vector %1 %2) [1 2 3] [4 5 6]) You can specify tests & pre/post-conditions. For example, ( d e f n c o n s t r a i n e d −s q r [ x ] { : p r e [ ( pos ? x ) ] : p o s t [( > % 1 6 ) , (< % 2 2 5 ) ] } (∗ x x ) ) Each function is compiled into separate Java class Each function implements Runnable interface Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 13 / 40

Metaprogramming & macros Macros receives source code and returns new code, that will compiled Big part of language is written using macros Variable number of arguments (same as functions) Code could be generated with list functions, but it’s much handy to use quasi-quote – ‘ and substitution operators – ~ and ~@ The # suﬃx in names is used to generate unique names macroexpand-1 & macroexpand are used to debug macros Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 14 / 40

Macros examples (1) The standard macros when: ( d e f m a c r o when [ t e s t & body ] ( l i s t ’ i f t e s t ( c o n s ’ do body ) ) ) when used as: ( when ( pos ? a ) ( p r i n t l n " p o s i t i v e ") (/ b a ) ) will expanded into: ( i f ( pos ? a ) ( do ( p r i n t l n " p o s i t i v e ") (/ b a ) ) ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 15 / 40

Macros examples (2) The standard macros and ( d e f m a c r o and ( [ ] true ) ([ x] x) ( [ x & next ] ‘ ( l e t [ and# ~x ] ( i f and# ( and ~@next ) and # ) ) ) ) will expanded into diﬀerent code, depending on number of arguments: user> ( macroexpand ’ ( and ) ) == t r u e > user> ( macroexpand ’ ( and (= 1 2 ) ) ) == (= 1 2 ) > user> ( macroexpand ’ ( and (= 1 2 ) (= 3 3 ) ) ) ==> ( let∗ [ and__4457__auto__ (= 1 2)] ( if and__4457__auto__ ( c l o j u r e . c o r e / and (= 3 3)) and__4457__auto__ ) ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 16 / 40

Polymorphism & multimethods Extensibility Multimethods aren’t bound exclusively to types/classes Dispatching is performed using results of user-specified dispatch function Dispatching on several parameters Differs from CLOS – absence of :before, :after, . . . defmulti – is analog of defgeneric in Common Lisp Defined as ( defmulti func−name dispatch−fn) + set of implementations (via defmethod) You can also define hierarchical relationships with derive, and use them in dispatch (via isa?) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 17 / 40

Multimethods examples (1) Simple dispatching using data type/class: −e ( d e f m u l t i m xample c l a s s ) −e ( d e f m e t h od m x a m p l e S t r i n g [ t h i s ] ( p r i n t l n " This i s s t r i n g ’" t h i s " ’")) −e ( d e f m e t h od m x a m p l e j a v a . u t i l . C o l l e c t i o n [ t h i s ] ( p r i n t " This i s c o l l e c t i o n ! " ) ) and we’ll get: −e (m x a m p l e " H e l l o " ) == " T h i s i s s t r i n g ’ H e l l o ’ " > −e (m x a m p l e [ 1 2 3 ] ) == " T h i s i s c o l l e c t i o n ! " > −e (m x a m p l e ’ ( 1 2 3 ) ) == " T h i s i s c o l l e c t i o n ! " > Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 18 / 40

Multimethods examples (2) ( defmulti encounter ( fn [ x y ] [ ( : Species x ) ( : Species y ) ] ) ) ( d e f m e t h od e n c o u n t e r [ : Bunny : L i o n ] [ b l ] : run−away ) ( d e f m e t h od e n c o u n t e r [ : L i o n : Bunny ] [ l b ] : e a t ) ( d e f m e t h od e n c o u n t e r [ : L i o n : L i o n ] [ l 1 l 2 ] : f i g h t ) ( d e f m e t h od e n c o u n t e r [ : Bunny : Bunny ] [ b1 b2 ] : mate ) ( d e f b1 { : S p e c i e s : Bunny } ) ( d e f b2 { : S p e c i e s : Bunny } ) ( def l 1 { : S p e c i e s : Lion }) ( def l 2 { : S p e c i e s : Lion }) ( encounter b1 b2 ) ==> : mate ( encounter b1 l1 ) ==> : run−away ( encounter l1 b1 ) ==> : eat ( encounter l1 l2 ) ==> : fight Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 19 / 40

Protocols & Datatypes Introduced in version 1.2 (will released shortly) Much faster than multimethods Allows to write “Clojure in Clojure” Dispatching is done only on data types Similar to type classes in Haskell Java interface is created for each protocol defrecord & deftype deﬁne new data types extend-protocol & extend-type bind protocol(s) with data types (not only with deﬁned in Clojure!) reify is used to implement protocols and interfaces for “once-used types” Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 20 / 40

Protocol’s examples ( d e f p r o t o c o l H e l l o " Test of p r o t o c o l " ( h e l l o [ t h i s ] " h e l l o function ")) ( d e f r e c o r d B [ name ] H e l l o ( h e l l o [ t h i s ] ( s t r " H e l l o " ( : name t h i s ) " ! " ) ) ) ( h e l l o (B . " U s e r " ) ) == " H e l l o U s e r ! " > ( e x t e n d −p r o t o c o l H e l l o S t r i n g ( hello [ this ] ( str " Hello " this "!"))) ( h e l l o " w o r l d " ) == " H e l l o w o r l d ! " > ( e x t e n d −p r o t o c o l H e l l o j a v a . l a n g . O b j e c t ( h e l l o [ t h i s ] ( s t r " H e l l o ’" t h i s " ’ ! (" ( type t h i s ) " ) " ) ) ) ( h e l l o 1 ) == " H e l l o ’ 1 ’ ! ( c l a s s j a v a . l a n g . I n t e g e r ) " > Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 21 / 40

Miscellaneous Metadata – #^{} in 1.0 & 1.1, or simply ^{} in 1.2 Optional type speciﬁcation (type hints) – #^Type or ^Type You can specify tests directly in function’s declaration Scope management Access and change of metadata from code Don’t change value equality Namespaces: are ﬁrst-class objects are used to organize code into libraries Data destructuring (function arguments, etc.) ( l e t [ [ a b & c : as e ] [1 2 3 4 ] ] [ a b c e ] ) == [ 1 2 ( 3 4 ) [ 1 2 3 4 ] ] > ( l e t [ { : k e y s [ a b c ] : a s m : o r {b 3}} { : a 5 : c 6 } ] [ a b c m] ) == [ 5 3 6 { : a 5 : c 6 } ] > Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 22 / 40

Interoperability with Java Two-way interoperability with target platform: Creation of instances – new or Class. Call of Java methods – ., .., doto Class & interface generation – gen-class, gen-interface, or proxy (anonymous class) You can execute Clojure code in Java programs Separate functions to work with Java arrays: make-array – array creation aget/aset – access to array’s elements amap/areduce – iteration over array’s elements memfn allows to use Java methods as arguments of map, filter, etc. The set! special form to set values inside class Generation and catch of exception – throw & catch Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 23 / 40

Interoperability examples Instance creation: ( new j a v a . u t i l . Date ) <== ( j a v a . u t i l . Date . ) > Call of Java methods and access to data: ( . s u b s t r i n g " H e l l o World " 0 5 ) == " H e l l o " > ( . " H e l l o World " s u b s t r i n g 0 5 ) == " H e l l o " > Math/ PI == 3 . 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3 > ( I n t e g e r / p a r s e I n t " 4 2 " ) == 42 > .. is used to chained calls: ( . . System g e t P r o p e r t i e s ( g e t " o s . name " ) ) == "Mac > System . g e t P r o p e r t i e s ( ) . g e t ( " o s . name " ) doto – call of several methods for one object: ( d o t o ( j a v a . u t i l . HashMap . ) ( . p u t " a " 1 ) ( . p u t "b" 2 ) ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 24 / 40

Concurrent programming Special language features, that provides data mutation: Refs – synchronous, coordinated change Agents – asynchronous, not coordinated Atoms – synchronous, not coordinated Vars – local for current thread @ (deref) is used to get access to data Parallel code execution future pmap, pvalues & pcalls Native threads Synchronization with promise Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 25 / 40

State and Identity Clojure separates concepts of state and identity State (value) isn’t changed! More information at http://clojure.org/state Code Code Code (past) (present) (future) Identity (ref/agent/atom name) State S1 State S2 State S3 [1 2 3 4] [1 2 3 4 5 6 7] [?????] Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 26 / 40

Refs & STM Coordinated change of data from several threads Software Transaction Memory provides atomicity, consistency, and isolation Changes could be made only inside transaction You can add function for data validation Ability to add watcher function ( def counters ( r e f {})) ( d e f n get −c o u n t e r [ k e y ] ( @counters key 0)) ( d e f n i n c r e m e n t −c o u n t e r [ k e y ] ( dosync ( a l t e r c o u n t e r s a s s o c key ( i n c ( @counters key 0 ) ) ) ) ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 27 / 40

Agents Asynchronous change of data – “ﬁre and forget” Thread pools for data update functions: send – for “fast” functions, and send-off – for “heavyweight” functions (separate thread pool) send & send-off get function that will applied to agent’s current state Validators and watchers You can wait until ﬁnish of all updates ( def acounters ( agent {})) ( d e f n i n c r e m e n t −a c o u n t e r [ k e y ] ( send a c o u n t e r s a s s o c key ( i n c ( @counters key 0 ) ) ) ) ( d e f n get −a c o u n t e r [ k e y ] ( @acounters key 0)) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 28 / 40

Vars & Atoms Atoms Synchronous data change without coordination Data are changed by applying function to current state of atom Function could be called several times Function shouldn’t have side eﬀects! Vars Provide data change only in current thread binding links new values with symbols Change aﬀects all functions, called from current thread Be careful with lazy sequences! ( def ∗ var ∗ 5) ( defn foo [ ] ∗ var ∗) ( f o o ) == 5 > ( b i n d i n g [ ∗ v a r ∗ 1 0 ] ( f o o ) ) == 10 > Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 29 / 40

Parallel code execution future executes given code in separate thread @ is used to get results of code execution @ blocks current thread, if results aren’t available yet results of code execution are cached promise is used for synchronization between parts of programs deliver sets value of promise @ reads value, or blocks execution if value isn’t set yet pmap, pvalues & pcalls are used for “heavyweight” operations, that could be performed in parallel Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 30 / 40

Clojure in real life Existing infrastructure: IDEs Build tools Debugging and related tools Libraries Code repositories Clojure/core – commercial support & consulting Clojure is used in commercial projects since 2008 Sources of information Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 31 / 40

Infrastructure: IDEs and build tools Supported by most of popular IDEs: Emacs + SLIME (most popular now) VimClojure NetBeans Eclipse IntelliJ IDEA Build tools: Clojure support in Maven, Ant and Gradle Leiningen – written in Clojure, extensible and very simple Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 32 / 40

Infrastructure: libraries & repositories Simple access to big number of existing Java libraries Clojure-speciﬁc libraries: Clojure-Contrib – “semi-standard” libraries Compojure, Ring, Scriptjure – Web development ClojureQL – databases Incanter – R-like environment and libraries other – see at http://clojars.org Repositories: build.clojure.org clojars.org Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 33 / 40

Commercial projects FlightCaster – machine learning, etc. Relevance, Inc. – consulting Runa – marketing services Sonian Networks – archiving solutions BackType – social media analytics DRW Trading Group – trading and ﬁnance DocuHarvest project by Snowtide Informatics Systems, Inc. ThorTech Solutions – scalable, mission-critical solutions TheDeadline project by freiheit.com (http://www.freiheit.com/) and many more. . . Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 34 / 40

Sources of information (1) Main sites: Site of the language – http://clojure.org Planet Clojure – http://planet.clojure.in the #clojure IRC channel at freenode.net The labrepl project (http://github.com/relevance/labrepl) – learning environment Try-Clojure (http://www.try-clojure.org/) – you can execute Clojure code via Web-browser http://clojuredocs.org/ – documentation and examples German resources: Book in German will released in 2010 fall (http://www.clojure-buch.de/) clojure-de mailing list (http://groups.google.com/group/clojure-de) videos about Clojure in German (http://www.rheinjug.de/videos/ gse.lectures.app/Talk.html#Clojure) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 35 / 40

Sources of information (2) Books: Programming Clojure – 2009th, Clojure v. 1.0 Practical Clojure. The Deﬁnitive Guide – 2010th, Clojure, v. 1.2 The Joy of Clojure (beta) Clojure in Action (beta) Clojure Programming on Wikibooks Clojure Notes on RubyLearning Video-lectures & screencasts Clojure user groups around the world Clojure study courses (on-site in USA & Europe, or online) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 36 / 40

Examples. Hadoop Hadoop word count example is only lines of code (using clojure-hadoop), instead dozens for Java... All low-level details are handled by macros ( n s c l o j u r e −hadoop . e x a m p l e s . w o r d c o u n t 3 ( : import ( java . u t i l StringTokenizer )) ) ( d e f n my−map [ k e y v a l u e ] ( map ( f n [ t o k e n ] [ t o k e n 1 ] ) ( e n u m e r a t i o n −s e q ( S t r i n g T o k e n i z e r . v a l u e ) ) ) ) ( d e f n my−r e d u c e [ k e y v a l u e s −f n ] [ [ k e y ( r e d u c e + ( v a l u e s −f n ) ) ] ] ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 37 / 40

Examples. Cascalog Domain speciﬁc language to query data in Hadoop with joins, aggregates, custom operations, subqueries, etc. Interactive work with data, using Clojure Available from http://github.com/nathanmarz/cascalog Example: query for persons younger than 30: (?<− ( s t d o u t ) [ ? p e r s o n ? age ] ( age ? p e r s o n ? age ) (< ? age 3 0 ) ) Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 38 / 40

Examples. Concurrency Ants: 320 lines of code, with commentaries and documentation Massive parallel execution without explicit threading Many independent objects Graphical interface Cluster analysis with K-means algorithm: Each cluster represented by agent Automatically uses all available cores http://www.informatik.uni-ulm.de/ni/staff/HKestler/ Reisensburg2009/talks/kraus.pdf Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 39 / 40

Thank you Questions Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010 u 40 / 40

http://coderwall.com	13
http://paper.li	3
http://www.linkedin.com	2

Clojure talk at Münster JUG

by Alex Ott

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 18

Statistics

Clojure talk at Münster JUG Presentation Transcript