Practical Magic - Clojure forData ScientistsHi!Iʼm Bruce Durling.@otfrom on twitter.CommunityI help run the London Clojuri...
mastodonc.com mastodonc.blogspot.com @mastodoncI already haveR/SPSS/PSSP/MatLab/Excel• You can do most (all?) of it in clo...
Getting Startedproject.clj(defproject quiddswic “0.0.3”description "Quick and Dirty Data Science withIncanter and Clojure"...
Strings are not dates$map and date-clj conj-cols(def dts (incanter/dataset [:dts] (incanter/$map (fn [d](tformat/parse (:y...
mean(stats/mean (incanter/$ :beer s3))sorting(def sm-ord (incanter/$order :season :asc sm))So, Iʼve got a lot of numbersvi...
Help us at our monthly sprints!Join the london-clojurians google group to find out more.Thanks!Bruce Durling, @otfromCTO w...
Upcoming SlideShare
Loading in …5
×

Practical Magic with Incanter

5,378 views

Published on

Presentation by Bruce Durling CTO Mastodon C at Data Science London

Published in: Technology, Travel
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,378
On SlideShare
0
From Embeds
0
Number of Embeds
3,152
Actions
Shares
0
Downloads
4
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Practical Magic with Incanter

  1. 1. Practical Magic - Clojure forData ScientistsHi!Iʼm Bruce Durling.@otfrom on twitter.CommunityI help run the London Clojurians.And help with the• London Java Community• London Python Dojo• London Salesforce User Group.Upcoming• EuroClojure 24-25 May http://euroclojure.com Rich Hickey and Stuart Halloway KeynotesWorkIʼm the CTO of Mastodon C.Weʼre going to cut global CO2 by 1% by greenifying you hadoopjobs in the cloud.
  2. 2. mastodonc.com mastodonc.blogspot.com @mastodoncI already haveR/SPSS/PSSP/MatLab/Excel• You can do most (all?) of it in clojure• And you get prolog• And you get hadoop w/Datalog (Cascalog)• And you get Storm (realtime stats)• And you get Web Apps• And it is a free general purpose language• Lots of developers love it• But it is still beautiful and understandableSo, where should I start?Incanter to the rescueWhat is Incanter?http://www.incanter.org http://data-sorcery.org/http://github.com/liebke/incanter
  3. 3. Getting Startedproject.clj(defproject quiddswic “0.0.3”description "Quick and Dirty Data Science withIncanter and Clojure"dependencies [[org.clojure/clojure "1.3.0"][incanter “1.3.0”exclusions [swank-clojureorg.clojure/clojure org.clojure/clojure-contriborg.clojars.bmabey/congomongo congomongo]] [clj-time “0.3.3”exclusions [org.clojure/clojureorg.clojure/clojure-contrib]] [congomongo “0.1.7”exclusions [org.clojure/clojureorg.clojure/clojure-contrib]]])What is required?(require ʻ[clj-time.format :as tformat]) (require ʻ[clj-time.core :astime]) (require ʻ[clj-time.coerce :as coerce]) (require ʻ[incanter.core:as incanter]) (require ʻ[incanter.stats :as stats]) (requireʻ[incanter.charts :as charts]) (require ʻ[incanter.io :as io]) (useʻclojure.repl)How do I get data in?(def series (io/read-dataset “./beer.csv” :header true))
  4. 4. Strings are not dates$map and date-clj conj-cols(def dts (incanter/dataset [:dts] (incanter/$map (fn [d](tformat/parse (:year-month-day tformat/formatters) d)) [:date]series)))(def t (incanter/dataset [:millis] (incanter/$map (fn [d] (coerce/to-long d)) [:dts] dts)))(def s1 (incanter/conj-cols series dts t))(incanter/head s1)(incanter/view s1)Now what do I do with itHow about a seasonal means?(def dow (incanter/dataset [:dow] (incanter/$map (fn [d] (time/day-of-week d)) [:dts] s1)))(def s2 (incanter/conj-cols s1 dow))(def sm (incanter/col-names (incanter/$rollup (fn [coll] (stats/meancoll)) :beer :dow s2) [:season :mean]))(def sm-ord (incanter/$order :season :asc sm))(def s3 (incanter/$join [:season :dow] sm s2))(incanter/head s3)
  5. 5. mean(stats/mean (incanter/$ :beer s3))sorting(def sm-ord (incanter/$order :season :asc sm))So, Iʼve got a lot of numbersview(incanter/view s1)Plotting and Graphics(def ts-plot (charts/time-series-plot (incanter/$ :seconds s1)(incanter/$ :metric s1)))(incanter/view ts-plot)The Future• Make it faster (mostly remove reflection)• Integrate with frinj• Big Data (hadoop, datomic)
  6. 6. Help us at our monthly sprints!Join the london-clojurians google group to find out more.Thanks!Bruce Durling, @otfromCTO www.mastodonc.com @MastodonCEuroClojure 24-25 May: http://euroclojure.com  

×