• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Practical Magic with Incanter

Practical Magic with Incanter



Presentation by Bruce Durling CTO Mastodon C at Data Science London

Presentation by Bruce Durling CTO Mastodon C at Data Science London



Total Views
Views on SlideShare
Embed Views



1 Embed 2,665

http://datasciencelondon.org 2665


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Practical Magic with Incanter Practical Magic with Incanter Document Transcript

    • Practical Magic - Clojure forData ScientistsHi!Iʼm Bruce Durling.@otfrom on twitter.CommunityI help run the London Clojurians.And help with the• London Java Community• London Python Dojo• London Salesforce User Group.Upcoming• EuroClojure 24-25 May http://euroclojure.com Rich Hickey and Stuart Halloway KeynotesWorkIʼm the CTO of Mastodon C.Weʼre going to cut global CO2 by 1% by greenifying you hadoopjobs in the cloud.
    • mastodonc.com mastodonc.blogspot.com @mastodoncI already haveR/SPSS/PSSP/MatLab/Excel• You can do most (all?) of it in clojure• And you get prolog• And you get hadoop w/Datalog (Cascalog)• And you get Storm (realtime stats)• And you get Web Apps• And it is a free general purpose language• Lots of developers love it• But it is still beautiful and understandableSo, where should I start?Incanter to the rescueWhat is Incanter?http://www.incanter.org http://data-sorcery.org/http://github.com/liebke/incanter
    • Getting Startedproject.clj(defproject quiddswic “0.0.3”description "Quick and Dirty Data Science withIncanter and Clojure"dependencies [[org.clojure/clojure "1.3.0"][incanter “1.3.0”exclusions [swank-clojureorg.clojure/clojure org.clojure/clojure-contriborg.clojars.bmabey/congomongo congomongo]] [clj-time “0.3.3”exclusions [org.clojure/clojureorg.clojure/clojure-contrib]] [congomongo “0.1.7”exclusions [org.clojure/clojureorg.clojure/clojure-contrib]]])What is required?(require ʻ[clj-time.format :as tformat]) (require ʻ[clj-time.core :astime]) (require ʻ[clj-time.coerce :as coerce]) (require ʻ[incanter.core:as incanter]) (require ʻ[incanter.stats :as stats]) (requireʻ[incanter.charts :as charts]) (require ʻ[incanter.io :as io]) (useʻclojure.repl)How do I get data in?(def series (io/read-dataset “./beer.csv” :header true))
    • Strings are not dates$map and date-clj conj-cols(def dts (incanter/dataset [:dts] (incanter/$map (fn [d](tformat/parse (:year-month-day tformat/formatters) d)) [:date]series)))(def t (incanter/dataset [:millis] (incanter/$map (fn [d] (coerce/to-long d)) [:dts] dts)))(def s1 (incanter/conj-cols series dts t))(incanter/head s1)(incanter/view s1)Now what do I do with itHow about a seasonal means?(def dow (incanter/dataset [:dow] (incanter/$map (fn [d] (time/day-of-week d)) [:dts] s1)))(def s2 (incanter/conj-cols s1 dow))(def sm (incanter/col-names (incanter/$rollup (fn [coll] (stats/meancoll)) :beer :dow s2) [:season :mean]))(def sm-ord (incanter/$order :season :asc sm))(def s3 (incanter/$join [:season :dow] sm s2))(incanter/head s3)
    • mean(stats/mean (incanter/$ :beer s3))sorting(def sm-ord (incanter/$order :season :asc sm))So, Iʼve got a lot of numbersview(incanter/view s1)Plotting and Graphics(def ts-plot (charts/time-series-plot (incanter/$ :seconds s1)(incanter/$ :metric s1)))(incanter/view ts-plot)The Future• Make it faster (mostly remove reflection)• Integrate with frinj• Big Data (hadoop, datomic)
    • Help us at our monthly sprints!Join the london-clojurians google group to find out more.Thanks!Bruce Durling, @otfromCTO www.mastodonc.com @MastodonCEuroClojure 24-25 May: http://euroclojure.com