Your SlideShare is downloading. ×
Cascalog présenté par Bertrand Dechoux
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Cascalog présenté par Bertrand Dechoux

1,459
views

Published on

Published in: Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,459
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. HUG France #1 - 15 Mars 2012 Cascalog des MapReduces concisBertrand DECHOUX@BertrandDechoux
    • 2. Casca... quoi? Map/Reduce : pertinent mais insuffisant Des alternatives : Pig, Hive... et Cascading ! Cascalog : Cascading + Datalog + Clojure
    • 3. Cascading : les concepts tuple : liste de valeurs nommées tap : source et sink de données ▶ par défaut : LFS et HFS ▶ par la communauté : Avro, HBase, JDBC, JSON, Kryo, Memcached, Membase, ElasticSearch, SimpleDB, Solr, Thrift... flow : tap(s) source + des opérations + tap(s) sink
    • 4. Cascading : les opérations (Cascading 1.2 - User Guide)
    • 5. Clojure un dialect de Lisp sur la JVM ▶ environnement connu ▶ interoperabilité avec Java ▶ homoiconicité ou code-as-data : fait pour les DSLs
    • 6. Clojure : syntaxe 1/2 (http://lisperati.com/syntax.html)
    • 7. Clojure : syntaxe 2/2 (http://lisperati.com/syntax.html)
    • 8. Cascalog : select ... from ... (?<- (stdout) [?person] (person ?person)) (?<- (stdout) [?person ?age] (age ?person ?age)) (?<- (stdout) [?age] (age _ ?age))
    • 9. Cascalog : select ... from ... where ... (?<- (stdout) [?person ?age] (age ?person ?age) (< ?age 30))
    • 10. Cascalog : select ... as ... from ... (?<- (stdout) [?person ?junior] (age ?person ?age) (< ?age 30 :> ?junior))
    • 11. Cascalog : select count(*) from ... group by ... (?<- (stdout) [?count] (age _ _) (c/count ?count)) (?<- (stdout) [?junior ?count] (age _ ?age) (< ?age 30 :> ?junior) (c/count ?count)) disponible : count, sum, min, max
    • 12. Cascalog : select ... from ... join ... (?<- (stdout) [?person ?age ?gender] (age ?person ?age) (gender ?person ?gender))
    • 13. Cascalog : select ... from ... (select ...) (let [many-follows (<- [?person] (follows ?person _) (c/count ?count) (> ?count 2))] (?<- (stdout) [?personA ?personB] (many-follows ?personA) (many-follows ?personB) (follows ?personA ?personB)))
    • 14. Cascalog : définir vos fonctions (defn toUpperCase [person] (.toUpperCase person)) (?<- (stdout) [?PERSON] (person ?person) (toUpperCase ?person :> ?PERSON))
    • 15. Plus d’informations? Cascalog ▶ github ▶ wiki ▶ google group  Clojure et cie ▶ midje, framework de test ▶ Pallet, DevOps pour la JVM
    • 16. #1 HUG France 15 Mars 2012 Cascalog Merci pour votre attentionBertrand DECHOUX@BertrandDechoux