Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@Xebiconfr #Xebicon18 @scauglog
Build the future
Des notebooks pour le monitoring avec Zeppelin
Romain Sagean, Développeur...
@Xebiconfr #Xebicon18 @scauglog
Qui suis-je ?
Scauglog (Romain Sagean)
Développeur
Xebia
@scauglog
2
@Xebiconfr #Xebicon18 @scauglog
Let’s Monitor
the performance of your machine learning model
3
@Xebiconfr #Xebicon18 @scauglog
Monitor?
● once a day
● mean errors
● sell
● data are on HDFS, SQL,
Cassandra
● reality an...
@Xebiconfr #Xebicon18 @scauglog
Why not Jupyter?
5
@Xebiconfr #Xebicon18 @scauglog
Jupyter?
● notebook
● language oriented
● no database connection
● can’t hide code
● no se...
@Xebiconfr #Xebicon18 @scauglog
What about kibana?
7
@Xebiconfr #Xebicon18 @scauglog
ELK?
● near real time monitoring
● too much stuff to install
● need creation of pipeline t...
@Xebiconfr #Xebicon18 @scauglog
Zeppelin???
9
@Xebiconfr #Xebicon18 @scauglog
Zeppelin?
● big data notebook
● apache project
● written in Java (nobody is perfect)
● Bac...
@Xebiconfr #Xebicon18 @scauglog 11
Demo
@Xebiconfr #Xebicon18 @scauglog
There is an interpreter
for that.
● Spark Scala, Python, R
● JDBC
● Markdown
● Shell
● Ang...
@Xebiconfr #Xebicon18 @scauglog
Interpreter
● Auto shutdown
● shared, scoped, isolated
● spark context is shared among
lan...
@Xebiconfr #Xebicon18 @scauglog
Dashboarding
1. mix language
○ JDBC
○ Scala
○ Cassandra
14
@Xebiconfr #Xebicon18 @scauglog
Dashboarding
1. mix language
2. choose your dataviz
○ table
○ bar
○ line
○ create your own...
@Xebiconfr #Xebicon18 @scauglog
Dashboarding
1. mix language
2. choose your dataviz
3. hide code
16
@Xebiconfr #Xebicon18 @scauglog
Dashboarding
1. mix language
2. choose your dataviz
3. hide code
4. create form
17
@Xebiconfr #Xebicon18 @scauglog
Production
1. schedule
18
@Xebiconfr #Xebicon18 @scauglog
Production
1. schedule
2. REST API
19
@Xebiconfr #Xebicon18 @scauglog
Production
1. schedule
2. REST API
3. Secure
○ login
20
@Xebiconfr #Xebicon18 @scauglog
Production
1. schedule
2. REST API
3. Secure
○ login
○ manage interpreter
credential
21
@Xebiconfr #Xebicon18 @scauglog 22
@Xebiconfr #Xebicon18 @scauglog
Thank you
Any questions?
23
Upcoming SlideShare
Loading in …5
×

XebiCon'18 - Des notebook pour le monitoring avec Zeppelin

97 views

Published on

Depuis quelques années, les notebook ont gagné en popularité. Apache Zeppelin, avec son architecture modulaire, offre une solution de notebook et de dashboarding, le tout sans mettre de côté la sécurité. Cette présentation vous fera découvrir cet outil merveilleux dont vous ne pourrez bientôt plus vous passer.

Par Romain Sagean, Data Engineer chez Xebia

Toutes les informations sur xebicon.fr

Published in: Technology
  • Be the first to comment

  • Be the first to like this

XebiCon'18 - Des notebook pour le monitoring avec Zeppelin

  1. 1. @Xebiconfr #Xebicon18 @scauglog Build the future Des notebooks pour le monitoring avec Zeppelin Romain Sagean, Développeur Xebia 1
  2. 2. @Xebiconfr #Xebicon18 @scauglog Qui suis-je ? Scauglog (Romain Sagean) Développeur Xebia @scauglog 2
  3. 3. @Xebiconfr #Xebicon18 @scauglog Let’s Monitor the performance of your machine learning model 3
  4. 4. @Xebiconfr #Xebicon18 @scauglog Monitor? ● once a day ● mean errors ● sell ● data are on HDFS, SQL, Cassandra ● reality and prediction are in separate file ● result are for dev and business 4
  5. 5. @Xebiconfr #Xebicon18 @scauglog Why not Jupyter? 5
  6. 6. @Xebiconfr #Xebicon18 @scauglog Jupyter? ● notebook ● language oriented ● no database connection ● can’t hide code ● no security out of the box ● no auto refresh ● doesn’t free ressource after use 6
  7. 7. @Xebiconfr #Xebicon18 @scauglog What about kibana? 7
  8. 8. @Xebiconfr #Xebicon18 @scauglog ELK? ● near real time monitoring ● too much stuff to install ● need creation of pipeline to feed the dashboard ● not enough flexibility 8
  9. 9. @Xebiconfr #Xebicon18 @scauglog Zeppelin??? 9
  10. 10. @Xebiconfr #Xebicon18 @scauglog Zeppelin? ● big data notebook ● apache project ● written in Java (nobody is perfect) ● Backed by Hortonworks Engineer ● v0.8.0 ● shipped with HDP ● designed for Big Data 10
  11. 11. @Xebiconfr #Xebicon18 @scauglog 11 Demo
  12. 12. @Xebiconfr #Xebicon18 @scauglog There is an interpreter for that. ● Spark Scala, Python, R ● JDBC ● Markdown ● Shell ● Angular ● Many More ● Create Your own interpreter 12
  13. 13. @Xebiconfr #Xebicon18 @scauglog Interpreter ● Auto shutdown ● shared, scoped, isolated ● spark context is shared among language ● Configure interpreter ○ library ○ multiple spark conf 13
  14. 14. @Xebiconfr #Xebicon18 @scauglog Dashboarding 1. mix language ○ JDBC ○ Scala ○ Cassandra 14
  15. 15. @Xebiconfr #Xebicon18 @scauglog Dashboarding 1. mix language 2. choose your dataviz ○ table ○ bar ○ line ○ create your own dataviz 15
  16. 16. @Xebiconfr #Xebicon18 @scauglog Dashboarding 1. mix language 2. choose your dataviz 3. hide code 16
  17. 17. @Xebiconfr #Xebicon18 @scauglog Dashboarding 1. mix language 2. choose your dataviz 3. hide code 4. create form 17
  18. 18. @Xebiconfr #Xebicon18 @scauglog Production 1. schedule 18
  19. 19. @Xebiconfr #Xebicon18 @scauglog Production 1. schedule 2. REST API 19
  20. 20. @Xebiconfr #Xebicon18 @scauglog Production 1. schedule 2. REST API 3. Secure ○ login 20
  21. 21. @Xebiconfr #Xebicon18 @scauglog Production 1. schedule 2. REST API 3. Secure ○ login ○ manage interpreter credential 21
  22. 22. @Xebiconfr #Xebicon18 @scauglog 22
  23. 23. @Xebiconfr #Xebicon18 @scauglog Thank you Any questions? 23

×