Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kamal Hakimzadeh – Reproducible Distributed Experiments


Published on

Flink Forward 2015

Published in: Technology
  • Dating direct: ♥♥♥ ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❤❤❤ ❤❤❤
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Kamal Hakimzadeh – Reproducible Distributed Experiments

  1. 1. Reproducible Distributed Experiments Kamal Hakimzadeh PhD Student 1 Jim Dowling Associate Professor
  2. 2. Agenda 2 • Motivation • Reproducibility • Demo: Simple experiment 30-40 min • Karamel Rep. • Karamel Engine • Orchestration • Challenges
  3. 3. Motivation 3 Analytical vs Empirical proof DS supports many scientific advancements Scheduling, fault tolerant, scalability … Extremely complex
  4. 4. Reproducible vs. Replicable 4 1. Laboratory 2. Experimenter 3. Apparatus Reproducible Replicable Computational Reproducibility: Infrastructure, software, experiment and data
  5. 5. Demo : Word Count 5 Text Generator Text Generator Text Generator Word Count
  6. 6. Karamel: Rep. in different layers 6 Bare Metal Google Compute Engine Virtual Machine is and abstract entity Software is defined in Chef It is publicly available in Github
  7. 7. Karamel Engine 7 DSL Service Cloud Clients Karamel Engine Physical Mapping Orchestrator
  8. 8. Orchestration – queuing model
  9. 9. Result
  10. 10. Challenges and future work 10 Scalability Fault Recovery Model Elasticity – Handle Churn Instrumentation Recommendation System Language Support Load generators Scheduling Container base machines Result Management Debugging
  11. 11. Team members 11 Kamal Hakimzadeh PhD Student at KTH Alberto Lorente Leal Software Developer at Comeon Jim Dowling Associate Professor at KTH Hooman Peiro Sajjad PhD Student at KTH Abhimanyu Babbar Backend Developer at Wrap
  12. 12.