Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Infinite power at your fingertips with Microsoft Azure Cloud & ActiveEon

Joint talk Microsoft-ActiveEon at Cloud Expo Europe, Big Data Analytics and Cloud management theater. Presenters: Christopher Plieger, Microsoft Azure Product Marketing Manager, and Denis Caromel , CEO - ActiveEon

  • Login to see the comments

  • Be the first to like this

Infinite power at your fingertips with Microsoft Azure Cloud & ActiveEon

  1. 1. ✔ X ✔ ✔ X X ✔ ✔ ✔ ✔ X X ✔ X X ✔ X ✔ ✔ X X
  2. 2. Global Locations Partnerships Key information Management Denis Caromel, CEO François Tournesac, CSO Brian Amedro, CTO  Founded in 2007 by Denis Caromel in Sophia-Antipolis, Spin-off of INRIA  Addressing $80 Billion Hybrid Cloud Market with 27% CAGR  Disruptive Patented Technology w/ Exceptional Business Outcomes  60% of the revenue from international Sophia-Antipolis (France) Paris (France) London (United Kingdom) San-Jose (United States) Montreal (Canada) Fribourg (Switzerland) Dakar (Senegal) ProActive Solution Job Scheduling, Workload Automation Orchestration & Meta-Scheduling On-premises and on all clouds Open Source
  3. 3. REST APIsApp-specific Interfaces Integrated Web Portals ProActive Workflows Scheduling Orchestration Meta-scheduling Resource Allocation Big Data, Data Science, Third Party Software Local Scheduler Resource Manager Fault Tolerance Cloud bursting Resource agnostic Micro-service Etc. Workflow Automation Cloud Data Lake LSF Clusters
  4. 4. Legal & General $1,200,000 Azure Computing per Year • Utilize ActiveEon’s ProActive to distribute the load • Burst calculations into Azure • “Infinite capacity” reduce Time To Results
  5. 5. Legal & General CHALLENGES Azure Migration Resource allocation according to CPU and memory available Solvency II analysis on 2.5 million Monte Carlo scenarios RESULTS Optimized resource allocation Algorithm acceleration Shorter scenario analysis Scalable solution • Migration from private datacenter • Replacing old Tibco Datasynapse • Replacing old IBM Algobatch MAIN DRIVER REQUIREMENTS • Cloud capable • Dynamically define and prioritize workloads • Minimize time to delivery of results • Save resources COMPANY PROFILE • Industry: Insurance • Product: Compliance algorithm
  6. 6. Legal & General Time (hours) 105 160 1024Workers Profile Results Time (hours) 1024Workers Profile Results 105 140 I/O intensive tasks Aggregation & Reporting Low Priority CPU-Intensive tasks Risk Watch Simulation Medium Priority CPU-Intensive tasks Risk Watch Simulation High Priority CPU-Intensive tasks Risk Watch Simulation “More Value, Faster” Full computation without intermediate result High Priority Results Full Report 2H 5H Batch optimization with ProActive: • Enforce strong priorities • Optimal compact execution • Start tasks as early as possible • Pipeline and co-allocate • CPU-intensive with I/O intensive tasks 18H
  7. 7. End-to-end execution from 18h on-prem to 5h on Azure Legal & General with Azure Task monitoring with ProActive Automated grid start on Azure Life cycle management of Azure nodes Fault-Tolerance: this host died, and ProActive rescheduled the tasks executing on it, and routed Tasks around the faulty node afterwards Execution on 1 024 Azure nodes
  8. 8. CHALLENGES Multiple secured environments behind a firewall Scale to support Big Data needs Connect to various databases with specific RBAC RESULTS Secured communication between environments Shared resources for maximum performances • Allocation of resources across environments • Respect RBAC during compute time MAIN DRIVER REQUIREMENTS • Support use of Docker • Support Hadoop, Python, SAS, Spotfire, Greenplum • High availability COMPANY PROFILE • Industry: Government • Product: Data analysis for criminality reduction
  9. 9. Main Benefits • Central Orchestration Tool • Workflow expressiveness: Universal & Comprehensive • Management of Security for highly sensitive environments • Management of Resources for all appliances (SAS, GREENPLUM, TIBCO, …) Execution prod Analytical prod Staging Dev Virtualized Infrastructure using Docker 4 000 Physical Cores
  10. 10. PEPS: Sentinel Satellite Image Analysis ProActive task ProActive nodes
  11. 11. Data from Mining Machine Sensors Data Processing on Premises & in the Clouds Health and performance of machines Schedule data analytics hourly or on events Data Real-Time Control & Optimizations IoT Automation in the Cloud ActiveEon allowed to migrate from AWS to Azure
  12. 12. Scheduler Passive Main Benefits  Deployed On Premise (Capex) or on a Hosting Service (Opex)  Auto-scaling on infrastructure to match capacity and demand  Huge costs optimization using only the VMs needed and interruptible low cost instances (e.g. EC2 Spot instances)  Capacity to deliver the system on any third party Mediametrie customer TV Audience Measurement Scheduler Active EC2 Spot Instances Low costs EC2 Instances Regular costs IaaS On-Prem
  13. 13. Azure PoC in the Box Azure Node Source InfrastructureAzure Node Source Infrastructure Scale automatically Leverage Azure services Existing Resources Local & Network Resources Private Cloud, HPC & Others Resource Manager Workflow Scheduler </> Azure Node Source Infrastructure LSF Using Azure Scale Sets Deploy over 150 k nodes
  14. 14. 150 K Nodes Benchmarks on Azure
  15. 15. 150 K Nodes Benchmarks on Azure • Up to 150 000 Nodes with a single Scheduler instance on Azure BIG COMPUTE READY RESPONSIVE & RELIABLE FAST & SCALABLE • Average Response time of the scheduler: 6 ms (Min 4 ms, Max 38 ms) • Time to deploy Nodes: • 10K: 5 mn • Scale up to 40K: 10 mn • Scale up to 80K: 15 mn • Scale up to 150K: 25 mn (To be compared: Average Time to buy a 150-Cores Servers/Cluster: 1 Year!)