Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data Analytics at Vestas Wind Systems


Published on

Présentation "Know the wind" - Big Data Analytics - Smarter Analytics Live - 5 juin 2012 - par Anders Rhod Gregersen - Vestas

Published in: Business, Technology
  • Be the first to comment

Big Data Analytics at Vestas Wind Systems

  1. 1. Know the windBig data analytics at Vestas Wind Systems A/SAnders Rhod Gregersenagreg@vestas.comSenior specialist, Vestas Wind Systems A/S5th of June 2012, Anders Rhod Gregersen, Vestas Wind Systems A/S
  2. 2. Presentation Outline Introduction• Introduction to Vestas What are we trying to achieve?• Finding a good site and HPC How do we do it? Q&A• Forecasting• Usability• Building HPC capability• Road ahead2 Vestas Wind Systems A/S
  3. 3. Marked leader in Wind turbines Wind only company Install base approx. 50GW / 50.000 wind turbines 22.000 employees World wide3 Vestas Wind Systems A/S
  4. 4. Why high performance data heavy computing?4 Vestas Wind Systems A/S
  5. 5. Why HPC and Big data?Renewable vs base production • Predictability • IntegrationSignificant investmentWhat matters to the customer?5 Vestas Wind Systems A/S
  6. 6. The complexity of business case certainty Cost of Energy Wind Resource Service Cost Turbulence Complexity Height contours Site picture6 Vestas Wind Systems A/S
  7. 7. Finding a good siteTraditional process:Point measurements (met mast)Point estimate of wind resourcePoint estimate of turbulenceDrawbacks:CostlyTime consumingPoint measurement of flowNo weather context7 Vestas Wind Systems A/S
  8. 8. Understand wind ressourceHindcasting the weatherVery high area resolutionVery high time resolutionUnlike weather services, the model is sensitive to windTime series from turn of millenium onwardsApprox. 200 parametersWind measurements in contextExtreme events8 Vestas Wind Systems A/S
  9. 9. Understand wind turbulence Complex vs simple terrain Turbulence and fattigue Point measurement of turbulence Modeling via Computational Fluid Dynamics Moving from point to flow Wind shear In-flow angles9 Vestas Wind Systems A/S
  10. 10. Flow over terrain10 Vestas Wind Systems A/S
  11. 11. Architecture11 Vestas Wind Systems A/S
  12. 12. Architecture overviewCompute:1.306 compute nodes2.612 Intel X5670 CPUs15.672 X5670 cores>34TiB RAM>161TFLOPS10.752 M2070Q coresInterconnects:4xQDR Infiniband (40Gb)2:1 over-subscribedEthernetStorage:1.680 spindles14 I/O servers>20GB/s12 Vestas Wind Systems A/S
  13. 13. Primary memory Modeling fluid flow means Solving Navier-Stokes equeations which means Memory bound code On die memory controller One bank, three ranks/CPU Remote memory via Message passing interface (MPI) MPI via Infiniband Low latency / high bandwidth13 Vestas Wind Systems A/S
  14. 14. Secondary memory GPFS – general parallel filesystem Freedom in both bandwidth and capacity Singular name space I/O moves via Infiniband Exposed to the OS as POSIX Backup vs snapshots14 Vestas Wind Systems A/S
  15. 15. Analytics at Vestas15 Vestas Wind Systems A/S
  16. 16. Building database technology that scales beyond petabytesUse cases:Point queries: user wants the full time series for the four adjacent pointsData exploration: meteorologist queries region or world for existence or frequency of phenomenonForensics: Engineer is doing a post-mortem on high frequency data from a wind turbineRegular database/warehouse has problems scaling (conceptually/economically)16 Vestas Wind Systems A/S
  17. 17. Business challengeWhat we have200 parametersHourly measurementsTime series since 2000What we need:Relational database functionalityFast point queriesScalable queries17 Vestas Wind Systems A/S
  18. 18. Facts:Nature of data:Volume, VelocityAssumption, all data time seriesAssumption, common query full time seriesOptimization: partition eliminationOptimzation: parameter eliminationSolution: a column based18 Vestas Wind Systems A/S
  19. 19. Enter IBM InfoSphere BigInsightsPartnership with IBMJoint development withIBM Almaden (research) andIBM Silicon Valley Lab (productization)Weekly meetings19 Vestas Wind Systems A/S
  20. 20. Advantages of a partnership:Software developed that matches our needSoftware lifecycle in handled by IBMSoftware productization20 Vestas Wind Systems A/S
  21. 21. Thank you for your attentionCopyright NoticeThe documents are created by Vestas Wind Systems A/S and contain copyrighted material, trademarks, and other proprietary information. All rights reserved. No part of the documents may be reproduced or copied in any form or by anymeans - such as graphic, electronic, or mechanical, including photocopying, taping, or information storage and retrieval systems without the prior written permission of Vestas Wind Systems A/S. The use of these documents by you, oranyone else authorized by you, is prohibited unless specifically permitted by Vestas Wind Systems A/S. You may not alter or remove any trademark, copyright or other notice from the documents. The documents are provided “as is” andVestas Wind Systems A/S shall not have any responsibility or liability whatsoever for the results of use of the documents by you.
  22. 22. Usability challengeHPC grew out of academiaWhat to expect from users?Users range from PhDs ….to saleforceOne common tool for common users ….enables tracabilityPoint and click HPC for regular usersTerminal based interaction for the power user22 Vestas Wind Systems A/S
  23. 23. Building up HPC capabilityQ1/2003First commercial license for CFDQ4/2006First cluster (40 cores)Q3/2007CFD model validated.Q4/2008Second cluster (15 TFLOPS, TiBs)Q2/2011Third cluster (3rd largest industrial HPC, PiBs)23 Vestas Wind Systems A/S