Technology in support of utilities challenges


Published on

Oracle Technology to support Smart Grid, New Challenges for Utilities

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 16/09 – replaced ‘clients’ by ‘customers’ everywhere (telcos jargon)
  • Integrationisbroad and functionallycomplex(much more thanthishighlevelviewwhichalready shows somelevel of complexity)Questions to Iberdrola:Is your system integrated? With WMS? With AM? With CIS? How?How do you set up the data in the HES and MDM?How do you commission meterstoday?
  • BI adaptadoparatrabajar en memoria.In-memory Cache: decide quédatoscachearTimesten – DB generalEssbase – analítica multidimensional, predicciones, en memoria.Interfaces de conexiónnInfiniband: HA con otrosExalytics o paraExadata
  • Exalytics & Exadata: plataforma de BI más rápida del mundo.
  • Hadoop: was inspired by Google's MapReduce and Google File System (GFS) papers.Apache Cassandra: was developed at Facebook to power their Inbox Search feature (it is a columnar nosql DB)Cloudera: . It contributes to Hadoop and related Apache projects and provides a distribution for Hadoop for the enterpriseHive: DW on top of HadoopHbase: Nosql (columnar) – based on Googles BigtableVoldemort is a distributed data store that is designed as a key-value store used by LinkedIn for high-scalability storage (NoSql key value)
  • Business intelligence will be also extremely challenged: more info then ever, more complexity than before, and more operationality than before (less for off line BI, more participating to operational processes).BI is key to provide cohesive view of the world : assets, crews, outages, and take the right decisionsAll components will need to support this.
  • Qué pasa con los datos? va a existir muchísima necesidad de Integración de Datos, Distribución Vs Comercial, movimiento de ficheros, contingencia,…GG: Réplica casi en tiempo real, no intrusiva, heterogénea.
  • CEP for me focusing on criticalevent drivent near real time smart grids scenarios-Night Expend > 10 KW-average # of Elect. Power expended in the last 24 hours-Outage A, B and C occurred within 30 minute window
  • Technology in support of utilities challenges

    1. 1. <Insert Picture Here>Technology for Utilities Transformation and newChallenges: Integration, Big Data, ExtremePerformance.Aitor IbañezEnterprise Architect for Utilities
    2. 2. Contents <Insert Picture Here>• Utilities Business Transformation• Need for Technology• Extreme Performance• No-limit Integration• Applications: Architectural Challenges 2
    3. 3. <Insert Picture Here>Utilities BusinessTransformation 3
    4. 4. Data Growth defines new Challenges… DATA GROWTH IMPACT IT CHALLENGES • Multiple Devices: • Agility for Business Meters, Apps, SCAD Processes on As,… Transactions between Devices and Applications • Info Processed by multiple • New Scalable Departments: Risk Architecture supporting Mgmt., Billing, Dema new Analysis and nd Transaction Growth Analysis, Network. Reqs. • Growth of Corporate • New Data Mgmt Reqs. DW for Reports and for Storage, Master Trend Analysis. Data, Synchronization & Sharing of Information Fuente: Accenture “Challenges for Utilities” 2010, Oracle Insight4 4
    5. 5. Challenges that arise new Biz & IT Needs Business Perspective IT Perspective External Pressure Strategy Levers Implications Managing efficiently Scalable growing Data Volumes Infrastructure Managing Data Explosion Improving Customer Business Processes without cost increments Care & Go-to-market Processes Automation Growing SalesGrowing Market Real-time Data Improving Availability Access, Load Share balancingDefense against Improving Competency Performance and On-demand security of growing, Business Cost Savings Transactions Continuity Reducing Operational Simplifying Storage Costs and improving Energy Efficiency and Servers Faster processing to improve Billing, Demand More Power Analysis, … Utilities need New Technology to face up New Challenges 5 5
    6. 6. … Needs translated into new Requirements • Scalable Infrastructure • Horizontal growing Performance • Business Processes Automation • Faster deployment of processes • Process Governance • Transactions • Real-time Data Access, Load Balancing Protecting • Fast response to data needs Investment • Grid computing • On-demand growing, Business continuity • Right-sizing, growing infrastructure as the Business demands • SLAs Compliance Scalability • Simplifying Storage and Servers • Reducing HW Installed base: best platform for every purpose • Energy Savings • More Power • Unprecedented Processing Power Manageability • Driving new Business opportunities6 6
    7. 7. <Insert Picture Here>Need for Technology 7
    8. 8. Smart Grid Challenges Managing the Exponential Growth of Data Volumes Much more data… Much more processing… Break Point Cost System Cost inviable Processing Power required for Interval based billingSource: EPRI Scalable Scalability break points: System storage, scalability cost Meters 8
    9. 9. Integration beyond BoundariesResponding to events and information from avariety of sources 9
    10. 10. So… Need for Technology!• What do we need? Extreme Performance Engineered Massive Data Volume Systems Unstructured Data Analysis No-Limit Integration Data Integration Biz Processes Automation Services Architecture BPM, SOA, EDA Events Processing 10
    11. 11. <Insert Picture Here>Extreme Performance 11
    12. 12. Exadata Database Machine • Scalable Grid of industry standard servers for Compute and Storage • Eliminates long-standing tradeoff between Scalability, Availability, Cost Database Grid Intelligent Storage Grid • 8 Dual-processor x64 • 14 High-performance low-cost database servers storage servers OR • 2 Eight-processor x64 • 100 TB High Speed disk, or database servers 336 TB High Capacity disk • 5.3 TB PCI Flash InfiniBand Network • Data mirrored across storage • Redundant 40Gb/s switches servers • Unified server & storage networkCopyright © 2010, Oracle Corporation and/or its affiliates – 12 – 12
    13. 13. Traditional Vs Smart Scan Processing Traditional Scan processing Exadata Smart Scan processing  SELECT  customer_name SELECT  FROM calls customer_name Rows Returned Rows Returned WHERE amount > FROM calls 200;WHERE amount > 200;   DB Host reduces  Consolidated terabyte of data to 1000 Smart Scan Result Set  Constructed And customer names that Built From All Table Sent To Cells Cells are returned to client Extents Identified  Smart Scan    identifies rows and 2MB of data I/Os Issued I/Os Executed: columns within returned to server 1 terabyte of data terabyte table that returned to hosts match request© 2008 Oracle Corporation – Proprietary and Confidential – 13 – 13
    14. 14. Exadata Storage Software Unique Features • Exadata Smart Scans • Exadata Smart Flash Cache • Breaks random I/O bottleneck by • 10X or greater reduction in data sent increasing IOPs by up to 20X to database servers • Doubles user data scan bandwidths • Exadata Storage Indexes • Eliminate unnecessary I/Os • I/O Resource Manager (IORM) • Enables storage grid by prioritizing I/Os to ensure predictable • Hybrid Columnar Compression performance • Efficient compression increases effective storage capacity and • Quality of Service (QoS) increases user data scan bandwidths by a factor of up to 10X • Actively meet and maintain SLAs • Memory Guard to protect existing current transactions from memory- based failuresCopyright © 2010, Oracle Corporation and/or its affiliates – 14 – 14
    15. 15. Proof of Concepts Results Referencia “As IS” Exadata Optimizacion 1500 1000 500 0 "01" "07" "OF " Queries 47 veces más 68 veces más 60 veces más 2 veces más rápido rápido rápido rápido 15
    16. 16. Oracle Exalogic Elastic Cloud Application Grid, Integrated Compute, I/O, Networking and Storage Integrated Storage WebLogic • Central storage for Coherence Server system images • Clustered for HA • 40 TB SAS diskEnterprise Manager JRockit and HotSpot Tuxedo • 4 TB read cache • 72 GB write cache Exalogic Elastic Cloud Software I/O Fabric Oracle Linux and/or Solaris • 40 Gb/sec links • 10 Gigabit Ethernet connectivity to Exalogic Elastic Cloud Hardware EL X2-2 datacenter X2-2 EL X2-2 • 30 x86 compute nodes • 360 Xeon cores (2.93 GHz) • 2.8 TB DRAM • 960 GB SSD Copyright © 2011 Oracle Corporation - Proprietary and Confidential 16
    17. 17. Oracle Exalogic and Exadata Together Designed for Seamless Interoperability, Unmatched Performance 960 Gigabits/second (Maximum: 24x InfiniBand QDR) • Exclusive: direct InfiniBand integration! • Extreme performance, reliability, security • Simple to deploy and manageExalogic Exadata Up to 3X OLTP Performance Active GridLink for RAC • Resource-aware load balancing • Instantaneous connection failover • Transaction affinity to RAC nodes • SQLnet optimized for InfiniBand SDPCopyright © 2011 Oracle Corporation - Proprietary and Confidential 17
    18. 18. Technologies in SupportExaMDM Concept: Technical Architecture 10 M Meters Benchmark • ½ Exadata + ½ Exalogic + Oracle MDM + CC&B • 500.000 bills  1 hour • Daily meter reads upload  2,6 hours • VEE processing  4 hours • Usage Aggregations  30 min. • Daily Total: 8 hours. • Scale: ½  Full ExaMDM = 4 hours 18
    19. 19. Oracle Exalytics – Under the Hood TimesTen for Exalytics 1 TB RAM Essbase 40 Processing Cores InfiniBand High Speed Networking Adaptive In-Memory Tools In-Memory In-Memory Oracle Business Intelligence Foundation Suite Analytics Software Analytics Hardware Speed-of-thought enterprise Extreme performance Business Intelligence platform via ExadataFor information purposes only, not a commitment to the development, release, and timing of any features or functionality. 19
    20. 20. Oracle ExalyticsExtreme Performance for Enterprise Analytics• Speed-of-Thought Analysis at Enterprise Scale • 20x better response times • Sub-second performance enables speed-of-thought analytics • Grow to 50,000 enterprise users on one appliance• What-if Analysis and Forecasting • 16x better response times • More accurate – use more detail, include more scope, evaluate wider operations • More scale – more users and higher volumes at busy periods • Greater agility – do daily instead of monthly• Enterprise Planning • Reduce planning cycle times with 5x better response time and 6x better throughput • Run more planning models with finer grained operational detail to improve accuracy • Extend beyond Finance to every LOB, with 10,000 Planning users on one appliance 20
    21. 21. Big Data for Smart Grid Analysis from unstructured data• Unstructured data coming from • Smart Meters & Devices, Domotic,… • Info coming from Network & Generation SCADAs • Substations, other equipments,… • Meteo Data.• BigData Appliance • 18 nodes, 216 CPU cores. • 864 GB RAM, 648 TB storage • 40 Gb/s Infiniband• Extreme Performance for Unstructured Data processing. 21
    22. 22. Big Data for Smart Grid Think about the Future…• Driving Immediate Analysis with unprecedented processing power • Social Networks: mentions [ourCompany] • Sentimental Analysis, Complaints • Social Networks: mentions #Meter, [ourCompany], #power  Trending Topics. • Targeted programs for each consumer • Understanding Customer: CRM, Meter load, Internet logs, Twitter,… • Power Demand Planning: processing trends and actual consumption by geography/concentrator. • Prediction of Network failures or bottlenecks. • Geographical & Meteo Data 22
    23. 23. Divided Solution Spectrum Data VarietyUnstructured Distributed NoSQL File Systems Flexible MapReduce Specialized Solutions Developer Transaction (Key-Value) Centric Stores SQL DBMS ETL DBMS Advanced Trusted (DW) Analytics Secure Schema (OLTP) Administered Information Density Acquire Organize Analyze 23
    24. 24. Oracle’s Big Data Solution Data Variety Big Data ApplianceUnstructured HDFS• Hadoop Hadoop • NoSQL Database In-DB • Oracle Loader for hadoop Loader Analytics Oracle • Oracle Data Integrator Hadoop Oracle for “R” Exalytics NoSQL DB Mining • Speed of Text Thought Oracle Graph Analytics Data Integrator Spatial Oracle Exadata • OLTP & DW Oracle Oracle Database Database • Data Mining & Oracle R (DW) Oracle (OLTP) BI EE Schema • Semantics • Spatial Information Density Acquire Organize Analyze 24
    25. 25. Smart Metering Architecture DecisionsBig Data Business Intelligence SOA Framework Middleware & Bus. Orchestration Services AMI Self Service / portal Work & Asset Mobile meter Head-End NRT MDM CIS TBs Distribution management Outage Management 25
    26. 26. <Insert Picture Here>No-Limit Integration 26
    27. 27. Oracle GoldenGate The Solution for Enterprise-wide Real Time Data Needs Database and Mission Criticalapplications, Mixed Applications &sources, distributed Real-time Data, Business systems, legacy, information Intelligence, Reporti OLTP, OLAP ng for Customers, Partner s & Employees Real-time Access Oracle GoldenGate delivers real-time access of real-time information, enabling companies to dramatically improve the availability, reliability, and performance of critical data across enterprise systems. 27
    28. 28. Oracle Data Integrator Data Integrator• Data Integration Solution E-LT Transformation Realtime Data Business Enterprise SOA | BAM | Hubs Intelligence Warehouse CEP• Declarative Design• No-intrusive• Change Data Capture Batch Data Flow Oracle Data Integrator Real Time Data• Architecture hot- Flow Services changes E-LT Events• Multi-DB support: CDC Knowledge modules Logs Transactional Logs Logs CDC CDC RDBMS 28
    29. 29. Supply Real Time Business IntelligenceNon-invasive Capture + E-LT ProcessingApplication Real Time BI Analytic BI (using Data Copy) (Facts & Dims) Consistency Window E-LT (Mini-Batch + Transforms) 29
    30. 30. <Insert Picture Here>Applications:Architectural Challenges 30
    31. 31. Technologies in Support Oracle Integration Platform for Utilities AIA Foundation Pack for UtilitiesObjective: Reference Process ModelProvide a comprehensive, consistent, and flexibleintegration approach across the Utilities SuiteApproach: Industry Common Object Model & Services • Built on Oracle’s Fusion Middleware Standard s • Leveraging AIA Methodology • Expand AIA Foundation Pack for Utilities: –Common Objects and Enterprise Services –Compliant with utility industry standards –Deliver best-practice process models for key processes across utility segments (water, gas, electric, waste) Oracle Utilities Integrated Solution Suite - WAM MWM ML / AMI CC&B CRMBenefits: • Provide a common integration approach across SOA Suite, AIA (ESB + FP for CIM and MultiSpeak) all Oracle applications. • Accelerate deployments and provide flexibility NMS SCADA QM MDM GIS / Spatial Model for project-specific extensions. 31
    32. 32. Oracle Utilities Integration Strategy Oracle AIA Integrations Benefits $ Value Open Ecosystem Pre-built integrations  AIA Centers of using leading Oracle Excellence Fusion middleware Extensibility Methodology  AIA Partner Co-  Lower TCO  Loose coupling of apps development  Industry best Supported products through SOA practices  UGBU Partners to  Designed for next  Focus on  Prescriptive certify with SAP + generation apps configurations not other 3rd Parties rebuilds methodology compatibility  Successfully deployed  Industry standards  Reusability adoption such as in complex, high- volume environments CIM for Utilities  Proven by Oracle 32
    33. 33. Business Process Management Processes Life Cycle Business Analyst Process Developer Process Modeling, simulating and Documentation Process Development Process and Systems Integration Execution and ManagementBusiness Owner Users Interaction Business Participants Corporate Applications Dashboards, tools and historical And Databases trends 33
    34. 34. Oracle Technologies in support Event-Driven Architecture Event Sources Response Services event Complex Event Streams ProcessorCoherence eventReal Time • Event sourcingListeners • Continuous query language to event define event BAM Dashboards • Correlation Applications • Sequencing event • Window of time evaluationDatabases • Pattern matching event • Event publishing BAM Alerts • Real time Java infrastructure BPM Processes BPM/BPEL Processes JMS closed loop Messaging 34
    35. 35. Business Impact of Events, Oracle BAM Meaningful, Event-driven Intelligence for End-Users Monitor business processes & services in real-time  Key Performance Indicators (KPIs)  Service-Level Agreements (SLAs) Analyze events as they occur  Correlate events & KPIs  Identify trends as they emerge  Alert users to bottlenecks & solutions Act on current conditions  Event-driven alerts  Real-time dashboards  BPEL processes & web services integration 35
    36. 36. Questions? 36