Downloaded 79 times
Uploaded byVoltDB
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
The document discusses building an IoT architecture using HPE Vertica and VoltDB, highlighting the potential economic impact of IoT, which is estimated to reach $11 trillion annually. It covers the importance of real-time data processing at the edge for efficient decision-making and analytics, showcasing VoltDB's capabilities in handling high-velocity data and providing insights. Additionally, it mentions various industries benefiting from IoT solutions and emphasizes the performance advantages of integrating machine learning and real-time analytics into IoT applications.
More Related Content
Similar to VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
- 1. page VOLTDB AND HPEVERTICA: BUILDING AN IOT ARCHITECTURE FOR FAST + BIG DATA
- 2. page© 2016 VoltDB OURSPEAKERS 2 Chris Selland VP Business Development Hewlett Packard Enterprise Dennis Duckworth, Director Product Marketing VoltDB
- 3. page© 2016 VoltDBpage AGENDA • Introduction • What is the Internet of Things? • HPE Vertica and IoT • VoltDB and IoT • Demo of VoltDB and HPE Vertica • Q&A 3
- 4. Building an IoT Architecturefor Big & Fast Data Chris Selland VP Business Development
- 5. What is theInternet of Things? “The Internet of Things is the network of physical objects that contains embedded technology to communicate and sense or interact with the objects' internal state or the external environment.”1 Consumer “things” Personal sensing with remote monitoring and control 1 Gartner. IT Glossary – Internet of Things. Industrial “things” From individual sensors to entire power plants
- 6. Business outcomes through bigdata analytics How can value be created from IoT data? Up to $11 trillion per year in potential economic impact by 20251 • Increased revenue • Optimized operations • New products and services • Workforce productivity • Reduced risks • Reduced operating costs • Optimized maintenance • Optimized network throughput Smart citiesManufacturing Consumer Smart homes Healthcare Oil & gas Aviation Energy & utilities Transportation Retail Industries that can benefit Insurance Agriculture 1 McKinsey Global Institute. The Internet of Things: Mapping the Value Beyond the Hype, June 2015. Connected devices source of contextual data
- 7. IoT is notthe future. It’s already here. Business Outcomes • Understand driver behaviors and patterns • To improve efficiency of the company and drivers Business Outcomes • Safety and well-being of citizens • Analyzing safety threats • Accurate identification and scene analysis Business Outcomes • Increases customer value with equipment monitoring, event mitigation, & energy performance services • Helps customers reduce energy & maintenance costs 7 Equipment Manufacturers Enterprises Service-based Businesses Integrated sensor and telemetry analytics in HP Fleet vehicles Monitors 2000+ cameras and performs video analytics Analytics enhance the Trane Intelligent Services business Successes in IoT
- 8. HPE Vertica andIoT 8
- 9. Computing at theEdge – “shifting left” for strategic advantage Goal – Processing streaming data as close to the sensor as possible creates new opportunities Advantage – Processing data streams in real time, before the data is stored for additional analysis, creates advantages – Example: Transformer or turbine thermal runaway, requires immediate action Result – Fast action prior to data storage prevents data obsolescence and lost opportunities/alerts HPE Confidential | Share under NDA 9 “Things” generate data and need control Edge IT, data center and cloudOperations technology Early analytics and compute Deep analytics and compute Data is sensed, Things controlled Data acquired and aggregated The Edge Opportunity Accelerate insight by moving compute from the data center to the Edge Data flow Control flow Augment the data center with processing at the edge
- 10. The Vertica Real-TimeAnalytics Engine Native High Availability Standard SQL Interface Column Orientation Auto Database Design Advanced Compression MPP Massive Parallel Processing Leverages BI, ETL, Hadoop/MapReduce and OLTP investments No disk I/O bottleneck simultaneously load & query Native DB-aware clustering on low-cost x86 Linux nodes Built-in redundancy that also speeds up queries Automatic setup, optimization, and DB management Up to 90% space reduction using 10+ algorithms ü 50x – 1000x faster than traditional RDBMS ü Scales from TB to PB with industry- standard hardware ü Simple integration with existing ETL and BI solutions ü SQL-99+ compliant ü Ultimate deployment flexibility ü Extended advanced analytics ü 24/7 Load & Query Confidential 10
- 11. Taking IoT analyticsto the next level – Integrated Machine Learning Native Machine Learning algorithms run in database (Vertica) • K-means (anomalies) • Linear Regression (risks, trends) • Logistic Regression Train predictive analytics models in the datacenter, and easily run at the edge Utilizes the same hardware as the database itself • Lower cost solution • No transfer of data required • Large data sets lead to more accurate models 11
- 12. Taking IoT analyticsto the next level – High Performance Messaging Vertica Streaming Adapter for Apache Kafka Kafka replaces custom data ingest solutions with a robust open-source implementation Enables distributed data pipelines for high throughput and low latency ingest into Vertica • Time to insight reduced from hours to seconds (micro-batch loading) • Easy handling of data bursts (10m messages per minute) 12
- 13. Taking IoT analyticsto the next level – In-database Sensor Data Analytics Powerful time series and window functions (gap filling, interpolation) for data quality management at the edge1 Live Aggregate Projections (personalized billing on demand) Log text analytics and pattern matching (SIEM – security, intrusion detection) Geospatial analytics (location based services, asset management) 13 1 HPE internal testing of Vertica on EL4000 shows it can: - Repair 5.7 million readings per second - Load, repair and analyze a reading every 282 nanoseconds
- 14. www.hpe.com/events/bdc Big Data Conference2016 Boston August 29- September 1
- 15. page BUILDING AN IOTARCHITECTURE FOR FAST + BIG DATA DENNIS DUCKWORTH DIRECTOR OF PRODUCT MARKETING
- 16. page© 2016 VoltDB VOLTDBDOESN’T MAKE THE APPS, WE MAKE THE APPS… 16 • Real-time intelligence and context for richer interactions • Allows unique decisions on each individual event or person • Analyze and act on real-time/streaming data • More efficient use of hardware • 100X faster than traditional transactional databases • World record performance in the cloud (YCSB) • Millisecond response times • Hundreds of thousands to millions of transactions per second • High-speed data ingestion • Simpler apps, easier to test and maintain • Easier to program with SQL + Java • Seamless ecosystem integration • Data is always consistent and correct, never lost Smarter Faster Simpler 1/10 of the Resources Needed 100X Traditional Transactional DB 100% Consistent, Correct
- 17. page© 2016 VoltDB Low Complexity Rich, Smart Value of Individual Data Item Value of Data Collection DataValue Data Warehouse Hadoop, etc.NoSQL Data in Motion Data at Rest Fast Data Big Data Feeds, Collectors CEP CEP + DB DataInteraction FAST DATA + BIG DATA
- 18. page© 2016 VoltDB Streaming Analytics - Filtering - Windowing - Aggregation - Enrichment - Correlations Deep Analytics - Machine Learning - Statistical correlations - Multi-dimensional analysis - Predictive/Prescriptive analytics Operational Interactions / Transactions - Context-aware - Personal - Real-time + Big Data FAST DATA + BIG DATA Fast Data
- 19. page© 2016 VoltDB CUSTOMERSUSING VOLTDB FOR IOT TODAY • Smart Meters (over 60 million meters under VoltDB management) • Smart Energy GB • Smart Metering in Japan (Shikoku and Hokkaido) • Wearables • Home Security • Equipment Service Automation
- 20. page© 2016 VoltDB EQUIPMENTSERVICE AUTOMATION: LEADING STORAGE EQUIPMENT VENDOR USING IOT
- 21. page© 2016 VoltDB21 #Simplify
- 22. page© 2016 VoltDB Proven ü Jepsen Tested Simpler ü Fewer components and clusters needed ü Easier to write, test and maintain applications Familiar ü SQL and Java Glue Code Glue Code Zookeeper “DIY” STREAMING DATA INFRASTRUCTURE VS VOLTDB
- 23. page© 2016 VoltDB EXAMPLE:LARGE TELCO IOT PLATFORM 23 Challenges: • High volume and velocity of data from smart devices • Complexity (multiple ingest points, apps, databases) • Performance – need to automate action on inbound data at the velocity of the feeds Notes • Applications have their own database • Each database replicates to Cassandra and Hadoop • In-memory grid used to maintain logic and publish ‘state’ back and forth • Rules engine for intra-day data to trigger actions (e.g., ‘turn lights on’) • PostgreSQL used for dimension data
- 24. page© 2016 VoltDB BEFORE 24 AFTER Results ü Simplified system architecture ü Single ingest point for high-velocity feeds of inbound data ü Faster time to value
- 25. page© 2016 VoltDB BEFORE 25 ORPOSSIBLY Results ü Simplified system architecture ü Single ingest point for high-velocity feeds of inbound data ü Faster time to value
- 26.
- 27.
- 28. page© 2016 VoltDB QUESTIONS? • Use the chat window to type in your questions • Download our ebook, Architecting for the Internet of Things (learn.voltdb.com/EbookIoT.html) • Try VoltDB yourself Free trial of the Enterprise Edition: www.voltdb.com/Download • Email us at: info@voltdb.com 28