More Related Content
Similar to Quby - we create toon - Enabling smart energy services using scalable data science
Similar to Quby - we create toon - Enabling smart energy services using scalable data science (20)
More from BigDataExpo
More from BigDataExpo (20)
Recently uploaded
Recently uploaded (20)
Quby - we create toon - Enabling smart energy services using scalable data science
- 1. 21 September 2018 Enabling smart energy services using scalable data science Stephen Galsworthy
- 3. 2004 Home Automation Europe is founded 2012 Home Automation Europe partners with Eneco to create Toon 2013 Home Automation Europe becomes Quby Our history in milestones 2018 launch security service ThuisWacht with Interpolis 2018 Launch Toon2 2006 Home control alternatives are prototyped
- 4. 4 Launched in 2016 Launched in 2017 Launched in 2012 Testing phaseOur partners:
- 5. SMART THERMOSTAT & APP TOON SOLAR BOILER MONITORING WASTE CHECKER MONTHLY ENERGY INSIGHT WATER INSIGHT SECURITY PACKAGE & APP SMART METER DONGLE & APP ENERGY INSIGHTS APP DATA SERVICES
- 6. Z-Wave Meter adaptor Boiler adapter Gas sensor Philips Hue Z-Wave Central heating system Solar panels Click data Smart plugs & smoke detectors Electricity sensor
- 7. 100 MB data per user collected per month Over 350,000 displays installed 300 types of sensor and user interaction data
- 8. For production grade algorithms and R&D For machine learning on distributed data - Spark cluster management - Collaboration using shared notebooks - Scheduling complex workflows - Dashboards API management Big data storage and processing Our Data Science platform
- 10. Traditional billing relationship Historical energy insight Real time information Appliance level diagnostics <4% 4-6% 8-10% 20%+ Maximizing energy savings and customer engagement 1Armel et. al., Stanford University (2011) Increasingengagement betweencustomerandutility Increasing energy savings potential1
- 11. Energy Waste Checker “We don’t always notice how much energy we’re wasting. Toon can now expose the energy guzzlers in your home.” Launched in December 2017 to all Eneco Toon users
- 12. Toon detects inefficient appliances and behaviours & more coming soon
- 13. Electricity data Appliance signals Key technology: load disaggregation algorithms
- 14. Quby’s disaggregation algorithms DishwasherWashing machine Washing machine(only night hours shown) DryerDryer Patent pending algorithms can detect appliances from 10 second resolution electricity meter data
- 15. Quby’s disaggregation algorithms Washing machine Dryer DishwasherDryerWashing machine Washing machine Dryer Dishwasher Dryer But users try to make it complicated for us…
- 16. Use case example: Inefficient dishwasher diagnosis Disaggregation algorithms run on the 10s electricity meter data Compared with industry energy consumption standards and peers Translated to personalised advice for the end user Toon determines the “fingerprint” of the appliance through features
- 17. Scale of the Waste checker Each day we detect over 75,000dishwasher cycles That’s 13 years of dishwashers running continuously and over 25% are used inefficiently
- 18. POP QUIZ Which is the most popular day of the week to do the laundry? ?
- 19. It’s Sunday! 5.0 3.7 3.1 3.5 3.3 3.6 4.9 Sunday Monday Tuesday Wednesday Thursday Friday Saturday k 100k 200k 300k 400k 500k 600k 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 Sunday Monday Tuesday Wednesday Thursday Friday Saturday with 5.0 million of the 27 million washing machine detections we’ve made in 2018 so far
- 20. POP QUIZ When is the most popular time to start the dishwasher? Day and hour ?
- 21. It’s Monday at 18:00! with 330k of the 20 million dishwasher detections we’ve made in 2018 so far k 50k 100k 150k 200k 250k 300k 350k 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 0 2 4 6 8 10121416182022 Sunday Monday Tuesday Wednesday Thursday Friday Saturday
- 22. 5 things we learned
- 23. #1: We love Spark Machine learning on distributed data is key to all of this working at scale. An example: Spark’s window functions work great for time series data.
- 24. #2: We prefer Scala now Moving from Python to Scala takes effort but it’s worth it. An example: We define our dataset schemas statically. Static typing and type inference in Scala helps enforce these schemas.
- 25. #3: We live in the cloud Rather than taking care of the infrastructure, data engineers can concentrate on how to make data available to the business. The Databricks platform gives us cluster timeouts, autoscaling, spot instances… ….all without the need for additional FTEs.
- 26. #4: ‘In production’ is not the end Continuous feedback from hundreds of thousands of users means there’s always improvements to make. An example: Our users told us that they wanted more information about their efficient appliances, such as heat pump dryers. We productionized a deep learning model to give them this info.
- 27. #5: Full-stack data scientists rock! Build and run, from initial concept to end result. Making services that our end users will love. Working together we deliver state-of-the-art algorithms operating at scale in production.
- 28. Looking for a new challenge? Look no further! Quby – we create Toon, is looking to hire a DATA SCIENTIST YOU - are a problem-solving data genie - get excited about creating your own features and product - want to put all that theory into practice - love to visualize your data for storytelling - are a learning addict - are our future team member? Talk to me or e-mail our recruiter Tatjana at Careers@quby.com
- 29. With thanks to our data science partners: Stephen Galsworthy Head of Data Science +31 (0) 20 462 1680 stephen.galsworthy@quby.com
- 31. 1. Boiler and thermostat data are constantly monitored by Toon 4. Toon user is alerted 2. Toon determines when the system leaves its normal operating range Example: Heating system anomaly detection 3. Severity and user preferences taken into account
Editor's Notes
- how databricks is valuable : cluster management, scheduling, shared notebooks, plotting functons, intellij integration, dashboards,…
- What does the end user want? What can be created to solve that? What can be sold?
- Concept of load disaggregation … How do we do it?
- Deep learning
- Ideal case. All appliances separate. Spark:Time series data. Window functions. By user and date/time Datasets: type info on top of dataframes
- Timeliness Definition: The degree to which data represent reality from the required point in time. Correctness/Accuracy Definition: The degree to which data correctly describes the "real world" object or event being described. Completeness Definition: The proportion of stored data against the potential of "100% complete”. Consistency Definition: The absence of difference, when comparing two or more representations of a thing against a definition. also Uniqueness Definition: No thing will be recorded more than once based upon how that thing is identified. Validity Definition: Data are valid if it conforms to the syntax (format, type, range) of its definition.