This document discusses big data and how organizations can gain insights from data. It notes that by 2015, organizations that have built a modern information system will financially exceed their competitors by 20%. It describes different types of structured and unstructured data organizations are dealing with, including machine-generated data from sensors, satellites, science experiments, videos, and more. It also lists common uses of big data like recommendations, smart meter monitoring, equipment monitoring, advertising analysis, and more. The document then discusses how Microsoft can help users gain better insights through self-service BI, connecting and collaborating in Office 365, and answering questions. It outlines different data sources including non-relational data stored in HDInsight on Azure.
Case of success: Visualization as an example for exercising democratic transp...Big Data Spain
The document discusses creating an interactive dashboard to visualize Spain's state budget data in order to promote democratic transparency. The dashboard aims to [1] analyze and normalize available budget data files, [2] create an interactive tool for citizens to better understand budget information, and [3] empower citizens to make their own conclusions about budget spending. Big data and open source technologies would be used to extract, transform, analyze, and visualize the budget data in the dashboard.
Enabling the Bank of the Future by Ignacio BernalBig Data Spain
BBVA is transforming into a digital bank through building a global cloud banking platform. The platform utilizes new technologies including a global hybrid cloud infrastructure, global data and PaaS platforms, and an embedded security platform. It integrates legacy systems through a global service layer and real-time data integration. A new operating model features DevOps, everything as a service through a single API catalog, and an Ubuntu-like global developer community. Developing great talent is also a focus through a different approach to talent development and strategic partnerships with startups and other partners.
This document discusses Google Cloud Platform's Internet of Things (IoT) architecture and services. It describes how IoT data can be captured using protocols and streaming into Google Cloud Pub/Sub. Machine learning algorithms can then detect patterns in real-time streams. Data is also archived in Cloud Storage. Google Cloud Dataflow is highlighted for processing both batch and stream workloads, with features like autoscaling, intuitive programming model, and unified processing of data.
Denodo Platform for AWS 30 day free trial: https://goo.gl/15JPHS
Denodo Platform for Azure 30 day free trial: https://goo.gl/xEzTki
The Denodo Cloud Survey was conducted in December, 2017 to gain insight into how organizations are using Cloud platforms, such as Amazon AWS and Microsoft Azure.
The survey sought input from a diverse group of technical people, including enterprise architects, data architects, IT heads of department, such as Head of Analytics or BI Director, and some VP/CTO level respondents (see Slide 3 for breakdown).
The respondents were largely located throughout North America and Europe.
Reinventing Payments at HSBC with a Unified Platform for Data and AI in the C...Databricks
HSBC is using a unified data analytics platform on Azure to drive machine learning-powered innovation in payments. This platform brings together diverse data sources in a single data lake for real-time data masking, discovery, model development and deployment. It has enabled richer business insights, more timely analytics using streaming data, and faster iterations for data scientists, analysts and engineers. This unified approach is reinventing payments by powering improved products with data-driven insights.
AI as Driver of Transformation - Didier Ongena @CONNECT19Codit
During Didier his session you’ll discover how companies and partners that embrace AI will clearly create a competitive advantage and will grow at a faster pace than those who resist it. Future-orientated companies are already laying the foundations for this transformation, by conducting pilot projects and releasing AI applications for their daily operations. Of course, our technology evolves and we’re moving towards working with our partners and customers to create artificial intelligence that truly augments human capabilities.
Building Identity Graph at Scale for Programmatic Media Buying Using Apache S...Databricks
The proliferation of digital channels has made it mandatory for marketers to understand an individual across multiple touchpoints. In order to develop market effectiveness, marketers need have a pretty good sense of its consumer’s identity so that it can reach him via mobile device, desktop or a big TV screen on living room. Examples of such identity tokens include cookies, app IDs etc.A consumer can use multiple devices at the same time and so the same consumer should not be treated as different people in the advertising space. The idea of identity resolution comes with this mission and goal to have an omnichannel view of a consumer.
This document discusses big data and how organizations can gain insights from data. It notes that by 2015, organizations that have built a modern information system will financially exceed their competitors by 20%. It describes different types of structured and unstructured data organizations are dealing with, including machine-generated data from sensors, satellites, science experiments, videos, and more. It also lists common uses of big data like recommendations, smart meter monitoring, equipment monitoring, advertising analysis, and more. The document then discusses how Microsoft can help users gain better insights through self-service BI, connecting and collaborating in Office 365, and answering questions. It outlines different data sources including non-relational data stored in HDInsight on Azure.
Case of success: Visualization as an example for exercising democratic transp...Big Data Spain
The document discusses creating an interactive dashboard to visualize Spain's state budget data in order to promote democratic transparency. The dashboard aims to [1] analyze and normalize available budget data files, [2] create an interactive tool for citizens to better understand budget information, and [3] empower citizens to make their own conclusions about budget spending. Big data and open source technologies would be used to extract, transform, analyze, and visualize the budget data in the dashboard.
Enabling the Bank of the Future by Ignacio BernalBig Data Spain
BBVA is transforming into a digital bank through building a global cloud banking platform. The platform utilizes new technologies including a global hybrid cloud infrastructure, global data and PaaS platforms, and an embedded security platform. It integrates legacy systems through a global service layer and real-time data integration. A new operating model features DevOps, everything as a service through a single API catalog, and an Ubuntu-like global developer community. Developing great talent is also a focus through a different approach to talent development and strategic partnerships with startups and other partners.
This document discusses Google Cloud Platform's Internet of Things (IoT) architecture and services. It describes how IoT data can be captured using protocols and streaming into Google Cloud Pub/Sub. Machine learning algorithms can then detect patterns in real-time streams. Data is also archived in Cloud Storage. Google Cloud Dataflow is highlighted for processing both batch and stream workloads, with features like autoscaling, intuitive programming model, and unified processing of data.
Denodo Platform for AWS 30 day free trial: https://goo.gl/15JPHS
Denodo Platform for Azure 30 day free trial: https://goo.gl/xEzTki
The Denodo Cloud Survey was conducted in December, 2017 to gain insight into how organizations are using Cloud platforms, such as Amazon AWS and Microsoft Azure.
The survey sought input from a diverse group of technical people, including enterprise architects, data architects, IT heads of department, such as Head of Analytics or BI Director, and some VP/CTO level respondents (see Slide 3 for breakdown).
The respondents were largely located throughout North America and Europe.
Reinventing Payments at HSBC with a Unified Platform for Data and AI in the C...Databricks
HSBC is using a unified data analytics platform on Azure to drive machine learning-powered innovation in payments. This platform brings together diverse data sources in a single data lake for real-time data masking, discovery, model development and deployment. It has enabled richer business insights, more timely analytics using streaming data, and faster iterations for data scientists, analysts and engineers. This unified approach is reinventing payments by powering improved products with data-driven insights.
AI as Driver of Transformation - Didier Ongena @CONNECT19Codit
During Didier his session you’ll discover how companies and partners that embrace AI will clearly create a competitive advantage and will grow at a faster pace than those who resist it. Future-orientated companies are already laying the foundations for this transformation, by conducting pilot projects and releasing AI applications for their daily operations. Of course, our technology evolves and we’re moving towards working with our partners and customers to create artificial intelligence that truly augments human capabilities.
Building Identity Graph at Scale for Programmatic Media Buying Using Apache S...Databricks
The proliferation of digital channels has made it mandatory for marketers to understand an individual across multiple touchpoints. In order to develop market effectiveness, marketers need have a pretty good sense of its consumer’s identity so that it can reach him via mobile device, desktop or a big TV screen on living room. Examples of such identity tokens include cookies, app IDs etc.A consumer can use multiple devices at the same time and so the same consumer should not be treated as different people in the advertising space. The idea of identity resolution comes with this mission and goal to have an omnichannel view of a consumer.
This document provides an overview and agenda for a presentation on product data management using Neo4j graph databases. The presentation will include an introduction to graph databases and Neo4j by Bruno Ungermann from Neo4j, followed by a discussion of using graph databases for product data management by Dr. Andreas Weber from semantic PDM. Examples will be provided of graph models and how they can be used for various domains including logistics, manufacturing, and customer relationships. Attendees will have an opportunity to ask questions and discuss use cases.
Using the Yahoo Cloud Storage Benchmark (YCSB) , we show that Xanadu outperforms other NoSQL databases while offering strong consistency, high throughput, low latency and high scalability.
Big Data and Fast Data combined – is it possible ? Introduction aux architectures Big Data. M. Ulises Fasoli, Senior Consultant Trivadis. Conférence donnée dans le cadre du Swiss Data Forum du 24 novembre 2015 à Lausanne
Evolving From Monolithic to Distributed Architecture Patterns in the CloudDenodo
Watch full webinar here: https://goo.gl/rSfYKV
Gartner states in its Predicts 2018: Data Management Strategies Continue to Shift Toward Distributed,
“As data management activities are becoming more widespread in both distributed processing use cases, like IoT, and demands for new types of data, emerging roles such as data scientists or data engineers are expected to be driving the new data management requirements in the coming two years. These trends indicate that both the collection of data as well as the need to connect to data are rapidly becoming the new normal, and that the days of a single data store with all the data of interest — the enterprise data warehouse — are long gone.”
Data management solutions are becoming distributed, heterogeneous and extremely diverse.
Attend this session to learn:
• How to evolve architecture patterns in the cloud using data virtualization.
• How data virtualization accelerates cloud migration and modernization.
• Successful cloud implementation case studies.
Brian Greig gave a presentation on visualizing data in realtime using WebSockets and D3. He discussed collecting and consuming data from various sources, performing data analytics and visualizations using the DADA loop, using WebSockets for bidirectional data transmission, manipulating the DOM with D3 for data visualization, and presented a case study on building a simulation.
Delivering Quality Open Data by Chelsea UrsanerData Con LA
Abstract:- The value of data is exponentially related to the number of people and applications that have access to it. The City of Los Angeles embraces this philosophy and is committed to opening as much of its data as it can in order to stimulate innovation, collaboration, and informed discourse. This presentation will be a review of what you can find and do on our open data portals as well as our strategy for delivering the best open data program in the nation.
Michael will discuss some of the issues and challenges around Big Data. It is all very well building Big Data friendly databases to manage the tidal wave of real-time data that the IoT inevitably creates but this must also be incorporated into legacy data to deliver actionable insight.
Why Business Intelligence Should Consider Agile Modern Data Delivery Platformsyed_javed
Modern data solution like Lyftron provides high availability and concurrency at all scales for modern analytical and business intelligence applications such as Looker, Tableau, PowerBI, Sisence, PeriscopeData etc. and can deliver timely results for you.
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?PROIDEA
This document discusses data scalability and introduces GridwiseTech, a vendor-independent scalable technology expert. It explains that IT systems are constantly growing due to increased users, applications, and data which can lead to infrastructure bottlenecks. To improve efficiency, GridwiseTech introduces scalability through distributed processing, load balancing, and scaling out data. It then summarizes a case study where GridwiseTech helped an electronic manufacturer scale its infrastructure to ensure scalability on each functional layer and achieve significant performance improvements like 10x faster data processing.
Webinar: The 5 Most Critical Things to Understand About Modern Data IntegrationSnapLogic
In this webinar, we talk to industry analyst, author and practitioner David Linthicum who provides a state-of-the-technology explanation of big data integration.
David also provides 5 critical and lesser known data integration requirements, how to understand today's requirements, and guidance for choosing the right approaches and technology to solve these problems.
To learn more, visit: www.snaplogic.com/big-data
Pushing the boundaries with IoT - Glenn Colpaert @CONNECT19Codit
With over 20 years’ experience in the field, Codit is helping customers get into Azure IoT Solution. New evolutions like Azure IoT Edge and Digital Twins are real game-changers for business and open up a whole range of new possibilities. Glenn will give a behind the curtains look on success stories, so you can get ideas about how IoT can be used for your business to drive revenue, discover new business models, and optimize business processes.
CTO of ParStream Joerg Bienert hold a presentation on February 25, 2014 about Big Data for Business Users. He talked about several use cases of current ParStream customers and ParStreams' technology itself.
Multi-Cloud Data Integration with Data Virtualization (APAC)Denodo
Watch full webinar here: https://bit.ly/3cnw5MW
More and more organization are adopting multi-cloud strategies to provide greater flexibility, cost savings, and performance optimization. Even when organizations commit to a single cloud provider, they often have data and applications spread across different cloud regions to support different business units or geographies. The result of this is a high distributed infrastructure that makes finding and accessing the data needed for reporting and analytics even more challenging.
The Denodo Platform Multi-Location Architecture provides quick and easy managed access to data while still providing local control to the 'data owners' and complying with local privacy and data protection regulations (think GDPR and CCPA!).
In this on-demand session, you will learn about:
- The challenges facing organizations as they adopt multi-cloud data strategies
- How the Denodo Platform provides a managed data access layer across the organization
- The different multi-location architectures that can maximize local control over data while still making it readily available
- How organizations have benefited from using the Denodo Platform as a multi-cloud data access layer
Cloud Modernization with Data VirtualizationDenodo
Watch full webinar here: [https://buff.ly/2sLhFAc]
TransAlta is an electric power generator company headquartered in Calgary, Alberta. TransAlta's IT department initiated "Zero Data Center" project to move their entire data layer to the cloud for flexibility, agility and lower TCO. Data virtualization technology played a central role in TransAlta's real-time data integration, while helping them move to the cloud with zero down-time
Attend this Denodo DataFest 2018 session to learn:
Who is TransAlta and why TransAlta wanted to move their entire enterprise data layer to the cloud
Why data virtualization played a critical role in TransAlta's cloud modernization effort
How TransAlta uses DV in their energy trading, wind icing forecast and HR fuctions
Agile Data Management with Enterprise Data Fabric (Middle East)Denodo
Watch full webinar here: https://bit.ly/3td9ICb
In a world where machine learning and artificial intelligence are changing our everyday lives, digital transformation tops the strategic agenda in many private and government organizations. Data is becoming the lifeblood of a company, flowing seamlessly through it to enable deep business insights, create new opportunities, and optimize operations.
Chief Data Officers and Data Architects are under continuous pressure to find the best ways to manage the overwhelming volumes of the data that tend to become more and more distributed and diverse.
Moving data physically to a single location for reporting and analytics is not an option anymore – this is the fact accepted by the majority of the data professionals.
Join us for this webinar to know about the modern virtual data landscapes including:
- Virtual Data Fabric
- Data Mesh
- Multi-Cloud Hybrid architecture
and to learn how to leverage Denodo Data Virtualization platform to implement these modern data architectures.
Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)Jorge Millán Cabrera
En esta breve charla, Carlos Sacristán nos mostrará qué es ADF, sus componentes principales y cómo podemos sacarle partido empleándolo en algunos escenarios de uso típicos.
Basic concepts, best practices, pricing of using BigQuery the analytic data platform at petabyte scale from Google Cloud Platform. There is a lot things to learn about this tool and its features such as BI engine and AI Platform.
Oxalide MorningTech #1 - BigData
1er MorningTech @Oxalide, animé par Ludovic Piot (@lpiot), le 15 décembre 2016.
Pour cette 1ère édition du Morning Tech nous vous proposons une overview sur un des thèmes du moment : le Big Data.
Au delà de ce buzz word nous aborderons :
Les grands concepts
Les étapes clés des projets Big Data et les technologies à utiliser (stockage, ingestion, …)
Les enjeux des architectures Big Data (architecture lambda, …)
L'intelligence artificielle (machine learning, deep learning, …)
Et nous finirons par un cas d'usage du big data sur AWS autour de l'utilisation des données gyroscopiques de vos internautes mobiles
Subject: Oxalide's 1st MorningTech talk about BigData.
Date: 15-dec-2016
Speakers: Ludovic Piot (@lpiot, @oxalide)
Language: french
Lien SpeakerDeck : https://speakerdeck.com/lpiot/oxalide-morningtech-number-1-bigdata
Lien SlideShare : https://www.slideshare.net/LudovicPiot/oxalide-morningtech-1-bigdata
YouTube Video capture: https://youtu.be/7O85lRzvMY0
Main topics:
* Les grands enjeux du BigData
** les 3 V du Gartner : volume, variété, vélocité
* Le stockage des données
** datalake
** les technos
* L'ingestion des données
** ETL
** datastream
** les technos
* Les enjeux du compute
** map-reduce
** spark
** lambda architecture
* Démo d'une plateforme BigData sur AWS
* L'intelligence artificielle
** datascience exploratoire et notebooks,
** machine learning,
** deep learning,
** data pipeline
** les technos
* Pour aller plus loin
** La gouvernance des données
** La dataviz
This document provides an overview and agenda for a presentation on product data management using Neo4j graph databases. The presentation will include an introduction to graph databases and Neo4j by Bruno Ungermann from Neo4j, followed by a discussion of using graph databases for product data management by Dr. Andreas Weber from semantic PDM. Examples will be provided of graph models and how they can be used for various domains including logistics, manufacturing, and customer relationships. Attendees will have an opportunity to ask questions and discuss use cases.
Using the Yahoo Cloud Storage Benchmark (YCSB) , we show that Xanadu outperforms other NoSQL databases while offering strong consistency, high throughput, low latency and high scalability.
Big Data and Fast Data combined – is it possible ? Introduction aux architectures Big Data. M. Ulises Fasoli, Senior Consultant Trivadis. Conférence donnée dans le cadre du Swiss Data Forum du 24 novembre 2015 à Lausanne
Evolving From Monolithic to Distributed Architecture Patterns in the CloudDenodo
Watch full webinar here: https://goo.gl/rSfYKV
Gartner states in its Predicts 2018: Data Management Strategies Continue to Shift Toward Distributed,
“As data management activities are becoming more widespread in both distributed processing use cases, like IoT, and demands for new types of data, emerging roles such as data scientists or data engineers are expected to be driving the new data management requirements in the coming two years. These trends indicate that both the collection of data as well as the need to connect to data are rapidly becoming the new normal, and that the days of a single data store with all the data of interest — the enterprise data warehouse — are long gone.”
Data management solutions are becoming distributed, heterogeneous and extremely diverse.
Attend this session to learn:
• How to evolve architecture patterns in the cloud using data virtualization.
• How data virtualization accelerates cloud migration and modernization.
• Successful cloud implementation case studies.
Brian Greig gave a presentation on visualizing data in realtime using WebSockets and D3. He discussed collecting and consuming data from various sources, performing data analytics and visualizations using the DADA loop, using WebSockets for bidirectional data transmission, manipulating the DOM with D3 for data visualization, and presented a case study on building a simulation.
Delivering Quality Open Data by Chelsea UrsanerData Con LA
Abstract:- The value of data is exponentially related to the number of people and applications that have access to it. The City of Los Angeles embraces this philosophy and is committed to opening as much of its data as it can in order to stimulate innovation, collaboration, and informed discourse. This presentation will be a review of what you can find and do on our open data portals as well as our strategy for delivering the best open data program in the nation.
Michael will discuss some of the issues and challenges around Big Data. It is all very well building Big Data friendly databases to manage the tidal wave of real-time data that the IoT inevitably creates but this must also be incorporated into legacy data to deliver actionable insight.
Why Business Intelligence Should Consider Agile Modern Data Delivery Platformsyed_javed
Modern data solution like Lyftron provides high availability and concurrency at all scales for modern analytical and business intelligence applications such as Looker, Tableau, PowerBI, Sisence, PeriscopeData etc. and can deliver timely results for you.
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?PROIDEA
This document discusses data scalability and introduces GridwiseTech, a vendor-independent scalable technology expert. It explains that IT systems are constantly growing due to increased users, applications, and data which can lead to infrastructure bottlenecks. To improve efficiency, GridwiseTech introduces scalability through distributed processing, load balancing, and scaling out data. It then summarizes a case study where GridwiseTech helped an electronic manufacturer scale its infrastructure to ensure scalability on each functional layer and achieve significant performance improvements like 10x faster data processing.
Webinar: The 5 Most Critical Things to Understand About Modern Data IntegrationSnapLogic
In this webinar, we talk to industry analyst, author and practitioner David Linthicum who provides a state-of-the-technology explanation of big data integration.
David also provides 5 critical and lesser known data integration requirements, how to understand today's requirements, and guidance for choosing the right approaches and technology to solve these problems.
To learn more, visit: www.snaplogic.com/big-data
Pushing the boundaries with IoT - Glenn Colpaert @CONNECT19Codit
With over 20 years’ experience in the field, Codit is helping customers get into Azure IoT Solution. New evolutions like Azure IoT Edge and Digital Twins are real game-changers for business and open up a whole range of new possibilities. Glenn will give a behind the curtains look on success stories, so you can get ideas about how IoT can be used for your business to drive revenue, discover new business models, and optimize business processes.
CTO of ParStream Joerg Bienert hold a presentation on February 25, 2014 about Big Data for Business Users. He talked about several use cases of current ParStream customers and ParStreams' technology itself.
Multi-Cloud Data Integration with Data Virtualization (APAC)Denodo
Watch full webinar here: https://bit.ly/3cnw5MW
More and more organization are adopting multi-cloud strategies to provide greater flexibility, cost savings, and performance optimization. Even when organizations commit to a single cloud provider, they often have data and applications spread across different cloud regions to support different business units or geographies. The result of this is a high distributed infrastructure that makes finding and accessing the data needed for reporting and analytics even more challenging.
The Denodo Platform Multi-Location Architecture provides quick and easy managed access to data while still providing local control to the 'data owners' and complying with local privacy and data protection regulations (think GDPR and CCPA!).
In this on-demand session, you will learn about:
- The challenges facing organizations as they adopt multi-cloud data strategies
- How the Denodo Platform provides a managed data access layer across the organization
- The different multi-location architectures that can maximize local control over data while still making it readily available
- How organizations have benefited from using the Denodo Platform as a multi-cloud data access layer
Cloud Modernization with Data VirtualizationDenodo
Watch full webinar here: [https://buff.ly/2sLhFAc]
TransAlta is an electric power generator company headquartered in Calgary, Alberta. TransAlta's IT department initiated "Zero Data Center" project to move their entire data layer to the cloud for flexibility, agility and lower TCO. Data virtualization technology played a central role in TransAlta's real-time data integration, while helping them move to the cloud with zero down-time
Attend this Denodo DataFest 2018 session to learn:
Who is TransAlta and why TransAlta wanted to move their entire enterprise data layer to the cloud
Why data virtualization played a critical role in TransAlta's cloud modernization effort
How TransAlta uses DV in their energy trading, wind icing forecast and HR fuctions
Agile Data Management with Enterprise Data Fabric (Middle East)Denodo
Watch full webinar here: https://bit.ly/3td9ICb
In a world where machine learning and artificial intelligence are changing our everyday lives, digital transformation tops the strategic agenda in many private and government organizations. Data is becoming the lifeblood of a company, flowing seamlessly through it to enable deep business insights, create new opportunities, and optimize operations.
Chief Data Officers and Data Architects are under continuous pressure to find the best ways to manage the overwhelming volumes of the data that tend to become more and more distributed and diverse.
Moving data physically to a single location for reporting and analytics is not an option anymore – this is the fact accepted by the majority of the data professionals.
Join us for this webinar to know about the modern virtual data landscapes including:
- Virtual Data Fabric
- Data Mesh
- Multi-Cloud Hybrid architecture
and to learn how to leverage Denodo Data Virtualization platform to implement these modern data architectures.
Un orquestador en la nube: Azure Data Factory (por Carlos Sacristán)Jorge Millán Cabrera
En esta breve charla, Carlos Sacristán nos mostrará qué es ADF, sus componentes principales y cómo podemos sacarle partido empleándolo en algunos escenarios de uso típicos.
Basic concepts, best practices, pricing of using BigQuery the analytic data platform at petabyte scale from Google Cloud Platform. There is a lot things to learn about this tool and its features such as BI engine and AI Platform.
Oxalide MorningTech #1 - BigData
1er MorningTech @Oxalide, animé par Ludovic Piot (@lpiot), le 15 décembre 2016.
Pour cette 1ère édition du Morning Tech nous vous proposons une overview sur un des thèmes du moment : le Big Data.
Au delà de ce buzz word nous aborderons :
Les grands concepts
Les étapes clés des projets Big Data et les technologies à utiliser (stockage, ingestion, …)
Les enjeux des architectures Big Data (architecture lambda, …)
L'intelligence artificielle (machine learning, deep learning, …)
Et nous finirons par un cas d'usage du big data sur AWS autour de l'utilisation des données gyroscopiques de vos internautes mobiles
Subject: Oxalide's 1st MorningTech talk about BigData.
Date: 15-dec-2016
Speakers: Ludovic Piot (@lpiot, @oxalide)
Language: french
Lien SpeakerDeck : https://speakerdeck.com/lpiot/oxalide-morningtech-number-1-bigdata
Lien SlideShare : https://www.slideshare.net/LudovicPiot/oxalide-morningtech-1-bigdata
YouTube Video capture: https://youtu.be/7O85lRzvMY0
Main topics:
* Les grands enjeux du BigData
** les 3 V du Gartner : volume, variété, vélocité
* Le stockage des données
** datalake
** les technos
* L'ingestion des données
** ETL
** datastream
** les technos
* Les enjeux du compute
** map-reduce
** spark
** lambda architecture
* Démo d'une plateforme BigData sur AWS
* L'intelligence artificielle
** datascience exploratoire et notebooks,
** machine learning,
** deep learning,
** data pipeline
** les technos
* Pour aller plus loin
** La gouvernance des données
** La dataviz
DNA - Einstein - Data science ja bigdataRolf Koski
This document discusses DNA's journey in data science and big data. It summarizes that the big things driving change were the omnichannel customer demanding more data and analytics, and new technologies like cloud computing and data science providing endless scale and processing power. It outlines DNA's achievements in using these technologies to understand customers better, increase sales and marketing ROI, and automate many processes. Upcoming areas discussed include expanding into artificial intelligence, chatbots, and understanding speech. Culture aspects emphasized include having thinker-doers who can code leading projects and openly demonstrating work to connect with others.
Verso i bigdata giudiziari? (Nexa Torino, luglio 2016)Simone Aliprandi
Le slides utilizzate da Simone Aliprandi per il seminario "Verso i bigdata giudiziari? Problemi di privacy e copyright nella diffusione di sentenze sul web" tenutosi al Centro Nexa del Politecnico di Torino (info: http://juriswiki.it/news/al-politecnico-di-torino-si-parla-di-bigdata-giudiziari-e-di-juriswiki)
위 자료는 BOAZ 2016 하반기 프로젝트 주제의 하나로, Advanced 정규세션 동안 Base 정규세션에서 배웠던 다양한 이론들과 기본 지식들, 그리고 툴 활용능력들을 직접 실행하며 진행한 결과물입니다.
*** 서울시 2030 나홀로족을 위한 라이프 가이드북 ***
서울에 거주하는 2030 나홀로족을 위해 제작된 라이프 가이드북. 이 가이드북의 주목적은 먹는 것(식) 그리고 사는 것(주)에 대해서 그에 관한 정보를 주는 것임.
6기 김승효 중앙대학교 응용통계학과
6기 김재은 이화여자대학교 시각디자인과
7기 박다혜 한국외국어대학교 통계학과
** 국내 최초 대학생 빅데이터 연합동아리 BOAZ **
Blog : http://BOAZbigdata.com
Facebook : http://fb.com/BOAZbigdata
This very short document contains a link to a Facebook page called "Hadoopers" and the username of the person who posted it, @secooler. It ends with the phrase "Good luck."
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0Denodo
In this presentation, you will see the new functionalities of the Denodo 6.0 detailing dynamic query optimization engine, managing enterprise deployments, and using information self-service for discovery and search.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/DzRtkg.
The document outlines the history of building a big data platform from 2014 to 2016, starting with building a Hadoop cluster in 2014, creating the first data report page in 2015, launching products based on big data also in 2015, developing data analysis products in 2016, and making changes to the platform in 2016. It then transitions to discussing the current state of the big data platform.
SAMOA: A Platform for Mining Big Data Streams (Apache BigData Europe 2015)Nicolas Kourtellis
A general overview of the APACHE SAMOA platform for mining big data streams using machine learning algorithms running on distributed stream processing platforms such as Apache STORM, Apache Flink, Apache Samza and Apache Apex.
Results are shown from experimentation with VHT, the Vertical Hoeffding Tree proposed in "VHT: Vertical Hoeffding Tree." N. Kourtellis, G. De Francisci Morales, A. Bifet, A. Mordupo. IEEE BigData 2016.
Presentation in APACHE BIG DATA Europe 2015
SAMOA: A Platform for Mining Big Data Streams (Apache BigData North America 2...Nicolas Kourtellis
A general overview of the APACHE SAMOA platform for mining big data streams using machine learning algorithms running on distributed stream processing platforms such as Apache STORM, Apache Flink, Apache Samza and Apache Apex.
Results are shown from experimentation with VHT, the Vertical Hoeffding Tree proposed in "VHT: Vertical Hoeffding Tree." N. Kourtellis, G. De Francisci Morales, A. Bifet, A. Mordupo. IEEE BigData 2016.
Presentation in APACHE BIG DATA North America 2016
Презентация Виталия Никитина о возомжностях платформы HPE Idol для работы с BigData в современном кол-центре. Аналитика аудио и текстовой информации на базе платформы HPE IDOL
This document provides biographical information about Dr. Dinh Le Dat, the co-founder and CEO of ANTS, a Big Data advertising and data-driven marketing solution company. It outlines his educational background, including a PhD in Physics and Mathematics from Moscow State University, and over 15 years of experience working for technology companies in Vietnam, including roles as CTO of FPT Online Service JSC and co-founder of Yola JSC. It also lists his contact information and links to his LinkedIn profile and website.
This document provides an overview of Spark and using Spark on HDInsight. It discusses Spark concepts like RDDs, transformations, and actions. It also covers Spark extensions like Spark SQL, Spark Streaming, and MLlib. Finally, it highlights benefits of using Spark on HDInsight like integration with Azure services, scalability, and support.
This document discusses scaling IoT solutions. It outlines four main challenges in scaling IoT: 1) current infrastructure does not scale well for time to market and predictability, 2) current team structures do not scale as operations become a bottleneck, 3) solutions become too customer-specific instead of tenant-agnostic, and 4) communication with devices over VPN is not scalable and causes cascading failures. It proposes solutions like using continuous delivery, developer-led operations, tenant-based architectures, and alternative communication methods like MQTT to address these challenges in scaling IoT solutions.
Evento exclusivo organizado por target para que nuestros clientes estén al día con los actuales referentes del panorama nacional y se oxigenen tomando un refresco inmersos en el mejor networking para su empresa.
The document outlines the agenda for the GOTO Night: Elasticsearch event, including keynotes, workshops on various technical topics, and new speakers. It then discusses why Elasticsearch is a worthwhile technology to focus on, citing trends from Gartner and the job market as well as the large community and number of deployments behind Elasticsearch. The presentation will feature speakers from companies like Facebook, Elastic, and Wikimedia discussing their experiences with Elasticsearch and what the future may hold for the technology.
Big Data Experience Sharing: Building Collaborative Data Analytics Platform -...Amazon Web Services
Speaker: Kenny Kwan, Head of Software and Cloud Engineering, Gibson Innovations Limited
Here from Gibson Innovations engineering on how they build, deploy and manage their Data Analytics Platforms and IoT Hub on AWS and get business insights.
Did you know that THIS MORNING
there is more data in the world
than EVER BEFORE?!
By 2018, 40% of enterprise architecture teams will be distinguished as leaders by their primary focus on applying disruptive technologies and the power of Big Data to drive business innovation.
Building Large-Scale Applications for the Internet of Things at BoschMongoDB
The document discusses the opportunities and challenges presented by the Internet of Things (IoT) and big data. It notes that by 2020, over 50 billion devices will be connected as part of the IoT, generating huge volumes of data. This data will need to be managed and analyzed at massive scales. New database technologies are required that can handle the flexibility, scalability, analytics and unified views needed for IoT applications. The document outlines Bosch's IoT solutions and provides two use cases, one for managing tool asset data and one for capturing field data from vehicles, that demonstrate how their IoT platform addresses these big data challenges.
Innogy - data als inspiratie - jachtdagRaaf & Wolf
innogy aims to use customer data from smart meters and other sources to develop new services and gain insights into customer behavior. They plan to enrich customer profiles using first, second, and third party data, and to partner with other companies to build common data platforms and pursue cross-industry business models. The company's bit.B sensors can monitor energy consumption in real-time at the appliance level or for industrial processes, helping customers reduce costs through optimization.
Presentation: Study: #Big Data in #Austria, Mario Meir-Huber, Big Data Leader Eastern Europe, Teradata GmbH & Martin Köhler, Austrian Institute of Technology, AIT (AT), at the European Data Economy Workshop taking place back to back to SEMANTiCS2015 on 15 September 2015 in Vienna.
Quby - we create toon - Enabling smart energy services using scalable data sc...BigDataExpo
Toon, a leading European smart home platform, implements AI and machine learning to generate actionable insights for users. Stephen Galsworthy will describe the data driven services we’ve built which ensure that Toon users do not needlessly waste energy and can receive real-time alerts about problems with their heating systems.
Big data analytics and building intelligent applicationsFlytxt
This document discusses big data analytics, intelligent applications, and the future of artificial intelligence. It covers the current state of the art in machine learning, data mining, and data science. It also addresses challenges in scaling artificial intelligence to meet future data generation rates, which will outpace computation and communication speeds. New distributed computation models that move code to data instead of vice versa will be needed. Fundamental limits of physics, networks and computation must also be considered in architecting intelligent systems of tomorrow.
Internet of Things and Big Data: Vision and Concrete Use CasesMongoDB
This document discusses Internet of Things (IoT) and big data. It provides an overview of key concepts in IoT such as the growing number of connected devices, drivers in the IoT ecosystem including enterprises and users, and examples of IoT applications from Bosch Group. It also discusses how big data and evolving data models are driving new requirements for databases including scalability, flexibility, support for analytics, and providing a unified view of data. The document promotes an upcoming webinar series on IoT and big data.
Interoperability challenges & solutions in the EW-Shopp H2020 innovation action: tool-supported interoperability; exchange of event data and custom event ontology for data analytics; reconciliation across systems of spatial identifiers.
BICS empowers predictive analytics and customer centricity with a Hadoop base...DataWorks Summit
BICS uses a Hadoop data lake powered by the Informatica Big Data Platform to enable predictive analytics and customer centricity. The data lake provides scalable storage and processing for billions of telecommunications transactions. BICS aims to migrate more analytics and reporting from its Teradata data warehouse to Hadoop to gain cost efficiencies and handle increasing data volumes and complex analytics. The roadmap includes moving near real-time subscriber tracking to Hadoop while maintaining low latency, as well as computing new analytics and providing longer term historical reporting from Hadoop.
The New Role of Data in the Changing Energy & Utilities LandscapeDenodo
Watch full webinar here: https://bit.ly/3PrxEx2
Energy companies - both producers and utilities - are facing a challenging and changing business and regulatory environment over the next decade or so. As governments around the world pledge to be 'net zero' by 2050, new regulations are putting pressure on energy companies to accelerate the move to renewable energy sources whilst at the same time gearing up for more widespread electrification as consumers move away from carbon fuels.
The growth of renewable energy sources has also changed the way that utilities manage demand response. The old way of bringing generating units (typically coal or gas-fueled generators) online for peak demand hours no longer works. The distributed utility infrastructure that is used today requires a lot more flexibility and planning to meet - and to shape - consumer demand.
At the heart of the energy company challenges is data. Data to better manage and optimize the generating resources. Data to better inform the consumers about their energy consumption. And data to deliver better services and new product offerings to those consumers.
In this webinar, we will look at how energy companies and utilities can liberate and democratize their data to better utilize the strategic data assets that they already own. We will look at how the Denodo Platform, powered by Data Virtualization, has helped energy companies around the world access real-time data to drive their operations and allow them to respond to the ever-changing business environment.
Key Data Management Requirements for the IoTMongoDB
The document discusses key data management requirements for Internet of Things (IoT) applications. It notes that IoT will generate massive amounts of structured and unstructured data from a large number of connected devices and sensors. This data must be managed in a way that allows for rich applications, a unified view of data, real-time operational insights, business agility, and continuous innovation. It argues that traditional relational databases may not be well-suited for IoT data management and that NoSQL databases can provide scalability, flexibility, analytics and a unified view of data from multiple sources.
Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of T...Demetris Trihinas
An overview of monitoring techniques used on the edge to lower big data and energy efficiency barriers for IoT. To achieve this we introduce the AdaM and ADMin frameworks. This presentation is from a talk given at the University of Cyprus (March 2017). If used, please cite one of the following:
- "Adam: An adaptive monitoring framework for sampling and filtering on IoT devices", D. Trihinas et al., IEEE BigData 2015, 10.1109/BigData.2015.7363816
- "ADMin: Adaptive Monitoring Dissemination for the Internet of Things", D. Trihinas et al., IEEE INFOCOM 2017, to appear
This document summarizes the enCOMPASS project, which aims to stimulate behavioral change for energy saving through innovative digital tools. The project will integrate IoT technologies to collect energy usage data and combine it with persuasive technologies to provide personalized energy saving recommendations. It will validate this approach through pilots in three European countries testing it on residential, school, and office buildings. The project aims to demonstrate that energy can be saved without sacrificing comfort through engaging users with gamified energy visualizations and recommendations.
SMi Group's 3rd annual Meter Asset Management 2016 conferenceDale Butler
This document provides information about the 3rd annual Meter Asset Management conference taking place on June 20-21, 2016 in London. It includes details about registration, discounts for booking early, speakers, sessions, sponsors and exhibitors. The conference will focus on optimizing asset management and preparing for the evolution of smart meters by discussing topics such as smart meter implementation programmes, progress in different countries, using smart meter data to influence customer behavior and determining asset life. A half-day workshop on smart meter asset management through data use will also be held on June 22nd.
Utilities are embarking on a digital transformation focused on customer centricity. They are building digital ecosystems centered around customer analytics, grid edge network operations and analytics, and customer portals. These ecosystems leverage data from smart meters, weather data, and other sources. Utilities are digging into data analytics to better understand their data, reduce platforms, and improve capabilities through data science. Building a data-driven culture focused on customers will be key to success in utility digitization efforts.
1) Statistics Netherlands is working on several Big Data projects to produce new official statistics in a timely manner using large alternative data sources like road sensors.
2) Their Center for Big Data Statistics aims to reduce response burden, deepen methodological knowledge, and stimulate cooperation using an ecosystem of partners.
3) As a proof of concept, they have produced the first Big Data-based official statistic on regional traffic intensity using minute-level road sensor data from 20,000 sensors on Dutch highways. This required data cleaning, transformation, estimation techniques, and integrating skills from statistics, IT, and subject-matter expertise.
Datahub – towards future electricity retail marketFingrid Oyj
Vaasa EnergyWeek 2017
Cyber Security & Digitalization 22.3.2017
Datahub – towards future electricity retail market
Pasi Aho
Project Manager of Fingrid Datahub program
Managing Director at Fingrid Datahub Oy
Monetizing the Internet of Things: Creating a Connected Customer ExperienceZuora, Inc.
Customers today have new expectations. And never before has the customer experience been so critical than in the world IoT. Learn monetization strategies as well as the infrastructure require for delivering memorable customer experiences.
Similar to Bigdata analytics and our IoT gateway (20)
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Zilliz
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
2. Agenda
• A little bit about us
• Who is Quby & what is Toon?
• Our big data journey
• Data collection & services
• Architecture & technologies
• Demo
• Questions
@javafreekNL @twoxey
3. A little bit about us
FreekAemro
@javafreekNL @twoxey
8. The service center
Service Center
Toon Displays
Mobile devices
Product
Applications
Back Office
Applications
Mobile
Backend
Internet Internet
VPN
Access
Data collector
11. Our journey towards bigdata
30 March,
2017 12
2014
• 30K customers
• 40 types of sensor
data from 3K
customers
• up to 1 minutes
resolution
2012
• 4K customers
• Aggregated
Energy data
collected(DAY,WE
EK,MONTH)
• Benchmarking
functionality for
customers
2013
• 90K customers
• Data collection
from 30K
customers
• Event and
Measurement
data
12. Steps moving towards bigdata(cont’d)
30 March,
2017 13
2017
• 300K customers
• Near Real Time
aggregation with
segmentation
• 400 signal/5 minutes
• Sharing
Measurement data
with tenants
2015
• 190K customers
• Apache storm for Near
Real Time Energy
consumption (3K
customers)
• Data analysis using
Cloudera stack
• Up to 10 seconds
resolution
2016
• 330K customers
• Collect data from
230K customers
anonymously
• Data as a service
• Analytics platform
as a service
13. Toon®
Types of data
More than 300types of sensor and user interaction data. If you want to know more please contact us : dteam@quby.com
14
Energy CH-boiler (OT) Thermostat
Gas
consumption
Electricity
consumption
Water
temperature
Service
messages
Burner
information
Room
temperature
Thermostat
program
Manual
settings
Plug-data
Electricity
consumption
On/off
Water pressure
Energy
production
Energy
feed-in
Boiler status
Touch events
Thermostat
Mobile
1 2 3 4 5
Thermostat
setting
Hue data
14. Anonymous Data Collection
30 March,
2017 15
Service CenterCustomer
Always anonymous data
collected
Customer may opt in for services and
send keys
Customer may opt out and change
keys
Data can’t be de-anonymized again
15. Sample Services with Toon’s Data?
30 March,
2017 16
1. Benchmarking with daily energy meter data
16. Sample Services with Toon’s Data? ( cont’d)
30 March,
2017 17
2. Nearly Real Time Forecasting : Energy data Aggregation
17. Sample Services with Toon’s Data? ( cont’d)
30 March,
2017 18
3. Home Appliances Energy Consumption Detection: Energy Data disaggregation
18. Sample Services with Toon’s Data ?(cont’d)
30 March,
2017 19
4. Boiler Maintenance Prediction (KetelIQ)
19. Sample Services with Toon’s Data? ( cont’d)
30 March,
2017 20
5. Customer’s Behavioral studies
Click data Thermostat programs and co2 emission
20. What are we collecting?
24K measurements /
day / customer
600KB (zipped)
230K customers
21
24. Things to mention
• ssl mutual authentication
• raw-data stored partitioned in better way
• Customers can control access to their own data
30 March,
2017 27
25. What are we doing with all that data?
• Firehose – data as a service for B2B (and researchers)
• Analytic platform as a service
• New data driven services for Toon users
• Improve existing services
• Marketing strategy (understand your customers)
30 March,
2017 29
Freek van Gool – Cloud Platform Architect
We are in talks with many companies and people who have a broader perspective than us on the what is happening in the IoT world. We were told and noticed ourselves we’re doing something remarkable.
We want to share our learnings and pitfalls with a larger audience. We don’t have all the answers and solutions and see the IoT field is still changing very quickly.
But by sharing our knowledge and experiences we hope we can help more people being active in the IoT industry. Finally we hope we can also learn from your experiences and solutions.
About Quby
Started in 2004 in Amsterdam as Home Automation Europe BV
Currently a company of 150 people in Amsterdam with more than 25 nationalities
Focus on smart energy & smart living
We have built out the whole stack to bring you Toon, from hardware to cloud
Smart home before smart home
IoT before IoT
Difficult for end users
No engagement
Smart thermostat and IoT gateway
Consumers currently use Toon® with….
Smart thermostat
Smart home hub (solar monitor, mazout, smart plugs, smoke detector)
So… How do you sell this as a small company?
The solution which started the success is this.. (explain the logical setup and users in the solution without diving into all the details)
The second pivot:
how to find the right business model
Via energy utilities
Help utilities make theenergy transition
From utility toenergy service provider
We’ve celebrated numerous big successes which were dwarfed already by the next big success in a short time. You can imagine It’s very exciting to be part of a company which experiences exponential growth. However, it also required us to change a lot of things in our organization in order to cope with the growth we experienced.
##Number of customers derived from database
SELECT YEAR(complete.tijdstip) as jaar, COUNT(*) as number_of_activations FROM ( SELECT STARTDATE as tijdstip FROM quby.DEVICE_PARAMETER_VALUE_HISTORY asstat_hist where value = 'CONNECTED_TO_CLIENT' union SELECT TIMETAKEN as tijdstip FROM quby.DEVICE_PARAMETER_VALUE as stat where value = 'CONNECTED_TO_CLIENT' )as complete group by YEAR(complete.tijdstip) order by jaar desc