At WomenWhoCode, 2019 https://india.womenwhocode.dev/agenda/
Edge Computing: What, Why, How and Where
Edge Analytics: What, Why, benefits, limitations
Edge computing vs Edge Analytics
Edge Analytics use-cases
IoT and connected devices: an overviewPascal Bodin
This is the presentation I use as a support to my 9 hour-long talk to postgraduate students of a French Telecom and Electronics Master. The idea is to provide them with a broad view, including some non-technical domains.
The slides defines IoT and show the differnce between M2M and IoT vision. It then describes the different layers that depicts the functional architecture of IoT, standard organizations and bodies and other IoT technology alliances, low power IoT protocols, IoT Platform components, and finally gives a short description to one of IoT low power application protocols (MQTT).
With the expansion of big data and analytics, organizations are looking to incorporate data streaming into their business processes to make real-time decisions.
Join this webinar as we guide you through the buzz around data streams:
- Market trends in stream processing
- What is stream processing
- How does stream processing compare to traditional batch processing
- High and low volume streams
- The possibilities of working with data streaming and the benefits it provides to organizations
- The importance of spatial data in streams
IoT and connected devices: an overviewPascal Bodin
This is the presentation I use as a support to my 9 hour-long talk to postgraduate students of a French Telecom and Electronics Master. The idea is to provide them with a broad view, including some non-technical domains.
The slides defines IoT and show the differnce between M2M and IoT vision. It then describes the different layers that depicts the functional architecture of IoT, standard organizations and bodies and other IoT technology alliances, low power IoT protocols, IoT Platform components, and finally gives a short description to one of IoT low power application protocols (MQTT).
With the expansion of big data and analytics, organizations are looking to incorporate data streaming into their business processes to make real-time decisions.
Join this webinar as we guide you through the buzz around data streams:
- Market trends in stream processing
- What is stream processing
- How does stream processing compare to traditional batch processing
- High and low volume streams
- The possibilities of working with data streaming and the benefits it provides to organizations
- The importance of spatial data in streams
on successful go through of this complete PPT, the learners can be able to understand the Raspberry PI, Raspberry Pi Interfaces(Serial, SPI,I2C) Programming, Python programming with Raspberry PI with the focus of Interfacing external gadgets
Controlling output Reading input from pins.
In general, data can be broken into two categories – data in motion vs data at rest. Learn the difference between these two types of data and the best infrastructure options to get optimal performance.
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends:
Exposing the device to a management framework
Exposing that management framework to a business centric logic
Exposing that business layer and data to end users.
This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles between cloud, APIs and native hardware/software configurations.
With the increase in need of water for irrigation, there is also a case where we use more water for irrigation than it’s needed for crops. That results in the wastage of water and causes the problem in the growth of crops. To overcome this problem, this paper puts together a study of a system based on Irrigation using IOT (Internet of things). This system targets on sensing the soil moisture and temperature using the sensors and provide the data to the Thing speak server after which the farmer can decide whether to ON or OFF the pump.
on successful go through of this complete PPT, the learners can be able to understand the Raspberry PI, Raspberry Pi Interfaces(Serial, SPI,I2C) Programming, Python programming with Raspberry PI with the focus of Interfacing external gadgets
Controlling output Reading input from pins.
In general, data can be broken into two categories – data in motion vs data at rest. Learn the difference between these two types of data and the best infrastructure options to get optimal performance.
How do APIs and IoT relate? The answer is not as simple as merely adding an API on top of a dumb device, but rather about understanding the architectural patterns for implementing an IoT fabric. There are typically two or three trends:
Exposing the device to a management framework
Exposing that management framework to a business centric logic
Exposing that business layer and data to end users.
This last trend is the IoT stack, which involves a new shift in the separation of what stuff happens, where data lives and where the interface lies. For instance, it's a mix of architectural styles between cloud, APIs and native hardware/software configurations.
With the increase in need of water for irrigation, there is also a case where we use more water for irrigation than it’s needed for crops. That results in the wastage of water and causes the problem in the growth of crops. To overcome this problem, this paper puts together a study of a system based on Irrigation using IOT (Internet of things). This system targets on sensing the soil moisture and temperature using the sensors and provide the data to the Thing speak server after which the farmer can decide whether to ON or OFF the pump.
Microservices: The Future-Proof Framework for IoTCapgemini
Dr Michael Capone Principal Analyst - Capgemini
The data generated by IoT-enabled machines, vehicles and devices can provide companies with insight into user behaviour that they can use to create a personal connection with their customers. Companies are, therefore, scrambling to implement IoT systems in order to generate, capture, protect, and analyse this valuable data. But the insights created are only valuable when they trigger consequent decisions and timely actions. There are many potential users of IoT data such as marketing, sales, held service, product
development, customer support, operations, and supply chain not to mention external users like vendors and partners. Each user group needs to be able to access and select different data and apply different logic and analytic approaches to perform specific tasks.
Furthermore, each group can have unique usability requirements. As companies become more IoT mature and start to plan for “data actionability,” the disadvantages of a homogenous IoT stack or departmental systems become obvious. The best option from a data quality, user acceptance, and ROI perspective is a microservices IoT platform.
Approaches to gather business requirements, defining problem statements, business requirements for
use case development, Assets for development of IoT solutions
An emulation framework for IoT, Fog, and Edge ApplicationsMoysisSymeonides
In this talk, we presented an emulation framework that eases the modeling, deployment, and large-scale experimentation of fog and 5G testbeds. The framework provides a toolset to (i) model complex fog topologies comprised of heterogeneous resources, network capabilities, and QoS criteria; (ii) abstractions for physical 5G infrastructure concepts such as radio units, edge servers, mobile nodes, user equipment, and node trajectories; (iii) deploy the modeled configuration and services using popular containerised descriptions to a cloud or
local environment; (iv) experiment, measure and evaluate the deployment by injecting faults, adapting the configuration at runtime, real-time updates of the radio network (i.e., signal strength) and respective network QoS to test different “what-if” scenarios that reveal the limitations of service before introduced to the public. The framework has been used for studying the performance of Intelligent transportation services, Industrial IoT micro-service applications, geo-distributed deployments of big data engines, and many more.
The presentation took place at Athens Demokritos Research Center organised by SKEL | The AI Lab
video: https://www.youtube.com/watch?v=z37I1QVFabg
Train & Sustain: Why data leaders need to pay attention to HITLCloudFactory
Humans are still very much needed in AI, ML, and automation. In this presentation, our Senior Solutions Consultant, Matt Beale, discusses why deep learning needs human expertise to succeed and why blending human intelligence and technology to build scalable models for production is the best approach.
Introduction to IoT, Current trends and challenges. It also describes some of the industry standard platforms such as Microsoft Azure IoT Edge and AWS IoT. Trends described includes Edge computing, Security, Cognitive Computing, Analytics, Containers and Microservices
Data Analytics in your IoT SolutionFukiat Julnual, Technical Evangelist, Mic...BAINIDA
Data Analytics in your IoT SolutionFukiat Julnual, Technical Evangelist, Microsoft (Thailand) Limited ในงาน THE FIRST NIDA BUSINESS ANALYTICS AND DATA SCIENCES CONTEST/CONFERENCE จัดโดย คณะสถิติประยุกต์และ DATA SCIENCES THAILAND
Powering the Internet of Things with Apache HadoopCloudera, Inc.
Without the right data management strategy, investments in Internet of Things (IoT) can yield limited results. Apache Hadoop has emerged as a key architectural component that can help make sense of IoT data, enabling never before seen data products and solutions.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Quantitative Data AnalysisReliability Analysis (Cronbach Alpha) Common Method...2023240532
Quantitative data Analysis
Overview
Reliability Analysis (Cronbach Alpha)
Common Method Bias (Harman Single Factor Test)
Frequency Analysis (Demographic)
Descriptive Analysis
3. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Includes
- 2000 IoT sensors
- Smart meters
For What?
- Predicting demands
- Improved Energy and Water
management
Link
3
8. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Cloud computing limitations
➔ Speed - Higher time to react (TTR)
➔ Real-time Decision-making and actuation delayed
➔ Connectivity issues
➔ Privacy concerns
Gartner states 90% of the data created is unusable, 32% is inaccurate
8
9. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Edge Computing
Edge computing is the process of offloading compute and storage from the
centralized Cloud to the network’s logical extremes
➔ Provides ultra-low latency ⚡
➔ Eases the network bottleneck
➔ Supports local data processing
➔ Enables previously impossible applications
9
11. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Fog Computing (termed by Cisco) Edge Computing
- Nodes, mini-cloud
- Compute and storage- Local network
assets, micro-data centres
- Insights from mini cloud
- Device
- Compute and storage resides at the
edge level, close to the device
- Insights directly from device
Edge Platforms- Microsoft Azure IoT Edge, Lambda@Edge, Dell’s Edge Gateways,
VMware's Pulse IoT Center, IBM Watson IoT Platform
e.g. Connected hospital, Surveillance e.g. Connected valves, sensors
Layers: Edge and Fog
11
12. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Edge Analytics
Edge analytics is an automated analytical computation of data at a sensor,
network switch or other device level
➔ Analyzing data as it's generated, near to the source
➔ Decreases latency in the decision-making process
➔ Powers scalability, subset of data sent to cloud for storage and to power Big
Data Analytics
Platform- IBM's IoT analytics, AWS IoT Analytics, SAS
Models from cloud to edge- PMML
12
13. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Home: Water use info and data
Cloud
Smart Water Meter
City Water department:
Water use info and data
Computing
➔ Collect data, log, track
➔ Send data to cloud for
storage, enable to train
models
13
Society Water
Management
Leaks/overflow- needs
immediate action (low TTR)
14. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Home: Water use info and data
Cloud
Smart Water Meter
City Water department:
Water use info and data
14
Society Water
Management
Analytics
➔ Deploy models trained in cloud at the edge
➔ Water meter sends alert to homeowner
➔ Meter automatically controls the valve to shut off
water supply
Leaks/overflow- needs
immediate action (low TTR)
15. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Edge Computing Edge Analytics
Edge computing is using devices strictly to
act as computers to
- log events
- performing inter-device
communication
- tracking location etc
Edge analytics is leveraging that same
device or devices and utilizing them to
process the data that has been computed
and turn it into actionable information
directly on that device
Perform analytical functions- what should
go to cloud, what shouldn’t?
Descriptive OR predictive OR prescriptive
analytics at the edge
Edge: Computing vs Analytics
➔ Computing and analytics are not mutually exclusive
➔ 2 sides of the same coin- one cannot exist without the other
15
16. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Analytical Requirements
Descriptive
● Analyze stored data at rest to provide overview of historical outcomes and performance
● Analyze data in-stream to provide real-time understanding of current state operations
Predictive
● Identify probabilities of potential outcomes and/or likely results of specific operations
● Provide real-time predictive indicators and likely portfolio of outcomes
Prescriptive
(Automation)
● Provide specific recommendations based on live streaming data to an employee or to
automatically initiate a process
● Contextualize live stream events within current business requirement
● Optimize future recommendations based on trained models
16
17. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Home: Water use info and data
Cloud
Smart Water Meter
City Water department:
Water use info and data
17
Society Water
Management
Descriptive
➔ Month wise water
consumption patterns
➔ Plans for the next
season
Predictive
➔ Society: How many
tankers do we need to
order
➔ Water dept: Plans for
the next season
Leaks/overflow- needs
immediate action (low TTR)
Prescriptive (Automation)
➔ Water meter sends alert to
homeowner
➔ Meter automatically
controls the valve to shut
off water supply
18. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Edge Analytics: Benefits
- On-time decision making
- Removes attic data (Gartner states 90% of the data created is unusable, 32%
is inaccurate)
- Saves processing and uploading time
- Preserves privacy- only processed, compliant data sent to cloud
- Greater production capacity planning and prescriptive maintenance
- More efficient service and warranty processes
- Lower impact of connectivity issues
Limitation: Lost data- subset sent to the cloud
18
20. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Appendix
Smart Water Meter:
https://smarterhomes.com/?gclid=Cj0KCQjwhPfkBRD0ARIsAAcYycFvvj6qLS1mXbqC0iW8agguMZ48t3rdnzSuepHckHZBBRgO1FfnvS
QaApEMEALw_wcB
Cloud vs Fog: https://www.winsystems.com/cloud-fog-and-edge-computing-whats-the-difference/
Edge Platforms- Microsoft Azure IoT Edge, AWS Lambda@Edge, Dell’s Edge Gateways, VMware's Pulse IoT Center, IBM Watson IoT
Platform
Edge Analytics Platform- IBM's IoT analytics, AWS IoT Analytics, SAS
Models from cloud to edge- PMML
Edge Analytics Pros, cons: https://www.rightscale.com/lp/state-of-the-cloud
https://internetofthingsagenda.techtarget.com/blog/IoT-Agenda/The-pros-and-cons-of-IoT-edge-analytics
Other awesome reads: https://www.zdnet.com/article/10-edge-computing-vendors-to-watch/
https://www.slideshare.net/mKrishnaKumar1/open-source-edge-computing-platforms-overview?qid=559f4ab8-ed8c-4928-bbbb-2cddd
723c9e9&v=&b=&from_search=4
https://www.cmswire.com/analytics/what-is-edge-analytics/
https://www.rightscale.com/lp/state-of-the-cloud
https://www.sas.com/content/dam/SAS/en_us/doc/whitepaper1/manage-analytical-life-cycle-continuous-innovation-106179.pdf
https://www.kdnuggets.com/2017/10/edge-analytics.html
https://www.sas.com/content/dam/SAS/en_us/doc/research2/iot-analytics-in-practice-107941.pdf
https://www.gartner.com/ngw/globalassets/en/information-technology/documents/insights/100-data-and-analytics-predictions.pdf
https://searchbusinessanalytics.techtarget.com/definition/edge-analytics
https://www.kdnuggets.com/2016/09/evolution-iot-edge-analytics.html
20
21. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Cloud
Surveillance Control room
- Image processing
- Encryption
- Uploading to cloud
Edge Computing vs Analytics: Surveillance
21
22. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Cloud
Surveillance Control room
- Image processing
- Encryption
- Uploading to cloud
Surveillance: Edge Computing
➔ Collect data, log, track
➔ Send data to cloud for storage,
enable to train models
22
23. WWCode CONNECT 2019 Shreya Mukhopadhyay, Intuit
Cloud
Surveillance Control room
- Image processing
- Encryption
- Uploading to cloud
Surveillance: Edge Analytics
➔ Deploy models trained in cloud at the edge
➔ Send out alerts to hospitals, fire brigade, police
station in case of emergency
➔ Anticipate camera failures- send out maintenance
alerts
➔ Identify blind spots- move cameras in advance
23