IIoT_ML_Architechure_AWS

•Download as PPTX, PDF•

0 likes•147 views

This document discusses architectures for industrial IoT and machine learning platforms using AWS services. It covers key considerations for domains like control, operations, and information. Example use cases include predictive quality, asset condition monitoring, and predictive maintenance using industrial data. The document outlines how to collect machine data from sources like sensors using protocols and technologies like MQTT, OPC-UA, and AWS IoT SiteWise. It proposes architectures with edge gateways, industrial data centers, and streaming the data to AWS services for storage, processing, and machine learning including AWS Kinesis, Kafka, S3, Glue, Athena, SageMaker and QuickSight.

Software

Data Analytics Platform for
Machine Learning
IoT(IIoT) Solution with AWS IoT
Narendra Ganesana

Key Design
Considerations
IIRA(Industrial Internet
Reference Architecture) Design
Architechure must consider
below points:
• Control domain
• Operations domain
• Information domain
• Application domain
• Business domain
ViewPoint in IIRA

IIoT Reference
Architechure & usecases
• Industrial Data for Predictive Quality
• Industrial Data for Asset Condition
Monitoring
• Industrial data for Predictive Maintenance
• Industrial Data for Asset Tracking TPT
Monitoring
• Industrial Data for Pricing Model

Sources of Machine Data
Sensors Caluclations
Predictions
Automation
Metadata Decisions
Transactions

KeyTerminologies
• Industry 4.0
• IoT Communication Standards

How to Collect the Machine data ?
• KEPServerEX
• Particle
• Samsara
• AWS IoT SiteWise,
deviceWISE and Many
more
• MQTT
• OPC-UA
• OPC-DA
HMI

Industrial DataCenter
Plant
Edge Gateway Servers
connectors:
• AWS Sitewise connector
• AWS greengrass&sitewise
connector
• Losant
Servers:
• Moxa
• OnLogic
• Aws Sitewise edge

Sample Industrial DataCenter Architechure
**Source: Siemens Edge Solution for IIOT

Highlevel Flow of Machine data collection

AWS IoT Greengrass Architechure
OPC-DA machine
OPC UA machine
Other Assets
Gateway Server
Ethernet
AWS IoT Greengrass
Industrial Datacenter
Other protocols
OPC UA protocols
OPC DA protocols
OPC UA STREAM
OWN STREAMING
PROTOCOL
Machine to cloud
data publisher
Streaming Manager
Connection Layer
OPC DA STREAM
IOT Connectivity HUB

IIoT Data architechure in AWS
1
2
3
4
5
6
7
8
9
10

AWS Lambda
Function
AWS Kinesis
IIoT Streaming Layer
AWS Kafka AWS Lambda AWS Firehose AWS S3
AWS Glue
AWS Lake
Formation
AWS Athena
AWS Redshift
AWS EMR
AWS Sagemaker
AWS QuickSight
IOT Core Rule AWS Simple
Storage Service
Producers Consumers Event Source Downstream
workflows
Machine data
Sensors
Windmills
Etc..
Event1
Event2
Event3
Etc..
Enterprise DataLake
AWS API Gateway
AWS Cognito

IIoT ML Layer
1. Setting up storage
2. Moving data
3. Preparing and
cataloguing data
4. Configuring
security policies
5. Making data
available for
consumption

Three Tier Architechure to the Functional Domains
• Edge Tier
• Platform Tier
• Enterprise Tier

Data Driven Architectures
• Data Mesh vs. Data Fabric
• Data Lake vs Data Hub
• How to Choose ?
Data Mesh: API-driven [solution] for developers, unlike [data] fabric
Data Fabric: low-code, no-code, which means that the API integration is
happening inside of the fabric without actually leveraging it directly
Data hub as an architectural concept is different from data hub as a database
Finally, they are architectural frameworks, not architectures. You don’t have
architecture until the frameworks are adapted and customized to your needs, your
data, your processes, and your terminology

Technologies Involved and Justification
Technologies Tools
Architechure Framework for IIOT Data Mesh
Streaming IOT layer rules AWS MSK and Kinesis
Serverless Lambda
ML Engine workspace Sagemaker, EMR, AWS ECS,
Database DynamoDB, Redshift
Dashboards Amazon Quick Sight
ETL & ELT Aws Glue, EMR
Binary Management Jfrog Artifactory
Monitor Cloudwatch & Prometheus
CI/CD Pipelines Jenkins
Graphdatabases Neptune
Serverless Query AWS Athena
Infrastructure as Code Terraform
Cataloging AWS LakeFormation
Why Kafka ?
1. Supports replay, analytics, and
machine learning based use cases.
2. Reduced Cost
3. Integration with Enterprise Data
Lake
4. Optimize Machine Learning
Workloads
5. Compliance Backloading data to
other system

Modern race cars produce lot of data, and all this in real time. In this presentation I will show you how data could be generated and used by various applications in the car, on the track or team head quarter. The demonstration will show how to move data using messaging systems like Apache Kafka, process the data using Apache Spark and use various storage technics: Distributed File System, NoSQL Database. This presentation is a great opportunity to see how to build a " near real time big data application". The code from this talk will be made available as open source.

Customer Experience at Disney+ Through Data Perspective

Databricks

Disney+ has rapidly scaled to provide a personalized and seamless experience to tens of millions of customers. This experience is powered by a robust data platform that ingests, processes and surfaces billions of events per hour using Delta lake, Databricks, and AWS technologies. The data produced by the platform is used by multitude of services including a recommendation engine for personalized experience, optimizing watch experience including group watch, and fraud and abuse prevention. In this session, you will learn how Disney+ built these capabilities, the architecture, technologies, design principles, and technical details that make it possible.

Data Privacy & Governance in the Age of Big Data: Deploy a De-Identified Data...

Amazon Web Services

Come to this session to learn a new approach in reducing risk and costs while increasing productivity, organizational alacrity, and customer experience, resulting in a competitive advantage and assorted revenue growth. We share how a de-identified data lake on AWS can help you comply with General Data Protection Regulation (GDPR) and California Consumer Protection Act requirements by solving the issue at its causal element.

ABD330_Combining Batch and Stream Processing to Get the Best of Both Worlds

Amazon Web Services

This document discusses implementing the Lambda architecture for combining batch and stream processing on AWS. It provides an overview of the Lambda architecture and its components. It then describes how to implement each layer (batch, speed, serving) using various AWS services like Kinesis, Athena, EMR, and Glue. It also provides an example of merging batch and streaming data using Spark SQL. Finally, it discusses security options for encrypting data in transit and at rest.

Democratizing data science Using spark, hive and druid

DataWorks Summit

MZ is re-inventing how the entire world experiences data via our mobile games division MZ Games Studios, our digital marketing division Cognant, and our live data platform division Satori. Growing need of data science capabilities across the organization requires an architecture that can democratize building these applications and disseminating insight from the outcome of data science applications to the wider organization. Attend this session to learn about how we built a platform for data science using spark, hive, and druid specifically for our performance marketing division cognant.This platform powers several data science application like fraud detection and bid optimization at large scale. We will be sharing lessons learned over past 3 years in building this platform by also walking through some of the actual data science applications built on top of this platform. Attendees from ML engineering and data science background can gain deep insight from our experience of building this platform. Speakers Pushkar Priyadarshi, Director of Engineer, Michaine Zone Inc Igor Yurinok, Staff Software Engineer, MZ

R, Spark, Tensorflow, H20.ai Applied to Streaming Analytics

Kai Wähner

Build and Run Streaming Applications with Apache Flink and Amazon Kinesis Dat...

Flink Forward

Stream processing facilitates the collection, processing, and analysis of real-time data and enables the continuous generation of insights and quick reactions to emerging situations. Yet, despite these advantages compared to traditional batch-oriented analytics applications, streaming applications are much more challenging to operate. Some of these challenges include the ability to provide and maintain low end-to-end latency, to seamlessly recover from failure, and to deal with a varying amount of throughput. We all know and love Flink to take on those challenges with grace. In this session, we explore an end to end example that shows how you can use Apache Flink and Amazon Kinesis Data Analytics for Java Applications to build a reliable, scalable, and highly available streaming applications. We discuss how you can leverage managed services to quickly build Flink based streaming applications and show managed services can help to substantially reduce the operational overhead that is required to run the application. We also review best practices for running streaming applications with Apache Flink on AWS. So you will not only see how to actually build streaming applications with Apache Flink on AWS, you will also learn how leveraging managed services can help to reduce the overhead that is usually required to build and operate streaming applications to a bare minimum.

Cloud-based Data Lake for Analytics and AI

Torsten Steinbach

IBM's Cloud-based Data Lake for Analytics and AI presentation covered: 1) IBM's cloud data lake provides serverless architecture, low barriers to entry, and pay-as-you-go pricing for analytics on data stored in cloud object storage. 2) The data lake offers SQL-based data exploration, transformation, and analytics capabilities as well as industry-leading optimizations for time series and geospatial data. 3) Security features include customer-controlled encryption keys and options to hide SQL queries and keys from IBM.

Jules Damji and Denny Lee from Databricks Developer Relations will recap some keynote highlights, and each will briefly present personal picks from sessions that resonated well with them. Next, Jacek Laskowski, an independent consultant, will speak about Spark 3.0 internals, and Scott Haines from Twilio, Inc. will give a talk about structured streaming microservice architectures. This live coding session and technical deep dive are not to be missed!

Columbia Migrates from Legacy Data Warehouse to an Open Data Platform with De...

Databricks

How Workato creates robust data pipelines and automations for you?

Jeraldine Phneah

Preparing Data for the Lake

Amazon Web Services

Welcome & AWS Big Data Solution Overview

Amazon Web Services

講師: Ivan Cheng, Solution Architect, AWS Join us for a series of introductory and technical sessions on AWS Big Data solutions. Gain a thorough understanding of what Amazon Web Services offers across the big data lifecycle and learn architectural best practices for applying those solutions to your projects. We will kick off this technical seminar in the morning with an introduction to the AWS Big Data platform, including a discussion of popular use cases and reference architectures. In the afternoon, we will deep dive into Machine Learning and Streaming Analytics. We will then walk everyone through building your first Big Data application with AWS.

Welcome and AWS Big Data Solution Overview

Amazon Web Services

This document discusses how enterprises are being disrupted by new technologies and how they can leverage data and analytics on the cloud. It introduces the concepts of the "Third Platform" of mobile, social, and cloud technologies driving IT spending. It then outlines how companies can build a modern data architecture on AWS to ingest, store, process, analyze and gain insights from data in real-time and at scale. This will allow them to enhance applications and digital services with data-driven insights.

Business Intelligence in Minutes with Amazon Athena and Amazon QuickSight - A...

Amazon Web Services

Business Intelligence in Minutes with Amazon Athena and Amazon QuickSight In this session you will learn how to leverage Amazon S3, Amazon Athena, and Amazon QuickSight to explore and visualise data without having to manage a database or spin up a server. We will show you how to upload a dataset to a Data Lake in the AWS Cloud, optimise it in a format that enables high speed queries using SQL, and create rich web-based visualisations from those results. Warren Paull, Solutions Architect, Amazon Web Services

AWS Community Day Nordics 2018 - Aino Health: Transition to serverless and le...

Rolf Koski

Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...

Amazon Web Services

Turn Big Data Into Big Value On Informatica and Amazon

Amazon Web Services

講師: Bob Yin, Senior Product Specialist, Informatica These Informatica Cloud offerings are pre-built packages for quick time-to-value for customers looking to fast-track cloud data management initiatives. For example, customers can quickly kick start a new Amazon Redshift data warehouse project and use Informatica Cloud Connector for Amazon Redshift to load it with meaningful connected data from cloud sources such as Salesforce.com or on-premises sources such as relational databases -- all within hours, not months.

GPSWKS301_Comprehensive Big Data Architecture Made Easy

Amazon Web Services

A modern Big Data architecture involves extending your on-premises data management to AWS, implementing a data pipeline to stream real-time data into cloud data warehouse Amazon Redshift, perform data transformation, discovery, predictive analytics through machine learning, visualize complex information and be notified to respond to business events. This session is for APN Consulting Partners and organizations looking for ways to accelerate and modernize their Big Data projects. You will learn how to deploy and integrate AWS Services with Third-party Solutions in AWS Marketplace. Reduce your time to market by combining AWS services, open source software and ready-to-run on AWS solutions. Familiarity with Database technologies required. The session includes demonstrations and cooperative learning group activities.

Using Big Data to Driving Big Engagement

Amazon Web Services

Building a Modern Data Architecture on AWS - Webinar

Amazon Web Services

Amazon Web Services gives you fast access to flexible and low cost IT resources, so you can rapidly scale and build virtually any big data application including data warehousing, clickstream analytics, fraud detection, recommendation engines, event-driven ETL, serverless computing, and internet-of-things processing regardless of volume, velocity, and variety of data. https://aws.amazon.com/webinars/anz-webinar-series/

How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017

Amazon Web Services

As data volumes grow and customers store more data on AWS, they often have valuable data that is not easily discoverable and available for analytics. The AWS Glue Data Catalog provides a central view of your data lake, making data readily available for analytics. We introduce key features of the AWS Glue Data Catalog and its use cases. Learn how crawlers can automatically discover your data, extract relevant metadata, and add it as table definitions to the AWS Glue Data Catalog. We will also explore the integration between AWS Glue Data Catalog and Amazon Athena, Amazon EMR, and Amazon Redshift Spectrum.

ABD312_Deep Dive Migrating Big Data Workloads to AWS

Amazon Web Services

ABD311_Deploying Amazon QuickSight For Enterprise

Amazon Web Services

One of the biggest tradeoffs customers usually make when deploying BI solutions at scale is agility versus governance. Large-scale BI implementations with the right governance structure can take months to design and deploy. In this session, learn how you can avoid making this tradeoff using Amazon QuickSight. Learn how to easily deploy Amazon QuickSight to thousands of users using Active Directory and Federated SSO, while securely accessing your data sources in Amazon VPCs or on-premises. We also cover how to control access to your datasets, implement row-level security, create scheduled email reports, and audit access to your data.

How Amazon.com uses AWS Analytics

Amazon Web Services

The document discusses Amazon's use of AWS analytics technologies. It describes Amazon's enterprise data warehouse, which stores over 5 petabytes of integrated data from multiple sources. It faces challenges from rapid data growth and limited IT budgets. Amazon is addressing this by building a data lake called "Andes" that stores data in S3 and serves as a common source. Teams can use services like Redshift, EMR, and Athena to analyze the data through subscriptions that synchronize datasets. This approach aims to provide scalability and choices for analytics at Amazon.

Earth on AWS - Next-Generation Open Data Platforms

Amazon Web Services

Making Earth observation data available by using Amazon S3 is accelerating scientific discovery and enabling the creation of new products. Attend and learn how the scale and performance of Amazon S3 lets earth scientists, researchers, startups, and GIS professionals gather and analyse planetary-scale data without worrying about limitations of bandwidth, storage, memory, or processing power. Co-presented with support of the Australian Geoscience Data Cube collaboration, DigitalGlobe’s Geospatial Big Data Platform and the developer of the popular ObservedEarth mobile app. Speakers: Craig Lawton, Public Sector Solutions Architect, Amazon Web Services Lachlan Hurst, Observed Earth Matt Paget, Senior Experimental Scientist, CSIRO Dan Getman, Digital Globe

(BDT306) Mission-Critical Stream Processing with Amazon EMR and Amazon Kinesi...

Amazon Web Services

Organizations processing mission critical high-volume data must be able to achieve high levels of throughput and durability in data processing workflows. In this session, we will learn how DataXu is using Amazon Kinesis, Amazon S3, and Amazon EMR for its patented approach to programmatic marketing. Every second, the DataXu Marketing Cloud processes over 1 Million ad requests and makes more than 40 billion decisions to select and bid on ad impressions that are most likely to convert. In addition to addressing the scalability and availability of the platform, we will explore Amazon Kinesis producer and consumer applications that support high levels of scalability and durability in mission-critical record processing.

Regulatory Reporting of Asset Trading Using Apache Spark-(Sudipto Shankar Das...

Spark Summit

1) Financial institutions must report asset trades within a short regulatory window based on changing rules. 2) A leading financial institution executes around 6 million trades daily and must report qualifying trades in a specified format within 15 minutes. 3) The institution transforms, enriches, de-duplicates trades and applies business rules to identify reportable trades for generation within 35 seconds for 30,000 records using Spark on a 10 node cluster.

Leveraging Operational Data in the Cloud

Inductive Automation

Travis Cox from Inductive Automation and Arlen Nipper from Cirrus Link Solutions discusses the various ways that tag data can be leveraged through cloud services provided by Amazon Web Services and Microsoft Azure. These experts will also show you different ways to get data up to the cloud in a simple, efficient, and secure manner. Learn more about cloud services such as: - Machine learning - Analytics - Business intelligence - Data lakes - Cloud databases - And more

Leveraging Operational Data in the Cloud

Inductive Automation

What's hot

Building a Streaming Microservices Architecture - Data + AI Summit EU 2020

Databricks

Columbia Migrates from Legacy Data Warehouse to an Open Data Platform with De...

Databricks

How Workato creates robust data pipelines and automations for you?

Jeraldine Phneah

Preparing Data for the Lake

Amazon Web Services

Welcome & AWS Big Data Solution Overview

Amazon Web Services

Welcome and AWS Big Data Solution Overview

Amazon Web Services

Business Intelligence in Minutes with Amazon Athena and Amazon QuickSight - A...

Amazon Web Services

AWS Community Day Nordics 2018 - Aino Health: Transition to serverless and le...

Rolf Koski

Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...

Amazon Web Services

Turn Big Data Into Big Value On Informatica and Amazon

Amazon Web Services

GPSWKS301_Comprehensive Big Data Architecture Made Easy

Amazon Web Services

Using Big Data to Driving Big Engagement

Amazon Web Services

Building a Modern Data Architecture on AWS - Webinar

Amazon Web Services

How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017

Amazon Web Services

ABD312_Deep Dive Migrating Big Data Workloads to AWS

Amazon Web Services

ABD311_Deploying Amazon QuickSight For Enterprise

Amazon Web Services

How Amazon.com uses AWS Analytics

Amazon Web Services

Earth on AWS - Next-Generation Open Data Platforms

Amazon Web Services

(BDT306) Mission-Critical Stream Processing with Amazon EMR and Amazon Kinesi...

Amazon Web Services

Regulatory Reporting of Asset Trading Using Apache Spark-(Sudipto Shankar Das...

Spark Summit

What's hot (20)

Building a Streaming Microservices Architecture - Data + AI Summit EU 2020

Columbia Migrates from Legacy Data Warehouse to an Open Data Platform with De...

How Workato creates robust data pipelines and automations for you?

Preparing Data for the Lake

Welcome & AWS Big Data Solution Overview

Welcome and AWS Big Data Solution Overview

Business Intelligence in Minutes with Amazon Athena and Amazon QuickSight - A...

AWS Community Day Nordics 2018 - Aino Health: Transition to serverless and le...

Building Data Lakes That Cost Less and Deliver Results Faster - AWS Online Te...

Turn Big Data Into Big Value On Informatica and Amazon

GPSWKS301_Comprehensive Big Data Architecture Made Easy

Using Big Data to Driving Big Engagement

Building a Modern Data Architecture on AWS - Webinar

How to build a data lake with aws glue data catalog (ABD213-R) re:Invent 2017

ABD312_Deep Dive Migrating Big Data Workloads to AWS

ABD311_Deploying Amazon QuickSight For Enterprise

How Amazon.com uses AWS Analytics

Earth on AWS - Next-Generation Open Data Platforms

(BDT306) Mission-Critical Stream Processing with Amazon EMR and Amazon Kinesi...

Regulatory Reporting of Asset Trading Using Apache Spark-(Sudipto Shankar Das...

Similar to IIoT_ML_Architechure_AWS

Leveraging Operational Data in the Cloud

Inductive Automation

Leveraging Operational Data in the Cloud

Inductive Automation

IoTSummit: Create iot devices connected or on the edge using ai and ml

Marco Dal Pino

Azure iot edge and AI enabling the intelligent edge

Marco Dal Pino

Marco Dal Pino presented on Azure IoT Edge and AI capabilities at the edge. He discussed Microsoft's IoT product portfolio including Azure Sphere, IoT Edge, IoT Hub, and Edge appliances. Dal Pino also covered built-in AI capabilities like anomaly detection on IoT Edge as well as cognitive services containers. Finally, he demonstrated Nvidia Deepstream running computer vision models on IoT Edge and discussed resiliency, observability, and storage options for IoT Edge deployments.

Breaking down an Industrial IoT reference architecture.pptx

Neel Sendas

The document presents an industrial IoT reference architecture for connecting industrial facilities to the cloud. It includes: - A breakdown of the common business and technical challenges faced in industrial digital transformation. - A low-level reference architecture showing how to create a unified OT/IT data backbone integrating edge, cloud, security, data management, applications and insights. - Examples of how the architecture enables use cases across industries like predictive maintenance, quality control and remote monitoring. - Recommendations around executing a modern data strategy and taking a holistic approach to maximize insights, minimize technical debt and future-proof the solution.

05 internet-of-things-io t-cloudcomputing

John Soldatos

This document discusses the integration of IoT and cloud computing. It provides overviews of cloud computing, including characteristics, stakeholders, and service delivery models. It discusses the motivation for integrating IoT and cloud, including leveraging cloud benefits and addressing conflicting properties. Integration models are described including adapting cloud models (IaaS, PaaS, SaaS) to IoT. Challenges of integration like data quality and lack of interoperability are covered along with solutions like abstraction layers and adapting existing cloud models. Edge/fog computing is introduced as an approach to address limitations of full cloud integration. Popular public IoT cloud platforms and their architectures are analyzed.

AWS re:Invent 2016: IoT: Build, Test, and Securely Scale (GPST302)

Amazon Web Services

With the rapid adoption of IoT services on AWS, how do partners and organizations effectively build, test, scale, and secure these highly transaction-data laden systems? This session is a deep dive on the API, SDK, device gateway, rules engine, and device shadows. Consulting and Technology Partner customers share their experiences as we highlight lessons learned and best practices to increase audience efficacy.

Io t solutions world congress 2018 review Henk Jan van Wijk Conclusion Connect

Conclusion Connect enabling industry 4.0 with IoT

Programma 16.30 Ontvangst 17.00 Robbrecht van Amerongen - Head of IoT, Conclusion Connect • Welkom en opvallendste zaken van het congres • Stand van zaken PaaS/SaaS, Digital Twin, Context IoT, Big Data en Machine Learning • IoT en Security • IoT-projecten: stand van zaken, opvallende voorbeelden en best practices • Waar staan de vendors/ platformleveranciers 18.00 Diner 18.45 Gertjan van het Hof - IoT Solutions Architect, Conclusion Connect • Platformen: Microsoft Azure IoT Reference Architecture en Google Cloud IoT • CrateDB • EdgeX Foundry en NetFoundry • Intel Video processing • Beacons • Security Maturity Model • Industrial Internet Consortium (IIC) 19:45 Pauze 20:00 Henk Jan van Wijk - IoT Solution Engineer, Conclusion Connect • Microsoft Azure Sphere (connected devices) • Low Power Connectivity (Thingstream) • Digital Twin • Anomaly detection at Intel (predict robots failure) 21.00 Afsluiting en borrel

IoT

Arun Kumar

This document summarizes an IoT and Cortana Analytics presentation. It introduces IoT and the opportunity it presents, describes a typical Azure IoT architecture including IoT Hub, Event Hubs, HDInsight and storage, and provides examples of IoT use cases. It also discusses the Cortana Analytics Suite and its ability to transform data into intelligent action. The presentation includes a demo of a smart building application and a healthcare use case.

Discover MongoDB Atlas and MongoDB Stitch - DEM02-S - Mexico City AWS Summit

Amazon Web Services

Learn about the modernization of application development using the MongoDB platform on AWS. In this session, discover key capabilities of MongoDB Atlas for on-demand cluster deployment, high availability, horizontal scalability, and geographically distributed operations. Additionally, learn how to quickly build a website or mobile application that is backed by MongoDB and that uses the MongoDB Stitch serverless platform.

Azure and Predix

Altoros

This document discusses the partnership between Microsoft Azure and GE's Predix platform for industrial IoT. For Microsoft, the partnership will help existing industrial customers build and operate IIoT solutions using Azure's capabilities in artificial intelligence, data analytics, and security. For GE, Predix will benefit from Azure's large global footprint and hybrid cloud capabilities. The combination of Predix and Azure aims to bridge the gap between operational technology and information technology for industrial customers worldwide.

Internet of Things A Vision, Architectural Elements, and Future Directions

Mostafa Arjmand

Present paper aboat Internet of Things (IoT) A Vision, Architectural Elements, and Future Directions Overall IoT vision and the technologies that will achieve the it Application domains in IoT with a new approach in defining them Cloud centric Internet of Things realization and challenges Case study of data analytics on the Aneka/Azure cloud platform Open Challenges and Future Directions Smart environment application domains Cloud computing Cloud centric Internet of Things Microsoft Azure

Latest trendsincloud computing

Liliana Ignat

Operational information processing: lightning-fast, delightfully simple

Xylos

How can you as an industrial company or service provider collect and analyse data from your operational production equipment in a simple, fast and smart way? During this session, HPE gives you some practical examples – and what’s more, you’ll discover the underlying reference architecture. As a result, you’ll boost the efficiency of your production process and tune the services you provide to the needs of your customers.

Edge Computing and 5G, a powerful digital mix for IoT - AIT

hubraum IoT Academy

The document discusses edge computing and 5G networks for IoT applications. It notes that using cloud computing alone has limitations like wasted bandwidth, high latency, inefficient storage, and limited flexibility for privacy. Edge computing processes data at network edges using devices like gateways and small data centers to address these issues. 5G networks are also discussed as enabling technologies that can help address IoT needs through capabilities like low latency, high data rates, and unified connectivity frameworks. Specific applications highlighted include smart manufacturing using edge and cloud, and autonomous vehicles combining 5G and edge computing.

Soldatos io t-academy-cosmote-231117-v-final

John Soldatos

The document discusses edge computing and 5G networks for IoT applications. It notes that using cloud computing alone has limitations like wasted bandwidth, high latency, inefficient storage, and limited flexibility for privacy. Edge computing processes data at network edges using devices like gateways and small data centers to address these issues. The document also explores several EU-funded projects applying edge computing for applications in manufacturing and autonomous vehicles. It outlines how 5G networks can further support IoT by enabling more data and lower latency connectivity.

Digital Reinvention by NRB

William Poos

How to reinvent your organization in an iterative and pragmatic way? This is the result of using our digital toolbox. It allows you to transform your business model, expand your ecosystem by setting up your digital platform. This reinvention is also supported by the adaptation of your governance allowing you to innovate while guaranteeing the performance of your organization. For any information / suggestion / collaboration - william.poos@nrb.be Comment réinventer votre organisation de manière itérative et pragmatique ? C'est le résultat de l'utilisation de notre boîte à outils digitale. Elle vous permet de transformer votre modèle métier, d'étendre votre écosystème en mettant en place votre plateforme digitale. Cette réinvention est également supportée par l'adaptation de votre gouvernance vous permettant d'innover tout en garantissant la performance de votre organisation. Pour toute information / suggestion / collaboration - william.poos@nrb.be

AggreGate IoT Integration Platform

Tibbo

AggreGate IoT Integration Platform is a white-labeled Internet of Things platform that employs modern communication technologies to control, configure, monitor and service different electronic devices. It also helps you to aggregate device data into a common database, where you can "slice and dice" it according to your needs, as well as let other enterprise applications transparently access it. AggreGate introduces new concepts in M2M (Machine-2-Machine) technology, also known as Industrial Control or SCADA. In a complex network environment your devices may be distributed over multiple segments, located behind firewalls, routers, bridges, etc. AggreGate Server provides a rich set of features for managing device networks and routing data. The core components are Java-based and may be deployed on most present-day hardware and operating systems. Connecting your devices to the system is easy and cost-effective. Any existing device may work with AggreGate regardless of its communication protocol even if it's not network-enabled. You can bridge your existing devices into the system using a programmable controller, by implementing AggreGate communication protocol in your own device, or by protocol conversion via software device drivers. For new designs, the programmable controller can be built directly into the product. Devices that use standard automation, control and monitoring protocols (such as OPC, Modbus, BACNet, SNMP, etc.) are supported directly, no software/hardware protocol conversion is required. AggreGate helps many different industries and companies to benefit from direct management of large networks of legacy devices or connecting them to the enterprise systems based on open standards. Typical solutions include network monitoring, data center management, industrial and home automation, machinery, telecommunications, security and access control, time and attendance control, medical devices, laboratory equipment, life sciences, retail, transportation, etc.

Best Practices for Streaming IoT Data with MQTT and Apache Kafka

Kai Wähner

Organizations today are looking to stream IoT data to Apache Kafka. However, connecting tens of thousands or even millions of devices over unreliable networks can create some architecture challenges. In this session, we will identify and demo some best practices for implementing a large scale IoT system that can stream MQTT messages to Apache Kafka. We use HiveMQ as open source MQTT broker to ingest data from IoT devices, ingest the data in real time into an Apache Kafka cluster for preprocessing (using Kafka Streams / KSQL), and model training + inference (using TensorFlow 2.0 and its TensorFlow I/O Kafka plugin). We leverage additional enterprise components from HiveMQ and Confluent to allow easy operations, scalability and monitoring.

Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...

confluent

This document discusses best practices for streaming IoT data with MQTT and Apache Kafka. It begins with an overview of a use case involving a global automotive company building a connected car infrastructure. An architecture is presented showing how sensor data from cars can be ingested via MQTT into Apache Kafka and then processed using tools like Kafka Streams, TensorFlow, and ElasticSearch for analytics and alerts. A live demo is described that implements this full pipeline. The document concludes with a discussion of best practices around choosing the right tools, separation of concerns, data types, and next steps.

Similar to IIoT_ML_Architechure_AWS (20)

Leveraging Operational Data in the Cloud

IoTSummit: Create iot devices connected or on the edge using ai and ml

Azure iot edge and AI enabling the intelligent edge

Breaking down an Industrial IoT reference architecture.pptx

05 internet-of-things-io t-cloudcomputing

AWS re:Invent 2016: IoT: Build, Test, and Securely Scale (GPST302)

Io t solutions world congress 2018 review Henk Jan van Wijk Conclusion Connect

IoT

Discover MongoDB Atlas and MongoDB Stitch - DEM02-S - Mexico City AWS Summit

Azure and Predix

Internet of Things A Vision, Architectural Elements, and Future Directions

Latest trendsincloud computing

Operational information processing: lightning-fast, delightfully simple

Edge Computing and 5G, a powerful digital mix for IoT - AIT

Soldatos io t-academy-cosmote-231117-v-final

Digital Reinvention by NRB

AggreGate IoT Integration Platform

Best Practices for Streaming IoT Data with MQTT and Apache Kafka

Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka (Kai Waehn...

Recently uploaded

How Can Hiring A Mobile App Development Company Help Your Business Grow?

ToXSL Technologies

Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...

The Third Creative Media

"Navigating Invideo: A Comprehensive Guide" is an essential resource for anyone looking to master Invideo, an AI-powered video creation tool. This guide provides step-by-step instructions, helpful tips, and comparisons with other AI video creators. Whether you're a beginner or an experienced video editor, you'll find valuable insights to enhance your video projects and bring your creative ideas to life.

KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD

rodomar2

Microservice Teams - How the cloud changes the way we work

Sven Peters

A lot of technical challenges and complexity come with building a cloud-native and distributed architecture. The way we develop backend software has fundamentally changed in the last ten years. Managing a microservices architecture demands a lot of us to ensure observability and operational resiliency. But did you also change the way you run your development teams? Sven will talk about Atlassian’s journey from a monolith to a multi-tenanted architecture and how it affected the way the engineering teams work. You will learn how we shifted to service ownership, moved to more autonomous teams (and its challenges), and established platform and enablement teams.

The Rising Future of CPaaS in the Middle East 2024

Yara Milbes

The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...

kalichargn70th171

Energy consumption of Database Management - Florina Jonuzi

Green Software Development

INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES

anfaltahir1010

Image: Include an image that represents the concept of precision, such as a AI helix or a futuristic healthcare setting. Objective: Provide a foundational understanding of precision medicine and its departure from traditional approaches Role of theory: Discuss how genomics, the study of an organism's complete set of AI , plays a crucial role in precision medicine. Customizing treatment plans: Highlight how genetic information is used to customize treatment plans based on an individual's genetic makeup. Examples: Provide real-world examples of successful application of AI such as genetic therapies or targeted treatments. Importance of molecular diagnostics: Explain the role of molecular diagnostics in identifying molecular and genetic markers associated with diseases. Biomarker testing: Showcase how biomarker testing aids in creating personalized treatment plans. Content: • Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and potential misuse of genetic information. • Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard patient rights. • Visuals: Include images or icons representing ethical considerations. Content: • Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and potential misuse of genetic information. • Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard patient rights. • Visuals: Include images or icons representing ethical considerations. Content: • Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and potential misuse of genetic information. • Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard patient rights. • Visuals: Include images or icons representing ethical considerations. Real-world case study: Present a detailed case study showcasing the success of precision medicine in a specific medical scenario. Patient's journey: Discuss the patient's journey, treatment plan, and outcomes. Impact: Emphasize the transformative effect of precision medicine on the individual's health. Objective: Ground the presentation in a real-world example, highlighting the practical application and success of precision medicine. Data challenges: Address the challenges associated with managing large sets of patient data in precision medicine. Technological solutions: Discuss technological innovations and solutions for handling and analyzing vast datasets. Visuals: Include graphics representing data management challenges and technological solutions. Objective: Acknowledge the data-related challenges in precision medicine and highlight innovative solutions. Data challenges: Address the challenges associated with managing large sets of patient data in precision medicine. Technological solutions: Discuss technological innovations and solutions

Oracle 23c New Features For DBAs and Developers.pptx

Remote DBA Services

Measures in SQL (SIGMOD 2024, Santiago, Chile)

Julian Hyde

SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries. SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL. To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context. A talk at SIGMOD, June 9–15, 2024, Santiago, Chile Authors: Julian Hyde (Google) and John Fremlin (Google) https://doi.org/10.1145/3626246.3653374

UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem

Peter Muessig

GreenCode-A-VSCode-Plugin--Dario-Jurisic

Green Software Development

ppt on the brain chip neuralink.pptx

Reetu63

Unveiling the Advantages of Agile Software Development.pdf

brainerhub1

What’s New in Odoo 17 – A Complete Roadmap

Envertis Software Solutions

Odoo releases a new update every year. The latest version, Odoo 17, came out in October 2023. It brought many improvements to the user interface and user experience, along with new features in modules like accounting, marketing, manufacturing, websites, and more. The Odoo 17 update has been a hot topic among startups, mid-sized businesses, large enterprises, and Odoo developers aiming to grow their businesses. Since it is now already the first quarter of 2024, you must have a clear idea of what Odoo 17 entails and what it can offer your business if you are still not aware of it. This blog covers the features and functionalities. Explore the entire blog and get in touch with expert Odoo ERP consultants to leverage Odoo 17 and its features for your business too. An Overview of Odoo ERP Odoo ERP was first released as OpenERP software in February 2005. It is a suite of business applications used for ERP, CRM, eCommerce, websites, and project management. Ten years ago, the Odoo Enterprise edition was launched to help fund the Odoo Community version. When you compare Odoo Community and Enterprise, the Enterprise edition offers exclusive features like mobile app access, Odoo Studio customisation, Odoo hosting, and unlimited functional support. Today, Odoo is a well-known name used by companies of all sizes across various industries, including manufacturing, retail, accounting, marketing, healthcare, IT consulting, and R&D. The latest version, Odoo 17, has been available since October 2023. Key highlights of this update include: Enhanced user experience with improvements to the command bar, faster backend page loading, and multiple dashboard views. Instant report generation, credit limit alerts for sales and invoices, separate OCR settings for invoice creation, and an auto-complete feature for forms in the accounting module. Improved image handling and global attribute changes for mailing lists in email marketing. A default auto-signature option and a refuse-to-sign option in HR modules. Options to divide and merge manufacturing orders, track the status of manufacturing orders, and more in the MRP module. Dark mode in Odoo 17. Now that the Odoo 17 announcement is official, let’s look at what’s new in Odoo 17! What is Odoo ERP 17? Odoo 17 is the latest version of one of the world’s leading open-source enterprise ERPs. This version has come up with significant improvements explained here in this blog. Also, this new version aims to introduce features that enhance time-saving, efficiency, and productivity for users across various organisations. Odoo 17, released at the Odoo Experience 2023, brought notable improvements to the user interface and added new functionalities with enhancements in performance, accessibility, data analysis, and management, further expanding its reach in the market.

ALGIT - Assembly Line for Green IT - Numbers, Data, Facts

Green Software Development

J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...

Bert Jan Schrijver

WWDC 2024 Keynote Review: For CocoaCoders Austin

Patrick Weigel

Mobile App Development Company In Noida | Drona Infotech

Drona Infotech

Malibou Pitch Deck For Its €3M Seed Round

sjcobrien

Recently uploaded (20)

How Can Hiring A Mobile App Development Company Help Your Business Grow?

Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...

KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD

Microservice Teams - How the cloud changes the way we work

The Rising Future of CPaaS in the Middle East 2024

The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...

Energy consumption of Database Management - Florina Jonuzi

INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLES

Oracle 23c New Features For DBAs and Developers.pptx

Measures in SQL (SIGMOD 2024, Santiago, Chile)

UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem

GreenCode-A-VSCode-Plugin--Dario-Jurisic

ppt on the brain chip neuralink.pptx

Unveiling the Advantages of Agile Software Development.pdf

What’s New in Odoo 17 – A Complete Roadmap

ALGIT - Assembly Line for Green IT - Numbers, Data, Facts

J-Spring 2024 - Going serverless with Quarkus, GraalVM native images and AWS ...

WWDC 2024 Keynote Review: For CocoaCoders Austin

Mobile App Development Company In Noida | Drona Infotech

Malibou Pitch Deck For Its €3M Seed Round

IIoT_ML_Architechure_AWS

1. Data Analytics Platform for Machine Learning IoT(IIoT) Solution with AWS IoT Narendra Ganesana

2. Key Design Considerations IIRA(Industrial Internet Reference Architecture) Design Architechure must consider below points: • Control domain • Operations domain • Information domain • Application domain • Business domain ViewPoint in IIRA

3. IIoT Reference Architechure & usecases • Industrial Data for Predictive Quality • Industrial Data for Asset Condition Monitoring • Industrial data for Predictive Maintenance • Industrial Data for Asset Tracking TPT Monitoring • Industrial Data for Pricing Model

4. Sources of Machine Data Sensors Caluclations Predictions Automation Metadata Decisions Transactions

5. KeyTerminologies • Industry 4.0 • IoT Communication Standards

6. How to Collect the Machine data ? • KEPServerEX • Particle • Samsara • AWS IoT SiteWise, deviceWISE and Many more • MQTT • OPC-UA • OPC-DA HMI

7. Industrial DataCenter Plant Edge Gateway Servers connectors: • AWS Sitewise connector • AWS greengrass&sitewise connector • Losant Servers: • Moxa • OnLogic • Aws Sitewise edge

8. Sample Industrial DataCenter Architechure **Source: Siemens Edge Solution for IIOT

9. Highlevel Flow of Machine data collection

10. AWS IoT Greengrass Architechure OPC-DA machine OPC UA machine Other Assets Gateway Server Ethernet AWS IoT Greengrass Industrial Datacenter Other protocols OPC UA protocols OPC DA protocols OPC UA STREAM OWN STREAMING PROTOCOL Machine to cloud data publisher Streaming Manager Connection Layer OPC DA STREAM IOT Connectivity HUB

11. IIoT Data architechure in AWS 1 2 3 4 5 6 7 8 9 10

12. AWS Lambda Function AWS Kinesis IIoT Streaming Layer AWS Kafka AWS Lambda AWS Firehose AWS S3 AWS Glue AWS Lake Formation AWS Athena AWS Redshift AWS EMR AWS Sagemaker AWS QuickSight IOT Core Rule AWS Simple Storage Service Producers Consumers Event Source Downstream workflows Machine data Sensors Windmills Etc.. Event1 Event2 Event3 Etc.. Enterprise DataLake AWS API Gateway AWS Cognito

13. IIoT ML Layer 1. Setting up storage 2. Moving data 3. Preparing and cataloguing data 4. Configuring security policies 5. Making data available for consumption

14. Core AWS Technologies for IIoT Platform

15. Three Tier Architechure to the Functional Domains • Edge Tier • Platform Tier • Enterprise Tier

16. IIOT AWS ML Architecture birds eye view

17. Data Driven Architectures • Data Mesh vs. Data Fabric • Data Lake vs Data Hub • How to Choose ? Data Mesh: API-driven [solution] for developers, unlike [data] fabric Data Fabric: low-code, no-code, which means that the API integration is happening inside of the fabric without actually leveraging it directly Data hub as an architectural concept is different from data hub as a database Finally, they are architectural frameworks, not architectures. You don’t have architecture until the frameworks are adapted and customized to your needs, your data, your processes, and your terminology

18. Technologies Involved and Justification Technologies Tools Architechure Framework for IIOT Data Mesh Streaming IOT layer rules AWS MSK and Kinesis Serverless Lambda ML Engine workspace Sagemaker, EMR, AWS ECS, Database DynamoDB, Redshift Dashboards Amazon Quick Sight ETL & ELT Aws Glue, EMR Binary Management Jfrog Artifactory Monitor Cloudwatch & Prometheus CI/CD Pipelines Jenkins Graphdatabases Neptune Serverless Query AWS Athena Infrastructure as Code Terraform Cataloging AWS LakeFormation Why Kafka ? 1. Supports replay, analytics, and machine learning based use cases. 2. Reduced Cost 3. Integration with Enterprise Data Lake 4. Optimize Machine Learning Workloads 5. Compliance Backloading data to other system

19. Questions ?

IIoT_ML_Architechure_AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to IIoT_ML_Architechure_AWS

Similar to IIoT_ML_Architechure_AWS (20)

Recently uploaded

Recently uploaded (20)

IIoT_ML_Architechure_AWS