Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ABD322_Implementing a Flight Simulator Interface Using AI, Virtual Reality, and Big Data on AWS

357 views

Published on

This workshop explores the technology options, architectures, and implementations associated with instrumenting AR, VR, and simulated worlds. Using flight simulation as the primary use case, you learn to consume, process, store, and analyze high velocity telemetry as well as exploring control plane implementations using AWS IoT, AWS Lambda, Amazon Kinesis, and Amazon SNS. This is a hands-on workshop and you need a laptop (tablets are not suitable). You should have a solid understanding of AWS products and Node.js.

  • Be the first to comment

ABD322_Implementing a Flight Simulator Interface Using AI, Virtual Reality, and Big Data on AWS

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:INVENT Implementing a Flight Simulator Interface Using AI, Virtual Reality, and Big Data on AWSM a r c T e i c h t a h l — M a n a g e r , S o l u t i o n s A r c h i t e c t u r e M a r t e n P a y n e — T e c h n i c a l A c c o u n t M a n a g e r , A N Z N o v e m b e r 2 8 , 2 0 1 7 A B D 3 2 2
  2. 2. Your Hosts for the Day Marc Teichtahl • Manager Solutions Architect, Melbourne Australia
  3. 3. Your Hosts for the Day Marc Teichtahl • Manager Solutions Architect, Melbourne Australia Marten Payne • Technical Account Manager , Melbourne Australia
  4. 4. Your Hosts for the Day Marc Teichtahl • Manager Solutions Architect, Melbourne Australia Marten Payne • Technical Account Manager , Melbourne Australia Daniel O‘Brien • Solutions Architect, Wellington, New Zealand
  5. 5. Your Hosts for the Day Having trouble? The proctors are here to help! https://pixabay.com/en/balloon-talk-language-german-males-2557031/
  6. 6. Today Section 1: Introduction to AR, VR, and Synthetic Worlds Section 2: Architectures and Technology Break Section 3: Hands-On Workshop
  7. 7. Today Section 1: Introduction to AR, VR, and Synthetic Worlds Section 2: Architectures and Technology Break Section 3: Hands-On Workshop
  8. 8. Today Section 1: Introduction to AR, VR, and Synthetic Worlds Section 2: Architectures and Technology Break Section 3: Hands-On Workshop
  9. 9. Today Section 1: Introduction to AR, VR, and Synthetic Worlds Section 2: Architectures and Technology Break Section 3: Hands-On Workshop
  10. 10. Today Section 1: Introduction to AR, VR, and Synthetic Worlds Section 2: Architectures and Technology Break Section 3: Hands-On Workshop
  11. 11. https://pixabay.com/en/network-cable-ethernet-plug-1027308/ Shameless plug!
  12. 12. Come See It All In Action re:Invent Builders Fair https://pixabay.com/en/network-cable-ethernet-plug-1027308/ The Quad @ Aria
  13. 13. Come See It All In Action re:Invent Builders Fair https://pixabay.com/en/network-cable-ethernet-plug-1027308/
  14. 14. Section 1: Introduction to AR, VR, and Synthetic Worlds
  15. 15. Our Changing Reality
  16. 16. Our Changing Reality
  17. 17. Our Changing Reality
  18. 18. Our Changing Reality
  19. 19. Our Changing Reality Synthetic worlds
  20. 20. These Worlds Are Similar but Different Virtual reality Virtual “things” in a virtual world
  21. 21. These Worlds Are Similar but Different Virtual reality Virtual “things” in a virtual world
  22. 22. These Worlds Are Similar but Different Virtual reality Augmented reality Virtual “things” in a virtual world Virtual “things” in the real world
  23. 23. These Worlds Are Similar but Different Virtual reality Augmented reality Virtual “things” in a virtual world Virtual “things” in the real world
  24. 24. These Worlds Are Similar but Different Virtual reality Augmented reality Simulators Virtual “things” in a virtual world Virtual “things” in the real world Real “things” in a virtual world
  25. 25. These Worlds Are Similar but Different Virtual reality Augmented reality Simulators Virtual “things” in a virtual world Virtual “things” in the real world Real “things” in a virtual world
  26. 26. These Worlds Are Similar but Different Virtual reality Augmented reality Simulators Virtual “things” in a virtual world Virtual “things” in the real world Real “things” in a virtual world Synthetic worlds
  27. 27. However, They Have Similar Attributes Many commonalities
  28. 28. However, They Have Similar Attributes Many commonalities Visual
  29. 29. However, They Have Similar Attributes Many commonalities Visual
  30. 30. However, They Have Similar Attributes Many commonalities Visual Tactile
  31. 31. However, They Have Similar Attributes Many commonalities Visual Tactile Audible
  32. 32. How do we think about these worlds? https://pixabay.com/en/steelwork-engineering-structure-1031611/
  33. 33. A Framework for Synthetic Worlds Simulate • Models • Systems • Coordinates • Inputs • Outputs
  34. 34. A Framework for Synthetic Worlds Simulate Visualize • Views and cameras • Landscapes • Objects • AI
  35. 35. A Framework for Synthetic Worlds Simulate Visualize Interface • Human • Tactile • Switches, dials • Buttons • Motors • Lights
  36. 36. A Framework for Synthetic Worlds Simulate Visualize Interface Process • Mediate • Normalize • Transform • Log and record
  37. 37. A Framework for Synthetic Worlds Simulate Visualize Interface Process Interact • Influence • Impact • Human factors
  38. 38. A Framework for Synthetic Worlds Simulate Visualize Interface Process Interact Learn Machine learning Predictive analytics Feedback Reporting
  39. 39. A Reference Architecture Sim hub toCloudWatch toDynamo CloudWatch Dynamo DB Redshift toRedshift kinesisAnalytics Aggregatedata raw data AML S3 Lex CloudWatch Dashboard kinesis raw data raw data Simulator Visuals Simulate Interface/interact Interact Visualize Learn Process
  40. 40. A Reference Architecture Sim hub toCloudWatch toDynamo CloudWatch Dynamo DB Redshift toRedshift kinesisAnalytics Aggregatedata raw data AML S3 Lex CloudWatch Dashboard kinesis raw data raw data Simulator Visuals Interface/interact Interact Visualize Learn Process
  41. 41. A Reference Architecture Bus toCloudWatch toDynamo CloudWatch Dynamo DB Redshift toRedshift kinesisAnalytics Aggregatedata raw data AML S3 Lex CloudWatch Dashboard kinesis raw data raw data Simulator Visuals Interact Visualize Learn Process Switches, inputs, outputs
  42. 42. A Reference Architecture Bus toCloudWatch toDynamo CloudWatch Dynamo DB Redshift toRedshift Aggregatedata raw data AML S3 Lex CloudWatch Dashboard Kinesis raw data raw data Simulator Visuals Interact Visualize Learn Switches, inputs, outputs Kinesis Analytics
  43. 43. A Reference Architecture Bus Amazon CloudWatch Dynamo DB Redshift Aggregatedata raw data AML S3 Lex CloudWatch Dashboard Kinesis raw data raw data Simulator Visuals InteractLearn Switches, inputs, outputs toCloudWatch toDynamotoRedshift Kinesis Analytics
  44. 44. A Reference Architecture Bus Dynamo DB Redshift Aggregatedata raw data AML S3 Lex CloudWatch Dashboard Kinesis raw data raw data Simulator Visuals Learn Switches, inputs, outputs Amazon CloudWatch toCloudWatch toDynamotoRedshift Kinesis Analytics
  45. 45. A Reference Architecture Bus DynamoDBAmazon Redshift Aggregatedata raw data AML S3 Lex CloudWatch Dashboard Kinesis raw data raw data Simulator Visuals Switches, inputs, outputs toCloudWatch toDynamotoRedshift Kinesis Analytics Amazon CloudWatch
  46. 46. A Reference Architecture Bus Kinesis Analytics Aggregatedata raw data AML S3 Lex CloudWatch Dashboard Kinesis raw data raw data Simulator Visuals Switches, inputs, outputs toCloudWatch toDynamotoRedshift +Greengrass IoT GG DynamoDBAmazon CloudWatch Amazon Redshift
  47. 47. But When We Develop https://pixabay.com/en/entrepreneur-start-start-up-career-696976/
  48. 48. But When We Develop We care about a number https://pixabay.com/en/entrepreneur-start-start-up-career-696976/ of important considerations
  49. 49. What must we consider? Volume, velocity, frequency
  50. 50. What must we consider? Volume, velocity, frequency • The amount of data • The rate at which data is produced/consumed • Frequency of data generated
  51. 51. What must we consider? Volume, velocity, frequency • Kinesis or IoT • Control plane and data plane • Sharding, costs • Threads, concurrency, and locking (mutexes/semaphores)
  52. 52. What must we consider? Latency
  53. 53. What must we consider? Latency • Network • Application • Processing • Human factors
  54. 54. What must we consider? Latency • Network partitions—IoT and AWS Greengrass • Application design and efficiencies • AWS Lambda (warm-up time, and so on) • Motion sickness
  55. 55. What must we consider? Cost, deployment, security
  56. 56. What must we consider? Cost, deployment, security • Choice of technology • Ease of deployment • Continuous deployment • Lambda
  57. 57. You Can Choose Your Own Adventure Visualize Interface Process InteractSimulate
  58. 58. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact
  59. 59. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact SurgerySim TouchSurgery NurseSim
  60. 60. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact
  61. 61. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Lumberyard Unity CryEngine UnrealEngine
  62. 62. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  63. 63. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact RSLogix SOIC Step7 Bus
  64. 64. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  65. 65. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  66. 66. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact AWS IoT Amazon Kinesis Analytics AWS Lambda Amazon Athena AWS CloudWatch Bus
  67. 67. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  68. 68. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  69. 69. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact RaspberryPi Intel Edison Arduino Bus
  70. 70. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Bus
  71. 71. You Can Choose Your Own Adventure Simulate Visualize Interface Process Interact Amazon Lex Amazon Polly Bus
  72. 72. Section 2: Architectures and Technology
  73. 73. Introduction—The Problem Statement Traditionally, it has been complex and expensive to: Securely acquire “near” real-time data from simulator engines Process and store large volumes of data to enable the extraction of meaningful insights Deliver a user experience that intuitively exposes near real-time information
  74. 74. Introduction—What You Will Do Today • Uses AWS Kinesis, AWS Lambda, AWS CloudFormation and AWS API Gateway to securely acquire, process, and publish near real-time flight data from a simulator data source • Implements AWS Lambda and Amazon Kinesis to process flight data using serverless architectures • Stores the flight data for real-time using Amazon DynamoDB • Deploys a user interface to visualize the flight data You will build a system that:
  75. 75. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources
  76. 76. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment
  77. 77. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing
  78. 78. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing Serverless processing
  79. 79. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing User interfaceServerless processing
  80. 80. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing User interface Acquire Serverless processing
  81. 81. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing Serverless processing User interface Acquire Process
  82. 82. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Data sources Time Flight Environment Preprocessing Serverless processing User interface Acquire Process Present
  83. 83. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Time Flight Environment • Various types of data from sensors and systems • Send raw data from the simulator • Real time and simulator time • Pitch, roll, yaw, airspeed, heading, and so on • Wind speed and direction, turbulence • Aggregation of raw data Acquire
  84. 84. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Time Flight Environment • Processing • Storage • Publishing Amazon Aurora Amazon S3 Amazon Redshift AWS Lambda Amazon Kinesis Streams Amazon Kinesis Analytics Amazon Kinesis Firehose Amazon SNS ProcessAcquire
  85. 85. How the Pieces Fit Together https://pixabay.com/en/airplane-plane-aircraft-sign-99047/ Icons made by Freepik from www.flaticon.com Time Flight Environment Final UI image here Acquire Process Present
  86. 86. Data Lifecycle Process Present • A powerful user interface to view flight data • A map, location, and flight info UI to visualize the simulated world Acquire Processing of raw data into final simulation dataset • Smoothed data • Aggregates • Averages Acquisition of data from remote simulator data sources • Flight • Environmental/weather • Time
  87. 87. Data Lifecycle Process PresentAcquire Amazon S3 AWS Lambda Amazon Kinesis Streams
  88. 88. Data Lifecycle Process PresentAcquire Amazon S3 Amazon DynamoDB AWS Lambda Amazon Kinesis Streams Amazon Kinesis Firehose JavaScript SDK
  89. 89. Data Lifecycle Process PresentAcquire Amazon API Gateway AWS Lambda AWS S3 JavaScript SDK
  90. 90. Data Lifecycle Process PresentAcquire Amazon API Gateway AWS Lambda AWS S3 JavaScript SDK Amazon S3 AWS Lambda Amazon Kinesis Streams Amazon S3 Amazon DynamoDB AWS Lambda Amazon Kinesis Streams Amazon Kinesis Firehose JavaScript SDK
  91. 91. The Architecture simDataProducer simDataConsumer MQTT Amazon Kinesis AWS Lambda Amazon DynamoDB Amazon API Gateway Acquire Process Present
  92. 92. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB Amazon API Gateway Process Present
  93. 93. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB Amazon API Gateway Present
  94. 94. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront
  95. 95. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFrontData sources
  96. 96. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFrontData sources Preprocessing
  97. 97. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFrontData sources Preprocessing Serverless processing
  98. 98. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFrontData sources Preprocessing User interface Serverless processing
  99. 99. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront
  100. 100. The Architecture simDataProducer simDataConsumer MQTT Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront
  101. 101. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront Trigger MQTT
  102. 102. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront Trigger MQTT SDK
  103. 103. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront Trigger Integration SDK MQTT
  104. 104. The Architecture simDataProducer simDataConsumer Amazon Kinesis AWS Lambda Amazon DynamoDB AWS Lambda Amazon API Gateway Amazon S3 Amazon CloudFront
  105. 105. Break
  106. 106. Section 3: Hands-On Workshop
  107. 107. https://..................... Student guide https://..................... Copy and paste guide ……………. Wi-Fi SSID
  108. 108. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you!

×