Recommended
Recommended
More Related Content
Similar to IoT Enabled smoker for Great BBQ
Similar to IoT Enabled smoker for Great BBQ (20)
More from Jimmy Dahlqvist
More from Jimmy Dahlqvist (8)
Recently uploaded
Recently uploaded (20)
IoT Enabled smoker for Great BBQ
- 1. IoT-enabled smoker for great BBQ Jimmy Dahlqvist Øredev - 2022-11-09
- 3. Hello, I'm JIMMY DAHLQVIST Father of two girls Serverless enthusiast AWS Ambassador AWS Community Builder Head of AWS Technologies at Sigma Technology Cloud
- 4. BACKGROUND
- 5. @jimmydahlqvist BBQ Low and slow • Low even temperature for a long time Different styles It’s an art form
- 6. @jimmydahlqvist Smoker types Ceramic – Kamado UDS – Ugly drum smoker Electric pellets Offset
- 8. IoT device
- 11. @jimmydahlqvist IoT Device Software AWS IoT Greengrass 2.0 - Core Custom component Hardware Raspberry Pi 4 model B 2.5mm food probes MCP3008 – AD converter
- 12. @jimmydahlqvist IoT device software First iteration Python application Updated over SSH Connected directly to AWS IoT Core
- 15. @jimmydahlqvist Lesson learned • IoT Rules as router • Small objects in S3 • Hard to change data format • Extending was a challenge
- 16. @jimmydahlqvist Improvements Cloud architecture • Data transformation • Event driven architecture • Decouple services IoT Device • Software development lifecycle • Log handling • Configuration
- 18. @jimmydahlqvist AWS Greengrass Interact with AWS services AWS provided components • Log manager Custom components • Publish new versions of components Support for AWS Lambda
- 20. @jimmydahlqvist Cloud architecture Data service Detection service Notification service Data augmentation service
- 21. Cloud deep dive
- 22. @jimmydahlqvist Cloud architecture • Reliably capture data • Managed services • Powerful Data service Detection service Notification service Data augmentation service Storage first
- 23. @jimmydahlqvist Cloud architecture Data service Detection service Notification service Data augmentation service
- 24. @jimmydahlqvist Data augmentation • Data transform pattern • Fetch data from DynamoDB
- 25. @jimmydahlqvist Cloud architecture Data service Detection service Notification service Data augmentation service
- 26. @jimmydahlqvist Detection service • Threshold breaches • Temperature trends • The dreadful stall
- 27. @jimmydahlqvist Cloud architecture Data service Detection service Notification service Data augmentation service
- 28. @jimmydahlqvist Serverless and event-driven Loosely coupled Scale and fail independently Cost effective Extensibility Highly available
- 30. I would say so
- 32. @jimmydahlqvist Choreography + Orchestration Data storage service Detection service Notification service Data transform service
- 33. @jimmydahlqvist AWS StepFunctions workflow types Express workflow • Short running – 5 minutes • 100K starts per second • Pay for duration Standard workflow • Long running - 1 year • 2000 starts per second • Pay per state transition
- 34. @jimmydahlqvist Express workflows Express Workflows employ an at-least-once model for asynchronous invocation
- 35. @jimmydahlqvist Create subscriptions Creates coupling on the filter Complex to add more targets Changing the filter affect all Coupling is on the event Easy to add more targets Filters can change independently
- 36. @jimmydahlqvist Default bus Avoid using the default bus for custom applications!
- 37. @jimmydahlqvist Transform don’t transport Use AWS Lambda to transform data, not transport.
- 39. @jimmydahlqvist Next step • Add camera support • Lambda functions on the device • Train an ML model for detection • Alexa integration
- 40. @jimmydahlqvist Summary • Building an IoT system • Serverless and event-driven • Get great BBQ with help of technology
Editor's Notes
- Hi!! So exited to be back here at Öredev as a speaker. It’s really great to see so many BBQ enthusiasts in the same room. I’m the guy between you and lunch…. In the next 40 minutes will we talk about how to get great BBQ by using tech. We will touch topics like BBQ, IoT, BBQ, Serverless and eventdriven architecture, and did I say BBQ? This is Iot Enabled Smoker for great BBQ and my name is Jimmy and I will be you pitmaster today!
- So the agenda for the day is: Background to the project Overview of the architecture and how it has evolved over time Cloud deep dive and – talk about changes I made and the benefits of that Summary – and final thoughts Before we dive into everything, who am I?
- As I said, my name is Jimmy Dahlqvist!! Father of two daughter, afraid of nothing!! IoT, Serverless, BBQ fantast! Serverless since 2016 – lambda Old Day Job – Head of sigma – (cheer) collegues in audience AWS Ambassador + Community Builder (cheer ??)
- Let’s jump into the background so we all have the same context.
- What is ? Low & Slow – Vs hot and fast I normally target ~120-125c – Reason for doing that is….. Tenderize, tissue breakdown Styles – Us (varies by state, Texas, Tenesee, Nort caroline, New york…), Jamaica, Austrailia, UK – Sweden most US styles Not cooking – Art! But doesn’t mean we can’t use Tech…..
- Audience Poll!! Kamado ? UDS? Electric ? Offset ?
- My offset smoker – Isn’t she a beuti! How does an offset work + IoT Device – ANIMATION IoT Device – Watch fire, help from technology, What does it do!!
- This is an overview of the IoT Device and the food probes I use, standard 2.5mm thermistor probes. What is a Thermistor?
- Background and context done Let’s look at the components, and architecture
- Two parts – IoT device + cloud Example services for both CLICK!!! Let us take a closer look at the IoT Device
- Two parts – HW + SW HW: Rasp-Pi 4 2.5mm food probes – thermistor! MCP3008 (10bit 8 channel)– AD converter – read voltage SW: AWS IoT Greengrass 2.0 – Core Custom component – math voltage to temp
- Initialy simple Python app updated over SSH Problems – Logs and Hard to update
- Done – Looked at IoT device CLICK Let’s talk about Cloud – where we will spend most of our time
- First version of cloud looked like this… Happy little man to the left I guess is me… Device data -> IoT Core IoT Core -> Rules -> Storage + Athena + Dynamo IoT Core -> Rules -> Several SF business logic (thresholds, trends…) API GW RESTful…
- IoT rules as router –> on mqtt –> not on payload –> messages discarded in business logic Each event –> one OBJECT in s3 –> Glue/Athena not optimized for that Data written directly to storage (Storage First is good but…) –> Format dictated by the device –> need transform Hard to extend –> Several services did same thing –> notification –> or needed to implement API
- So I had a couple of areas that I wanted to improve. I wanted to add the possibility to do a proper ETL and data transformation. So it would be easy to change how data is stored and presented in the cloud withour having to change the device, Introduce a event driven architecture with EventBridge as the event router instead of relying to heavily on rules in IoT core. The rules are great but at this point they didn’t really fulfill what I wanted to accomplish. Lastly decouple the services. Break the mini-monolith... All changes was to create a more flexible system that was easy to extend and manage.
- Let’s start by looking at the changes made for the IoT device
- With the Initial problems I decided to test out Greengrass. Interact with AWS Services – s3 config etc AWS provided components – Log Manager Build SW as components - Easy to push and publish new versions AWS Lambda support
- Now we move over to the cloud part what was done there
- Second iteration several improvments. IoT Core no longer primary message router -> EventBridge introduced -> EventDriven architecture -° Rules / targets / subscription Business Logic -> Microservice pattern – with clear responsibility -> Communicating over EB and API Transformation service -> EB Transform / augemnt EB – PayLoad filtering
- Let’s take a closer look at some parts of the architecture….. And start with the ingress part…… CLICK! -> Animate Favorite pattern – Storage First Create reliable way to capture data – prevent data loss Use managed services Very powerful when incoming data doesn’t require instant transformation
- The next part we should look at is the data augmentation CLICK This part became very important in the new design… CLICK
- ….. It allowed me to decouple cloud development from device development Data transform pattern Data augmentation -> Additional information fetched from DynamoDB. Data is transformed to an internal format -> Decouple from the IoT device Almost no code. StepFunctions integration to other services
- One of the most important services are the detection service CLICK This is where all the BBQ magic happens….. CLICK
- Threshold breach Trends Stall – Happens around 70c (160f). What is it?
- Next part to look at is actually the entire system. CLICK!! Everything is built on a serverless and event-driven approach.
- Reason You build an serverless and eventdriven architecture. Loosely coupled services Scale and fail independently Cost effective – pay for what you use Extensibility – easy and fast to extend HA – built in
- So have technology help me to become a better pitmaster and to get some great BBQ? Some may say it’s cheating but why not use tech to help?
- …. I let the result speak for it self!
- EventBridge Choreographs Four bounded contexts represented by each service (orchestrated) Stepfunctions The service has unique business logic that need to be implemented and happen in a certain order, when a event is invoked.
- StepFunctions Express Workflow has an invocation model of “At least once” which means that it’s possible that your workflow get invoked twice. In dev and test it’s unlikely that it will happen, even in my small project I never saw it. But if you run it in a large enough scale it will happen. So make sure your workload is idempotent and can handle it. This makes them ideal for orchestrating idempotent actions such as transforming input data and storing via PUT in Amazon DynamoDB.
- When building on eventbridge I would recommend that you create subscriptions. With that I mean that you create one event rule for one target. Even if EB support up to 5 targets per rule I still say this should be a 1-2-1 mapping. Why? If you create one rule with multiple targets you will create a coupling on the event filter. And if you hit the 5 target limit what are you suppose to do then? Create a second rule with the same filter and start adding new targets? What if you need to update the filter? You will impact several targets that might not be what you wanted to do in the first place, leaving you to start breaking things apart again. So instead we create subscriptions where we create the coupling on the event it self, and we set one target for one rule. It’s easy to add more target, just create a second rule and add the target to it. And the filter in every rule can change without affecting any other target. However…. This of course can lead to several rules having the same filter and that could create problems on it’s own. But in my opinion it’s still better to create subscriptions and deal with a rule explosion.
- there is a default EB bus in the region already existing that AWS services post events to. Should you use that as your bus in your application? My recommendation is NO. Leave that bus to be used by AWS services and create your own custom buses for you application. The reason for that is that in that case will become easier to extend and ass busses later. I have seen the default bus being used and the mess it was when then moving to a custom bus. So can’t repeat it enough… Leave the default bus alone! Create custom buses! It’s like using the default VPC…. We don’t do that either!
- So instead of using SQS + Lambda to transport. It would be possible to have IoT Core invoke a Express StepFunction with only one step which would be a SDK integration and that integration would just post the event to EB. No code, only 100% managed AWS services doing all of the work for me behind the scene. That is a really nice way to be able to use the SDK integration in StepFunctions. This wcould however break my storage first approach, but since there is no code involved, just AWS calling the SDK on my behalf I could totally live with that. It truly show the power and flexibility in a serverless approach and a service like AWS StepFunctions.
- So what is next in this project then?
- So to summarize the last 40 minutes. Building an IoT system Serverless and event-driven Get great BBQ with help of technology And with that I say! CLICK!!