Presentation from the WeTeach_CS Summit 2023

1. COMBINING MACHINE
LEARNING WITH PHYSICAL
COMPUTING
Hal Speed
2023 June 16
https://www.slideshare.net/Hal_Speed

3. Breaking News
WEF Future of Jobs
weforum.org/reports
page 39
https://teachai.org/

4. Five Big Ideas in AI
● Framework to guide
the development of
standards and learning
materials
● For use by
governmental
organizations, curricula
developers, CSTA,
educators, etc.

5. Machine Learning Primer

6. What is Machine Learning?
● Machine Learning is a subfield
of Artificial Intelligence focused
on developing algorithms that
learn to solve problems by
analyzing data for patterns.
● Deep Learning is a type of
Machine Learning that
leverages Neural Networks and
Big Data
Artificial Intelligence
Deep Learning
Machine Learning
Source: EdX tinyML course

7. What is Machine Learning?
Computer
Computer
Human Programming
Machine Learning
Input Data (ingredients)
Human Program (recipe)
Input Data (ingredients)
Output (cake)
Desired Output (cake)
Program (recipe)
or Model

8. What is Machine Learning?
Computer
Computer
Human Programming
Machine Learning
Input Data (2,2)
Human Program (x+y)
Input Data (2,2)
Output (4)
Desired Output (4)
Program (x+y, x*y, x2, y2, ?)
or Model

9. Example from Harvard

10. Example from Harvard

11. Once We Have A Trained Model
Trained Model
Unlabeled or
Unclassified Data
Inferences or Classifications
(guesses with a degree of
confidence)

12. How Do ML Models “See” and “Hear”?
spectrogram

13. Large Language Models
● Examples
○ GPT-3 and GPT-4 from OpenAI
○ BERT, LaMDA, PaLM 2 from Google
○ LLaMA from Meta
● Uses massively large datasets to train a language model using deep learning
○ GPT-3 was trained on about 45TB of text data
○ For comparison, the English version of Wikipedia is estimated to be around 50GB
○ Wikipedia makes up 3% of GPT-3 training data
● Size of neural network parameters increasing at an exponential rate
13

14. https://code.org/ai https://tinymledu.org/4K12
AI Resource Lists
https://raise.mit.edu/resources.html
https://www.actua.ca/ai/
https://ai4k12.org/ https://aiforteachers.org/

15. Machine Learning Workflow
Collect Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Data
Engineering
or Data
Science
Model
Engineering
Model
Deployment
Source: EdX tinyML and Google TensorFlow
Usage

16. Jupyter Notebook/Colab
Edge Impulse
Seeed Studio Codecraft
ML Machine & PlushPal
Google Teachable Machine
Microsoft Lobe
ML for Kids
Scratch + Teachable Machine
smartphone
Wio
Terminal
Pico4ML
micro:bit
others
Arduino IDE
Microsoft
MakeCode
MicroPython/
CircuitPython
Code.org
Building a tinyML Pipeline for K-12
Collect Data
Preprocess
Data
Design a
Model
Train a
Model
Evaluate
Optimize
Convert
Model
Deploy
Model
Make
Inferences
Nano 33

17. ML Machine for micro:bit
● Easy to use gesture model
● Web based, no login
● See the data in real time
● Collect, train, test
● Have another micro:bit
respond to model
predictions
https://ml-machine.org/

18. PlushPal
18
https://www.plushpal.app/

19. EdX Tiny Machine Learning
https://www.edx.org/professional-certificate/harvardx-tiny-machine-learning

20. Arduino Tiny Machine Learning Kit
● Supported hardware kit for the
EdX courses
● Arduino Nano 33 BLE Sense
○ IMU (motion sensor)
○ microphone
● Plus OV7675 camera and
breakout board

21. ● No-code ML platform
● Broad hardware support
including:
○ Smartphones
○ Arduino
○ micro:bit
○ Pico4ML
○ Wio Terminal
○ …and many more
● Coursera courses
● micro:bit V2 tutorials
○ Voice activated
○ Dance move detector
https://www.edgeimpulse.com/

22. Wio Terminal
https://www.seeedstudio.com/wio-terminal-tinyml.html

23. CodeCraft
● The power of Edge Impulse
with the simplicity to Scratch
● Learning resources
● Get Started with TinyML ebook
● Hackster project by Marcelo
Rovai
https://ide.tinkergen.com/

24. Other Hardware

25. Pico4ML
● Includes camera, IMU (motion), and
microphone
● Bluetooth optional
● Three pre-trained models and magic
wand tutorial available
https://www.arducam.com/pico4ml-an-rp2040-based-platform-for-tiny-machine-learning/

26. BrainCraft HAT + RPi with Lobe
Three different step-by-step
tutorials for image detection:
● Getting started
● Rock, paper, scissors
game
● Package detector
https://learn.adafruit.com/machine-learning-101-lobe-braincraft

27. Microsoft Farm Beats for Students
The easy-to-use FarmBeats kit includes
● preconfigured Microsoft Azure cloud services
● A Raspberry Pi with soil moisture, light, ambient
temperature, and humidity sensors to collect data.
● The data is then visualized in an online dashboard
that provides insights to help students.
Partnership
Future Farmers of America and Microsoft are working
together to create activity guides and resources to help
chapters get started with using the technology.
https://aka.ms/farmbeatsforstudents

28. AutoAuto
https://www.autoauto.ai/

29. Calypso for Cozmo
● A robot intelligence framework that
Incorporates multiple AI technologies:
○ Computer vision; face recognition
○ Speech recognition and generation
○ Landmark-based navigation
○ Path planning
○ Object manipulation
● Rule-based pattern matching language
inspired by Microsoft’s Kodu Game Lab
● Teaches computational thinking: “Laws of
Calypso”, idioms, etc.
world map
perception
rules
speech
recognition
https://calypso.software/

30. DeepRacer
● Intended for
professionals
● Support for advanced
sensors
https://aws.amazon.com/deepracer/

31. OpenBot
● Turning smartphone into robots
● Software stack for Android
● Real-time autonomous navigation
https://www.openbot.org/

32. HUSKEYLENS
● AI vision sensor
● Built-in training
algorithms
● Connect to micro:bit
and use MakeCode
or Mind+
● User guide and
tutorials wiki
https://www.dfrobot.com/product-1922.html

33. KOI Camera Module
● Works with or without micro:bit
● Progammable with MakeCode,
KittenBlock, and MicroPython
● Quick start guide and tutorials
https://www.kittenbot.cc/products/kittenbot-koi-artificaial-intelligence-module

34. Experiments with TF Lite for Microcontrollers
● Well-documented projects with
Tensorflow Lite for Microcontrollers
● Includes code & instructions
● Projects use the
Arduino Nano 33 BLE Sense
https://experiments.withgoogle.com/collection/tfliteformicrocontrollers

35. AIY - With Google
https://aiyprojects.withgoogle.com/

36. EdgeBadge
● Step-by-step tutorials for gesture
and speech detection
https://learn.adafruit.com/tensorflow-lite-for-edgebadge-kit-quickstart

37. ● Introduction to ML: Image Classification
● Personal Image Classifier: PICaboo
● Personal Audio Classifier
● Voice Calculator Tutorial
● Therapist Bot Tutorial
● Awesome Dancing with AI Tutorial
● Facemesh Filter Camera
● Rock Paper Scissors Tutorial
https://appinventor.mit.edu/explore/ai-with-mit-app-inventor

38. Other AI Resources

39. Teachable Machine
https://teachablemachine.withgoogle.com/

40. https://www.lobe.ai/

41. Machine Learning for Kids
https://machinelearningforkids.co.uk

42. http://cognimates.me
Cognimates offers AI extensions
for Scratch, such as:
● speech recognition
● sentiment analysis
● visual pattern detection
● robot control

43. Face Sensing https://lab.scratch.mit.edu/face/

44. Other Scratch-based Editors w/ML Extensions
https://scratch.techpark.jp/ https://stretch3.github.io/
https://mblock.makeblock.com/
https://www.kittenbot.cc/pages/software

45. Google Quick, Draw!
https://quickdraw.withgoogle.com/

46. TensorFlow Playground
Tutorial: https://cloud.google.com/blog/products/gcp/understanding-neural-networks-with-tensorflow-playground
https://playground.tensorflow.org

47. ● Data collection and analysis tool
https://codap.concord.org/

48. Code.org Resources
AI and Machine Learning Module
● ~ 5 week curriculum
● Standalone or optional unit in CS
Discoveries
https://code.org/ai
AI for Oceans
Classifier
How AI Works
Videos
AI and Ethics

49. High School Curriculum Unit
http://www.exploringcs.org/for-teachers-districts/artificial-intelligence
Intended to be an alternative unit to either unit 5 or 6 of the ECS course

50. AI in Education
https://iste.org/areas-of-focus/AI-in-education

51. AI4All: Online Learning
http://ai-4-all.org/open-learning
AI4ALL Open Learning empowers high school teachers of all
subjects to bring AI education to their classrooms through a free,
adaptable AI curriculum and teacher resources.
Interdisciplinary, Approachable AI Curriculum

52. ReadyAI Resources
https://www.readyai.org/
AI Picture
Books
AI Teaching &
Learning Kits
Self-paced
Courses
Lesson Plans Unplugged
Lessons
Teacher
Training

53. aiEDU
53
https://www.aiedu.org/

54. Other Books for Young Students
https://tinkertoddlers.com/

55. https://www.markcubanai.org/

56. https://www.waicy.org/
World AI Competition for Youth
● For students ages 6-18
● 2023 details coming soon