How to create your Smart Toy with bluemix & 7688 Duo board

633 views

Published on

Create your own smart toy with tiny 7688 duo board and IBM Cloud Bluemix

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
633
On SlideShare
0
From Embeds
0
Number of Embeds
157
Actions
Shares
0
Downloads
27
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

How to create your Smart Toy with bluemix & 7688 Duo board

  1. 1. © 2014 IBM Corporation How to Create your Smart Toy with IBM bluemix & MTK 7688 Duo Tommy Wu , IBM Cloud. Senior Cloud Architect Document number
  2. 2. © 2014 IBM Corporation 準備清單(Prerequisite) MTK 7688 Duo 一塊 USB Sound Card 外接音效卡 Speaker 喇叭 Mic 麥克風 Ragdoll 布娃娃 SD卡 2
  3. 3. © 2014 IBM Corporation 架構示意圖 -Device Publish STT Result To Cloud Q&A Logic iot_client.publish('iot-2/evt/status/fmt/json', '{"d":{"status": "connected" }}'); 3 7688 Duo DEIVCE Node-RED 你好 我是ooo MQTT Q&A Logic 小腦 大腦
  4. 4. © 2014 IBM Corporation 架構示意圖 -Device 監聽雲端 Q&A Response Data to Speaker Out 4 speaker Node-RED 7688 Duo deviceMQTT 大腦 小腦 iot_client.on("message", function(topic,payload){ console.log('received topic:'+topic+', payload:'+payload);}); Q&A Logic
  5. 5. © 2014 IBM Corporation Bluemix 準備工作
  6. 6. © 2014 IBM Corporation BLUEMIX Registration 事前準備! 6 前往BLUEMIX官網申請帳號
  7. 7. © 2014 IBM Corporation 建立Speech To Text 服務  Bluemix > Category > Watson > Speech To Text > Create Services  Bluemix > Dashboard > Servies > Speech To Text-ro – Get Your Username/Password 7
  8. 8. © 2014 IBM Corporation 建立IOT應用程式 8
  9. 9. © 2014 IBM Corporation Go To Bluemix Dashboard – Create Your application 9
  10. 10. © 2014 IBM Corporation Choose Template - 選擇樣板 10
  11. 11. © 2014 IBM Corporation Choose IOT Starter Template 應用程式 11
  12. 12. © 2014 IBM Corporation 完成建立IOT AP 應用程式 12  建立完成後,Go To Bluemix > Dashboard > Application > Your IOT Application
  13. 13. © 2014 IBM Corporation 選擇 IOT 服務 13
  14. 14. © 2014 IBM Corporation GO To IOT Service Dashboard -前往儀表板 14
  15. 15. © 2014 IBM Corporation Select Devices Tab -點選裝置 15
  16. 16. © 2014 IBM Corporation Add Your Device -新增裝置 16 Ex :7688duo robot device 2016年6月10日 下午10:20:55
  17. 17. © 2014 IBM Corporation Please keep the device token -請保留這張資訊 17
  18. 18. © 2014 IBM Corporation Device 端 (7866 Duo) 邏輯開發
  19. 19. © 2014 IBM Corporation Initialize MTK LinkIt 7688 Duo 19  參考CAVEDU 基本設置 http://blog.cavedu.com/%E7%89%A9%E8%81%AF%E7%B6%B2/linkit- 7688duo%E6%9D%BF%E4%BB%8B%E7%B4%B9%E8%88%87arduino- %E8%A8%AD%E7%BD%AE%E6%95%99%E5%AD%B8/  安裝所需npm package : mqtt/watson-developer-cloud/node-aplay/request/fs/querystring
  20. 20. © 2014 IBM Corporation Cross-Compiler for NPM Packages 20  本身只有 32mb flash 以及 128 mb Memory,所以需要Cross-Compiler來加快速度,有二個方 案:  https://iamblue.gitbooks.io/linkit-smart-nodejs/content/zh-TW/emulator/ 開個7688模擬器  https://medium.com/maker-cup/mt7688-%E6%8A%97%E7%97%9B%E5%AF%A7- %E4%BA%A4%E5%8F%89%E7%B7%A8%E8%AD%AF-node-js- %E5%8E%9F%E7%94%9F%E6%A8%A1%E7%B5%84-69abc6258bce#.ggasla7qu建 立7688 cross-compiler環境  安裝所需npm package : mqtt/watson-developer-cloud/node-aplay/request/fs/querystring  安裝 madplay for playing TTS voice.  空間使用資源如下:可以把所有東西移到SD卡,下次複製比較方便 (Put all your resource under SD card for easier replication)
  21. 21. © 2014 IBM Corporation 以npm安裝mqtt package 並撰寫code var clientId = [‘d’, “(組織ID)”, “robot”, “(裝置ID)”].join(‘:’); //7688 Duo Device Client ID iot_client = mqtt.connect("mqtt://(組織ID).messaging.internetofthings.ibmcloud.com:1883",// 連結mqtt伺服器的認證 { "clientId" : clientId, "keepalive" : 30, "username" : "use-token-auth", "password" : "(鑑別記號)" }); 21
  22. 22. © 2014 IBM Corporation 傳送一個Event至BLUEMIX的Status節點 22 iot_client.on('connect', function() {//如果連接上的話要做的事情 console.log(‘STT client connected to IBM IoT Cloud.'); iot_client.publish('iot-2/evt/status/fmt/json', '{"d":{"status": "connected" }}'); //發布一個event至伺服器, event的名子叫做status, format為json,其中值為 {“status”,”connected”} iot_client.subscribe('iot-2/cmd/+/fmt/+', function(err, granted){ console.log('subscribed command, granted: '+ JSON.stringify(granted)); }); //訂閱來自Cloud的response command, 允許為任何format
  23. 23. © 2014 IBM Corporation23 7688 Duo Device 端程式邏輯 (1) - Watson STT Enablement Use 8bit 22k recording to avoid overrun & delay Enable Watson Speech-to-Text Streaming Service
  24. 24. © 2014 IBM Corporation 7688 Duo Device 端程式邏輯 (2) -Watson STT Listen Event & IOT Publish  7688 Duo subscribe Cloud Q&A Logic and Response in Google TTS (中文) 24 Websocket listen for STT results and Publish to Bluemix IOT
  25. 25. © 2014 IBM Corporation Device端 7688 Duo 執行命令及結果檢視(monitor)  啟動Node.js AP  語音問答 (Q&A Results) 25
  26. 26. © 2014 IBM Corporation 雲端(Bluemix) 對談流程設計
  27. 27. © 2014 IBM Corporation 前往IOT application - Node-RED based Application 27
  28. 28. © 2014 IBM Corporation Design your Simple Q&A Logic with Node-RED Flow 28
  29. 29. © 2014 IBM Corporation 執行程式碼就可以看到資訊 29 You should see the status connected sent from 7688 Duo
  30. 30. © 2014 IBM Corporation 雲端大腦 Simple Q&A Editor  Node-Red : 4 Nodes to create your own dialog 30 Voice STT-In Google TTS-Out 自行編輯欲對話的簡單QA
  31. 31. © 2014 IBM Corporation Next Step  Connect To Watson Service : Dialog, Natural Language Classifier , Conversation.  Add Grove Pi Sensors to 7688 Duo Arduino MCU board 31
  32. 32. © 2014 IBM Corporation Resources  Bluemix Registration : https://console.ng.bluemix.net/registration  Source Code on Github : https://github.com/tommywu052/7688duoDoll  Demo Video on Youtube : https://youtu.be/aQ24_Zdb9Ps 32
  33. 33. © 2014 IBM Corporation 謝謝觀賞! 33

×