TJBot is a DIY open source kit that allows you to build your own programmable cardboard robot powered by Watson. It consists of a cardboard cutout (which can be 3D printed or laser cut), Raspberry Pi and a variety of add-ons – including a RGB LED light, a microphone, a servo motor, and a camera. This presentation provides an overview of how Watson Cognitive Services are leveraged to create capabilities within TJBot, and how to build simple applications for TJBot using Node.js.
YouTube Link: https://youtu.be/sHeJgKBaiAI
** Python Certification Training: https://www.edureka.co/python **
This Edureka video on 'Speech Recognition in Python' will cover the concepts of speech recognition module in python with a program using speech recognition to translate speech into text. Following are the topics discussed:
How Speech Recognition Works?
How To Install SpeechRecognition In Python?
Working With Microphones
How To Install Pyaudio In Python?
Use case
YouTube Link: https://youtu.be/sHeJgKBaiAI
** Python Certification Training: https://www.edureka.co/python **
This Edureka video on 'Speech Recognition in Python' will cover the concepts of speech recognition module in python with a program using speech recognition to translate speech into text. Following are the topics discussed:
How Speech Recognition Works?
How To Install SpeechRecognition In Python?
Working With Microphones
How To Install Pyaudio In Python?
Use case
Prototype Summer School: How to make an interactive light displayNeil Winterburn
Prototype, initiated by Neil Winterburn and Laura Pullig for FACT explores digital making as a creative tool to explore art. The summer school was led by Neil Winterburn, Hwa Young Jung, Stef Jayne and FACTLab’s Radamas Anja.
Over three days young people tinkered with Raspberry Pi’s, Arduinos and Mbots, prototyping robots that water plants, greet visitors to FACT, navigate mazes and send messages to Donald Trump.
Our local content strategy is based on creating meaningful content that adds value to our local customers around our thought leaders’ expertise resulting in new business opportunities & recruits.
Accenture-insights.nl is a content hub that reinforces Accenture’s next gen website. On this platform we feature:
- Insightful content demonstrating thought leadership of our Dutch Experts (authors, thought leaders & recruiters) about their expertise, the impact they’ve made, and service offerings relevant for the local market.
- Content that has originally been published on external platforms (earned media) about our people, brand, or our added value as a business partner.
- Content which has been published on https://www.accenture.com/nl-en enriched by a local context to make it more relevant and impactful for our market & clients.
- Blogs about career advise given by recruiters, first impressions, and career development by new hires.
- Articles about local business -, and recruitment events.
IBM Watson IoT - New Possibilities in a Connected WorldCasey Lucas
Cognitive IoT enables us to learn from, and infuse intelligence into, the physical world to transform business and enhance the human experience.
Find out how you can transform with Watson IoT: http://ibm.com/IoT
Accenture Greenlight Insights Conference November 1st 2016Sunny Webb
Accenture Labs Industry X.0 presentation at Greenlight Insights 2016 Conference. This discusses how the Intelligent Worker will use new tech applications in product development, including mixed reality / augmented reality / virtual reality.
Simtelligent is excited to lead this talk at IBM InterConnect 2017 Conference in Las Vegas.
If you want to outperform your competitors, set up an appointment with us to discuss viable options to implement a successful cloud migration strategy.
We look forward to seeing you there.
Description
Smart, connected devices are completely changing the game.
• Cisco predicts that there will be 50B connected devices worldwide by 2020
• Machina Research analysts predict that the global IoT market opportunity will reach USD 4 trillion by 2025
The emergence of smart, connected devices, which are increasingly embedded in broader systems, is radically reshaping companies and competition. This is forcing companies to redefine their industries and rethink nearly everything they do, beginning with their strategies.
Companies will need new types of relationships with customers. They will need new product capabilities, infrastructure, and processes; entirely new structures, functions, and new forms of cross-functional collaboration.
We discuss how smart, connected products are impacting business strategies and transforming the entire value chain.
About Simtelligent
Simtelligent helps midmarket organizations enhance their devices to deliver smart, connected products customers love to increase profit and achieve business growth.
About Hector Del Castillo
Hector Del Castillo has over 15 years of experience collaborating with executives and entrepreneurs to transform ideas into a great business. He’s passionate about building and marketing products customers love. He has launched over 30 global products resulting in over $350M of achieved sales revenue.
Hector is Chief Product Officer, at Simtelligent, where he's busy building a platform for smart connected devices that improve the customer experience, eliminate human error and create new revenue streams for clients.
Need help identifying viable options to grow your product-centric business?
Connect with Hector and inquire how he can help build smart connected devices to grow your business.
Connect: linkd.in/hdelcastillo Follow: @hmdelcastillo Browse: www.hmdelcastillo.com
The rapidly advancing mobile communication technology and the decrease in costs make it possible to incorporate mobile technology into home automation systems. The aim of this project is to develop a system that will provide remote control of home appliances and also provide security against intrusion when the home host is not at home. We propose a mobile-based home automation system that consists of a mobile phone with windows or android platform, raspberry pi, topping pi for led instead of home appliance, fire sensor, and proximity sensor, relay. Led is controlled by the raspberry pi, which operates according to the user commands received from the mobile phone via Cortana (windows platform) and an application (android platform). In our proposed system the raspberry pi will receive command from cloud services like Microsoft Azure for Windows platform and MQTT for Android platform created by user from mobile app. We have also ensure the security aspects which includes intrusion detection using proximity sensor and fire alarm using fire sensor. All sensors are connected to raspberry pi. If any of the previously mention event occurs it will immediately inform user through text sms and also email notification.
Using Alexa, IFTTT and Watson IoT Platform to control ble devicesMarkus Van Kempen
This presentation shows how you can use Alexa,IFTTT and Watson IoT Platform to control devices via MQTT.
video: https://www.youtube.com/watch?v=odaeuO2Yx7g&feature=youtu.be
github: https://github.com/markusvankempen/playbulb
node-red flow: https://github.com/markusvankempen/playbulb/blob/master/node-red-flows/alexa.flow
In the age of darkness light plays an important role in terms of keeping the human race motivated for the good. Projection mapping has been around for some time now. There is hardware, software and possibilities might seem limitless. How about the aspiring artist or an institution with limited funding for anything, anyone who wants to play with projection mapping? One does not want to leave her computer in a space for a month or more. An institution might not have resources to purchase hardware and software.
OfxPiMapper is an addon for the openFrameworks creative coding toolkit that lets one use the Raspberry Pi mini-computer for serious projection mapping projects. It is open source and available on GitHub. It can not only map videos and images, but can map anything openFrameworks is capable of, which includes data driven visuals, interactive motion graphics or generative 3D scenes.
Objective
Inspire, introduce an alternative tool and invite to join ofxPiMapper workshops
Target Audience
Creative coders, designers and artists working with code, the bored front-end developer
Assumed Audience Knowledge
Familiarity with or interest in one of the creative coding frameworks will do
Five Things Audience Members Will Learn
Scenarios where ofxPiMapper can be used
How to use ofxPiMapper
How to create custom generative source to be used with ofxPiMapper
How to prepare a SD card for the Raspberry Pi
Where to get additional information
Today Home Automation is one of the growing requirement in the society. This paper presents the implementation of Home Automation using Raspberry Pi. The Raspberry Pi is a basic embedded system and being a low cost single-board computer used to reduce the complexity of systems in real time applications. This application mainly serves as an efficient base to control various home appliance like Fan, Tube light, Refrigerator through mobile based application. The application is designed to provide a facility to user to access control of many appliances used in homes.
The MD Anderson / IBM Watson Announcement: What does it mean for machine lear...Health Catalyst
It’s been over six years since IBM’s Watson amazed all of us on Jeopardy, but it has yet to deliver similar breakthroughs in healthcare. The headlines in last week’s Forbes article read, “MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine.” Is it really a setback for the entire industry or not? Health Catalyst’s EVP for Product Development, Dale Sanders, believes that the challenges are unique to IBM’s machine learning strategy in healthcare. If they adjust that strategy and better manage expectations about what’s possible for machine learning in medicine, the future will be brighter for Watson, their clients, and AI in healthcare, in general. Watson’s success is good for all of us, but it’s failure is bad for all of us, too.
Join Dale as he discusses:
The good news: Machine learning technology is accelerating at a rate beyond Moore’s Law. Dale believes that machine learning algorithms and models are doubling in capability every six months.
The bad news: The healthcare data ecosystem is not nearly as rich as many would believe, and certainly not as rich as that used to train Watson for Jeopardy. Without high-volume, high-quality data, Watson’s potential and the constant advances in machine learning algorithms will hit a glass ceiling in healthcare.
The best news: By adjusting strategy and expectations, there are still plenty of opportunities to do great things with machine learning by using the current data content in healthcare, while we build out the volume and breadth of data we need to truly understand the patient at the center of the healthcare picture… and you don’t need an army of PhD data scientists to do it.
Raspberry Pi - Unlocking New Ideas for Your LibraryBrian Pichman
Join Brian Pichman as he uncovers the world of microcomputing; which are low cost, small computers (ranging from the size of a credit card to a stack of credit cards). Brian will cover Raspberry Pi's and how they can be used in your library; from attendance counters, event displays, library programming and more.
IoT, AI, ML Mix or How to Deal with New Technologies (Borys Pratsiuk Technolo...IT Arena
Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneurs and startuppers. Annually it takes place at the beginning of October in Lviv at Arena Lviv stadium. In 2016 the conference gathered more than 1800 participants and over 100 speakers from companies like Microsoft, Philips, Twitter, UBER and IBM. More details about the conference at itarena.lviv.ua.
TJBOT is an interactive Open Source robot which is designed to encourage All individuals to build TJBOT in fun way with cognitive services. Where TJBOT can be 3D printed and which comes with set of recipes from which we can build TJBOT. We can program the TJBOT to speak, listen, Shine Led, see, recognize, Understand the emotions, wave arms, play games Play Rock Paper Scissors in Node RED. Basically TJBOT was designed for two communities makers, who enjoy the DIY aspects of building and programming novel devices, and students, who can learn about programming cognitive systems. Haleema Khatoon | Prof. Praveen Kumar Pandey "TJBOT - An Open Source Cardboard Robot" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-3 , April 2020, URL: https://www.ijtsrd.com/papers/ijtsrd30701.pdf Paper Url :https://www.ijtsrd.com/engineering/computer-engineering/30701/tjbot--an-open-source-cardboard-robot/haleema-khatoon
Prototype Summer School: How to make an interactive light displayNeil Winterburn
Prototype, initiated by Neil Winterburn and Laura Pullig for FACT explores digital making as a creative tool to explore art. The summer school was led by Neil Winterburn, Hwa Young Jung, Stef Jayne and FACTLab’s Radamas Anja.
Over three days young people tinkered with Raspberry Pi’s, Arduinos and Mbots, prototyping robots that water plants, greet visitors to FACT, navigate mazes and send messages to Donald Trump.
Our local content strategy is based on creating meaningful content that adds value to our local customers around our thought leaders’ expertise resulting in new business opportunities & recruits.
Accenture-insights.nl is a content hub that reinforces Accenture’s next gen website. On this platform we feature:
- Insightful content demonstrating thought leadership of our Dutch Experts (authors, thought leaders & recruiters) about their expertise, the impact they’ve made, and service offerings relevant for the local market.
- Content that has originally been published on external platforms (earned media) about our people, brand, or our added value as a business partner.
- Content which has been published on https://www.accenture.com/nl-en enriched by a local context to make it more relevant and impactful for our market & clients.
- Blogs about career advise given by recruiters, first impressions, and career development by new hires.
- Articles about local business -, and recruitment events.
IBM Watson IoT - New Possibilities in a Connected WorldCasey Lucas
Cognitive IoT enables us to learn from, and infuse intelligence into, the physical world to transform business and enhance the human experience.
Find out how you can transform with Watson IoT: http://ibm.com/IoT
Accenture Greenlight Insights Conference November 1st 2016Sunny Webb
Accenture Labs Industry X.0 presentation at Greenlight Insights 2016 Conference. This discusses how the Intelligent Worker will use new tech applications in product development, including mixed reality / augmented reality / virtual reality.
Simtelligent is excited to lead this talk at IBM InterConnect 2017 Conference in Las Vegas.
If you want to outperform your competitors, set up an appointment with us to discuss viable options to implement a successful cloud migration strategy.
We look forward to seeing you there.
Description
Smart, connected devices are completely changing the game.
• Cisco predicts that there will be 50B connected devices worldwide by 2020
• Machina Research analysts predict that the global IoT market opportunity will reach USD 4 trillion by 2025
The emergence of smart, connected devices, which are increasingly embedded in broader systems, is radically reshaping companies and competition. This is forcing companies to redefine their industries and rethink nearly everything they do, beginning with their strategies.
Companies will need new types of relationships with customers. They will need new product capabilities, infrastructure, and processes; entirely new structures, functions, and new forms of cross-functional collaboration.
We discuss how smart, connected products are impacting business strategies and transforming the entire value chain.
About Simtelligent
Simtelligent helps midmarket organizations enhance their devices to deliver smart, connected products customers love to increase profit and achieve business growth.
About Hector Del Castillo
Hector Del Castillo has over 15 years of experience collaborating with executives and entrepreneurs to transform ideas into a great business. He’s passionate about building and marketing products customers love. He has launched over 30 global products resulting in over $350M of achieved sales revenue.
Hector is Chief Product Officer, at Simtelligent, where he's busy building a platform for smart connected devices that improve the customer experience, eliminate human error and create new revenue streams for clients.
Need help identifying viable options to grow your product-centric business?
Connect with Hector and inquire how he can help build smart connected devices to grow your business.
Connect: linkd.in/hdelcastillo Follow: @hmdelcastillo Browse: www.hmdelcastillo.com
The rapidly advancing mobile communication technology and the decrease in costs make it possible to incorporate mobile technology into home automation systems. The aim of this project is to develop a system that will provide remote control of home appliances and also provide security against intrusion when the home host is not at home. We propose a mobile-based home automation system that consists of a mobile phone with windows or android platform, raspberry pi, topping pi for led instead of home appliance, fire sensor, and proximity sensor, relay. Led is controlled by the raspberry pi, which operates according to the user commands received from the mobile phone via Cortana (windows platform) and an application (android platform). In our proposed system the raspberry pi will receive command from cloud services like Microsoft Azure for Windows platform and MQTT for Android platform created by user from mobile app. We have also ensure the security aspects which includes intrusion detection using proximity sensor and fire alarm using fire sensor. All sensors are connected to raspberry pi. If any of the previously mention event occurs it will immediately inform user through text sms and also email notification.
Using Alexa, IFTTT and Watson IoT Platform to control ble devicesMarkus Van Kempen
This presentation shows how you can use Alexa,IFTTT and Watson IoT Platform to control devices via MQTT.
video: https://www.youtube.com/watch?v=odaeuO2Yx7g&feature=youtu.be
github: https://github.com/markusvankempen/playbulb
node-red flow: https://github.com/markusvankempen/playbulb/blob/master/node-red-flows/alexa.flow
In the age of darkness light plays an important role in terms of keeping the human race motivated for the good. Projection mapping has been around for some time now. There is hardware, software and possibilities might seem limitless. How about the aspiring artist or an institution with limited funding for anything, anyone who wants to play with projection mapping? One does not want to leave her computer in a space for a month or more. An institution might not have resources to purchase hardware and software.
OfxPiMapper is an addon for the openFrameworks creative coding toolkit that lets one use the Raspberry Pi mini-computer for serious projection mapping projects. It is open source and available on GitHub. It can not only map videos and images, but can map anything openFrameworks is capable of, which includes data driven visuals, interactive motion graphics or generative 3D scenes.
Objective
Inspire, introduce an alternative tool and invite to join ofxPiMapper workshops
Target Audience
Creative coders, designers and artists working with code, the bored front-end developer
Assumed Audience Knowledge
Familiarity with or interest in one of the creative coding frameworks will do
Five Things Audience Members Will Learn
Scenarios where ofxPiMapper can be used
How to use ofxPiMapper
How to create custom generative source to be used with ofxPiMapper
How to prepare a SD card for the Raspberry Pi
Where to get additional information
Today Home Automation is one of the growing requirement in the society. This paper presents the implementation of Home Automation using Raspberry Pi. The Raspberry Pi is a basic embedded system and being a low cost single-board computer used to reduce the complexity of systems in real time applications. This application mainly serves as an efficient base to control various home appliance like Fan, Tube light, Refrigerator through mobile based application. The application is designed to provide a facility to user to access control of many appliances used in homes.
The MD Anderson / IBM Watson Announcement: What does it mean for machine lear...Health Catalyst
It’s been over six years since IBM’s Watson amazed all of us on Jeopardy, but it has yet to deliver similar breakthroughs in healthcare. The headlines in last week’s Forbes article read, “MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine.” Is it really a setback for the entire industry or not? Health Catalyst’s EVP for Product Development, Dale Sanders, believes that the challenges are unique to IBM’s machine learning strategy in healthcare. If they adjust that strategy and better manage expectations about what’s possible for machine learning in medicine, the future will be brighter for Watson, their clients, and AI in healthcare, in general. Watson’s success is good for all of us, but it’s failure is bad for all of us, too.
Join Dale as he discusses:
The good news: Machine learning technology is accelerating at a rate beyond Moore’s Law. Dale believes that machine learning algorithms and models are doubling in capability every six months.
The bad news: The healthcare data ecosystem is not nearly as rich as many would believe, and certainly not as rich as that used to train Watson for Jeopardy. Without high-volume, high-quality data, Watson’s potential and the constant advances in machine learning algorithms will hit a glass ceiling in healthcare.
The best news: By adjusting strategy and expectations, there are still plenty of opportunities to do great things with machine learning by using the current data content in healthcare, while we build out the volume and breadth of data we need to truly understand the patient at the center of the healthcare picture… and you don’t need an army of PhD data scientists to do it.
Raspberry Pi - Unlocking New Ideas for Your LibraryBrian Pichman
Join Brian Pichman as he uncovers the world of microcomputing; which are low cost, small computers (ranging from the size of a credit card to a stack of credit cards). Brian will cover Raspberry Pi's and how they can be used in your library; from attendance counters, event displays, library programming and more.
IoT, AI, ML Mix or How to Deal with New Technologies (Borys Pratsiuk Technolo...IT Arena
Lviv IT Arena is a conference specially designed for programmers, designers, developers, top managers, inverstors, entrepreneurs and startuppers. Annually it takes place at the beginning of October in Lviv at Arena Lviv stadium. In 2016 the conference gathered more than 1800 participants and over 100 speakers from companies like Microsoft, Philips, Twitter, UBER and IBM. More details about the conference at itarena.lviv.ua.
TJBOT is an interactive Open Source robot which is designed to encourage All individuals to build TJBOT in fun way with cognitive services. Where TJBOT can be 3D printed and which comes with set of recipes from which we can build TJBOT. We can program the TJBOT to speak, listen, Shine Led, see, recognize, Understand the emotions, wave arms, play games Play Rock Paper Scissors in Node RED. Basically TJBOT was designed for two communities makers, who enjoy the DIY aspects of building and programming novel devices, and students, who can learn about programming cognitive systems. Haleema Khatoon | Prof. Praveen Kumar Pandey "TJBOT - An Open Source Cardboard Robot" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-4 | Issue-3 , April 2020, URL: https://www.ijtsrd.com/papers/ijtsrd30701.pdf Paper Url :https://www.ijtsrd.com/engineering/computer-engineering/30701/tjbot--an-open-source-cardboard-robot/haleema-khatoon
Bot. You said bot? Let build bot then! - Laurent EllerbachITCamp
Bot is a new fashion word. This session will explain you what’s a bot, what’s not a bot, how to build a bot using the Microsoft Bot Framework, how to include language recognition using LUIS.ai. The session will be illustrated by a real bot built to connect various information from my own house, garden, wine cellar and health as well as a real production Lego bot!
ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...ITCamp
Bot is a new fashion word. This session will explain you what’s a bot, what’s not a bot, how to build a bot using the Microsoft Bot Framework, how to include language recognition using LUIS.ai. The session will be illustrated by a real bot built to connect various information from my own house, garden, wine cellar and health as well as a real production Lego bot!
Chat bots101 - practical insights on the business of botsBAM
Bots, AI and Messaging are a massive opportunity for businesses to connect with their customers. This deck introduces the concepts, statistics and processes needed to keep ahead of the competition
Chat bots101 - practical insights on the business of botsRoy Murphy
Conversational interfaces are the future of mobile. We've put together statistics, examples and insights into bots, ai and messaging to help businesses understand the landscape better – and see where the opportunities lie.
Verhaert Innovation day 2017 - conversational interfaces Jochem Grietens
Slides from my 2017 Innovation day talk on voice conversational interfaces. The presentation starts of by extrapolating and identifying future potential and halfway the presentation it turn really practical to show how you can start developing for your company.
I am always lecture or provide extra information on the topic.
Google Calendar is a versatile tool that allows users to manage their schedules and events effectively. With Google Calendar, you can create and organize calendars, set reminders for important events, and share your calendars with others. It also provides features like creating events, inviting attendees, and accessing your calendar from mobile devices. Additionally, Google Calendar allows you to embed calendars in websites or platforms like SlideShare, making it easier for others to view and interact with your schedules.
Building a Raspberry Pi Robot with Dot NET 8, Blazor and SignalR - Slides Onl...Peter Gallagher
In this session delivered at Leeds IoT, I talk about how you can control a 3D printed Robot Arm with a Raspberry Pi, .NET 8, Blazor and SignalR.
I also show how you can use a Unity app on an Meta Quest 3 to control the arm VR too.
You can find the GitHub repo and workshop instructions here;
https://bit.ly/dotnetrobotgithub
Building a Raspberry Pi Robot with Dot NET 8, Blazor and SignalR - Slides Onl...
Embedding Intelligence in Everyday Objects with TJBot
1. IBM Research
1
IBM Research
Victor Dibia
Embedding Intelligence in
Everyday Objects with TJBot.
An open source DIY project powered
by Watson Cognitive Services.
Human – Agent Collaboration Lab, IBM Research
dibiavc@us.ibm.com
@vykthur | github.com/victordibia
Feb 20, 2017
2. IBM Research
2
TJBot : What
and Why?
- Open source DIY project to get you engaged
with Watson Services
3. IBM Research
3
What is TJBot?
- A cardboard robot
- Simple, approachable
- Open Source (design, code)
- Cognitive (IBM Watson services)
- Extensible (prototyping platform)
Components: Raspberry Pi, LED, Camera,
Microphone, Speaker, Servo.
6. IBM Research
6
Recipes.
Step by step instructions +
Code (node.js) to help you
prototype capabilities for TJBot
powered by Watson services.
http://www.instructables.com/member/TJBot/
7. IBM Research
7
Project Goals
How can we make it easier to engage a
community of enthusiasts experimenting
with embodied cognition – the idea of
embedding intelligence in everyday
objects within the physical world?
8. IBM Research
8
Project Goals
Design principle – Approachable Design
- Use of familiar material (cardboard) that can be
altered with ease.
- Simplified part assembly: no soldering or
adhesive required.
- Simplified programming model and language
interface (JavaScript).
9. IBM Research
9
Project Outcome
A prototyping platform to help
democratize Embodied Cognition.
Target communities:
- Makers
- Developers
- Students (Education and Learning)
10. IBM Research
10
How Does Watson
Enable TJBot?
Listen
Watson Speech to Text
service converts spoken
speech to text that can be
analyzed
Speak
Watson Text to Speech
service service converts
text to sound using various
voices.
Understand
Emotions
Watson Tone Analyzer service can
infer the emotion within text. E.g.. it can tell if
a message contains emotions like happy , sad,
angry
Understand
Conversations
Watson Conversation Service can
respond to users in a way that simulates a
conversation between humans.
See
Watson Visual
Recognition service can
understand the content of an
image and describe it.
11. IBM Research
11TJBot
Sensors
Example
Capabilities
Example
Watson Services
Example
Use cases
LED
Speakers
CameraServo Motor Arm
Microphone Listen
Speak
Shine
Show emotion
Wave
See
Speech to text
Tone Analyzer
Vision Recognition
Conversation
Text to speech
Sentiment
Analysis
Virtual Agents
(eldercare,
home care)
Education
(language
learning)
14. IBM Research
14
IBM Watson Cognitive
Take your first step into the cognitive era with our variety of smart services.
Services.
- Natural interaction
- Semi-structured data processing
- Trained and continuously improved via machine learning and deep
learning.
- Restful API services with SDKs for node.js, java, python.
18. IBM Research
18
Speech to Text
Converts audio voice into written text.
• Transcription
• Voice-controlled applications: allows for custom
models
https://speech-to-text-demo.mybluemix.net/
19. IBM Research
19
Text To Speech
Converts written text into natural sounding
audio in a variety of languages and voices.
• Customize and control the pronunciation of specific words to
deliver a seamless voice interaction that catered s to your
audience.
• Interactive voice based applications.
https://text-to-speech-demo.mybluemix.net/
20. IBM Research
20
Tone Analyzer
Uses linguistic analysis to detect three types
of tones in written text: emotions, social
tendencies, and writing style.
• Understand emotional context in conversations or
communications
• Taylor interaction based on sentiment.
https://tone-analyzer-demo.mybluemix.net/
21. IBM Research
21
Visual Recognition
Understands the contents of images - visual
concepts tag the image, find human faces,
approximate age and gender, and find similar
images in a collection.
• Train the service by creating your own custom concepts.
Use Visual Recognition to detect a dress type in retail,
identify spoiled fruit in inventory, and more.
https://visual-recognition-demo.mybluemix.net/
22. IBM Research
22
AlchemyLanguage
Analyzes text to help you understand its
concepts, entities, keywords, sentiment,
and more.
• Additionally, you can create a custom model for some APIs
to get specific results that are tailored to your domain.
https://alchemy-language-demo.mybluemix.net/
23. IBM Research
23
Conversation
Quickly build, test and deploy a bot or virtual
agent across mobile devices, messaging
platforms like Slack or even on a physical robot.
• Visual dialog builder to help you create natural conversations
between your apps and users, without any coding experience
required.
https://conversation-demo.mybluemix.net/
27. IBM Research
27
Libraries Used
Depends on several npm packages.
- RGB LED – ws281x library
- Servo – pigpio software PWM library
- Microphone – mic library
- Speaker – aplay library
- Camera – raspistill wrapper
28. IBM Research
28
Code Walk through: Control
LED on TJBot using voice.
- http://www.instructables.com/i
d/Use-Your-Voice-to-Control-a-
Light-With-Watson/
- Code Walk through
29. IBM Research
29
TJBot Library [Beta]
- Experimental work to encapsulate basic
functions of the bot.
- https://github.com/ibmtjbot/tjbotlib
30. IBM Research
30
The TJBot Library
Encapsulate basic functions
for TJBot such as listening,
speaking, led color change,
waving, seeing.
31. IBM Research
31
The TJBot Library
tj.listen(transcript callback)
tj.speak(“text”)
tj.converse()
tj.see()
tj.shine(“red”)
32. IBM Research
32
Code Walk through: Control
LED using the TJBot library.
- https://github.com/ibmt
jbot/recipes
- Code Walk through
34. IBM Research
34
Improving Accuracy
How do we improve interaction (voice)
accuracy? Improving Speech-to-Text
models may not be enough!
- Customized language models?
- Intent Matching?
- Multi-turn conversations?
35. IBM Research
35
Bot “Interruptibility”
When and how should the robot be
interrupted (while performing an activity
like speaking, waving etc.)?
- Vision? (monitoring a user’s facial expression,
raised hand)
- Hardware button or sensor?
36. IBM Research
36
Latency Tolerance
Latency can severely degrade quality
of interaction. How do we minimize its
effect?
- Managing and ordering service responses
- Leverage cues to provide additional information
- Balancing capabilities – cloud vs local
processing.
38. IBM Research
38
Next Steps
3 pronged
- Conduct basic research that address open
issues.
- Make TJBot simpler and easier to use (tjbot
library, visual programming tool)
- Build and sustain the TJBot community.
Presentation Overview
Hi all, thanks for coming to this session, and thanks to the organisers for letting us share this project with this awesome audience.I am new to this meetup group, and I look forward to attending more events in the near future! By show of hands, just to know the audience – how many of us
I am a researcher with the Human Agent collaboration group over at IBM research, Yorktown and we are a group of individuals this background in software engineering, HCI,behavioural psychology, cognitive sciences, robotics and mathematics. Pretty diverse group. Some of the projects we have worked on include cognitive M&A, designing interfaces for wearables and wearable apps, learning and navigation systems for robots etc.
My background has elements of software engineering, HCI and behavioural psychology.
=====
Today, I am going to talk about a relatively new project some members of our group have worked on TJBot ….This effort is co-led by my colleague Maryam Ashoori and our current work focus is related to designing novel interfaces for cognitive application development.
What are the issues that arise ?
What are viable use cases ?
What are applicable design patterns
I’ll start with an
overview of the TJBot project, why we are working on it.
I’ll also go over Watson Services which is a key aspect of this project .
Finally, I ‘ll go over technilcal details for programming TJBot and discuss some open issues we hope to learn more about as the project proceeds.
Take away …
Additional infromation on Watson services
How Watson can be embedded in objects to enable capabilities
How you can start prototyping embodied cognition apps with TJBot
Im not a developer advocate … so there might be service speciifc questions I cannot answer at the moment, but if you leave your context or tweet at me, I’ll ensure I follow up.
After we came up with the initial design,
Simplicity
Hardware
LED and hardware connection
Snap to build approach, no tools, glue or adhessive required.
Software
Javascript, the most widely used programming language
Nodejs and watson developer
Simplicity
Hardware
LED and hardware connection
Snap to build approach, no tools, glue or adhessive required.
Software
Javascript, the most widely used programming language
Nodejs and watson developer
At our research group we care about questions related to understanding the various ways in which we as humans can can work with machine in a symbiotic computing environment. We explore how, intelligence can be embedded into spaces (rooms), objects, avatars and icons etc. A subdomain on these area has to do with the design of cognitive objects and embodied cognition. It is nice to develop web based or mobile phone based cognitive apps (e.g. chatbots)Embodied Cognition
Embedding intelligence in every day objects. Understanding how physical attributes integrate with intelligent services to craft engaging user experiences.
Simplicity
Hardware
LED and hardware connection
Snap to build approach, no tools, glue or adhessive required.
Software
Javascript, the most widely used programming language
Nodejs and watson developer
Embodied cognition is a research program comprising an array of methods from diverse theoretical fields (e.g., philosophy, neuroscience, psychology, etc.) held together by the key assumption that the body functions as a constituent of the mind rather than a passive perceiver and actor serving the mind.
Helping bridge gaps for two groups of people .. Makers and developers and also working as a tool for curriculum development around cognitive application development
Embodied Cognition
When we simplify the process sufficiently, what would people create and
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
Embodied Cognition
Natural interaction with humans
For example speech and emotions
Not only used by web applications, but also in physical world
Processing of semi-structured data and big amounts of data
For example language classifications and image recognitions
Trained and continuously improved via machine and deep learning
For example search (retrieve and rank)
Leverages context to improve service qualities
Embodied Cognition
Embodied Cognition
TranscriptionUse speech to text to create voice-controlled applications – even customize the model to improve accuracy for the language and content you care about most such as product names, sensitive subjects, or names of individuals.
converts written text into natural sounding audio in a variety of languages and voices. You can customize and control the pronunciation of specific words to deliver a seamless voice interaction that catered s to your audience. Use text to speech to develop interactive toys for children, automate call center interactions, and communicate directions hands-free. https://text-to-speech-demo.mybluemix.net/
Tone Analyzer uses linguistic analysis to detect three types of tones in written text: emotions, social tendencies, and writing style. Use the Tone Analyzer service to understand emotional context of conversations and communications. Use this insight to respond in an appropriate manner.
https://tone-analyzer-demo.mybluemix.net/
Visual Recognition understands the contents of images - visual concepts tag the image, find human faces, approximate age and gender, and find similar images in a collection. You can also train the service by creating your own custom concepts. Use Visual Recognition to detect a dress type in retail, identify spoiled fruit in inventory, and more.
analyze text to help you understand its concepts, entities, keywords, sentiment, and more. Additionally, you can create a custom model for some APIs to get specific results that are tailored to your domain. https://alchemy-language-demo.mybluemix.net/
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
raspberry pi node.js libra
Simplicity
Hardware
LED and hardware connection
Snap to build approach, no tools, glue or adhessive required.
Software
Javascript, the most widely used programming language
Nodejs and watson developer
raspberry pi node.js libra
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
Improving accuracyImproving accuracy of stt services is an ongoning effort, but this environment can be really noise so accurate speech models may not be enough.Gaol: Are there good ways to break down parts of an interaction such that we keep it natural but minimize errors
TJBot play smooth criminal by michael jackson from spotify
Play me some music
Sure, what song would you like?
Solution:
Intent based matching
Multi-turn conversation as a way to improve voice based accuracy
Intterrupt:
Goal: When and how should we interrupt the robot
Should we build in approaches closesly tied with humans - e.g an uncormfortable expression, a raised hand, frowning face etc ?
Solutions?
Hardware interrupt ?
Vision based interrupt ? (raised hand, frowning face)
Latency.
Can be a real problem, especially for a robot thats humanoid. He’s cute, but that will only take him so far.
A primary source of latency has to do with making calls to cognitiev services, and the goal is to arrive
Solution:
Tools to manage response arrival,
Leverage gestures (light, arm) to cue users
Balancing computation – how much can we put on the Pi itself without degrading performance?
Improving accuracyImproving accuracy of stt services is an ongoning effort, but this environment can be really noise so accurate speech models may not be enough.Gaol: Are there good ways to break down parts of an interaction such that we keep it natural but minimize errors
TJBot play smooth criminal by michael jackson from spotify
Play me some music
Sure, what song would you like?
Solution:
Intent based matching
Multi-turn conversation as a way to improve voice based accuracy
Intterrupt:
Goal: When and how should we interrupt the robot
Should we build in approaches closesly tied with humans - e.g an uncormfortable expression, a raised hand, frowning face etc ?
Solutions?
Hardware interrupt ?
Vision based interrupt ? (raised hand, frowning face)
Latency.
Can be a real problem, especially for a robot thats humanoid. He’s cute, but that will only take him so far.
A primary source of latency has to do with making calls to cognitiev services, and the goal is to arrive
Solution:
Tools to manage response arrival,
Leverage gestures (light, arm) to cue users
Balancing computation – how much can we put on the Pi itself without degrading performance?
Improving accuracyImproving accuracy of stt services is an ongoning effort, but this environment can be really noise so accurate speech models may not be enough.Gaol: Are there good ways to break down parts of an interaction such that we keep it natural but minimize errors
TJBot play smooth criminal by michael jackson from spotify
Play me some music
Sure, what song would you like?
Solution:
Intent based matching
Multi-turn conversation as a way to improve voice based accuracy
Intterrupt:
Goal: When and how should we interrupt the robot
Should we build in approaches closesly tied with humans - e.g an uncormfortable expression, a raised hand, frowning face etc ?
Solutions?
Hardware interrupt ?
Vision based interrupt ? (raised hand, frowning face)
Latency.
Can be a real problem, especially for a robot thats humanoid. He’s cute, but that will only take him so far.
A primary source of latency has to do with making calls to cognitiev services, and the goal is to arrive
Solution:
Tools to manage response arrival,
Leverage gestures (light, arm) to cue users
Balancing computation – how much can we put on the Pi itself without degrading performance?
The library is meant to encapsulate basic capabilities of the robot
Avoid code repetition
Streamline programming paradigm and patterns
Free developers from the nitty gritty of hardware controls so they can focus on being creative about usecases
Initial feedback has been postive .. (survey and twitter data)
Individuals appear to want one before they even know what it does. Engagement .. Check!
Wide availability – we have seen people download our 3d print and lasercut designs in places like south africa, pakistan, hong kong, chile, brazil, italy
We want to take this further … make TJBot available to more people
Solve some of the open problems and encapsulate what we learn in the TJBot library.
Build and sustain the TJBot community.