Rosie: The Ultimate Artificial Intelligence MachineIllustration By: Serena Lee & Maximillian Israel
Table of Contents• Artificial Intelligence• Visual Breakdown: Rosie• The Inspiration for Rosie• 1. How Rosie Detects the Tone of Human Voice and Responds by Changing Her Facial Features (Lu Chen) o Artificial Intelligence: Kimset to Rosie o Speech Detection and Vocalization o Facial Expression• 2. Rosie Detects Human Facial Expression and Responds Accordingly (Eric Jung) o Artificial Intelligence and Facial Recognition Software o Adaptive Advertising-Immersive Labs o Incorporation of Face Detection and Adaptive Advertising Technologies into Rosie• 3. Rosie Communicates via Siri Software (Erin Ward) o The Development of Siri o Siri, the New Eliza o How Rosie Interacts with Humans• 4. Rosies human-like traits and free will function via The Sims (Serena Lee) o The Sims & Artificial Intelligence o Rosies programmable personality o Rosie does laundry on Free Will o Rosie and Free Will• 5. Rosie: Motion (George Lai) o Features o Design o Combination of Functions• 6. Medical Monitoring & Digital Portability (Max Israel) o Medical Monitoring o Artificial Doctor o Digital Portability & Cyber Companionship• Bibliography
Artificial Intelligence• Alan Turing, a mathematician, analyst, and computer scientist, is credited with fathering artificial intelligence through visions of logic and computers in the early 1900s. Artificial intelligence models cognitive processes such as deduction, reasoning, planning, learning, perception, and even creativity. The creation of intelligent machines is a concept that may only trigger images of science fiction movies and futuristic robots, but artificial intelligence is actually an integral part of our society today.• Quite a few aspects of our present lives are improved by artificial intelligence. "Smart" technologies have made possible products like the Siri application for Apple iPhones, adaptive advertising on the web, and life- mimicking video games. We have pushed our own minds to create a thinking model of the human brain, and the potential possibilities are exciting."ACM Computing Classification System: Artificial intelligence". ACM. 1998. <http://www.acm.org/class/1998/I.2.html>.McCarthy, John. “What is Artificial Intelligence?” 12 Nov. 2007.Hodges, Andrew (1992) , Alan Turing: the enigma, London: Burnett Books
The Inspiration For Rosie• Alan Turings Artificial Intelligence inspired the Kismet robot, The Sims 3 Personality and Free Will feature, Apple iPhone 4S Siri, facial recognition software,and forms the foundation of medical technology. These five elements construct the ultimate AI machine, Rosie the robot. The name Rosie was inspired by Rosie the robot from the popular cartoon The Jetsons (shown in image above).• Rosies eyes and ears are programmed to reflect the Kismet robot. Rosie is able to cry, laugh, blink and make facial contortions. With facial recognition software adapted from Immersive Labs advertising technology, Rosie is able to react to human facial expression realistically.• Rosies interaction interface is programmed to react like Siri. When humans ask Rosie questions, Rosie is able to respond in the accent of the user. Rosie interacts with users through personal programmable information. She is able to give directions, send emails, send text messages, read messages aloud, call contacts, schedule appointments and more.• With the Sims 3 Personality and Free Will feature, users can have Rosie operate on free will. The user can pick four personality traits that will give Rosie her own personality. When Rosie operates on free will, she can do tasks on her own without human supervision.• Software developed by MIT which can detect human vitals through a simple webcam will be programmed to into Rosie to monitor its owners health. In addition, the owner can choose to "sync" Rosie to their smartphone, vehicle, or laptop providing the option of digital portability.
1. How Rosie Detects the Tone of Human Voice and Responds by Changing Her Facial FeaturesLu Chen
Artificial Intelligence: Kismet to Rosie• The idea of artificial intelligence was made popular by the father of computer science, Alan Turing. The world of artificial intelligence today focuses on many human-like qualities, which can then be engineered into a robot. There are many aspects that need to be considered, such as general intelligence, social intelligence, learning, perception, and planning.• Many robots have been made to model humans. One notable example is Kismet, a robot with highly developed social abilities. Kismet is unique in the fact that it can detect the tone of the person speaking to it, analyze it, and then react to that tone appropriately via its facial features. This feature gives Kismet an emotional quality, which can be a part of the ideal robot, Rosie, and give Rosie more human-like qualities. However, Kismet is not perfect, so Rosie can improve on Kismets existing features.
Speech Detection and Vocalization• Kismet detects speech through a wireless microphone. With that microphone, Kismet can not only have an input of information, but can also analyze that information and react to it. Specifically, Kismet can detect 5 categories of affects: approval, prohibition, attention, comfort, and neutral. So if a person came up to Kismet and said, "Good job, Kismet!" the robot would then curl its lips upward, and perk up its ears with happiness. But if you scolded Kismet, it would frown, lower its ears, and lower its eyes. If this technology were to be integrated into Rosie, Rosie would definitely expand on this idea. Rosie would not be limited to just 5 categories of affects, but will be programmed to detect every affect imaginable. Some examples would be detecting sarcasm, skepticism, and urgency.• Kismet’s voice output is done through an articulatory synthesizer, which is roughly modeled after the human articulatory tract. Kismet was specifically designed to sound like a child. Rosie would greatly expand on this idea. Although Kismet can talk, it would be best if Rosie sounded like a real human. It would also be interesting to have multiple Rosie voices. The owner of Rosie could then choose what voice Rosie can have for that particular day.
Facial Expression• There are 21 motors on Kismets face and neck in order to express emotion. The ears can do various things such as perk up (showing interest), or they can fold back (to show anger). The eyebrows can furrow in frustration, elevate with respect to its head to show surprise, and slant the inner corners of the eyebrows up to show sadness. The two eyes can either open and shut at the same time to blink, or at different times to wink. Lastly, the mouth can curve up to smile or curve down to frown.• This concept can be incorporated into Rosie as well. Rosie should be able to express the same emotions as Kismet, and more. Some examples are: confused, disgusted, and stressed. That way Rosie can better resemble the emotions of humans so it can be a sympathetic and understanding companion in the home.
2: Rosie Detects Human Facial Expression and Responds AccordinglyEric Jung
Artificial Intelligence and Facial Recognition Software• Many companies have turned to artificial intelligence in marketing to better promote their product and to reach more potential customers. Online marketing companies use contextual information to generate specific advertisements tailored to the individual surfing the web. For example, Yahoo has launched "SmartAds,1" a version of this contextual advertising that accesses the users search history, email topics, and other information, and "SoDash2" company has gone a step further to allow clients like FaceBook to be able to filter out a unwanted noise. The program would ignore comments such as people merely making a joke to allow a more accurate smart ad. SoDash allows companies to see whether their products are being talked about positively or negatively, how genuine these comments seem, and more by allowing this program to be "trained" in a way that utilizes artificial intelligence. The program incorporates information gleaned from social networking sites in addition to user history and email to generate a more complete picture of the customer.• This use of artificial intelligence and "people-reading" creates a much more personalized experience on the web for every surfer, and also makes it more likely that a customer will purchase that specific product.1 "Yahoo Launches SmartAds." Tech Crunch. http://techcrunch.com/2007/07/02/yahoo-launches-smart-ads/2 "SoDash: Bring Artificial Intelligence to Social Media Monitoring."Social Media Today. http://socialmediatoday.com/mattrhodes/311406/sodash-bringing-artificial...3 "Google Launches Boost." Read Write Web. http://www.readwriteweb.com/archives/google_launches_boost_easy_local_ad...
Adaptive Advertising - Immersive Labs• Further advanced innovations include Immersive labs,4 which is a face detection software for advertisers which uses the users computer webcam to immediately analyze demographics like gender, age, and ethnicity. This program also tracks how much time a user spends looking at a specific advertisement, to collect real-time data for the advertisers and companies. To take care of any privacy issues, they make sure not to record any sensitive information and no images are saved or recorded. These cameras are thus able to track and effectively read a persons face, and the Immersive Labs technology can then use this information to create a more personalized, adaptive experience for the consumer.4. Immersive Labs. http://www.immersivelabs.com/
Incorporation of Face Detection and Adaptive Advertising Technologies into Rosie• In the same way that Immersive Labs uses their face detection software to read and analyze a persons face, the home robot Rosie will incorporate this technology to be able to respond accurately to human facial expression. Rosie will be able to detect instantly to whom she is speaking, and then will be able to tailor her conversation accordingly. In addition, Rosie can detect when a person starts to lose interest by tracking the amount of time their eyes remain focused on her, and can tell whether someone is joking or being genuine through SoDash technology.• Rosie can also incorporate the technology from SmartAds contextual advertising by keeping a running history of various interactions with a specific person, and can thus stay relevant in her conversation with the human users. She can even offer new products and services that she thinks will interest someone in particular, just as contextual advertising aims to do.
3. Rosie Communicates via Siri SoftwareErin Ward
The Development of Siri • Siri (Speech Interpretation and Recognition Interface) is best known for being an “intelligent personal assistant,” an artificial intelligence software that debuted with the Apple iPhone 4S in 2011 to help mobile users multitask and speed up their searches, but the history of Siri goes beyond the popular Apple interaction interface. • Before Siri was integrated into the Apple iOS software, Siri was a separate application in the Apple App Store (for free) and was supposed to later partner with both Android and Blackberry, but was then bought out by Apple completely and prevented from expanding to other devices. • Even before Siri’s App Store debut, however, the A.I. research that sparked Siri’s birth was originally performed by DARPA (Defense Advanced Research Projects Agency) via the SRI (Stanford Research Institute) Artificial Intelligence Center.Dachis, Adam. "All About Siri, Your iPhones New Assistant." Image. lifehacker.com. 04 Oct. 2011. Web. 02 June 2012. <http://cache.lifehacker.com/assets/images/17/2011/10/1200-siri.jpg>.
Siri, the New Eliza• Siri is essentially how Turing envisioned the future. He once said, “One day, ladies will be walking their computers in the park and saying, ‘do you know, my little computer said such a funny thing to me this morning!” And with Siri’s ability to freely respond to the user without pre-programmed sets of responses, Turing’s vision is essentially coming true.• Simply put, Siri consists of three main layers: a speech-to-text analyzer, a grammar analyzer, and a set of service providers. This current version of Siri represents the first stage of a breakthrough in the AI field—intelligent systems that identify and react to the environment. Potts, Andy. "Alan Turing at 100." 2012. Photograph. Nature Journal. Web. 02 June 2012. <http://www.nature.com/news/specials/turing/index.html>.
How Rosie Interacts with Humans• After over a year of trial runs on Apple’s iPhone platform, Siri is being taken to the next level and incorporated into Rosie. Apple’s original contract with Siri expired, and Rosie Inc. drafted a new contract for the use of the Siri software use.• Siri has become the model for how humans interact with computers—Today, we are all experts with the computers and interacting with the keyboards, but tomorrow we will be conditioned to be more adept at dictation. The man-machine interface will evolve to incorporate natural communication as opposed to human-to-nonhuman commands, and artificial intelligence will be limitless in terms of its reactive capabilities in engagement with humans.• Rosie’s interaction interface will use a developed version of the Siri software, in which Rosie will be able to interpret the accent of the user, internalize their unique sound, and respond accordingly with the same accent. Likewise, Rosie will be able to personalize the user’s information after their first use through location services, and by syncing their personal information through their e-mail and phone. Rosie will be able to give directions, send e-mails, send text messages, read messages aloud, call contacts, schedule appointments, and more, merely through the user’s audio commands.Humans and Robots could walk together. Info News Technology. 2011. Photograph. 03 June 2012.<http://infonewstechnology.com/humans-and-robots-could-walk-together/humans-and-robots-could-walk-together-img03/>.
4: Rosies human-like traits and free will function via The SimsSerena Lee
The Sims & Artificial Intelligence• Richard Evans, the AI lead on Sims 3, wrote about artificial intelligence in “The Sims 3.” In the game, different sims have different personality traits. For example, some sims are family-oriented while other sims are flirty. These traits, according to Richard, influence autonomous behavior. The AI team for the Sims 3 focused on letting the sims work on their own. He said “One big thing we have insisted on from the start is that the Sims won’t need you to hold their hand every time they need to use the toilet. They are now able to take care of their basic needs on their own, leaving you to focus on more important things: building relationships, expanding your career opportunities, and exploring the world around you.”JudHudson. "Sims 3 VIP Blog: Artificial Intelligence in the Sims 3 ." Sim Programs. N.p., 26 02 2009. Web. 19 May 2012. <http://www.simprograms.com/4735/sims-3-vip-blog-artificial-intelligence-in-the-sims-3/>.
Rosies programmable personality • In the Sims 3, Sims can be programmed tocaring flirty have up to five personality traits. Similarly, Rosie can be programmed to have up to fivelazy hardworking personality traits. These personality traits will give Rosie a "human-like" personality.emotional careless Out of a selection of 20 traits (left), Rosie can be a mix of five distinctive traits. A usertech whiz book lover can prefer Rosie to be a hard working, caring, family-oriented, organized and anight owl morning robot good chef. A user may prefer Rosie to be more like a friend with traits like creative,chef creative whimsical, flirty, caring and creative. Or, the user can program Rosie to be lazy,distracted neurotic distracted, neurotic, emotional andorganized whimsical prankster.genius musical • The extensive personality trait combinationsprankster family-oriented will give Rosie her own personality whether she acts more like a housekeeper or more like a lazy college student.
Rosie does laundry on Free WillLee, Serena. Rosie at Free Will . 2012. Illustration.Laundry Room Main. 2012. Photograph. Elle Décor. Web. 02 June 2012. < http://www.elledecor.com/files/web/images/articles/laundry-room-main.jpg>.
Rosie and Free Will• Rosie can be programmed to operate on free will. When Rosie free will function is turned on, Rosie will busy herself with tasks. Since Rosie has personality traits, many of her tasks may be influenced by her characteristics. For example, if a Rosie is programed with lazy, musical, prankster and tech whiz, Rosie may update computer iOs programs, play music, prank the user and be unresponsive to household tasks such as laundry and cleaning the house. In contrast, if Rosie is programmed with productive and family-oriented traits, Rosie may do laundry, fold laundry, make meals, schedule appointments and play with the kids while a user is busy at work or at school.• The purpose of Rosies free will feature is to simplify the human users life. The user will not have to worry about watering plants, taking out the trash, doing laundry and cleaning the toilet. The free will feature can be abused if Rosie has a neurotic temper tantrum or is too lazy to do work.
6. Medical Monitoring & Digital PortabilityMaximillian IsraelIllustration Edited By: Serena Lee & Maximillian Israel
Medical Monitoring• Based on software developed by MIT, Rosie will be programmed with the capacity to monitor her owners vitals and health. With cameras embedded in Rosie’s eyes, she can measure pulse, respiration, blood pressure, and blood-oxygen measurements of her owner and other humans she encounters. Her software enables her to accurately measure human vitals by identify her subjects face as a digital video image, and then the digital information from this area is broken down into the separate red, green and blue portions of the video image (described in Figure 1 on the next slide). The software then measures slight variations in brightness produced by the flow of blood through blood vessels in the face. In addition to monitor vitals, Rosie can store her measurements in a digital file which the owner’s doctor has access to. This will help provide doctors with more medical information about their patients so that they can notice health patterns and develop more accurate diagnoses. With this software, Rosie will be programmed to dial 911 and summon an ambulance if she detects cardiac arrest within the household. This technology can save lives and enhance the quality of life for the owner. M. Poh, D. McDuff, and R. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt. Express 18, 10762-10774 (2010). Chandler, David L. "Your Vital Signs, on Camera." MITs News Office. MIT News, 4 Oct. 2010. Web. 01 June 2012. <http://web.mit.edu/newsoffice/2010/pulse- camera-1004.html>.
Medical Monitoring (con’t) M. Poh, D. McDuff, and R. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt.Express 18, 10762-10774 (2010).
Artificial Doctor• In addition to Rosie’s ability to detect and monitor human vitals, she will be programmed with medical software. This medical software will allow the owner to customize Rosie with their medical record so that her health monitoring becomes specialized for each household member. For example, if an owner has a heart condition he/she can customize Rosie’s programming to monitor that person and notify him/her of irregular heartbeats, high or low blood pressure, and provide nutritional recommendations and warnings. Furthermore, if someone in the house is not feeling well, they can consult Rosie and tell her their symptoms. She can then process that information with her vital measurements of the person and come up with a probable diagnosis. Rosies medical abilities do not stop here, if someone in the house suddenly gets a heart attack, in addition to her automatic summoning of the ambulance, she can direct and step others in the household through how to perform CPR on the victim. "A Doctor’s Office, Revisited." The New York Times. The New York Times, 24 June 2010. Web. 01 June 2012. <http://www.nytimes.com/interactive/2010/06/25/us/20100625- voice-graphic.html?ref=science>.
Digital Portability & Cyber Companionship• Rosie’s heart and soul is composed from her programming. Her physical form can vary from robot to laptop to smart phone to several other electronic devices. Her essence is contained within a knowledge base programmed into a master computer within the owner’s home. This master computer communicates wirelessly via Bluetooth with a computer embedded within the robotic structure, the owner’s smart phone, etc. All devices perform bidirectional communication with the master computer. However, they do not rely on the master computer for functioning. The communication is used to relay new information from the environment from one device to another. For example, if Rosie processes new music preferences from a household member, she will then send her data to the master computer who will store it and update these preferences on another device such as the person’s vehicle. I created a diagram on the next slide (Figure 2) to illustrate the communication relationship between devices.• Furthermore, this technological setup allows Rosie to be portable and become what I call a “Cyber Companion.” This way Rosie is digitally portable and can travel with you to work, on vacation, etc. Imagine you leave your home where the robot resides and as soon as you enter your vehicle you are greeted by Rosie. She then helps guide you to a meeting for work. When you get to out of your car and are headed to the meeting you want to set up a reminder to pick up a gift for your anniversary next week. So you take out your smart phone, activate Rosie, and say “Rosie, please set up a reminder for me to pick up champagne and flowers for next Monday at 5:00PM.” Rosie responds “I have setup your reminder. Would you like me to generate a list of all the local flower and champagne vendors?” Rosie is not just restricted to her robotic form, but is rather a digitally entity who can accompany you wherever you go and lend a helpful hand! Franklin, Curt, and Julia Layton. "How Bluetooth Works" 28 June 2000. HowStuffWorks.com. <http://electronics.howstuffworks.com/bluetooth.htm> 06 June 2012.
Digital Portability (con’t)Figure 2: Bidirectional Communication Diagram By: Maximillian Israel
Bibliography"A Doctor’s Office, Revisited." The New York Times. The New York Times, 24 June 2010. Web. 01 June 2012. <http://www.nytimes.com/interactive/2010/06/25/us/20100625-voice-graphic.html?ref=science>.“Alan Turing.” Wikipedia. Wikimedia Foundation, Inc. Web. 23 May 2012. <http://en.wikipedia.org/wiki/Alan_turing#Turingery>“Automatic Computing Engine.” Wikipedia. Wikimedia Foundation, Inc. Web. 23 May 2012. <http://en.wikipedia.org/wiki/Automatic_Computing_Engine>Bob. "The Cardiocam: Physiological Monitoring via Webcam." RDN Consulting. Software Development and Biomedical Engineering, 19 Dec. 2010. Web. 01 June 2012. <http://rdn-consulting.com/blog/2010/12/19/the-cardiocam- physiological-monitoring-via-webcam/>.Chandler, David L. "Your Vital Signs, on Camera." MITs News Office. MIT News, 4 Oct. 2010. Web. 01 June 2012. <http://web.mit.edu/newsoffice/2010/pulse-camera-1004.html>.Franklin, Curt, and Julia Layton. "How Bluetooth Works" 28 June 2000. HowStuffWorks.com. <http://electronics.howstuffworks.com/bluetooth.htm> 06 June 2012.“History of Robots.” Wikipedia. Wikimedia Foundation, Inc. Web. 1 Jun 2012. <http://en.wikipedia.org/wiki/History_of_robots>Hunter, Jenny. "IBM Research Unveils 3D Avatar to Help Doctors Visualize Patient Records and Improve Care." IBM News Room, 26 Sept. 2007. Web. 01 June 2012. <http://www-03.ibm.com/press/us/en/pressrelease/22375.wss>.Kain, Erik. "Apples Siri and the Future of Artificial Intelligence." Forbes. 15 Oct. 2011. Web. 19 May 2012. <http://forbes.com/sites/erikkain/2011/10/15/apples-siri-and-the-future-of-artificial-intelligence/>.Kismet, the Robot.” 20 May 2012. <http://www.ai.mit.edu/projects/humanoid-robotics-group/kismet/kismet.html>.“Kismet, the Robot.” National Public Radio. 9, April 2001. 20 May 2012. <http://www.npr.org/programs/morning/features/2001/apr/010409.kismet.html>.“Kismet – The Sociable Robot.” HowStuffWorks. 20 May 2012. <http://videos.howstuffworks.com/mit/2665-kismet-the-sociable-robot-video.htm>.M. Poh, D. McDuff, and R. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt. Express 18, 10762-10774 (2010).Medical Avatar. YouTube.com. MedicalAvatar, 25 Jan. 2011. Web. 28 Apr. 2012. <http://youtu.be/7sPDSRtp638>.“Meet Kismet.” 20 May 2012. <http://www.robotbooks.com/kismet-robot.htm>.Millian, Mark. "Apples Siri voice assistant based on extensive research." CNN.com. 04 Oct. 2011. Web. 20 May 2012. <http://articles.cnn.com/2011-10-04/tech/tech_mobile_iphone-siri_1_scott-forstall-iphone-apple-co-founder? _s=PM:TECH>.MIT Media Lab Medical Mirror. Prod. Melanie Gonick. Perf. Ming-Zher Poh. YouTube.com. MIT News Office, 01 Oct. 2010. Web. 28 Apr. 2012. <http://youtu.be/LyWnvAWEbWE>.OGrady, Jason. "Siri=Eliza? (Update: She knows her)." ZDNet.com. 16 Oct. 2011. Web. 19 May 2012. <http://www.zdnet.com/blog/apple/siri-eliza-update-she-knows-her/11413>.“Robots, Our New Friends Electric?” The Guardian. 13 April 2008. 20 May 2012. <http://www.guardian.co.uk/science/2008/apr/14/sciencenews.news>.Sofge, Erik. "Will Wrights 5 Prophecies about Artificial Intelligence ." Popular Mechanics. 18 12 2009: n. page. Web. 19 May. 2012.Still Gaming. "Gaming After 40: Eliza vs Turing Test."Gaming After 40. Blogspot, 18 09 2009. Web. 19 May 2012. <http://gamingafter40.blogspot.com/2009/09/eliza-vs-turing-test.html>.“Super Robust Robot Hand.” 25 Jan 2011. YouTube. Online Video Clip. Accessed on 23 May 2012. <http://www.youtube.com/watch?feature=fvwp&NR=1&v=YqmRKqFqiok>"Talk to Eliza." Eliza Program. Vistua, n.d. Web. 19 May 2012. <http://www.vistua.com/cgi-bin/eliza.shtml>."The Marketing Implications of Artificial Intelligence." IMedia Connection. http://www.imediaconnection.com/content/22336.aspTobak, Steve. "Inside Siri--Apples game-changing technology." CBS News. 12 Dec. 2011. Web. 20 May 2012. <http://cbsnews.com/8301-505125_162-57341019/inside-siri-apples-game-changing-technology/>.“Turing test.” Wikipedia. Wikimedia Foundation, Inc. Web. 23 May 2012. <http://en.wikipedia.org/wiki/Turing_Test>Wofford, Jeff. "How Siri Works." Wordpress. 05 Oct. 2011. Web. 19 May 2012. <http://www.jeffwofford.com/?p=817>.The Medical Mirror. Prod. Virgil Wong. YouTube.com. MedicalAvatar, 16 Mar. 2012. Web. 28 Apr. 2012. <http://youtu.be/p9HPxrg4uTY>.Wong, Virgil. "MEDICAL AVATAR." MEDICAL AVATAR // See a better you. Medical Avatar LLC, n.d. Web. 28 Apr. 2012. <http://medicalavatar.com/>.