Life Beyond Embodiment

Life Beyond Embodiment Leandro Agro - IxD Director Kallideas group How an Italian company is working to reduce this distance, by creating emotional virtual agents to move bytes toward atoms on one side, while moving atoms toward bytes, Second Life's human users, on the other.

Some people think Second Life is hype. Others think Second Life is falling down. A lot of people like virtual assistants. Others don't care about them. ...it is not important whether a specific technology is cool enough, but whether that technology is along the way between...

We live in a world of cognitive artifacts humans + machines are part of the same ecosystem

We live in a world of cognitive artifacts …and sometimes artifacts outnumber us

We are accidental users of many artifacts! This means user interfaces are... VERY important, because… the evolution of an artifact implies the evolution of the one who uses it

Do you feel the distance between bits and atoms? Earth: the world that we are living in

How to bridge the gap?

Perhaps we could port some popular Internet services to the real world… Aram Bartholl’s way

Hardware that show your IM status

how about equipping robots with better sensitivity?

Artifact’s UI A user interface is a system in which input devices and interaction techniques determine the tasks that users can handle. Old-fashioned green CRT UIs bring back to mind white lab coats. Meanwhile, today’s GUIs provide new ways to work for almost everybody in the world. …but now, what’s next? Task Interaction technique Input device UI

The gaming industry experimented with a lot of different input devices. Unfortunately, once the joystick became the standard , every game looked similar to the others. Revolutionary UI… It is not simply a new mouse! From Paddles to Pads: Is Controller Design Killing Creativity in Videogames? - Tom Armitage

Blade Runner

We already have a “Blade Runner” UI

In the world of User Generated Contents and long tails: ”markets are conversations” A complex, multimodal approach could fit professionals; while a “ conversational UI ” could fit interactions between companies and people, giving users more flexibility and freedom.

How to bridge the gap?

We feel this gap is too large to approach from a single side.

So, for communications over the Internet, we are creating emotional virtual agents ;

K-human ™

Which kind of assistants?

1988 200x Personal Assistant vs Profiled Users From a personal virtual assistant to an agent that is an extension of the bank, university, Italian restaurant, etc.

Which Technologies do we need to build a working system like the one in the Apple scenarios? First of all, we need:

Internet-based

Multimodal input—the ability to perceive the user... through touch, gesture, eye movement

Text2speech - Integration with standard Knowledge Base

Speech recognition (emotions detection) - AI (Brain)

Are we able to build an effective Virtual Assistant?

Back to Virtual Assistant’s Dream Julie: customer service Claudia: K-human First intelligent and emotive virtual assistant.

Whitch Brain? Anthropomorphous + Emotional Minsky rejects the idea of consciousness as a unitary "Self" in favor of "a decentralized cloud" of more than 20 distinct mental processes. In this view, emotional states like love and shame are not the opposite of rational cogitation; both, Minsky says, are ways of thinking. Multimodal approaches are changing the rules in UI design, and increasing the bio-diversity in the IT world.

Anthropomorphous software

Understand unformatted text

Understand the human voice (by means of Voice Recognition technology);

Accept other inputs (touch screen, eye-tracking, DTMF, etc.)

Read text (by means of Vocal synthesis technology.);

Write text and deliver documents (stored in a knowledge base system);

Perfome actions (like send e-mail, send SMS, print receipts …

Interact with users emulating humanity with personality + emotionality

Dialogue with users in a natural way with the goal of understanding

Emotions could fill the gap! Such a process should lead to machines that appear intelligent. Or at least to put up with machines much better than time of windowing interfaces. Roberto Ostinelli, Kallideas R&D Lab

Richer Knowledge requires AI

Your Customers needs AI Critical factors

Market Scenarios Present and upsell products Understand each customer’s needs, access corporate Knowledge Base to identify the best product or solution, propose additional opportunities. A sales agent that knows who you are and the best solution for you. Allow self-training for employees Real-time support in problem solving: answer queries and proactively present new content in an easy and interactive way. Learning by doing has never been so easy. Automate help desks Guide the user in a diagnostic process, increase automate ticket resolution proposing the right solution, best routing the residual escalation. Save you the time spent in finding “how is it broken?” Support customers on new products (DTT) Design a KB approach based on AI to simplify knowledge creation and evolution. Identify issues and automatically solve problems granting comparable user experience on different channels. Learn and improve from every problem. Increase Customer Satisfaction, Loyalty, Sales

Growing customers revenue through conversation Usefulness: New Frontiers of Support Incentive: New Opportunities for cross and up sell Dependence: Pervasive entertainment

Guide through your website

Assist in device and services configuration

Support form filling, registration, purchasing

Profile dynamically on the interaction content

Apply dynamic and adaptive schemes

React immediately on customers’ feedback

Interact proactively

Create an evolving relation

Customize contents on preferences

Innovate the way to offer ringtones, music, games, services

How it works BEHAVIOUR MODULE Describes the personality (one each virtual assistant) that filter the emotions through the environment. REPRESENTATION MODULE Analyzes the environment where the inputs are generate (i.e. dialog flow, dialog context). KNOWLEDGE MODULE Represents the inside and extern know how. OUTPUT MODULE Shows the result of emotional output computation and related rendering. EMOTIONS MODULE Represents the emotions set, its math description and the math functions to map the inputs and the representation into emotions. INPUT MODULE Collects all inputs (voice, text, touchscreen, etc...) and analyze their semantic

THE ARCHITECTURE This is Raul. A VA concept for ticketing…

Look behind the scenes

Pixels at your service Tanya interviewed her creators

How does this feel as a UI?

Case Studies

Silvia: Infopoint for customers

Silvia’s specs

Multichannel

Phone (Automatic Speech Recognition)

Web (chat)

Video Streaming

Horizontal Knowledge

Basic AI (rules engine)

Silvia’s Behavior

Discussion engine

Keep track of conversation

Managing the topic switch

Silvia’s Numbers

CartaSI holds 8.5 million credit card. The CartaSI call center gets 16 million incoming calls every year CartaSI hold 8,5million of credit card.

- During the first day (234 calls) she was unable to understand less than 6% of conversations . Silvia said “I don’t understand” more that 100 times.

After two weeks she miss the meaning of less that 6% of sentences, but can recover by asking the user a new yes/no question. Over 90% of errors are because she completely doesn’t know the topic!

Silvia understand about 96% of users conversations (first day 89%)

Silvia’s AI regardless of whether mobile or web users, and she handle more than 1000 calls by day.

Betti: intranet troubleshooter

Betti’s specs

Real-Time 3D Rendering

Text to Speech TTS

Specialized Knowledge

Machine-learning AI (graph navigation)

Betti’s Behavior

Disambiguation (she can handle hardware problems and lead users to understand exactly what is wrong).

Help desk ticket management (by email and IM)

Silvia & Betti

Are Similar

Same rendering engine (3D vs 2D)

Same text to speech (TTS) engine

Same AI engine

Are different

Stream vs Real Time

Horizontal Knowledge vs Specialized Knowledge

When people in CartaSI and other clients talks about K-humans, they don’t talk about pro and cons of their new CRM software. Our clients don’t experiece our VA like a cold piece of HW/SW. They like to talk just about Silvia or Betti.

Tracy Trailer: call her, to discover best movies

Tracy Trailer’s specs

She lives in a virtual TV-studio on a mobile phone GigaSIM!

She is an Intelligent Download Agent (background download of matching and newer trailers)

Tracy Trailer’s Behavior

She shows trailers according typologies and movie theatres’ locations

She updates the user’s profile according to

Feedback after the movie

Questions about previous films

Feedback about home video releases

What happens when a synthetic world (bytes) hosts humans (atoms)

Tryst: The game

A virtual place where it is possible to meet other players

A virtual home to set our place and host other players

Tryst’s specs

For mobile phones only

Videoportal (Italian tech that mixes video calls with a media player platform)

Insite eCommerce platform

Tryst’s Behavior

Real person gets an avatar to play

VAs (chat box) are avatar too

At the very beginning, when playing Tryst, you don’t feel a difference between human players and chat-boxes. Depending the user population, Tryst could be considered

Dating platform, dating simulation

Gaming platform (SIMs)

Tryst’s Shopping Engine

Digital is not “virtual”

Real digital music

Virtual object for your home

Buy for yourself or give a gift to someone else.

Do you remember Claudia?

She is a machine. Bytes (pixel) by bytes (AI).

When Claudia is working (in a call center, for example) she is a machine that replaces (well… actually supports) a human. Meanwhile when Claudia is going around in Second Life and meeting new friends, she is a real person. A human that plays the role of a machine, giving to us an opportunity to reflect about where our world is going. We are getting closer to a huge theatre, where human and machines plays together. ….to be continued (SL: the place where machines learn)

Feel free to ask to me anything you like… http://www.leeander.com As long you as you ask it in Italian :) Q&A We have computers that are more than 100.000 times faster (Moore’s Law) than Apple I (1976), huge and super-fast memory and broadband. The newest input devices can recognize eye gaze, gesture, and voice, not mechanically but taking into account human behaviors. Do you think that VAs are not intelligent enough? Probably, you’re right, but they just started to interact with humans and they are looking smarter every day.

Thank you ! - From GUI to E(motional) UI