SlideShare a Scribd company logo

Drawings for Alternative Communications Patent

•
•
R
robtedesco
1 of 43
Download to read offline
1 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH 22 FIG. 1 AT&T 3G 8:22 AM Text Calendar Photos Camera Stocks Maps WeatherRecorder Calculator Notes SettingsClock AAC+ Mail Safari iPodPhone 3 SMS 11 Thursday 73* + - x / 10 14 16 20 18 12 AAC+
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 2/43 FIG. 2 PRIOR ART MICROPHONE SPEAKER KEYBOARDDISPLAYCAMERA USER INTERFACE 10 24 16 26 14 18 12 CPU 28 ACCELEROMETER 34 MEMORY 30 SOFTWARE TRANSCEIVER CIRCUITRY 36 32 WIRED PORT 40 BATTERY 42 MISCELLANEOUS PERIPHERALS 44 38
3 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 3 PRIOR ART PSTN INTERNET PLMN 46 50 52 10D 56 10C 10B 10A 48 54
4 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH SMS 11 73* RH 10 58e 58d 58b FIG. 4 58c 58a 62 60 58f 64
5 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 5 CAREGIVER ACQUIRES MOBILE TERMINAL SOFTWARE LOADED ONTO MOBILE TERMINAL ESTABLISH A USER PROFILE STORE A CONTEXTUAL CRITERION FOR OUTPUTTING A SPEECH ICON OUTPUT THE SPEECH ICON 100 102 104 106 110 108 IS CRITERION SATISFIED? YES NO
6 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH AT&T 3G 8:22 AM Profiles AAC+BACK AAC+ 10 16 18 General Settings Profiles and Settings FIG. 6 AAC+ TO FIG. 8 “Caregiver Mode” Enabled (tap to change to “End User Mode”)
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 7/43 FIG. 7 USER SELECTS PROFILES COMMAND USER SELECTS PRE-EXISTING PROFILE OR TEMPLATE 150 152 USER CREATES NEW PROFILE 158 LOAD 154 GO TO BLOCK 108 OF FIGURE 5 EDIT 156 GIVE NAME TO PROFILE 160 GO TO BLOCK 104 OF FIGURE 5
8 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 8 AT&T 3G 8:22 AM AAC+ Profiles BACK Custom profiles Templates Nonverbal adolescent with autism Help with expressing feelings Adult user with semantic-pragmatic... Jimmy Smith Laura’s normal settings Settings for Laura’s vacation TO FIG. 10 TO FIG. 9 200 202 204
9 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 9 AT&T 3G 8:22 AM AAC+ Profile: Jimmy Smith BACK 208 220 218 EDIT Icon size: Large Default voice: Young Male Assoc. #1: “McD Menu” at McDonalds LOAD 210 212 206 Assoc. #2: “Help group” after “Help!” 214 Assoc. #3: “Basic conversation” near Mrs. Miller 216
10 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 10 AT&T 3G 8:22 AM AAC+ Create / Edit a Profile BACK Profile Name: Enter name here Speech/Language Impairment Info General Settings for This Profile Icons SAVE PROFILE To Fig. 11 To Fig. 15 To Fig. 30 User’s Schedule/Calendar Icon-Context Associations To Fig. 18 Contexts To Fig. 21
11 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 11 = AT&T 3G 8:22 AM AAC+BACK How would you like to provide speech/ language impairment information? Take a survey Enter specific impairment info Perform a user evaluation TO FIG. 12 TO FIG. 13 TO FIG. 14 Import data from another source
12 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 12 = AT&T 3G 8:22 AM Language Development SurveyBACK In response to the following questions, please mark “Yes” or “No”. Does the user know the correct names of some colors? YES NO Does the user say about 50 different words? YES NO Does the user say about 100 different words? YES NO Does the user say words that tell something about other words, such as “loud” in the utterance a “loud noise”?
13 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 13 = AT&T 3G 8:22 AM Enter Impairment InformationBACK Dyslexia Does user have dyslexia? Vocabulary General vocabulary level Known vocabulary words Stuttering and Annunciation Below Avg Avg Above Avg NO Does user stutter? YES Known annunciatory problem words: Type here (e.g., “bathroom”) CATEGORIES Colors Aqua Blue Brown Burgundy Crimson Forest Green Gold Green Lavender
14 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 14 = 3G 8:22 AM Say: “bathroom”BACK Say the word “bathroom”: PLAYSTOP SaveUndo RECREW 66 66a 14 Say, “bath-room” please.
15 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 15 = AT&T 3G 8:22 AM AAC+BACK Which general settings would you like to program for this profile? Caption font Caption size Icon style Icon size Language Voices Repetitions Participatory voice output Icons per screen TO FIG. 16 TO FIG. 17 Select a default voice: Young male (synthetic) Adult female (synthetic) Rosa Voice 124153 Jimmy Smith’s voice Save 68 Volume
16 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 16 = AT&T 3G 8:22 AM AAC+BACK Configure default “Repetitions” settings that should apply to this profile: Number of repetitions Accuracy threshold Penalties Rewards Prompt users to repeat voice outputs Number of required repetitions: Save ON 0 1 2 3 4 How accurate must the repetition be? Save 30% 40% 50% 60% 70% Configure “Repetitions” for a particular speech icon or group of icons: Browse categories Search Penalties Save Enforce a penalty if user fails to repeat Disable all voice output OFF Disable a speech icon Restrict game/Web access Rewards Save Give a reward if user completes repetition Tokens and points ON Messages Enable game/Web access TO FIG. 30
17 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 17 = AT&T 3G 8:22 AM AAC+BACK Configure default “Participatory voice output” settings that should apply to this profile: Frequency of participation Accuracy threshold Penalties Rewards Prompt users to participate in voice outputs Frequency of participation: Save ON How accurate must the participation be? Save 30% 40% 50% 60% 70% TO FIG. 30 Sometimes Word or phrase length Word or phrase type Always Browse categories Search Specific words, phrases, etc: Type of participation Save User finishes Step by step Increase MLU over time User begins Type of participation:
18 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 18 AT&T 3G 8:22 AM Icon-Context AssociationsBACK + “BK Menu” at 32 Main Street Burger King “Help group” after “Help!” Rule 1 (11/15/10 4:57 pm) “Basic conversation” near Mrs. Miller Rule 2 (11/23/10 6:18 am) “Recess icons” on Mon-Fri from 12:15 to... “Nervous emotions” based on heart rate and… TO FIG. 19 “Swing set” at McNamee Park
19 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 19 = AT&T 3G 8:22 AM BACK Create TO FIG. 20 WQ E R T Y U I O P SA D F G H J K L XZ C V B N M space return.?123 Name your Icon-Context Association: Icon-Context Assoc.
20 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 20 = AT&T 3G 8:22 AM Icon-Context AssociationsBACK How would you like to create an association between a speech icon and a particular context? Start with a context, then select icon(s) that should be presented in that context Start with an icon, then select context(s) in which that icon should be presented TO FIG. 21 TO FIG. 30
21 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 21 = AT&T 3G 8:22 AM ContextsBACK Sounds, noises or speech Physical movement Objects or people near this device TO FIG. 22 Temperature Lighting The location of this device The user’s body Times, dates and scheduled events + TO FIG. 29 Search
22 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 22 = AT&T 3G 8:22 AM Icon-Context AssociationsBACK Specify a location that should trigger output of a speech icon: Search/browse by location name Find a location on a map Draw a “zone” on a map How would you like to identify a location? 70 TO FIG. 23 Browse categories Search
23 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 23 AT&T 3G 8:22 AM Results for “McDonalds”BACK 1. 5065 Main Street | Anycity, USA | 2.84 mi 2. 390 Route 105 | Anycity, USA | 4.31 mi 3. 12 Drawley Lane | Nextown, USA | 5.39 mi 4. 101 Elm Street | Nextown, USA | 7.01 mi 5. 65 Far Drive | Farville, USA | 12.4 mi Show More Results TO FIG. 24View Selected
24 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 24 AT&T 3G 8:22 AM View/edit context (McDonald…)BACK McDonald’s 5065 Main Street | Anycity, USA General Information TO FIG. 25 ATM WIFI View/Edit Icons and Settings for “McDonald…” Add New Speech Icon(s) Browse categories Search More info TO FIG. 27No icons have been saved in association with this location.
25 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 25 AT&T 3G 8:22 AM Icon results for “fries”BACK FRENCH FRIES (1) Saved to 13 contexts Frequency of use: High Add Selected Icons to Context FRENCH FRIES (2) Saved to 4 contexts Frequency of use: Medium FRIED DOUGH Saved to 0 contexts Frequency of use: Low More info More info More info
26 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 26 AT&T 3G 8:22 AM Add “FRENCH FRIES (1) to…”BACK FRENCH FRIES (1) Add speech icon: FRENCH FRIES (1) to “McDonald’s 5065 Main St.”? TO FIG. 27 McDonald’s 5065 Main Street Save Icon to This Location
27 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 27 AT&T 3G 8:22 AM Icons and settings for “McD…”BACK BURGER FISH SANDWICH FRENCH FRIES Caption font Caption size Icon style Icon size Language Voices Repetitions Participatory voice output Icons per screen Icons (6): Settings for this context: Icon ranking Screen Editor TO FIG. 28 Volume
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 28/43 Screen Editor for “McDonald’s 5056 Main Street”BACK FIG. 28 Screen Editor for “McDonald’s 5056 Main Street”BACK BURGER FISH SANDWICH FRENCH FRIES ICE CREAM PLEASECOLA 72 74 76 78 80 82 SAVE SAVE
29 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 29 AT&T 3G 8:22 AM Create a new locationBACK Name this location: Drag and drop pins to define boundaries: SaveUndo Enter name here 84
30 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 30 AT&T 3G 8:22 AM IconsBACK + actions animals around the house art body parts clothing colors Search TO FIG. 31
31 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 31 AT&T 3G 8:22 AM Icon results for “burger”BACK BURGER Saved to 9 contexts Frequency of use: High TO FIG. 32 BURGER WITH BACON Saved to 2 contexts Frequency of use: Low HAM BURGER Saved to 0 contexts Frequency of use: Low
32 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 32 AT&T 3G 8:22 AM Icons: BURGERBACK BURGER Saved to 9 contexts Frequency of use: High Related icons Caption font (Arial) Caption size (Normal) Icon style (Photo) Icon size (Medium) Language (English) Voices (Young male) Repetitions (OFF) Participatory voice output (OFF) Settings for this icon: Volume (Default) TO FIG. 21Add “BURGER” to a new context
33 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 33 AT&T 3G 8:22 AM View/edit context (Low blood…)BACK Low blood sugar (beneath 70 mg/dL for more than 15 minutes) General Information View/Edit Icons and Settings for “Low blood…” Add New Speech Icon(s) Browse categories Search More info BURGER 70
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 34/43 FIG. 34A = 8:22 AM Create a new location Name this location: SaveUndo FIG. 34B FIG. 34C 10 Drag and drop pins to define boundaries: “Maiden Ridge” Neighborhood STEPHEN “Maiden Ridge” Speech Icons (Screen 1)BACK SALLY GO JOE PETER PASS TURN “Maiden Ridge” Speech Icons (Screen 2)BACK “Maiden Ridge” Speech Icons (Screen 3)BACK ME WANT I MRS. SMITH HOME PLEASE SKATEBOARD RAMP BASKETBALL FOOTBALL COOL FUN COOL FUN 86a 86b 86c 88a 88b 90 88c 92 16
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 35/43 FIG. 35A Set maximum levels for: FIG. 35B FIG. 35C 130 BPM Heart Rate Med-High Electrodermal Activity HeartRate(BPM): Levels Exceeded! Transmitting message... GSR: 8:22 AM AAC+ 10 58b 58d -5000 0 5000 10000 15000 = 8:22 AM 11 7 3 * MedLow Med-Low 54 AAC+ -- Caregiver Workstation Profile: Karen Settings History Progress Save Changes Undo Now editing Icon- Context Association: “Nervous emotions based on heart rate and electrodermal...” AAC+ Are you feeling NERVOUS or SCARED? “Nervous Emotions...” Speech Icons (Screen 1)BACK NOW 94b 94a I FEEL NERVOUS EXCITED SCARED FINE Active Voice OutputBACK NOW”“I FEEL NERVOUS 14 94c SPEAK 96 10 56
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 36/43 58e FIG. 36A = 8:22 AM “MILLER!” (voiceprint) Record audio for “MILLER!”: PLAYSTOP SaveUndo RECREW 62 Currentaudio: MATCH! Opening“BasicConv…” “MILLER!”: 8:22AM AAC+ FIG. 36B FIG. 36C “Basic Conversation near Mrs. Miller” Speech IconsBACK MILLERHI MRS. 14 Repetition Exercise!BACK MILLERHI MRS. JIMMY - NOW YOU SAY: Repetition Exercise!BACK MILLERHI MRS. Listening... 98a 98b 98c
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 37/43 58e FIG. 36D FIG. 36E FIG. 36F Repetition Exercise!BACK MILLERHELLO MRS. 14 Repetition Exercise!BACK JIMMY - NOW YOU SAY: Repetition Exercise!BACK 98i 98j 98k Repetition Exercise!BACK MILLERHI MRS. Listening... 98d Repetition Exercise!BACK MILLERHI MRS. Listening... 98e Repetition Exercise!BACK MILLERHI MRS. Repetition Complete! 98f Repetition Exercise!BACK Congratulations! 98g 14 MILLERHI MRS. You earned 10 points toward your goal. Goal: Trip to moviesBACK 98h 90% Your goal: Progress: TRIP TO MOVIES THERE MILLERHELLO MRS.THERE Congratulations! You earned: MILLERHELLO MRS.THERE 100% TRIP TO MOVIES
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 38/43 LOADED AS CURRENT PROFILE?: YES FIG. 37 I-001 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH HOT HOT.TXT NONE DEFAULT 0 OFF HOT.WAV I-002 FOOD FOOD.GIF NONE MEDIUM 0 OFF (TTS) I-003 LOST LOST.MOV C-001 LARGE 0 OFF LOST.MOV NONE NONE “EMERGENCY” I-004 REFRIGERATOR FRIDGE.JPG C-003 DEFAULT 1 OFF FRIDGE.MP3 I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC. “IN THE KITCHEN” ETC. ICON ID ICON LABEL ICON FILE ASSOCIATED CONTEXT ID ICON SIZE REPETITIONS PARTICPATION ASSOCIATED VOICE ASSOCIATED CONTEXT LABEL SPEECH ICON DATABASE
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 39/43 LOADED AS CURRENT PROFILE?: YES FIG. 38 C-001 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH EMERGENCY I-003, I-138, I-284, I-019... LARGE DEFAULT TO PROFILE BODY TEMP > 100* F PRESSURE SENSOR DETECTS... BRIGHTNESS “VERY HIGH” OR “VERY L…” C-002 BULLY NEARBY I-675, I-029, I-444, I-572 LARGE DEFAULT TO PROFILE MT-ID-001 WITHIN 10 METERS LOCATION IS “ELM SCHOOL” ETC. C-003 IN THE KITCHEN I-002, I-004, I-887,I-373... DEFAULT TO ICON DEFAULT TO PROFILE WORD “KITCHEN” IS UTTERED CAMERA DETECTS... ETC. 0 DEFAULT TO ICON DEFAULT TO ICON C-004 NERVOUS EMOTIONS BA... I-432, I-071, I-273, I884... DEFAULT TO PROFILE ON HEART RATE > 130 BPM GSR DATA INDICATES... ETC. I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC. DEFAULT TO ICON ETC. CONTEXT ID CONTEXT LABEL ASSOCIATED ICON ID(S) ICON SIZE PARTICPATION TRIGGER 1 TRIGGER 2 TRIGGER NREPETITIONS SAVED CONTEXT DATABASE
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 40/43 FIG. 39 WEDNESDAY FEBRUARY 1 3:13 PM PLAYDATE WITH SALLY 30- 40 DB “HELP,” CAR HORN, ... JIMMY, SALLY MCNAMEE PARK DATE TIME EVENT/ SCHEDULE AMBIENT VOLUME LAST 5 SOUNDS PEOPLE PRESENT LOCATION SLIDE MEDIUM-HIGH 77* F 99* F 82 MG/DL NORMAL NORMAL DAYLIGHT OBJECTS PRESENT MOTION ACTIVITY OUTSIDE TEMPERATURE BODY TEMPERATURE BLOOD GLUCOSE LEVEL BLOOD PRESSURE BRIGHTNESS LOADED AS CURRENT PROFILE?: YES PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH CURRENT CONTEXT DATABASE
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 41/43 LOADED AS CURRENT PROFILE?: YES I-039 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH SHARE ACTIONS, SOCIAL WEDNESDAY, 2/1, 1:13 PM VOICEPRINT OF SALLY SLIDE, BALL, SALLY, ... NONE 97* I-447 BULLY PEOPLE, SCHOOL, ... THURSDAY, 2/9, 12:17 PM 60 DB+ (UNINTEL...) UNKNOWN MT-ID-001 97* I-602 NONE DRINKS SUNDAY, 2/12, 11:15 AM (NONE) SUN NONE 100* PLAYGROUND SCHOOL BACKYARD ICON ID ICON LABEL ICON CATEGORY/ CATEGORIES DATE / TIME DETECTED BY MICROPHONE DETECTED BY CAMERA OTHER MOBILE TERMINALS NEARBY BODY TEMPERATURE LOCATION USAGE HISTORY DATABASE FIG. 40 I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC.ETC.
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 42/43 LOADED AS CURRENT PROFILE?: YES I-039 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH SHARE (21) ACTIONS (61), SOCIAL (17) WEDNESDAY (11) PLAYGROUND (13) SALLY (10) BALL (10) MOM’S PHONE (4) TUESDAY (5) HOME (6) “BALL” (5) OTHER (11) NONE (17) SATURDAY (5) OTHER (2) OTHER (6) 12:00 – 1:00 PM (8) 3:00 – 4:00 PM (7) 2:00 – 3:00 PM (6) ICON ID ICON LABEL ICON CATEGORY/ CATEGORIES DATE LOCATION DETECTED BY MICROPHONE DETECTED BY CAMERA OTHER MOBILE TERMINALS NEARBY TIME USAGE TRENDS DATABASE (INCIDENCE IN PARENTHESIS) FIG. 41
Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 43/43 LOADED AS CURRENT PROFILE?: YES FIG. 42 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH LANGUAGE DEVELOPMENT SURVEY QUESTION 1 YES LANGUAGE DEVELOPMENT SURVEY QUESTION 2 YES GENERAL VOCABULARY LEVEL 40TH PERCENTILE KNOWN VOCABULARY: COLORS BLUE, YELLOW KNOWN VOCABULARY: PEOPLE ME, DAD, MOM, SARAH... KNOWN VOCABULARY: TRANSPORT BIKE, CAR, TRAIN... DYSLEXIA NO LANGUAGE DEVELOPMENT SURVEY QUESTION 3 NO LARYNX TRAUMA NO LANGUAGE REGRESSION TRIGGER 1 LOUD AREAS LANGUAGE REGRESSION TRIGGER 2 NEW PEOPLE STUTTERING YES (MODERATE) STUTTERING PROBLEM WORD 1 BATHROOM STUTTERING PROBLEM WORD 2 BARKING STUTTERING PROBLEM WORD 3 CHICKEN STUTTERING PROBLEM WORD 4 CHURCH USER IMPAIRMENT DATABASE

Recommended

d&b audiotechnik Electroacoustics Seminar
d&b audiotechnik Electroacoustics Seminard&b audiotechnik Electroacoustics Seminar
d&b audiotechnik Electroacoustics Seminar
Jonathon Duncan
 
SaintyCo-Tablet&Capsule Counting Line
SaintyCo-Tablet&Capsule Counting LineSaintyCo-Tablet&Capsule Counting Line
SaintyCo-Tablet&Capsule Counting Line
Tony Zeng
 
Teoría 02
Teoría 02Teoría 02
Teoría 02
bdefig
 
DNA Methylation: An Essential Element in Epigenetics Facts and Technologies
DNA Methylation: An Essential Element in Epigenetics Facts and TechnologiesDNA Methylation: An Essential Element in Epigenetics Facts and Technologies
DNA Methylation: An Essential Element in Epigenetics Facts and Technologies
QIAGEN
 
África
ÁfricaÁfrica
África
Arpon Files
 
Daniels_MASc_thesis
Daniels_MASc_thesisDaniels_MASc_thesis
Daniels_MASc_thesis
Daniel Bondarenko
 
Science House
Science HouseScience House
Science House
robtedesco
 
Engineering Art House
Engineering Art HouseEngineering Art House
Engineering Art House
robtedesco
 

Recommended

d&b audiotechnik Electroacoustics Seminar
d&b audiotechnik Electroacoustics Seminard&b audiotechnik Electroacoustics Seminar
d&b audiotechnik Electroacoustics Seminar
Jonathon Duncan
 
SaintyCo-Tablet&Capsule Counting Line
SaintyCo-Tablet&Capsule Counting LineSaintyCo-Tablet&Capsule Counting Line
SaintyCo-Tablet&Capsule Counting Line
Tony Zeng
 
Teoría 02
Teoría 02Teoría 02
Teoría 02
bdefig
 
DNA Methylation: An Essential Element in Epigenetics Facts and Technologies
DNA Methylation: An Essential Element in Epigenetics Facts and TechnologiesDNA Methylation: An Essential Element in Epigenetics Facts and Technologies
DNA Methylation: An Essential Element in Epigenetics Facts and Technologies
QIAGEN
 
África
ÁfricaÁfrica
África
Arpon Files
 
Daniels_MASc_thesis
Daniels_MASc_thesisDaniels_MASc_thesis
Daniels_MASc_thesis
Daniel Bondarenko
 
Science House
Science HouseScience House
Science House
robtedesco
 
Engineering Art House
Engineering Art HouseEngineering Art House
Engineering Art House
robtedesco
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
Christy Abraham Joy
 
Time Management & Productivity - Best Practices
Time Management & Productivity - Best PracticesTime Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
Vit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
MindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
RachelPearson36
 

More Related Content

Featured

How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 

Featured (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity - Best PracticesTime Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Drawings for Alternative Communications Patent

  • 1. 1 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH 22 FIG. 1 AT&T 3G 8:22 AM Text Calendar Photos Camera Stocks Maps WeatherRecorder Calculator Notes SettingsClock AAC+ Mail Safari iPodPhone 3 SMS 11 Thursday 73* + - x / 10 14 16 20 18 12 AAC+
  • 2. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 2/43 FIG. 2 PRIOR ART MICROPHONE SPEAKER KEYBOARDDISPLAYCAMERA USER INTERFACE 10 24 16 26 14 18 12 CPU 28 ACCELEROMETER 34 MEMORY 30 SOFTWARE TRANSCEIVER CIRCUITRY 36 32 WIRED PORT 40 BATTERY 42 MISCELLANEOUS PERIPHERALS 44 38
  • 3. 3 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 3 PRIOR ART PSTN INTERNET PLMN 46 50 52 10D 56 10C 10B 10A 48 54
  • 4. 4 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH SMS 11 73* RH 10 58e 58d 58b FIG. 4 58c 58a 62 60 58f 64
  • 5. 5 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 5 CAREGIVER ACQUIRES MOBILE TERMINAL SOFTWARE LOADED ONTO MOBILE TERMINAL ESTABLISH A USER PROFILE STORE A CONTEXTUAL CRITERION FOR OUTPUTTING A SPEECH ICON OUTPUT THE SPEECH ICON 100 102 104 106 110 108 IS CRITERION SATISFIED? YES NO
  • 6. 6 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH AT&T 3G 8:22 AM Profiles AAC+BACK AAC+ 10 16 18 General Settings Profiles and Settings FIG. 6 AAC+ TO FIG. 8 “Caregiver Mode” Enabled (tap to change to “End User Mode”)
  • 7. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 7/43 FIG. 7 USER SELECTS PROFILES COMMAND USER SELECTS PRE-EXISTING PROFILE OR TEMPLATE 150 152 USER CREATES NEW PROFILE 158 LOAD 154 GO TO BLOCK 108 OF FIGURE 5 EDIT 156 GIVE NAME TO PROFILE 160 GO TO BLOCK 104 OF FIGURE 5
  • 8. 8 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 8 AT&T 3G 8:22 AM AAC+ Profiles BACK Custom profiles Templates Nonverbal adolescent with autism Help with expressing feelings Adult user with semantic-pragmatic... Jimmy Smith Laura’s normal settings Settings for Laura’s vacation TO FIG. 10 TO FIG. 9 200 202 204
  • 9. 9 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 9 AT&T 3G 8:22 AM AAC+ Profile: Jimmy Smith BACK 208 220 218 EDIT Icon size: Large Default voice: Young Male Assoc. #1: “McD Menu” at McDonalds LOAD 210 212 206 Assoc. #2: “Help group” after “Help!” 214 Assoc. #3: “Basic conversation” near Mrs. Miller 216
  • 10. 10 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 10 AT&T 3G 8:22 AM AAC+ Create / Edit a Profile BACK Profile Name: Enter name here Speech/Language Impairment Info General Settings for This Profile Icons SAVE PROFILE To Fig. 11 To Fig. 15 To Fig. 30 User’s Schedule/Calendar Icon-Context Associations To Fig. 18 Contexts To Fig. 21
  • 11. 11 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 11 = AT&T 3G 8:22 AM AAC+BACK How would you like to provide speech/ language impairment information? Take a survey Enter specific impairment info Perform a user evaluation TO FIG. 12 TO FIG. 13 TO FIG. 14 Import data from another source
  • 12. 12 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 12 = AT&T 3G 8:22 AM Language Development SurveyBACK In response to the following questions, please mark “Yes” or “No”. Does the user know the correct names of some colors? YES NO Does the user say about 50 different words? YES NO Does the user say about 100 different words? YES NO Does the user say words that tell something about other words, such as “loud” in the utterance a “loud noise”?
  • 13. 13 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 13 = AT&T 3G 8:22 AM Enter Impairment InformationBACK Dyslexia Does user have dyslexia? Vocabulary General vocabulary level Known vocabulary words Stuttering and Annunciation Below Avg Avg Above Avg NO Does user stutter? YES Known annunciatory problem words: Type here (e.g., “bathroom”) CATEGORIES Colors Aqua Blue Brown Burgundy Crimson Forest Green Gold Green Lavender
  • 14. 14 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 14 = 3G 8:22 AM Say: “bathroom”BACK Say the word “bathroom”: PLAYSTOP SaveUndo RECREW 66 66a 14 Say, “bath-room” please.
  • 15. 15 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 15 = AT&T 3G 8:22 AM AAC+BACK Which general settings would you like to program for this profile? Caption font Caption size Icon style Icon size Language Voices Repetitions Participatory voice output Icons per screen TO FIG. 16 TO FIG. 17 Select a default voice: Young male (synthetic) Adult female (synthetic) Rosa Voice 124153 Jimmy Smith’s voice Save 68 Volume
  • 16. 16 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 16 = AT&T 3G 8:22 AM AAC+BACK Configure default “Repetitions” settings that should apply to this profile: Number of repetitions Accuracy threshold Penalties Rewards Prompt users to repeat voice outputs Number of required repetitions: Save ON 0 1 2 3 4 How accurate must the repetition be? Save 30% 40% 50% 60% 70% Configure “Repetitions” for a particular speech icon or group of icons: Browse categories Search Penalties Save Enforce a penalty if user fails to repeat Disable all voice output OFF Disable a speech icon Restrict game/Web access Rewards Save Give a reward if user completes repetition Tokens and points ON Messages Enable game/Web access TO FIG. 30
  • 17. 17 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 17 = AT&T 3G 8:22 AM AAC+BACK Configure default “Participatory voice output” settings that should apply to this profile: Frequency of participation Accuracy threshold Penalties Rewards Prompt users to participate in voice outputs Frequency of participation: Save ON How accurate must the participation be? Save 30% 40% 50% 60% 70% TO FIG. 30 Sometimes Word or phrase length Word or phrase type Always Browse categories Search Specific words, phrases, etc: Type of participation Save User finishes Step by step Increase MLU over time User begins Type of participation:
  • 18. 18 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 18 AT&T 3G 8:22 AM Icon-Context AssociationsBACK + “BK Menu” at 32 Main Street Burger King “Help group” after “Help!” Rule 1 (11/15/10 4:57 pm) “Basic conversation” near Mrs. Miller Rule 2 (11/23/10 6:18 am) “Recess icons” on Mon-Fri from 12:15 to... “Nervous emotions” based on heart rate and… TO FIG. 19 “Swing set” at McNamee Park
  • 19. 19 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 19 = AT&T 3G 8:22 AM BACK Create TO FIG. 20 WQ E R T Y U I O P SA D F G H J K L XZ C V B N M space return.?123 Name your Icon-Context Association: Icon-Context Assoc.
  • 20. 20 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 20 = AT&T 3G 8:22 AM Icon-Context AssociationsBACK How would you like to create an association between a speech icon and a particular context? Start with a context, then select icon(s) that should be presented in that context Start with an icon, then select context(s) in which that icon should be presented TO FIG. 21 TO FIG. 30
  • 21. 21 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 21 = AT&T 3G 8:22 AM ContextsBACK Sounds, noises or speech Physical movement Objects or people near this device TO FIG. 22 Temperature Lighting The location of this device The user’s body Times, dates and scheduled events + TO FIG. 29 Search
  • 22. 22 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 22 = AT&T 3G 8:22 AM Icon-Context AssociationsBACK Specify a location that should trigger output of a speech icon: Search/browse by location name Find a location on a map Draw a “zone” on a map How would you like to identify a location? 70 TO FIG. 23 Browse categories Search
  • 23. 23 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 23 AT&T 3G 8:22 AM Results for “McDonalds”BACK 1. 5065 Main Street | Anycity, USA | 2.84 mi 2. 390 Route 105 | Anycity, USA | 4.31 mi 3. 12 Drawley Lane | Nextown, USA | 5.39 mi 4. 101 Elm Street | Nextown, USA | 7.01 mi 5. 65 Far Drive | Farville, USA | 12.4 mi Show More Results TO FIG. 24View Selected
  • 24. 24 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 24 AT&T 3G 8:22 AM View/edit context (McDonald…)BACK McDonald’s 5065 Main Street | Anycity, USA General Information TO FIG. 25 ATM WIFI View/Edit Icons and Settings for “McDonald…” Add New Speech Icon(s) Browse categories Search More info TO FIG. 27No icons have been saved in association with this location.
  • 25. 25 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 25 AT&T 3G 8:22 AM Icon results for “fries”BACK FRENCH FRIES (1) Saved to 13 contexts Frequency of use: High Add Selected Icons to Context FRENCH FRIES (2) Saved to 4 contexts Frequency of use: Medium FRIED DOUGH Saved to 0 contexts Frequency of use: Low More info More info More info
  • 26. 26 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 26 AT&T 3G 8:22 AM Add “FRENCH FRIES (1) to…”BACK FRENCH FRIES (1) Add speech icon: FRENCH FRIES (1) to “McDonald’s 5065 Main St.”? TO FIG. 27 McDonald’s 5065 Main Street Save Icon to This Location
  • 27. 27 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 27 AT&T 3G 8:22 AM Icons and settings for “McD…”BACK BURGER FISH SANDWICH FRENCH FRIES Caption font Caption size Icon style Icon size Language Voices Repetitions Participatory voice output Icons per screen Icons (6): Settings for this context: Icon ranking Screen Editor TO FIG. 28 Volume
  • 28. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 28/43 Screen Editor for “McDonald’s 5056 Main Street”BACK FIG. 28 Screen Editor for “McDonald’s 5056 Main Street”BACK BURGER FISH SANDWICH FRENCH FRIES ICE CREAM PLEASECOLA 72 74 76 78 80 82 SAVE SAVE
  • 29. 29 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 29 AT&T 3G 8:22 AM Create a new locationBACK Name this location: Drag and drop pins to define boundaries: SaveUndo Enter name here 84
  • 30. 30 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 30 AT&T 3G 8:22 AM IconsBACK + actions animals around the house art body parts clothing colors Search TO FIG. 31
  • 31. 31 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 31 AT&T 3G 8:22 AM Icon results for “burger”BACK BURGER Saved to 9 contexts Frequency of use: High TO FIG. 32 BURGER WITH BACON Saved to 2 contexts Frequency of use: Low HAM BURGER Saved to 0 contexts Frequency of use: Low
  • 32. 32 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 32 AT&T 3G 8:22 AM Icons: BURGERBACK BURGER Saved to 9 contexts Frequency of use: High Related icons Caption font (Arial) Caption size (Normal) Icon style (Photo) Icon size (Medium) Language (English) Voices (Young male) Repetitions (OFF) Participatory voice output (OFF) Settings for this icon: Volume (Default) TO FIG. 21Add “BURGER” to a new context
  • 33. 33 / 43 Inventor(s): TEDESCO et al. Title: METHODS FOR AUGMENTATIVE AND ALTERNATIVE COMMUNICATIONS INFORMED BY ENVIRONMENTAL, BEHAVIORAL, AND BIOLOGICAL CONTEXTS Docket No.: 2001-011.0001 Filing Date: FILED HEREWITH FIG. 33 AT&T 3G 8:22 AM View/edit context (Low blood…)BACK Low blood sugar (beneath 70 mg/dL for more than 15 minutes) General Information View/Edit Icons and Settings for “Low blood…” Add New Speech Icon(s) Browse categories Search More info BURGER 70
  • 34. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 34/43 FIG. 34A = 8:22 AM Create a new location Name this location: SaveUndo FIG. 34B FIG. 34C 10 Drag and drop pins to define boundaries: “Maiden Ridge” Neighborhood STEPHEN “Maiden Ridge” Speech Icons (Screen 1)BACK SALLY GO JOE PETER PASS TURN “Maiden Ridge” Speech Icons (Screen 2)BACK “Maiden Ridge” Speech Icons (Screen 3)BACK ME WANT I MRS. SMITH HOME PLEASE SKATEBOARD RAMP BASKETBALL FOOTBALL COOL FUN COOL FUN 86a 86b 86c 88a 88b 90 88c 92 16
  • 35. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 35/43 FIG. 35A Set maximum levels for: FIG. 35B FIG. 35C 130 BPM Heart Rate Med-High Electrodermal Activity HeartRate(BPM): Levels Exceeded! Transmitting message... GSR: 8:22 AM AAC+ 10 58b 58d -5000 0 5000 10000 15000 = 8:22 AM 11 7 3 * MedLow Med-Low 54 AAC+ -- Caregiver Workstation Profile: Karen Settings History Progress Save Changes Undo Now editing Icon- Context Association: “Nervous emotions based on heart rate and electrodermal...” AAC+ Are you feeling NERVOUS or SCARED? “Nervous Emotions...” Speech Icons (Screen 1)BACK NOW 94b 94a I FEEL NERVOUS EXCITED SCARED FINE Active Voice OutputBACK NOW”“I FEEL NERVOUS 14 94c SPEAK 96 10 56
  • 36. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 36/43 58e FIG. 36A = 8:22 AM “MILLER!” (voiceprint) Record audio for “MILLER!”: PLAYSTOP SaveUndo RECREW 62 Currentaudio: MATCH! Opening“BasicConv…” “MILLER!”: 8:22AM AAC+ FIG. 36B FIG. 36C “Basic Conversation near Mrs. Miller” Speech IconsBACK MILLERHI MRS. 14 Repetition Exercise!BACK MILLERHI MRS. JIMMY - NOW YOU SAY: Repetition Exercise!BACK MILLERHI MRS. Listening... 98a 98b 98c
  • 37. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 37/43 58e FIG. 36D FIG. 36E FIG. 36F Repetition Exercise!BACK MILLERHELLO MRS. 14 Repetition Exercise!BACK JIMMY - NOW YOU SAY: Repetition Exercise!BACK 98i 98j 98k Repetition Exercise!BACK MILLERHI MRS. Listening... 98d Repetition Exercise!BACK MILLERHI MRS. Listening... 98e Repetition Exercise!BACK MILLERHI MRS. Repetition Complete! 98f Repetition Exercise!BACK Congratulations! 98g 14 MILLERHI MRS. You earned 10 points toward your goal. Goal: Trip to moviesBACK 98h 90% Your goal: Progress: TRIP TO MOVIES THERE MILLERHELLO MRS.THERE Congratulations! You earned: MILLERHELLO MRS.THERE 100% TRIP TO MOVIES
  • 38. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 38/43 LOADED AS CURRENT PROFILE?: YES FIG. 37 I-001 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH HOT HOT.TXT NONE DEFAULT 0 OFF HOT.WAV I-002 FOOD FOOD.GIF NONE MEDIUM 0 OFF (TTS) I-003 LOST LOST.MOV C-001 LARGE 0 OFF LOST.MOV NONE NONE “EMERGENCY” I-004 REFRIGERATOR FRIDGE.JPG C-003 DEFAULT 1 OFF FRIDGE.MP3 I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC. “IN THE KITCHEN” ETC. ICON ID ICON LABEL ICON FILE ASSOCIATED CONTEXT ID ICON SIZE REPETITIONS PARTICPATION ASSOCIATED VOICE ASSOCIATED CONTEXT LABEL SPEECH ICON DATABASE
  • 39. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 39/43 LOADED AS CURRENT PROFILE?: YES FIG. 38 C-001 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH EMERGENCY I-003, I-138, I-284, I-019... LARGE DEFAULT TO PROFILE BODY TEMP > 100* F PRESSURE SENSOR DETECTS... BRIGHTNESS “VERY HIGH” OR “VERY L…” C-002 BULLY NEARBY I-675, I-029, I-444, I-572 LARGE DEFAULT TO PROFILE MT-ID-001 WITHIN 10 METERS LOCATION IS “ELM SCHOOL” ETC. C-003 IN THE KITCHEN I-002, I-004, I-887,I-373... DEFAULT TO ICON DEFAULT TO PROFILE WORD “KITCHEN” IS UTTERED CAMERA DETECTS... ETC. 0 DEFAULT TO ICON DEFAULT TO ICON C-004 NERVOUS EMOTIONS BA... I-432, I-071, I-273, I884... DEFAULT TO PROFILE ON HEART RATE > 130 BPM GSR DATA INDICATES... ETC. I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC. DEFAULT TO ICON ETC. CONTEXT ID CONTEXT LABEL ASSOCIATED ICON ID(S) ICON SIZE PARTICPATION TRIGGER 1 TRIGGER 2 TRIGGER NREPETITIONS SAVED CONTEXT DATABASE
  • 40. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 40/43 FIG. 39 WEDNESDAY FEBRUARY 1 3:13 PM PLAYDATE WITH SALLY 30- 40 DB “HELP,” CAR HORN, ... JIMMY, SALLY MCNAMEE PARK DATE TIME EVENT/ SCHEDULE AMBIENT VOLUME LAST 5 SOUNDS PEOPLE PRESENT LOCATION SLIDE MEDIUM-HIGH 77* F 99* F 82 MG/DL NORMAL NORMAL DAYLIGHT OBJECTS PRESENT MOTION ACTIVITY OUTSIDE TEMPERATURE BODY TEMPERATURE BLOOD GLUCOSE LEVEL BLOOD PRESSURE BRIGHTNESS LOADED AS CURRENT PROFILE?: YES PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH CURRENT CONTEXT DATABASE
  • 41. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 41/43 LOADED AS CURRENT PROFILE?: YES I-039 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH SHARE ACTIONS, SOCIAL WEDNESDAY, 2/1, 1:13 PM VOICEPRINT OF SALLY SLIDE, BALL, SALLY, ... NONE 97* I-447 BULLY PEOPLE, SCHOOL, ... THURSDAY, 2/9, 12:17 PM 60 DB+ (UNINTEL...) UNKNOWN MT-ID-001 97* I-602 NONE DRINKS SUNDAY, 2/12, 11:15 AM (NONE) SUN NONE 100* PLAYGROUND SCHOOL BACKYARD ICON ID ICON LABEL ICON CATEGORY/ CATEGORIES DATE / TIME DETECTED BY MICROPHONE DETECTED BY CAMERA OTHER MOBILE TERMINALS NEARBY BODY TEMPERATURE LOCATION USAGE HISTORY DATABASE FIG. 40 I-00N ETC. ETC. ETC. ETC. ETC. ETC. ETC.ETC.
  • 42. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 42/43 LOADED AS CURRENT PROFILE?: YES I-039 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH SHARE (21) ACTIONS (61), SOCIAL (17) WEDNESDAY (11) PLAYGROUND (13) SALLY (10) BALL (10) MOM’S PHONE (4) TUESDAY (5) HOME (6) “BALL” (5) OTHER (11) NONE (17) SATURDAY (5) OTHER (2) OTHER (6) 12:00 – 1:00 PM (8) 3:00 – 4:00 PM (7) 2:00 – 3:00 PM (6) ICON ID ICON LABEL ICON CATEGORY/ CATEGORIES DATE LOCATION DETECTED BY MICROPHONE DETECTED BY CAMERA OTHER MOBILE TERMINALS NEARBY TIME USAGE TRENDS DATABASE (INCIDENCE IN PARENTHESIS) FIG. 41
  • 43. Inventor(s):TEDESCOetal. Title:METHODSFORAUGMENTATIVEANDALTERNATIVECOMMUNICATIONSINFORMEDBYENVIRONMENTAL, BEHAVIORAL,ANDBIOLOGICALCONTEXTS DocketNo.:2001-011.0001 FilingDate:FILEDHEREWITH 43/43 LOADED AS CURRENT PROFILE?: YES FIG. 42 PROFILE ID: P-0001 PROFILE NAME: JIMMY SMITH LANGUAGE DEVELOPMENT SURVEY QUESTION 1 YES LANGUAGE DEVELOPMENT SURVEY QUESTION 2 YES GENERAL VOCABULARY LEVEL 40TH PERCENTILE KNOWN VOCABULARY: COLORS BLUE, YELLOW KNOWN VOCABULARY: PEOPLE ME, DAD, MOM, SARAH... KNOWN VOCABULARY: TRANSPORT BIKE, CAR, TRAIN... DYSLEXIA NO LANGUAGE DEVELOPMENT SURVEY QUESTION 3 NO LARYNX TRAUMA NO LANGUAGE REGRESSION TRIGGER 1 LOUD AREAS LANGUAGE REGRESSION TRIGGER 2 NEW PEOPLE STUTTERING YES (MODERATE) STUTTERING PROBLEM WORD 1 BATHROOM STUTTERING PROBLEM WORD 2 BARKING STUTTERING PROBLEM WORD 3 CHICKEN STUTTERING PROBLEM WORD 4 CHURCH USER IMPAIRMENT DATABASE