Proposal for A New Ideation Process
Harvard University, Independent Design Engineering Project
Togo Kida
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
2
Problem Domain
Creative teams working in
advertising industries.
How can I design something
that would solve their
problem in their job?
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
3
Why Creative Matters: Consulting Firms Buying Creative Agencies
Oct. 2019
Aug. 2019
Apr. 2019
Apr. 2019
Mar. 2019
Mar. 2019
Nov. 2018
Oct. 2019
Jul. 2017
Jul. 2017
Oct. 2016
May. 2015
Nov. 2016
Source: Press Release Information
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
4
Preliminary Research: Interviewing Stakeholders
T. K.
Dentsu Inc.
Creative Strategist
A. S.
Bytedance Japan
Creative Director
R. N.
Dentsu Inc.
Creative Director
A. S
Dentsu Inc.
Copywriter, Strategist
Y. K.
Dentsu Inc.
Creative Strategist
K. W.
Dentsu Inc.
Copywriter, Planner
Y. K.
Dentsu Inc.
Creative Planner
K. Y.
Dentsu Inc.
Creative Technologist
Advertising Agency Broader Creative (non-advertising) Domain Experts
M. S.
Dentsu Inc.
Creative Technologist
O.K.
Dentsu Inc.
Creative Director
Y. N.
Dentsu Creative X Inc.
Creative Techologist
N. W.
Dentsu Inc.
Comm. Planner
I. C.
Cash Studios
Creative Director
S. H.
Certain Measures
Computational Designer
P. R.
Pablo Rochat Studio
Creative / Art Director
R. C
Ovo Energy
Senior UX Designer
M. K.
Fahrenheit 212
Strategist
T. A.
Harvard BS
Professor
L. G.
Uber
Executive Cr. Director
R. K.
Drive Inc.
Art Director
S. O.
Harvard GSD
Researcher
M. W.
Normal Inc.
Creative Coder
M. C.
Deep Mind
Senior Visual Designer
S. A.
Harvard GSD
Researcher
Domain
Interviewee
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
5
Preliminary Research: Stakeholder Analysis
Creative Director
Art Director Copywriter/

TV Spot Planner
Creative
Strategist
Creative
Technologist
PR Planner Media Planner
Programmer /
Developper
Designer
Media
Representative
Account
Executives
Client
Client Domain Agency Domain Partners Domain
Client
Other Dept.
Client
Other Dept.
Client
Other Dept.
Client
Other Dept.
Videographer
Other partners
based on projects
…
=
communication
and/or
interaction
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
6
Preliminary Research: Work Flow Analysis
Briefing Ideation Pitch Building Pitch Revision Decision Execution Reporting
Planning Phase Pitching Phase Execution Phase
• Receive orientation /
brief from account team
• Understand what needs
to be pitched.
• Conduct necessary
research
• Study precedents
• Come up with ideas
individually
• Meet as a team to
discuss ideas.
• Conduct the above
multiple times to
converge on the basic
storyline for the pitch.
• Breakdown what needs
to be in the pitch deck,
and assign task
members to work on the
component.
• Report on progress,
converge as a team if
necessary.
• Talk with account
executive team members
to align on overall
direction
• Conduct the above
multiple times if
necessary.
Process
Detail
Players
• Present the idea to the
client.
• Seek feedback from the
client and make
revisions if necessary.
• Present again.
• Agree with the client
what idea is going to be
executed.
• Agree upon how to
measure success.
• Work with partnering
companies, entities to
execute the idea.
• Upon completion of the
project, return to the
client to report the result
of the campaign.
• Share key learnings
from the project with the
client.
Creative
Director
Account
Executives
Creative
Director
Art Director
Copywriter/
Planner
Strategist
Creative
Technologist
PR Planner
Media Planner
Creative Director
Client
Account
Executive
Creative Director
Client
Account
Executive
Creative Director
Client
Account
Executive
All Creative Team
All Creative Team
Client
Partners
Responsi-
bility
Creative
Director
Account
Executive
Duration 7 - 10 days 1day 7 - 10 days 7 - 10 days 90 - 180days 14 - 30days7 - 30 days 7 - 30 days
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
7
Preliminary Research: Ideation Phase Analysis
Briefing Ideation Pitch Building Pitch
Planning Phase Pitching Phase
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Creative
Director
Round 1 Round 2 Round 3
…
= 1 idea
Ideation Mode Individual Group Individual Group Individual Group
Diverge Converge Diverge Converge Diverge ConvergeIdeation Phase
Ideation Round
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
“
8
Preliminary Research: Expressed Pain Point
An interviewee in the advertising agency
*multiple other agency interviewees responded similarly
In certain circumstances, I need 

to formulate 20 - 30 ideas weekly.
My job would be so much easier, 

if there is something that could
help me.
”
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
9
Ideation Among the Creatives: Text-based (Digital / Non-digital Format)
Digital Non-Digital
An example of
ideating strategy
An example of
ideating tag lines
& TVC scripts
An example of
ideating campaign
An example of
ideating campaign
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
10
Insight and Problem Statement:
Insight
Problem Statement
Creative teams working in advertising industries 

are under strong time pressure to generate many
creative output in short time.
The teams lack the right tool to ideate more, 

better quality creative output.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
11
Solution Structure: Before
Staff in Creative
Agency
Creative
Creative Writing
Output
Solitary Process
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
12
Solution Structure: After
Staff in Creative
Agency
Application
+
Machine Learning
Model Backend
Text Input
Suggestion
Creative
Creative Writing
Output
Collaborative Process
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
13
Design Principle
• More output (quantity)
• In agency context
• Fun to use
A tool that enables the user to ideate more ideas through its usage.
A tool that that fits in the workflow of advertising agency staffs.
A tool that is genuinely fun to interact with to foster ideation.
*quantity eventually becomes quality
*
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
15
Solution Design: AI-powered Text Editor “Phantom”
Phantom is a text editor powered
by an AI backend.
There is a text editing section 

which the user can enter text,
pieces of storyline.
Phantom can suggest additional
text which would be source of
inspiration for additional ideas.
Powered by:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
16
Powered by:
The AI is created with an LSTM-RNN. Phantom is equipped with three different LSTM models by
default. These models are capable of generating sequence of text by giving a seed text.
Neural Network Architecture of LSTM-RNN
Models
Model 1
Model 2
Model 3{ }
Solution Design: AI-powered Text Editor “Phantom”
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
17
Powered by:
AI
Models
Model 1
Model 2
Model 3{ }“Love is…”
Generated Text
“an infinite amount of deepness…”
“something unknown…”
“is the toughest thing in this world”
Seed Text
Input Output
Solution Design: AI-powered Text Editor “Phantom”
The AI is created with an LSTM-RNN. Phantom is equipped with three different LSTM models by
default. These models are capable of generating sequence of text by giving a seed text.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
18
Solution Design: AI-powered Text Editor “Phantom”
Main Text
Entry Area
Generate Button
Suggestion Text Area
Model
Selection
Model
Settings
Powered by:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
19
Solution Design: AI-powered Text Editor “Phantom”
Powered by:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
19
The user can easily
specify which models
to use.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
20
Powered by:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
20
Independent Design Engineering Project
Harvard University Master of Design Engineering
20
“Proposal for A New Ideation Process”
Togo Kida
Solution Design: AI-powered Text Editor “Phantom”
The user can also
specify the parameters
of the LSTM-RNN
models.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
21
Solution Design: AI-powered Text Editor “Phantom”
Step 1: Write, and highlight.
Step 2: Hit generate.
Step 3: See suggestion text.
Powered by:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
22
Solution Design: AI-powered Text Editor “Phantom”
Powered by:
The user will start writing in the
main text area but the user can
also benefit from leveraging the
AI models to complete the
writing task.
Proposed Ideation Process with Phantom:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
23
Underlying Research Question
Staff in Creative
Agency
Application
+
Machine Learning
Model Backend
Text Input
Suggestion
Creative
Would this collaborative model
between a creative agent and the
user be effective?
“
”
Collaborative Model
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
24
Experiment Structure & Measure of Success
A Prompt
A Prompt
Participants Input Intervention Output
15 min. Experiment
Creative Staff
Group 1
Creative Staff

Group 2
No Tool
With AI Agent Writing
Writing
Evaluation
Quantitative Qualitative
Analytics upon using the tool. Comments from the participants.
Outcome 

Evaluation
Conducted series of 15 minutes online experiments with three groups and evaluate the outcomes.
A Prompt
Creative Staff

Group 3
With Dummy Agent
*random phrase Writing
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
25
Group Design Detail and it’s Difference
Group 1
Group 2
Group 3
Task Detail Duration
Writing a
poem of
COVID19
Tool Intervention
15 minutes
15 minutes
15 minutes
Tool Latency Task Category
3 AI models
trained in
JPOP lyrics
(LSTM-RNNs)
None
Dummy
System
(Picks random
JPOP lyrics
phrase)
5 - 10 sec.
N/A
0.1< sec.
Collaborative
Process
Solitary
Process
Collaborative
Process
Data Source
20MB lyrics
Text data
20MB lyrics
Text data
N/A
Tool Relevance
The tool will
try to generate
text related to
user input.
It has nothing
to do with
user input.
N/A
Writing a
poem of
COVID19
Writing a
poem of
COVID19
With AI
No AI
With Dummy
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
26
AI Dataset Design: Why JPOP Lyrics?
I’ve built other models with different dataset, but considering the prompt, I settled down to using JPOP lyrics as dataset for AI.
Dataset Used to Train AI(LSTM-RNN) Generated Output Analysis Is it Suitable for Experiment? Selected?
JPOP Lyrics
Fan Written Japanese Fantasy Novel
Contemporary Classic Japanese Lit.
Text generated from this model is
suggestive and seems wide applicability.
Text generated from this model is
too specific. A lot about magic spells
and swords and knights.
Text generated from this model is
too archaic.
Yes.
No.
No.
✅
👎
👎
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
27
Quantitative Metrics Collection
Quantitative metrics are recorded as .json file while each of the subjects interact with Phantom.
Participant .json file
Collected Metrics
• The amount of text input over time.
• What was the input text used to
generate suggestions from the AI/
dummy system.
• Parameter settings when user
asked for text suggestion.
Interaction During Experiment Analytics Data Recorded
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
28
Board of Judges
Asked user feedback after experiment. Also, had an independent group of judges to
judge the writing outputs and see if there is a noticeable difference in scores.
Creative
Writing
Score
Qualitative Metrics Collection
Participants
Score/Reflection
• Your self score on output?
• Was the generated text helpful?
• Rate the experience itself?
Post-experiment Questionnaire
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
Group Design to Answer Research Questions
Number of research questions can be answered through this experiment.
Question to be Answered
Would this collaborative model between a
creative agent and the user be effective?
Is the ML model better in some way than
simply randomly showing text as
inspiration?
How it Could be Answered?
Compare group 2 and group 1 + 3.
Rationale
Compare group 1 and group 3.
Group 2 is a solitary task.
Group 1 + 3 is a collaborative task
with the tool.
Group 1 uses AI(LSTM-RNN) and
Group 2 is a dummy system that
simply returns text randomly, not
relevant to the user input.
Is the quality of output different among
groups?
Scores from the external judges.
3 independent judges were
assigned to score the output. This
will give objective means to
evaluate the output.
29
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
30
Experiment Structure: Recruitment
Recruited 30 participants from the advertising industry to test the solution design in 3 groups.
Experiment Group
Total of
30 Participants
Group 1 = 10 Group 2 = 10 Group 3 = 10
Creative Strategist
Copywriter
Creative Tech Director
TVC Planner
Copywriter
Creative Manager
Creative Technologist
Creative Director
TVC Planner
Copywriter
Creative Strategist
Copywriter
Creative Tech Director
TVC Planner
Strategist
Creative Manager
Planner
Creative Director
TVC Planner
Copywriter
Creative Strategist
Copywriter
Executive Creative Director
TVC Planner
Copywriter
Creative Manager
Creative Technologist
Creative Director
TVC Planner
Copywriter
With AI No AI With Dummy
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
31
Experiment Result: From the Analytics
Collaborative task group (gr. 1 & 3) seem to yield more output than solitary task group (gr. 2).
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
With the assistance of AI models(group 1)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
Without any form of assistance(group 2)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
With the assistance of randomly selected phrases(group 3)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy)
Participants Participants Participants
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
32
Experiment Result: From the Analytics
While some participants are copying generated text, not all participants are doing so necessarily (simply viewing).
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
With the assistance of AI models(group 1)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
Without any form of assistance(group 2)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
0
200
400
600
0 250 500 750
Elapsed Seconds
TotalAmountofText
colour
01
02
03
04
05
06
07
08
09
10
With the assistance of randomly selected phrases(group 3)
Elapsed Seconds vs Total Amount of Text Entered
n = 10
Suggestion Copied
Suggestion Copied
Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy)
Participants Participants Participants
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
33
Experiment Result: From the Analytics
Collaborative task group (gr. 1 & 3) seem to yield more output than solitary task group (gr. 2).
745.4
626.4
825.3
0
200
400
600
800
group 1 group 2 group 3
Groups
TotalInputTime
group group 1 group 2 group 3
With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3)
Average Input Time Among Groups
301
215
371
0
100
200
300
group 1 group 2 group 3
Groups
TotalTextAmount
group group 1 group 2 group 3
With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3)
Average Text Amount Among Groups
0.4
0.34
0.45
0.0
0.1
0.2
0.3
0.4
group 1 group 2 group 3
Groups
TextInputRateperSecond
group group 1 group 2 group 3
With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3)
Average Rate of Text Input per Second
With AI No AI With Dummy With AI No AI With Dummy With AI No AI With Dummy
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
34
Experiment Result: From the Analytics
Looking at the data, some evidence to say that the more you generate, the more output quantity is achieved.
●
●
●
●
●
●
●
●
●
●
0
200
400
600
0 5 10 15 20 25
Number of Times Suggestion Generated
TotalAmountofText
With the assistance of AI models(group1)
Number of Times Suggestion Generated vs Total Amount of Text Entered
n = 10
●
●
●
●
●
●
●
●
●
0
200
400
600
0 5 10 15 20 25
Number of Times Suggestion Generated
TotalAmountofText
With the assistance of randomly selected phrases(group 3)
Number of Times Suggestion Generated vs Total Amount of Text Entered
n = 10
Group 1 (With AI) Group 3 (With Dummy)
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
35
Experiment Result: From the Analytics
Group 1 which is trying to generate on user context scored higher.
mean:mean:mean:mean:mean:mean:mean:mean:mean:
6.566.566.566.566.566.566.566.566.56
0
1
2
3
4
5
0 3 6 9
Score
Count
With AI(Group 1)
How Helpful Were the Generated Results?
mean:mean:mean:mean:mean:mean:mean:mean:mean:
4.444.444.444.444.444.444.444.444.44
0
1
2
3
4
5
0 3 6 9
Score
Count
With Dummy System(Group 3)
How Helpful Were the Generated Results?
Group 1 (With AI) Group 3 (With Dummy)
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
36
Experiment Result: From the Analytics
Very very positive result on the collaborative task model.
mean:mean:mean:mean:mean:mean:mean:mean:mean:
8.568.568.568.568.568.568.568.568.56
0
1
2
3
4
5
0 3 6 9
Score
Count
With AI(Group 1)
How Fun was it to Collaborate?
mean:mean:mean:mean:mean:mean:mean:mean:mean:
8.228.228.228.228.228.228.228.228.22
0
1
2
3
4
5
0 3 6 9
Score
Count
With Dummy System(Group 3)
How Fun was it to Collaborate?
Group 1 (With AI) Group 3 (With Dummy)
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
37
Experiment Result: From the Analytics
There is not that much difference among groups.
mean:mean:mean:mean:mean:mean:mean:mean:mean:
666666666
0
1
2
3
4
5
0 3 6 9
Score
Count
With AI(Group 1)
Self Reported Scores of the Creative Output
mean:mean:mean:mean:mean:mean:mean:mean:mean:mean:
6666666666
0
1
2
3
4
5
0 3 6 9
Score
Count
With No AI(Group 2)
Self Reported Scores of the Creative Output
mean:mean:mean:mean:mean:mean:mean:mean:mean:
6.226.226.226.226.226.226.226.226.22
0
1
2
3
4
5
0 3 6 9
Score
Count
With Dummy System(Group 3)
Self Reported Scores of the Creative Output
Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy)
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
38
Experiment Result: From the Analytics
Group 3, with dummy system, got the highest judged score.
5.1
5.47
6.87
0
2
4
6
group 1 group 2 group 3
Groups
Score
group group 1 group 2 group 3
With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3)
Final text output scored by 3 judges
With AI No AI With Dummy
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
39
Experiment Summary
Group 1
Group 2
Group 3
Total Text Output
745.4
Avg. Time Used Text per Sec.
626.4
825.3
301
215
371
0.4
0.34
0.45
Long
Long
Short
Long
Long
Short
Fast
Fast
Slow
# of Gen. vs. Text Amount
Strong Positive
Relation
Weak Postive
Relation
N/A
Self Score
6
6
6.22
Generation Relevancy Scr.
6.56
4.44
N/A
Highest
Lowest
Judge Score
5.1
5.47
6.87
Highest
With AI
No AI
With Dummy
Neutral
Neutral
Neutral
Lowest
Group 1 & 3 outperformed Group 2 with output metrics, and Group 3 got the highest score from the judges.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
40
Revisiting the Original Research Question
Question
Would this collaborative model between a
creative agent and the user be effective?
Is the ML model better in some way than
simply randomly showing inspiration?
Is the quality of output different among
groups?
Findings
Yes. Both groups (1 & 3) with collaborative models yield higher score on
output quantity and gives fun experience throughout the creative process.
Yes, in the sense that, user testing revealed that generated text from the AI
models were regarded more relevant to the prompt. However, No in the
sense that it did not translate by receiving the highest score from the
judges.
Yes. One of the groups with collaborative process yielded the highest score
from the judges.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
41
5.1
5.47
6.87
0
2
4
6
group 1 group 2 group 3
GroupsScore
group group 1 group 2 group 3
With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3)
Final text output scored by 3 judges
Why did the Group with AI Model Scored the Lowest?
Possible Reasons:
Latency Matters?
Bad Examples Help?
Due to it’s computationally heavy nature,
the AI model had a disadvantage in
comparison to the Dummy model (10 sec. vs.
0.1 sec). User being instantaneously shown
random phrases worked better in this case?
Dummy group had more complaints on the
suggested text not being relevant to the task
itself. This worked in favor for the user to
“eliminate” the bad examples from their mind?
With AI No AI With Dummy
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
42
Other Questions that Arise from the Current Results
Quality vs Quantity?
Broader Applicability?
Generalizability?
We saw collaborative models enable users to
be more prolific. How could this be translated
into quality?
Would the findings be consistent among age
groups, ethnicities, and industries? Or would
there be a different trend? If so, is there
something that can be generalized or not?
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
43
Further Ahead: Designers as Curators of Data when Using AI?
• Potential need for users to design their own dataset for the
models.
• There is no “universal model” that will help the ideation task
in general.
• Rather, better results could be achieved by specifically
designing a model for that particular task, with a particular
set of dataset.
• In that sense, the designer’s role will become more like an
“editor” who curates dataset to build the collaborative
agents.
Staff in Creative
Agency
Application
+
Machine Learning
Model Backend
Text Input
Suggestion
Creative
“Curate Data”
and Design the AI.
Curated Dataset
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
44
No tool is neutral, each is embedded
with the beliefs and biases of its creators.“
”Lauren McCarthy
(artist, creator of p5.js)
Thank you!
Appendix
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
46
Current Prototype
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
47
Preliminary Testing
Had 2 professional writers / 2
non-writers interact with the
prototype and asked about their
ideation process and the
experience.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
48
Preliminary Testing Responses
I can’t say I’m going to use the text, but it’s an
interesting prompt. It might affect the phrasing,
imagery, and the atmosphere of the writing.
Nothing similar to what I had in my mind.

I liked it because it was random.
Fun!
Response 1: Response 2:
Response 3:
It shows signs that the collaborative
model as a new process of ideation is
valid.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
49
How This Could Scale
•Further build an extension / plugin to connect with other existing writing
tools. (i.e. Google Document, Grammarly)
•Build analytic features to further quantify the tendency of the ideation
process that could be utilized for other means. (i.e. better team structures)
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
50
Measure of Success (Further Analysis)
Develop features to count various
types of interactions on the tool to
later on quantitatively analyze the
usage of the tool itself.
•How many times the model was
called?
•What models were used?
•What’s the amount of text written?
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
51
Solution Design, Data Preparation
Raw Data Process 1 Process 2 Model
TV Spots that won
Cannes Lions from 2013,
2014, 2015, 2018
Cut the individual TV spots
into images. (1image / 3sec.)
Apply image captioning to the
images to generate texts.
Data Building
RNN
(Recurrent
Neural Network)
Task
Detail
Conceptual
Diagram of 

the Process
Sample TV Spot

(~2,000 spots) …
“A guy is brushing
his teeth”
“Riding a bike on
road”
“A guy is on a TV”
}
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
52
What creatives do in advertising agencies
State A State BObstacle
Idea
{ } { }
Ideas can be:Obstacles can be:
• A Tagline
• A Script for TV spot
• A campaign plan
• A public relations plan
• A strategy
…
• Low brand recognition rate
• Low sales
• Low presence in the market
• Low brand engagement rate
…
Using ideas ( = “creative jump”) to overcome business obstacles to reach from starting state A to a better, preferred state B.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
53
How do creative teams in the advertising industry operate?
What is the typical workflow?
Research Question 1:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
54
Critical Point in the Workflow
The ideation part is critical because, the idea that comes in
this phase defines the quality of the output until the end.
“
”One interviewee in the advertising agency
*multiple other agency interviewees responded similarly
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
55
What does ideation process look like in the agency environment?
Research Question 2:
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
56
Noticed Similarities and Differences in Ideation
Receptive.
Agency Creative Staff Non-agency Creative Staff
“Idea is something
I need to get.”
Ideation as a proactive process.Characteristic
Comment
Ideation as a receptive process.
Personal and CollaborativeProcess Personal and Collaborative
Pen, Paper, DigitalTools Pen, Paper, Digital
Class
“Idea is something that
will come to me.”
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
57
Individual Process of Ideation
Input Output FormalizeSubphase
Ideation Phase Diverge
Ideation Round Round n
• Understand what is being required for
the next group meeting with the team.
• Collect relevant information from
various sources.
• Also view relevant case studies and
industry material to be up-to-date.
• Start writing out ideas.
• Keep on pushing.
• Push even further.
• Try different techniques to “squeeze”
ideas.
• Formalize the material into decks /
notes / memos so that it can be
shared with others during the next
meeting.
• Use text / images to effectively
communicate the ideas developed.
Task
Medium
Recent news
Case studies of comm. project
Content from various social media
Informal discussion with colleague
Creative brief
Look at art / design / precedents
Pen & paper
Digital entry
Idea deck
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
58
How ideas are recorded in individual phase
Digital Non-Digital
An example of
ideating strategy
An example of
ideating tag lines
& TVC scripts
An example of
ideating campaign
An example of
ideating campaign
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
59
How ideas are converged in group phase
Ideas formulated individually are being shared, discussed, and then converged into a direction for the next round iteration.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
60
Many Individual / Group Sessions with Numerous Ideas before Pitch
Briefing Ideation Pitch Building Pitch
Planning Phase Pitching Phase
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Art Director
Copywriter
Creative
Technologist
Strategist
Media Planner
PR Planner
Creative
Director
Creative
Director
Round 1 Round 2 Round 3
…
= 1 idea
Ideation Mode Individual Group Individual Group Individual Group
Diverge Converge Diverge Converge Diverge ConvergeIdeation Phase
Ideation Round
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
61
What the ideation process is in the agency environment conceptually
= 1 idea
Latent Idea Space
Round 1
Round 2
Round 3
Creative Direction
CreativeDirection
The Ideation phase is essentially creative teams
attempting to navigate possible latent idea space
to find the best idea that would make it to the pitch.
The best ideas
for the pitch
= Pitch Idea
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
62
Source: “The Team Creativity Model: An Exploratory Case Study ” by de Vreede et al, 2017
Reference: “Team Creativity Model”
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
63
Quantity brings quality
Latent Idea Space 2
Round 1
Round 2
Round 3Latent Idea Space 1
Round 1
Round 2
= 1 idea
= Pitch Idea
= 1 idea
= Pitch Idea
Reaching alternative, better ideas
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
64
Ideation methods / tools that are out there
Demo ScreenName
The Toolbox
By
Hyper Island
Overview
• Collection of numerous frameworks and methods for ideation
for individuals and groups, offered by Hyper Island (education
institution)
Source: “Toolbox” by Hyper Island
Issue
• Mainly targeted for service design, does not fit into the context
of advertising agency business landscape.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
65
Ideation methods / tools that are out there
DemoName
Oblique Strategies
By
Brian Eno
Overview
• Cards that have instructions for the users to think differently.
Source: “Toolbox” by Hyper Island
Issue
• Maybe helpful for ideation, but the content of the card does not
relate to the current ideation task the user is working on.
• The process of ideation becomes a mere serendipity.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
66
Problem Statement:
Creative teams working in advertising industries
can ideate more with the right tool, but no such tool exists
designed specifically for them.
An opportunity for a solution.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
67
Precedent 1:
Demo ScreenProject Name
Writing Machine
Project By Project Year
Robin Sloan 2016
Source: “The Writing with the Machine” by Robin Sloan, 2016
Project Overview
Reason for Selection
There is a potential of being a new tool / interaction model which
the user leverages this as an agent to ideate.
• Recurrent Neural Network (RNN) trained using Sci-Fi novel text
data
• The trained model is connected with a text editor, and with a
tab key pressed by the user, takes the previous text as an input
to generate additional sentence.
• The generated text is in Sci-Fi theme, which the user is free to
use as a source of inspiration.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
68
Precedent 2:
Demo ScreenProject Name
Suggestive Drawing
Project By Project Year
Nono Martínez Alonso 2017
Project Overview
Reason for Selection
• Utilized pix2pix and designed an interface that will work in
conjunction with the user to assist / suggest drawing process.
• The user is drawing the lines, but the filling patterns are
determined by the AI backend.
Also another potential of being a new tool / interaction model
which the user leverages this as an agent for a different creative
process.
Source: “Suggestive Drawing” by Nono Martínez Alonso, 2017
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
69
One particular user of this tool
Copywriter
TV Spot Planner
• A copywriter / TV Spot Planner in an agency.
• Needs to formulate (many) ideas for a TV Spot for client’s next
advertising campaigns.
• Usually employs digital tools writes out the ideas.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
70
One particular user of this tool
Sample screen of a copywriter ideating with a memo app
Copywriter
TV Spot Planner
The ideation process
is solitary, text-based.
Independent Design Engineering Project
Harvard University Master of Design Engineering“Proposal for A New Ideation Process”
Togo Kida
71
Data Preparation 1
January February March April May
ML Model
Const. 1
Interface 

Design/Dev 1
Workshop
(User Testing)
Data
Preparation 2
ML Model
Const. 2
Interface 

Design/Dev 2
★Completion
Month
Task
Further Plan

Independent Design Engineering Project II (ADV9382): Final Presentation

  • 1.
    Proposal for ANew Ideation Process Harvard University, Independent Design Engineering Project Togo Kida
  • 2.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 2 Problem Domain Creative teams working in advertising industries. How can I design something that would solve their problem in their job? Independent Design Engineering Project Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida
  • 3.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 3 Why Creative Matters: Consulting Firms Buying Creative Agencies Oct. 2019 Aug. 2019 Apr. 2019 Apr. 2019 Mar. 2019 Mar. 2019 Nov. 2018 Oct. 2019 Jul. 2017 Jul. 2017 Oct. 2016 May. 2015 Nov. 2016 Source: Press Release Information
  • 4.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 4 Preliminary Research: Interviewing Stakeholders T. K. Dentsu Inc. Creative Strategist A. S. Bytedance Japan Creative Director R. N. Dentsu Inc. Creative Director A. S Dentsu Inc. Copywriter, Strategist Y. K. Dentsu Inc. Creative Strategist K. W. Dentsu Inc. Copywriter, Planner Y. K. Dentsu Inc. Creative Planner K. Y. Dentsu Inc. Creative Technologist Advertising Agency Broader Creative (non-advertising) Domain Experts M. S. Dentsu Inc. Creative Technologist O.K. Dentsu Inc. Creative Director Y. N. Dentsu Creative X Inc. Creative Techologist N. W. Dentsu Inc. Comm. Planner I. C. Cash Studios Creative Director S. H. Certain Measures Computational Designer P. R. Pablo Rochat Studio Creative / Art Director R. C Ovo Energy Senior UX Designer M. K. Fahrenheit 212 Strategist T. A. Harvard BS Professor L. G. Uber Executive Cr. Director R. K. Drive Inc. Art Director S. O. Harvard GSD Researcher M. W. Normal Inc. Creative Coder M. C. Deep Mind Senior Visual Designer S. A. Harvard GSD Researcher Domain Interviewee
  • 5.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 5 Preliminary Research: Stakeholder Analysis Creative Director Art Director Copywriter/
 TV Spot Planner Creative Strategist Creative Technologist PR Planner Media Planner Programmer / Developper Designer Media Representative Account Executives Client Client Domain Agency Domain Partners Domain Client Other Dept. Client Other Dept. Client Other Dept. Client Other Dept. Videographer Other partners based on projects … = communication and/or interaction
  • 6.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 6 Preliminary Research: Work Flow Analysis Briefing Ideation Pitch Building Pitch Revision Decision Execution Reporting Planning Phase Pitching Phase Execution Phase • Receive orientation / brief from account team • Understand what needs to be pitched. • Conduct necessary research • Study precedents • Come up with ideas individually • Meet as a team to discuss ideas. • Conduct the above multiple times to converge on the basic storyline for the pitch. • Breakdown what needs to be in the pitch deck, and assign task members to work on the component. • Report on progress, converge as a team if necessary. • Talk with account executive team members to align on overall direction • Conduct the above multiple times if necessary. Process Detail Players • Present the idea to the client. • Seek feedback from the client and make revisions if necessary. • Present again. • Agree with the client what idea is going to be executed. • Agree upon how to measure success. • Work with partnering companies, entities to execute the idea. • Upon completion of the project, return to the client to report the result of the campaign. • Share key learnings from the project with the client. Creative Director Account Executives Creative Director Art Director Copywriter/ Planner Strategist Creative Technologist PR Planner Media Planner Creative Director Client Account Executive Creative Director Client Account Executive Creative Director Client Account Executive All Creative Team All Creative Team Client Partners Responsi- bility Creative Director Account Executive Duration 7 - 10 days 1day 7 - 10 days 7 - 10 days 90 - 180days 14 - 30days7 - 30 days 7 - 30 days
  • 7.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 7 Preliminary Research: Ideation Phase Analysis Briefing Ideation Pitch Building Pitch Planning Phase Pitching Phase Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Creative Director Round 1 Round 2 Round 3 … = 1 idea Ideation Mode Individual Group Individual Group Individual Group Diverge Converge Diverge Converge Diverge ConvergeIdeation Phase Ideation Round
  • 8.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida “ 8 Preliminary Research: Expressed Pain Point An interviewee in the advertising agency *multiple other agency interviewees responded similarly In certain circumstances, I need 
 to formulate 20 - 30 ideas weekly. My job would be so much easier, 
 if there is something that could help me. ”
  • 9.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 9 Ideation Among the Creatives: Text-based (Digital / Non-digital Format) Digital Non-Digital An example of ideating strategy An example of ideating tag lines & TVC scripts An example of ideating campaign An example of ideating campaign
  • 10.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 10 Insight and Problem Statement: Insight Problem Statement Creative teams working in advertising industries 
 are under strong time pressure to generate many creative output in short time. The teams lack the right tool to ideate more, 
 better quality creative output.
  • 11.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 11 Solution Structure: Before Staff in Creative Agency Creative Creative Writing Output Solitary Process
  • 12.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 12 Solution Structure: After Staff in Creative Agency Application + Machine Learning Model Backend Text Input Suggestion Creative Creative Writing Output Collaborative Process
  • 13.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 13 Design Principle • More output (quantity) • In agency context • Fun to use A tool that enables the user to ideate more ideas through its usage. A tool that that fits in the workflow of advertising agency staffs. A tool that is genuinely fun to interact with to foster ideation. *quantity eventually becomes quality *
  • 15.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 15 Solution Design: AI-powered Text Editor “Phantom” Phantom is a text editor powered by an AI backend. There is a text editing section 
 which the user can enter text, pieces of storyline. Phantom can suggest additional text which would be source of inspiration for additional ideas. Powered by:
  • 16.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 16 Powered by: The AI is created with an LSTM-RNN. Phantom is equipped with three different LSTM models by default. These models are capable of generating sequence of text by giving a seed text. Neural Network Architecture of LSTM-RNN Models Model 1 Model 2 Model 3{ } Solution Design: AI-powered Text Editor “Phantom”
  • 17.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 17 Powered by: AI Models Model 1 Model 2 Model 3{ }“Love is…” Generated Text “an infinite amount of deepness…” “something unknown…” “is the toughest thing in this world” Seed Text Input Output Solution Design: AI-powered Text Editor “Phantom” The AI is created with an LSTM-RNN. Phantom is equipped with three different LSTM models by default. These models are capable of generating sequence of text by giving a seed text.
  • 18.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 18 Solution Design: AI-powered Text Editor “Phantom” Main Text Entry Area Generate Button Suggestion Text Area Model Selection Model Settings Powered by:
  • 19.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 19 Solution Design: AI-powered Text Editor “Phantom” Powered by: Independent Design Engineering Project Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 19 The user can easily specify which models to use.
  • 20.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 20 Powered by: Independent Design Engineering Project Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 20 Independent Design Engineering Project Harvard University Master of Design Engineering 20 “Proposal for A New Ideation Process” Togo Kida Solution Design: AI-powered Text Editor “Phantom” The user can also specify the parameters of the LSTM-RNN models.
  • 21.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 21 Solution Design: AI-powered Text Editor “Phantom” Step 1: Write, and highlight. Step 2: Hit generate. Step 3: See suggestion text. Powered by:
  • 22.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 22 Solution Design: AI-powered Text Editor “Phantom” Powered by: The user will start writing in the main text area but the user can also benefit from leveraging the AI models to complete the writing task. Proposed Ideation Process with Phantom:
  • 23.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 23 Underlying Research Question Staff in Creative Agency Application + Machine Learning Model Backend Text Input Suggestion Creative Would this collaborative model between a creative agent and the user be effective? “ ” Collaborative Model
  • 24.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 24 Experiment Structure & Measure of Success A Prompt A Prompt Participants Input Intervention Output 15 min. Experiment Creative Staff Group 1 Creative Staff
 Group 2 No Tool With AI Agent Writing Writing Evaluation Quantitative Qualitative Analytics upon using the tool. Comments from the participants. Outcome 
 Evaluation Conducted series of 15 minutes online experiments with three groups and evaluate the outcomes. A Prompt Creative Staff
 Group 3 With Dummy Agent *random phrase Writing
  • 25.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 25 Group Design Detail and it’s Difference Group 1 Group 2 Group 3 Task Detail Duration Writing a poem of COVID19 Tool Intervention 15 minutes 15 minutes 15 minutes Tool Latency Task Category 3 AI models trained in JPOP lyrics (LSTM-RNNs) None Dummy System (Picks random JPOP lyrics phrase) 5 - 10 sec. N/A 0.1< sec. Collaborative Process Solitary Process Collaborative Process Data Source 20MB lyrics Text data 20MB lyrics Text data N/A Tool Relevance The tool will try to generate text related to user input. It has nothing to do with user input. N/A Writing a poem of COVID19 Writing a poem of COVID19 With AI No AI With Dummy
  • 26.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 26 AI Dataset Design: Why JPOP Lyrics? I’ve built other models with different dataset, but considering the prompt, I settled down to using JPOP lyrics as dataset for AI. Dataset Used to Train AI(LSTM-RNN) Generated Output Analysis Is it Suitable for Experiment? Selected? JPOP Lyrics Fan Written Japanese Fantasy Novel Contemporary Classic Japanese Lit. Text generated from this model is suggestive and seems wide applicability. Text generated from this model is too specific. A lot about magic spells and swords and knights. Text generated from this model is too archaic. Yes. No. No. ✅ 👎 👎
  • 27.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 27 Quantitative Metrics Collection Quantitative metrics are recorded as .json file while each of the subjects interact with Phantom. Participant .json file Collected Metrics • The amount of text input over time. • What was the input text used to generate suggestions from the AI/ dummy system. • Parameter settings when user asked for text suggestion. Interaction During Experiment Analytics Data Recorded
  • 28.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 28 Board of Judges Asked user feedback after experiment. Also, had an independent group of judges to judge the writing outputs and see if there is a noticeable difference in scores. Creative Writing Score Qualitative Metrics Collection Participants Score/Reflection • Your self score on output? • Was the generated text helpful? • Rate the experience itself? Post-experiment Questionnaire
  • 29.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida Group Design to Answer Research Questions Number of research questions can be answered through this experiment. Question to be Answered Would this collaborative model between a creative agent and the user be effective? Is the ML model better in some way than simply randomly showing text as inspiration? How it Could be Answered? Compare group 2 and group 1 + 3. Rationale Compare group 1 and group 3. Group 2 is a solitary task. Group 1 + 3 is a collaborative task with the tool. Group 1 uses AI(LSTM-RNN) and Group 2 is a dummy system that simply returns text randomly, not relevant to the user input. Is the quality of output different among groups? Scores from the external judges. 3 independent judges were assigned to score the output. This will give objective means to evaluate the output. 29
  • 30.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 30 Experiment Structure: Recruitment Recruited 30 participants from the advertising industry to test the solution design in 3 groups. Experiment Group Total of 30 Participants Group 1 = 10 Group 2 = 10 Group 3 = 10 Creative Strategist Copywriter Creative Tech Director TVC Planner Copywriter Creative Manager Creative Technologist Creative Director TVC Planner Copywriter Creative Strategist Copywriter Creative Tech Director TVC Planner Strategist Creative Manager Planner Creative Director TVC Planner Copywriter Creative Strategist Copywriter Executive Creative Director TVC Planner Copywriter Creative Manager Creative Technologist Creative Director TVC Planner Copywriter With AI No AI With Dummy
  • 31.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 31 Experiment Result: From the Analytics Collaborative task group (gr. 1 & 3) seem to yield more output than solitary task group (gr. 2). 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 With the assistance of AI models(group 1) Elapsed Seconds vs Total Amount of Text Entered n = 10 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 Without any form of assistance(group 2) Elapsed Seconds vs Total Amount of Text Entered n = 10 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 With the assistance of randomly selected phrases(group 3) Elapsed Seconds vs Total Amount of Text Entered n = 10 Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy) Participants Participants Participants
  • 32.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 32 Experiment Result: From the Analytics While some participants are copying generated text, not all participants are doing so necessarily (simply viewing). 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 With the assistance of AI models(group 1) Elapsed Seconds vs Total Amount of Text Entered n = 10 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 Without any form of assistance(group 2) Elapsed Seconds vs Total Amount of Text Entered n = 10 0 200 400 600 0 250 500 750 Elapsed Seconds TotalAmountofText colour 01 02 03 04 05 06 07 08 09 10 With the assistance of randomly selected phrases(group 3) Elapsed Seconds vs Total Amount of Text Entered n = 10 Suggestion Copied Suggestion Copied Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy) Participants Participants Participants
  • 33.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 33 Experiment Result: From the Analytics Collaborative task group (gr. 1 & 3) seem to yield more output than solitary task group (gr. 2). 745.4 626.4 825.3 0 200 400 600 800 group 1 group 2 group 3 Groups TotalInputTime group group 1 group 2 group 3 With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3) Average Input Time Among Groups 301 215 371 0 100 200 300 group 1 group 2 group 3 Groups TotalTextAmount group group 1 group 2 group 3 With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3) Average Text Amount Among Groups 0.4 0.34 0.45 0.0 0.1 0.2 0.3 0.4 group 1 group 2 group 3 Groups TextInputRateperSecond group group 1 group 2 group 3 With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3) Average Rate of Text Input per Second With AI No AI With Dummy With AI No AI With Dummy With AI No AI With Dummy
  • 34.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 34 Experiment Result: From the Analytics Looking at the data, some evidence to say that the more you generate, the more output quantity is achieved. ● ● ● ● ● ● ● ● ● ● 0 200 400 600 0 5 10 15 20 25 Number of Times Suggestion Generated TotalAmountofText With the assistance of AI models(group1) Number of Times Suggestion Generated vs Total Amount of Text Entered n = 10 ● ● ● ● ● ● ● ● ● 0 200 400 600 0 5 10 15 20 25 Number of Times Suggestion Generated TotalAmountofText With the assistance of randomly selected phrases(group 3) Number of Times Suggestion Generated vs Total Amount of Text Entered n = 10 Group 1 (With AI) Group 3 (With Dummy)
  • 35.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 35 Experiment Result: From the Analytics Group 1 which is trying to generate on user context scored higher. mean:mean:mean:mean:mean:mean:mean:mean:mean: 6.566.566.566.566.566.566.566.566.56 0 1 2 3 4 5 0 3 6 9 Score Count With AI(Group 1) How Helpful Were the Generated Results? mean:mean:mean:mean:mean:mean:mean:mean:mean: 4.444.444.444.444.444.444.444.444.44 0 1 2 3 4 5 0 3 6 9 Score Count With Dummy System(Group 3) How Helpful Were the Generated Results? Group 1 (With AI) Group 3 (With Dummy)
  • 36.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 36 Experiment Result: From the Analytics Very very positive result on the collaborative task model. mean:mean:mean:mean:mean:mean:mean:mean:mean: 8.568.568.568.568.568.568.568.568.56 0 1 2 3 4 5 0 3 6 9 Score Count With AI(Group 1) How Fun was it to Collaborate? mean:mean:mean:mean:mean:mean:mean:mean:mean: 8.228.228.228.228.228.228.228.228.22 0 1 2 3 4 5 0 3 6 9 Score Count With Dummy System(Group 3) How Fun was it to Collaborate? Group 1 (With AI) Group 3 (With Dummy)
  • 37.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 37 Experiment Result: From the Analytics There is not that much difference among groups. mean:mean:mean:mean:mean:mean:mean:mean:mean: 666666666 0 1 2 3 4 5 0 3 6 9 Score Count With AI(Group 1) Self Reported Scores of the Creative Output mean:mean:mean:mean:mean:mean:mean:mean:mean:mean: 6666666666 0 1 2 3 4 5 0 3 6 9 Score Count With No AI(Group 2) Self Reported Scores of the Creative Output mean:mean:mean:mean:mean:mean:mean:mean:mean: 6.226.226.226.226.226.226.226.226.22 0 1 2 3 4 5 0 3 6 9 Score Count With Dummy System(Group 3) Self Reported Scores of the Creative Output Group 1 (With AI) Group 2 (No AI) Group 3 (With Dummy)
  • 38.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 38 Experiment Result: From the Analytics Group 3, with dummy system, got the highest judged score. 5.1 5.47 6.87 0 2 4 6 group 1 group 2 group 3 Groups Score group group 1 group 2 group 3 With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3) Final text output scored by 3 judges With AI No AI With Dummy
  • 39.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 39 Experiment Summary Group 1 Group 2 Group 3 Total Text Output 745.4 Avg. Time Used Text per Sec. 626.4 825.3 301 215 371 0.4 0.34 0.45 Long Long Short Long Long Short Fast Fast Slow # of Gen. vs. Text Amount Strong Positive Relation Weak Postive Relation N/A Self Score 6 6 6.22 Generation Relevancy Scr. 6.56 4.44 N/A Highest Lowest Judge Score 5.1 5.47 6.87 Highest With AI No AI With Dummy Neutral Neutral Neutral Lowest Group 1 & 3 outperformed Group 2 with output metrics, and Group 3 got the highest score from the judges.
  • 40.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 40 Revisiting the Original Research Question Question Would this collaborative model between a creative agent and the user be effective? Is the ML model better in some way than simply randomly showing inspiration? Is the quality of output different among groups? Findings Yes. Both groups (1 & 3) with collaborative models yield higher score on output quantity and gives fun experience throughout the creative process. Yes, in the sense that, user testing revealed that generated text from the AI models were regarded more relevant to the prompt. However, No in the sense that it did not translate by receiving the highest score from the judges. Yes. One of the groups with collaborative process yielded the highest score from the judges.
  • 41.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 41 5.1 5.47 6.87 0 2 4 6 group 1 group 2 group 3 GroupsScore group group 1 group 2 group 3 With AI(group 1) vs With No AI(group 2) vs With Dummy(group 3) Final text output scored by 3 judges Why did the Group with AI Model Scored the Lowest? Possible Reasons: Latency Matters? Bad Examples Help? Due to it’s computationally heavy nature, the AI model had a disadvantage in comparison to the Dummy model (10 sec. vs. 0.1 sec). User being instantaneously shown random phrases worked better in this case? Dummy group had more complaints on the suggested text not being relevant to the task itself. This worked in favor for the user to “eliminate” the bad examples from their mind? With AI No AI With Dummy
  • 42.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 42 Other Questions that Arise from the Current Results Quality vs Quantity? Broader Applicability? Generalizability? We saw collaborative models enable users to be more prolific. How could this be translated into quality? Would the findings be consistent among age groups, ethnicities, and industries? Or would there be a different trend? If so, is there something that can be generalized or not?
  • 43.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 43 Further Ahead: Designers as Curators of Data when Using AI? • Potential need for users to design their own dataset for the models. • There is no “universal model” that will help the ideation task in general. • Rather, better results could be achieved by specifically designing a model for that particular task, with a particular set of dataset. • In that sense, the designer’s role will become more like an “editor” who curates dataset to build the collaborative agents. Staff in Creative Agency Application + Machine Learning Model Backend Text Input Suggestion Creative “Curate Data” and Design the AI. Curated Dataset
  • 44.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 44 No tool is neutral, each is embedded with the beliefs and biases of its creators.“ ”Lauren McCarthy (artist, creator of p5.js) Thank you!
  • 45.
  • 46.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 46 Current Prototype
  • 47.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 47 Preliminary Testing Had 2 professional writers / 2 non-writers interact with the prototype and asked about their ideation process and the experience.
  • 48.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 48 Preliminary Testing Responses I can’t say I’m going to use the text, but it’s an interesting prompt. It might affect the phrasing, imagery, and the atmosphere of the writing. Nothing similar to what I had in my mind.
 I liked it because it was random. Fun! Response 1: Response 2: Response 3: It shows signs that the collaborative model as a new process of ideation is valid.
  • 49.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 49 How This Could Scale •Further build an extension / plugin to connect with other existing writing tools. (i.e. Google Document, Grammarly) •Build analytic features to further quantify the tendency of the ideation process that could be utilized for other means. (i.e. better team structures)
  • 50.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 50 Measure of Success (Further Analysis) Develop features to count various types of interactions on the tool to later on quantitatively analyze the usage of the tool itself. •How many times the model was called? •What models were used? •What’s the amount of text written?
  • 51.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 51 Solution Design, Data Preparation Raw Data Process 1 Process 2 Model TV Spots that won Cannes Lions from 2013, 2014, 2015, 2018 Cut the individual TV spots into images. (1image / 3sec.) Apply image captioning to the images to generate texts. Data Building RNN (Recurrent Neural Network) Task Detail Conceptual Diagram of 
 the Process Sample TV Spot
 (~2,000 spots) … “A guy is brushing his teeth” “Riding a bike on road” “A guy is on a TV” }
  • 52.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 52 What creatives do in advertising agencies State A State BObstacle Idea { } { } Ideas can be:Obstacles can be: • A Tagline • A Script for TV spot • A campaign plan • A public relations plan • A strategy … • Low brand recognition rate • Low sales • Low presence in the market • Low brand engagement rate … Using ideas ( = “creative jump”) to overcome business obstacles to reach from starting state A to a better, preferred state B.
  • 53.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 53 How do creative teams in the advertising industry operate? What is the typical workflow? Research Question 1:
  • 54.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 54 Critical Point in the Workflow The ideation part is critical because, the idea that comes in this phase defines the quality of the output until the end. “ ”One interviewee in the advertising agency *multiple other agency interviewees responded similarly
  • 55.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 55 What does ideation process look like in the agency environment? Research Question 2:
  • 56.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 56 Noticed Similarities and Differences in Ideation Receptive. Agency Creative Staff Non-agency Creative Staff “Idea is something I need to get.” Ideation as a proactive process.Characteristic Comment Ideation as a receptive process. Personal and CollaborativeProcess Personal and Collaborative Pen, Paper, DigitalTools Pen, Paper, Digital Class “Idea is something that will come to me.”
  • 57.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 57 Individual Process of Ideation Input Output FormalizeSubphase Ideation Phase Diverge Ideation Round Round n • Understand what is being required for the next group meeting with the team. • Collect relevant information from various sources. • Also view relevant case studies and industry material to be up-to-date. • Start writing out ideas. • Keep on pushing. • Push even further. • Try different techniques to “squeeze” ideas. • Formalize the material into decks / notes / memos so that it can be shared with others during the next meeting. • Use text / images to effectively communicate the ideas developed. Task Medium Recent news Case studies of comm. project Content from various social media Informal discussion with colleague Creative brief Look at art / design / precedents Pen & paper Digital entry Idea deck
  • 58.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 58 How ideas are recorded in individual phase Digital Non-Digital An example of ideating strategy An example of ideating tag lines & TVC scripts An example of ideating campaign An example of ideating campaign
  • 59.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 59 How ideas are converged in group phase Ideas formulated individually are being shared, discussed, and then converged into a direction for the next round iteration.
  • 60.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 60 Many Individual / Group Sessions with Numerous Ideas before Pitch Briefing Ideation Pitch Building Pitch Planning Phase Pitching Phase Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Art Director Copywriter Creative Technologist Strategist Media Planner PR Planner Creative Director Creative Director Round 1 Round 2 Round 3 … = 1 idea Ideation Mode Individual Group Individual Group Individual Group Diverge Converge Diverge Converge Diverge ConvergeIdeation Phase Ideation Round
  • 61.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 61 What the ideation process is in the agency environment conceptually = 1 idea Latent Idea Space Round 1 Round 2 Round 3 Creative Direction CreativeDirection The Ideation phase is essentially creative teams attempting to navigate possible latent idea space to find the best idea that would make it to the pitch. The best ideas for the pitch = Pitch Idea
  • 62.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 62 Source: “The Team Creativity Model: An Exploratory Case Study ” by de Vreede et al, 2017 Reference: “Team Creativity Model”
  • 63.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 63 Quantity brings quality Latent Idea Space 2 Round 1 Round 2 Round 3Latent Idea Space 1 Round 1 Round 2 = 1 idea = Pitch Idea = 1 idea = Pitch Idea Reaching alternative, better ideas
  • 64.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 64 Ideation methods / tools that are out there Demo ScreenName The Toolbox By Hyper Island Overview • Collection of numerous frameworks and methods for ideation for individuals and groups, offered by Hyper Island (education institution) Source: “Toolbox” by Hyper Island Issue • Mainly targeted for service design, does not fit into the context of advertising agency business landscape.
  • 65.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 65 Ideation methods / tools that are out there DemoName Oblique Strategies By Brian Eno Overview • Cards that have instructions for the users to think differently. Source: “Toolbox” by Hyper Island Issue • Maybe helpful for ideation, but the content of the card does not relate to the current ideation task the user is working on. • The process of ideation becomes a mere serendipity.
  • 66.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 66 Problem Statement: Creative teams working in advertising industries can ideate more with the right tool, but no such tool exists designed specifically for them. An opportunity for a solution.
  • 67.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 67 Precedent 1: Demo ScreenProject Name Writing Machine Project By Project Year Robin Sloan 2016 Source: “The Writing with the Machine” by Robin Sloan, 2016 Project Overview Reason for Selection There is a potential of being a new tool / interaction model which the user leverages this as an agent to ideate. • Recurrent Neural Network (RNN) trained using Sci-Fi novel text data • The trained model is connected with a text editor, and with a tab key pressed by the user, takes the previous text as an input to generate additional sentence. • The generated text is in Sci-Fi theme, which the user is free to use as a source of inspiration.
  • 68.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 68 Precedent 2: Demo ScreenProject Name Suggestive Drawing Project By Project Year Nono Martínez Alonso 2017 Project Overview Reason for Selection • Utilized pix2pix and designed an interface that will work in conjunction with the user to assist / suggest drawing process. • The user is drawing the lines, but the filling patterns are determined by the AI backend. Also another potential of being a new tool / interaction model which the user leverages this as an agent for a different creative process. Source: “Suggestive Drawing” by Nono Martínez Alonso, 2017
  • 69.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 69 One particular user of this tool Copywriter TV Spot Planner • A copywriter / TV Spot Planner in an agency. • Needs to formulate (many) ideas for a TV Spot for client’s next advertising campaigns. • Usually employs digital tools writes out the ideas.
  • 70.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 70 One particular user of this tool Sample screen of a copywriter ideating with a memo app Copywriter TV Spot Planner The ideation process is solitary, text-based.
  • 71.
    Independent Design EngineeringProject Harvard University Master of Design Engineering“Proposal for A New Ideation Process” Togo Kida 71 Data Preparation 1 January February March April May ML Model Const. 1 Interface 
 Design/Dev 1 Workshop (User Testing) Data Preparation 2 ML Model Const. 2 Interface 
 Design/Dev 2 ★Completion Month Task Further Plan