Xavier Llorà
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
xllora@uiuc.edu

Subjective-based optimization schemes?
•
Examples
•
 Affinova
 The electronic sheep
 Emotional prosody
A common subjective-based optimization scheme
•
 Interactive genetic algorithms
 Limitations
Combating user fatigue and contradictions
•
 Worst-case scenario
 Partial-ordering graphs, surrogate fitness, and support-vector machines
Conclusions and current directions
•
Acknowledgments
•
September 21, 2006 Xavier Llorà 2

The goal: “Find a good solution in a given search space”
•
The handicap: “No quantitative measure of goodness is available”
•
The search space needs to be explored based on subjective criteria
•
 Qualitative fitness is provided by the user
• Sorting of several solutions
• Choosing one of solution out of a subset
• Based on global impression
 Limitations of subjective-based optimization:
• The time user can expend doing repetitive tasks (user fatigue)
• The progress achieved during the repetitive tasks (user frustration)
• Criteria adaptation (user’s criteria may change during the process as a
results of the process itself)
Let’s see some real examples
•
September 21, 2006 Xavier Llorà 3

Subjective-based optimization schemes? ✔
•
Examples
•
 Affinova
 The electronic sheep
 Emotional prosody
A common subjective-based optimization scheme
•
 Interactive genetic algorithms
 Limitations
Combating user fatigue and contradictions
•
 Worst-case scenario
 Partial-ordering graphs, surrogate fitness, and support-vector machines
Conclusions and current directions
•
Acknowledgments
•
September 21, 2006 Xavier Llorà 4

Concept design: Create new product candidates
•
Affinova (http://www.affinnova.com/)
•
1. Base Concept 2. Featurize 3. Create alternatives
4. Generate alternative, involve customers, and iterate
September 21, 2006 Xavier Llorà 5

Proposed by Scott Draves (http://electricsheep.org/)
•
A form of aesthetic evolution, a concept first realized by Karl Sims
•
The goal:
•
 Animate and evolve artificial life-forms know as sheep
 Reference to Philip K. Dirk’s novel
Use screensavers to:
•
 Create a distributed rendering farm
 Collect user votes on favorite sheep
Popular sheep life longer and generate new sheep
•
Search space:
•
 The space of parameters of fractal flames,a generalization and
refinement of the Iterated Function System (IFS).
 Each sheep is defined by 240 floating point values to optimize
September 21, 2006 Xavier Llorà 6

Sheep #1700 Sheep #110345149
September 21, 2006 Xavier Llorà 7

Cecilia Alm and Xavier Llorà [Alm & Llora, 2006] (http://www.i-discus.org)
•
The problem:
•
 Text-to-speech (TTS) synthesis (given a text the associated speech is synthesized)
 The lack of emotion (neutrality) makes the TTS synthesis sound unnatural
The goal:
•
 Adjust the TTS to incorporate emotional prosody
 Spoken text should sound sad, angry, happy, etc.
 Very useful for story telling or automated audio book generation
Users can easily discern the emotional prosody, but hard to explain why
•
Search space:
•
 Several parameters can be tweak to modify the prosody (6 per word)
 These parameters define the prosody search space
September 21, 2006 Xavier Llorà 8

Sad Angry
Strawberry Strawberry
Tan-tan-tan Tan-tan-tan
Bubhalos Bubhalos
September 21, 2006 Xavier Llorà 9

Subjective-based optimization schemes? ✔
•
Examples ✔
•
 Affinova ✔
 The electronic sheep ✔
 Emotional prosody ✔
A common subjective-based optimization scheme
•
 Interactive genetic algorithms
 Limitations
Combating user fatigue and contradictions
•
 Worst-case scenario
 Partial-ordering graphs, surrogate fitness, and support-vector machines
Conclusions and current directions
•
Acknowledgments
•
September 21, 2006 Xavier Llorà 10

No quantitative fitness is available
•
Qualitative fitness is provided by the user
•
 Sorting of several solutions
 Choosing one of solution out of a subset
 Based on global impressions
User fatigue
•
 Short time periods (1 hour)
 Repetitive task
Frustration
•
 Repeated evaluation of similar solutions
 No progress is appreciated
How can we help the user in the subjective-based optimization quest?
•
Hint: Big time-scale difference between user judgment and computer
•
response
September 21, 2006 Xavier Llorà 11

Interactive
Standard
Genetic Algorithms
Genetic Algorithms
Computer Aided Human-Based
Design (CAD) Genetic Algorithms
[Kosorukoff & Goldberg, 2002]
September 21, 2006 Xavier Llorà 12

Subjective-based optimization schemes? ✔
•
Examples ✔
•
 Affinova ✔
 The electronic sheep ✔
 Emotional prosody ✔
A common subjective-based optimization scheme ✔
•
 Interactive genetic algorithms ✔
 Limitations ✔
Combating user fatigue and contradictions
•
 Worst-case scenario
 Partial-ordering graphs, surrogate fitness, and support-vector machines
Conclusions and current directions
•
Acknowledgments
•
September 21, 2006 Xavier Llorà 13

✖ Users need a clear idea of the outcome
•
 Need for a clear criteria of the goal to reach
✖ A good visualization goes a long way
•
 A non-intuitive visualization may mislead user’s evaluations
✔ Lack of numerical fitness can be a problem
•
 No numeric form that can be optimize is available
✔ User fatigue needs to be minimized
•
 User may only be able to provide reliable evaluations for short
time periods (1 hour)
✔ Users tend to change their criteria along the way
•
 Easy to maintain an unique criteria for short time periods
September 21, 2006 Xavier Llorà 14

Can we learn user preferences from the interaction?
•
If so, we can build models of its preferences
•
Evaluate solutions before presenting them to the user
•
Reduce the number of solutions (combat fatigue)

Show progress during the process (fight user frustration)

The question to answer:
•
How to learn a model from a simple like/don’t like scenario?

September 21, 2006 Xavier Llorà 15

A minimal iGA scenario
•
Two solutions are presented for user evaluation

Three possible evaluation outcomes:

1. The first is better than the second
2. The second is better than the first
3. Don’t know, don’t care
Evaluation by comparison
•
The GA equivalent
•
Tournament selection s=2
•
Can we learn a numerical model out of partial user
•
evaluations?
September 21, 2006 Xavier Llorà 16

• Solution-quality order should be maintained
 Any synthetic fitness needs to maintain the solution ordering
provided by the user
If s1≥ s2 ≥ … ≥ sn then fs(s1) ≥ fs(s2) ≥ … ≥ fs(sn)
• Synthetic fitness should allow extrapolation
 Any synthetic fitness needs to be able to be a good predictor of
the ordering relation beyond the available evaluations collected
• Let’s make some assumptions:
 Imagine I have a few evaluated solutions :)
September 21, 2006 Xavier Llorà 17

Given a fs: X → Y
•
 X: problem attributes
 Y: numeric value
Set of evaluated solutions
•
New solution
•
 Compute the k-nearest neighbor
 Assign the fitness of the
weighted fitness of the k-nearest
neighbors
No accurate prediction beyond
•
the current limits
September 21, 2006 Xavier Llorà 18

• Regression model
• Extrapolates beyond the
evaluated solutions
• Provides a direction toward
improvements
• How many solutions we
need to have a reliable
model?
September 21, 2006 Xavier Llorà 19

• y = a*x + b
• a>1 guaranties the order
• Tournament only cares
about the partial order
among solutions!
• Low cost, high error models
• PAC-learning bounds tell us
how many instances I need
to properly train
September 21, 2006 Xavier Llorà 20

If we have a numerical fitness:
•
 We can build a regression model
 We can use such model for combating fitness fatigue
What is it available?
•
 Partial ordering of solutions (binary tournament)
 Incremental refinement (new evaluations)
The idea:
•
 Binary tournament produce partial order relations
 I can build a graph that contains all the user evaluations:
• Nodes = unique solutions
• Edges = user evaluations (I like the first one, the second one, or neither)
 Compute a numeric fitness for each solution based on the graph
structure
September 21, 2006 Xavier Llorà 21

• Given a set of solutions
 Start with the pair-wise comparisons on the leaves
 As you move up, compare the first leaves reach on a DFS for
each of the branches
 Obtain the partial ordering of the solutions
 Such ordering can be expressed in a graph form
September 21, 2006 Xavier Llorà 22

Nodes are solutions
•
Edges represent the evaluation provided by the user
•
1. One solution is better than the other (→)
2. Don’t know, don’t care ()
Transformed to contain only type 1 relations

Property: cycles detect user contradictions in evaluations

September 21, 2006 Xavier Llorà 23

• Dominance measure
Dominates Dominated by Difference
Real Synthetic
September 21, 2006 Xavier Llorà 24

1. Collect the user evaluations
2. Build the partial order graph
3. Compute the numeric fitness using the partial order
graph
4. Use ε-SVM for creating a regression model
5. Use the learned regression as the synthetic fitness
September 21, 2006 Xavier Llorà 25

• Cut down the number of human evaluations
• Exploit synthetic fitness
 Optimize it
 Sample the best candidates
 Show the best solutions to the user
• The sample best solutions help combating
 Fatigue (educated guess of user preference)
 Frustration (produce new eureka solutions)
September 21, 2006 Xavier Llorà 26

Evaluation
Partial order
update
Sampling Sampling
fs(x)
Optimizer
Synthetic fitness
September 21, 2006 Xavier Llorà 27

• Considerations in the design process
 Clear goal definition
 Impact of problem visualization
 Persistence of user criteria
• Focus
 Lack of numeric fitness
 User fatigue
• A simple controlled task
 One Max
September 21, 2006 Xavier Llorà 28

September 21, 2006 Xavier Llorà 29

• The system
 A simple web application
 OneMax task
 No linkage learning needed
 ε-SVM and a linear kernel
 The compact genetic algorithm (Harik, Cantú-Paz, Goldberg, & Miller,
1999)
• Set up
 One user with no relation to the research
 Repeated series of 10 independent runs for different problem
sizes {4, 8, 12, 16, 20, 24, 28, and 32 variables}
 Collect the data to compare it to a simple GA
September 21, 2006 Xavier Llorà 30

September 21, 2006 Xavier Llorà 31

• Subjective-based optimization is likely to be dynamic
• User criteria may vary as a results of:
 Lack of attention
 Difficulty to perform pair-wise accurate comparisons
 The solutions seen so far may be changing the target!
• As user criteria changes it is like to materialize as
contradictions
• These contradictions can be easily detected as cycles in
the partial-order graph
September 21, 2006 Xavier Llorà 32

[Alías & Llorà, 2006]
•
Iteration 1 Iteration 2 Iteration 3
September 21, 2006 Xavier Llorà 33

• Based on the partial-order graph
• A simple one: proportion of solutions in cycles
September 21, 2006 Xavier Llorà 34

Subjective-based optimization schemes? ✔
•
Examples ✔
•
 Affinova ✔
 The electronic sheep ✔
 Emotional prosody ✔
A common subjective-based optimization scheme ✔
•
 Interactive genetic algorithms ✔
 Limitations ✔
Combating user fatigue and contradictions ✔
•
 Worst-case scenario ✔
 Partial-ordering graphs, surrogate fitness, and support-vector machines ✔
Conclusions and current directions
•
Acknowledgments
•
September 21, 2006 Xavier Llorà 35

• Address the lack of numerical fitness and fatigue
• Synthetic fitness model of user preferences
• Optimize such model to take the advantage of the
timescale difference
• Remarkable speedups
• Measure of user consistency (reliability of the evaluations)
• Real-world applications confirm the theoretical results:
 Emotional text-to-speech synthesis (two research groups)
 Marketing campaign and product design (advertisement company)
 Tuning of text mining analysis tools (chance discovery consortium)
September 21, 2006 Xavier Llorà 36

• Hierarchical tournament impact
• Relation between graph structure and quality of the
regression method
• User consistency versus problem difficulty
• Managing large scale experimentation (metadata stores)
• Optimization involving multiple users
 How can we reliably combine partial-order graphs?
 Can we create combination of partially build synthetic fitnesses?
 How can we deal with multiple targets?
• Users may have multiple criteria in mind
• Users may not realize that even common criteria may have different
subjective interpretations
September 21, 2006 Xavier Llorà 37

• Sponsors:
 National Science Foundation
 Air Force Office of Scientific Research
 SALERO project funded by IST EU
• Not one man’s endeavor:
 David E. Goldberg, Kumara Sastry, Lalitha Lakshmi (IlliGAL/UIUC)
 Loretta Auvil, Michael Welge (ALG/NCSA)
 Cecilia Alm (Linguistics/UIUC)
 Francesc Alías, Lluís Formiga (URL)
 IlliGAL labbies
 Chance Discovery Consortium members
September 21, 2006 Xavier Llorà 38

DISCUS project web page
•
http://www.i-discus.org
IlliGAL web site
•
http://www-illigal.ge.uiuc.edu
IlliGAL blog
•
http://illigal.blogspot.com
Xavier’s blog
•
http://gal31.ge.uiuc.edu/xllora
September 21, 2006 Xavier Llorà 39