Schemas for the Real World [RubyConf AU 2013]

Schemas for
the Real World

Carina C. Zona
@cczona

Sunday, February 24, 13
[As Keith noted,] in addition to being a developer, I'm also a sex educator.

—xkcd #940

@cczona


I think a lot about how the two overlap.
They do. Quite a lot

Imagine walking through the world knowing that everyone’s ﬁrst assumptions
about how you see yourself, who you love, and what feels right for you are
completely wrong. Now imagine signing up for a cool website, and then being
required to select an option from a drop-down menu that doesn’t include
anything that represents you....[Y]ou’ll feel defeated.You’ll want to argue that
whatever they think they’re learning from that drop-down menu, it’s not really
true.You’ll want to tell them that they’re adding to your humiliation by making
you do this.You’ll want to tell them that they’re missing a huge part of you…
—Sarah Dopp

@cczona


Female Male

* *
@cczona


Married Divorced

In a Relationship It's Complicated
Widowed
Single

@cczona


Straight Gay
Bisexual

Pansexual

Asexual

@cczona


@cczona


@cczona


Users are giving pushback to assumptions that leave them out. Social apps, in particular, are
being pressed to adjust. Facebook, Google, and others have been dealing with these
questions for years and are still working it out. So if you feel out of depth, you are not the
only one.

Sociological
Normalization

Construction of an idealized norm

@cczona


When developers talk about normalization, we're talking about databases. But when dealing
with human attributes, we often wind up pursuing a different cause: sociological
normalization. [read]

Conﬂating the two works ﬁne if you only want users from among the select few who belong
to the idealized norm. But most of us are going for broad userbase.

Database
Normalization

Mirror real-world concepts and
their interrelationships.

@cczona


What we lose track of is one of the core objectives of database normalization. We're supposed
to [read]. When the database is in tension with people's own, real world -- then it's not
people who need to be ﬂexible.

@cczona


Why IS this hard? Because at ﬁrst glance, this stuff looks so easy. It's just forms, right? We've
done those a million times before.

But we get tripped up by some ﬂawed premises:

Gender is one of those things
everyone thinks they understand,
but most people don't. Like "Inception".
— Sam Killerman

@cczona


First, a premise that deeply personal stuff about humans can be reduced to lists.

'Hey, this is just a system I can ﬁgure out
easily!' is also a problem among engineers
ﬁrst diving into the stock market.
—xkcd #592

@cczona


Second, assumptions that canonical lists for these exist. Or are at least, SURELY, must be
creatable.

<option value="1">Mr</option> <option value="18">Dame</option> <option value="39">King of Kings and Lord

<option value="2">Mrs</option> <option value="19">Chief</option> of Lords</option>

<option value="3">Miss</option> <option value="20">Sister</option> <option value="40">Conquering Lion of the

<option value="4">Ms</option> <option value="21">Reverend</option> Tribe of Judah</option>

<option value="5">Dr</option> <option value="22">Major</option> <option value="41">Elect of God and Light

<option value="6">Professor</option> <option value="23">Other</option> of this World</option>

<option value="7">The Rt Revd Dr</option> <option value="24">Cllr</option> <option value="42">His Own Divine

<option value="8">The Most Revd</option> <option value="25">Sir</option> Majesty</option>

<option value="9">The Rt Revd</option> <option value="26">Rt Hon Lord</option> <option value="43">First Ancient King of

<option value="10">The Revd Canon</ <option value="27">Rt Hon</option> Creation</option>

option> <option value="28">The Lord </option> <option value="44">King Alpha</option>

<option value="11">The Revd</option> <option value="29">Viscount</option> <option value="45">Queen Omega</option>

<option value="12">The Rt Revd <option value="30">Viscountess</option> <option value="46">Beginning with Our End

Professor</option> <option value="31">Baroness</option> and First with Our Last</option>

<option value="13">The Ven</option> <option value="32">Captain</option> <option value="47">Protector of All Human

<option value="14">The Most Revd Dr</ <option value="33">Master</option> Faith</option>

option> <option value="34">Very Revd</option> <option value="48">Ruler of the

<option value="16">Rabbi</option> <option value="35">Lady</option> Universe</option>

<option value="17">Canon</option> <option value="38">MP</option> <option value="49">Dude</option>

@cczona


And the third problem is our faith that the ﬁrst two problems here can be easily solved. Just
add more list items!

"Seeing more than 20 uncategorized
options can be bewildering…"
—Smashing Magazine

@cczona


So that's not gonna work.

Also, it risks looking like a dumbass...

Social Networks


So here's the next ﬂawed premise: that we intuitively understand the nature of social
networks.

Stepping aside from how "social networking" has come to be implemented as technology --
what IS a social network? What is it in purely human terms?

Social network

Individuals and the dyadic ties between them.
Interactions within a
social group of exactly two people.

@cczona


[slide] That's the real life that our apps are meant to replicate and build upon. One-on-one.
As personal as our social selves get. In social, identity is a dependency.

Developer & user
are in dyad too
How can I be better 1:1 partner with this
person who…
• is a current user
• I hope will become a user

@cczona


As developers, we build -- or break -- community with each line of code.
What 1:1 relationships are we fostering between the individual and the app? What
relationships are we inhibiting between individuals?

In real life

"I know your personhood better than you"

sounds presumptuous
@cczona


In real life

sounds presumptuous
@cczona


In real life

"Who you are is invalid"

sounds arrogant
@cczona


In real life

sounds arrogant
@cczona


In real life

"Your existence isn't possible"

sounds clueless
@cczona


In real life

sounds clueless
@cczona


@cczona


It's not what we set out to do. It's not the community we seek to build. We want to create
things that feel cutting edge -- not that seem out of touch with modern reality.

Be conservative in what you do,
be liberal in what you accept from others.
-Postel's Law

@cczona


[READ]

How do we bring
modern realities into
the data, views, logic?

@cczona


How DO we bring modern realities into the data, views, and logic...? We start at the schema.

@cczona


As developers, we're tugged in two directions: keep the codebase manageable, yet design for
modern complexity.

Get schemas
into alignment.

@cczona


Mental Schema
•Set of pre-conceived ideas
•Framework for representing some
aspect of the world

•System of organizing & perceiving new
information.

@cczona


There are two kinds of schema. [read]

Database Schema
•Structure described in the database's
language

•Blueprint for database construction
•Describes how the real world is being
modeled

@cczona


Database schemas are basically the same thing. It's a mental schema translated into
blueprints for a database.

create_table "users", :force => true do |t|
t.string "username"
t.string "email"
t.string "password"
end

@cczona


When you use Rails generate scaffold or Rails generate model, it creates a migration. Maybe
you created something like this, for instance. Those migrations ultimately get translated into
a uniﬁed schema to be used by the environment's database.

@cczona


These are, simply, frontend manifestations of someone's mental schema.

Schemas are foundation for

expressing

things deeply intimate.

@cczona



expressing

self-image.

@cczona



expressing

identity.

@cczona



expressing

important relationships.

@cczona



expressing

values.

@cczona



expressing

spirit,

creativity,

uniqueness.
@cczona


Schemas deﬁne the

user experience.

@cczona


Schemas deﬁne user experience.

& UX

Our schemas are
leaving people behind.

@cczona


[read]
We can ﬁx that.

"What benefit will
the user notice?"

@cczona


When developing a schema that's going to ask a person about their experiences, feelings,
sense of self, there isn't a single "right way" I can tell to do this. What we can do is evaluate
tradeoffs, and ask "What benefit will the user notice?" That's not equivalent to "How will the
user benefit?", a question that grants too much latitude for assuming that what we want is of
course to their benefit because it's gonna help us deliver a product that's Awesome.

Evaluating from
user perspective
gives us focus.

@cczona


[read]

Coerced Guided Discretionary
Restricted Corrective Freeform

@cczona


There's no point on this chart where everybody is seeing maximum beneﬁt. So let's look at
the choices.

Checkbox Corrective Freeform
Radio
Select

@cczona


Checkboxs, radio buttons, and select menus imply that all possible values are represented.
Hey user, just pick the right one, (or ones). It's real world being rejected because it didn't
happen to look like our mental schema.

Checkbox Corrective Textarea
Radio Text
Select

@cczona


Checking a box is a one step action. Entering a text string is not. So a freeform solution
such as a textarea or text field isn't automatically exciting. Nevertheless, a freeform field can
deliver striking benefit to users.

11 years

11 years

@cczona


At MetaFilter, gender has been a text ﬁeld for over a decade.

…the early crowd at MeFi were often
programmers and they hated the idea of
"dirty" data collection…
—Matt Haughey, founder

@cczona


Initially, there were some who shuddered at the thought.

All girl Dangly bits
XY Chicklet
Braid runner Innie not outie
Fella Convex
Fembot Sideburns
Alto Ambisextrous
It depends Member of the
MYOB patriarchy
@cczona


For about 5 seconds. Then they jumped on board. Because they could be creative and silly.

I speak using the 50% quintessential
male gender, when tomboy, 50% total
required by girly-girl
language
Gender bo bender
M, but known to be banana fanna fo
ﬂexible lender fee ﬁ mo
@cczona


AND because they could express this THING about themselves fully. With authentic voice.

…it was one of the earliest indications I'd
landed in the right place.
—MeFi user

@cczona


That text ﬁeld grew into a beloved institution. What you as a user put into that ﬁeld says
something revealing who you are. That you're allowed to put in anything -- or put in nothing
-- says something revealing about what MetaFilter is meant to be for.

11 years

8 years
11 years
4 years

@cczona


That schema's trust in users was the foundation for the users to ASK to share themselves
even more.

@cczona


Today MetaFilter users are trusted with many free-form ﬁelds, including ones most
developers instinctually constrain or thoroughly validate.

@cczona


Nonsensical values? Hell yeah, go for it. Field values that blatantly contradict each other? No
objection to that either. The message is: hey, user, we get that this is YOUR OWN HOME.
Make it as comfortable, personal, and messy as you want to. WE, as developers, can work
with that.

Social Research


When collecting data on people, you're in a different realm. Social sciences. If you want to
run useful analytics about personal attributes and behavior, then data collection needs to
meet at least two minimum criteria:

Exhaustive
Every possible option


A ﬁeld's values must include every possible option.

Mutually Exclusive
No overlap exists between them


And the ﬁeld's options must be mutually exclusive.

How many social apps have both of those bare minimum criteria covered?

@cczona


We've seen this one enough. At least that set of values seems covered fully.

@cczona


But it's not.

@cczona


Still not...

We'll never get there by adding more list items. But that's ﬁne. Because instead we can
choose to look at human data from a different perspective...

Data doesn’t have
to be for analysis.

@cczona


It's easy to get into the habit of structuring data for easy analysis. But step back. Wallow in
the user's perspective:

Data can be sheer
expressiveness.

@cczona


That data can be sheer expressiveness. Data that has character, individualism,
distinctiveness.

- = select_tag 'person[profile][gender]',
options_for_select( ["","Female","Male"] )

+ = profile.text_field :gender

@cczona


More recently, years ago Diaspora's Sarah Mei turned gender into a text field there too.

@cczona


Just like on MetaFilter, users had fun with it.

Some developers
were not amused

@cczona


Some developers didn't. One of the major complaints raised was the affect on
internationalization of gendered pronouns.

Gendered pronouns
are a rat hole.

@cczona


And here's what I've got to say to that: Gendered pronouns are a rat hole. It gets hella messy,
hella fast, when you deal with internationalization.

The role of gender in society is the most
complicated thing I’ve ever spent a lot of
time learning about, and I’ve spent a lot of
time learning about quantum mechanics.
—Randall Munroe, xkcd

@cczona


That's a plan that comes with a level of complexity that English hasn't wrapped its head
around. In a lot of non-English languages, grammatical gender and what you and I would call
"gender" are independent of each other. So internationalization based on "Gender" will screw
you over. Think about how to avoid dealing with gendered pronouns at all. If it's a
requirement, then the way to cope is to ask.

"Which pronouns do
you prefer?"
(masculine) he/him/himself/his/his

(feminine) she/her/herself/hers/her

(androgynous) they/them/themself/theirs/their

(inanimate) it/it/itself/its/its

<full name> <fn>/<fn>/<fn>/<fn>'s/<fn>'s
@cczona


Randall Munroe, best known for xkcd, has examined the problem several times for English
language projects. Asking, straight up, "Which Pronouns Do You Prefer?" is truly the best he
could come up with. Oh, and as complicated as THIS matrix it, it grows even BIGGER in many
other languages.

Avoid. Or ask. Don't let pronouns become the driving force for bad UX.

@cczona


As developers, we have a vision of what a good codebase should be and not be.

What we want What we get

• Structured • Validations, exceptions
• Predictable • Conditionals, partials

• Relational
• Premature optimization
• Indexed

• Cultural variability
• Exhaustive? • Individual POV
• Moving target

• Easy analytics • Decisions based on false
• Data-driven decisions premises

@cczona


[READ!!]

In the most sensible of ways, we often arrive at solutions that are factually truthy while far
removed from real life utility.

"What is your religion,
if any?"

@cczona


ARIS is the largest ongoing survey of Americans' religious identiﬁcation. It asks this simple,
OPEN-ENDED question: [slide]

Which nets over 100 unique answers. Which, if you're making a form based on that, is tricky.
There's no form element that makes it easy for our users to pick themselves out of a list of so
many possibilities.

@cczona


ARIS found, though, that these could be compressed into 13 major categories. More
manageable list, right? We could use that for a form. But, eh, a lot of those are edge cases.
We'd rather want to focus on genuinely major groupings.

What is your religion, if any?

Christian 76%

Other 4%

None 15%

Don't Know or Refused 5%

@cczona


Which brings us down to this. At least a quarter of Americans are Christian. Done. Every other
religion would just be clutter. Edge cases.

And then there's this sort of crummy data with it. We'd probably assign nil value for more
than one of these categories, right?

What is your religion?

Christian 76%

Other 4%

n/a 20%

@cczona


Fixed. Which focuses attention on a problem here: 1 in 5 are not useful answers from an
advertiser's perspective or for out own analytics. The NILS -- gotta go.

What is your religion?

Christian 76%

Other 24%

@cczona


So what we're left with is a good, clear, list. It covers ALL the big stuff. When you get
reductive enough, for Americans: religion it a binary. Which, from a storage standpoint, is
great. Booleans! Score.

Religion:

@cczona


But we don't do THIS. It covers the biggest categories. But oh how it leaves people out.

People aren't
edge cases.

@cczona


People aren't edge cases.

And they're pushing back on apps that treat them that way.

@cczona


So reductive has big problems. So does scaling upward:

Balancing between
approaches

@cczona


As engineers, it's instinctually uncomfortable to step away from this. To deliberately NOT
structure data for easy analysis. I feel ya. I really do. This freaks me too. But, again, the
foundational question is "What beneﬁt will the user notice?"

@cczona


And if necessary, we CAN strike a middleground. This is where guided response comes in.
Autosuggest.

Required Autosuggest Optional
Checkbox Alteration Textarea
Radio Text
Select

@cczona


Required Autosuggest Optional
Checkbox Minimal Suggest Textarea
Radio Text
Select

@cczona


When there's a subset of values that you're most interested in, do minimal suggest instead.
Auto suggest using just the handful of values you care about. Structure from those who want
to give it, free form to incite expressiveness in those who want THAT.

Unguided Text
Of those who use MetaFilter's gender ﬁeld,
40% of responses are: f, m, female, male.


[read]
So structured data IS there. This can be a balanced solution in many cases where you're
willing to tolerate some ambiguity. Of course there are tradeoffs.
Data quantity is lower. Freed to opt out of proving personal info, many do.
On the other hand, data quality should improve.

Optional Select
60% of Facebook users select a relationship status


It's ﬁne to mix and match here. Find the right approach for your users and your app's
business objectives. Facebook, for instance, makes relationship status completely optional,
but coercive for those who do opt-in to setting a value. Most users do opt-in. 60% of them
select a relationship status.

We want everyone to
feel excited what we've
built

@cczona


The bottom line is that [read]

We want users to feel
passionate about their
involvement

@cczona


[read]

Analytics, investments,
& monetization are
based on a premise that
data is accurate
@cczona


[read!!!]

BUT the data has been collected by coercive approaches, the RISK is that--

Garbage

@cczona


this premise is garbage. Some people are lying, because lying has been made requirement
for getting past the barriers. So conclusions drawn from that bad data can misdirect
decision-making about the next stage of development.

Required Guided Discretionary
Checkbox Corrective Textarea
Radio Text
Select

@cczona


The restrictive options, that stuff at bottom left, those don't actually have to be marked
"required".

t.string "gender",
! ! ! ! :null => false,
! ! ! ! :default => "female",
! ! ! ! :limit =>6

@cczona


But the way we setup schema often embeds assumptions that we should, and we will. So we
do. A field that's not allowed to be null is destined to be mandatory. A field that sets a very
short length is asserting that any reasonable value is going to fit within it. This migration
implies "You're either male or you're female. Transgender is not a reasonable value." If you're
transgender, you'll wind up being coerced into a response that's inauthentic.

t.string "gender"

@cczona


BOOM. This is foundation for a whole different user experience. And the cool ninja move
was that we decided to do nothing.
Make this stuff ﬂexible upfront. Optimize storage later. Decide what's valid later.

Required Guided Optional
Checkbox Alteration Textarea
Radio Text
Select

@cczona


So there you have it: a discretionary ﬁeld. Whether to respond is left up to the user.

t.string "relationship_status",
! ! ! ! :null => true

@cczona


As developers, we may upon THIS expression as redundant, completely unnecessary. Duh,
null is true by default. But making that explicit is a communication to the team and to your
future self. It's a statement of intent. It's documenting a product decision.

Facebook
Single
In a relationship
Engaged
Married
It's complicated
Open relationship

@cczona


...What would a canonical set of relationship statuses look like...? Three years ago Facebook
ﬁgured this list was pretty good. Arguably pretty progressive too, right? Users disagreed.
Strongly.

Facebook
Single Widowed
In a relationship Separated
Engaged Divorced
Married Civil union
It's complicated Domestic partnership
Open relationship

@cczona


Under pressure, Facebook nearly doubled the options in just two years time.

Google+
Single Separated
In a relationship Divorced
Engaged Civil union
Married Domestic partnership
It's complicated I don't want to say
Open relationship
Widowed
@cczona


While Google+ has largely adopted that list, it has not included Separated or Divorced.
Notice that they also added something: choice. Opt out of labeling.

@cczona


Allowing users to identify their relationships with labels of greater personal signiﬁcance...That's being driven
by people rejecting a user experience that isn't working for them.

@cczona


How did some status seem universal, while others weren't? Naming a thing creates scope. The assumed
validity of a ﬁeld's values get constrained as soon as the ﬁeld is named.

Marital Status?
Single Separated
Engaged Civil union
Open relationship
Widowed
@cczona


"Marital status" for instance, might lead to a list such as this. In which one is assumed to be
either unmarried, preparing to be married, currently married, or formerly married.

Relationship Status?
Single Separated
Engaged Civil union
Open relationship
Widowed
@cczona


Whereas "relationship status" might lead to a list more like this one. In which one either has
a current relationship -- or is deﬁned by the absence of any.

But people's lives aren't actually arranged like these. We go through life experiencing many of
these. And new status doesn't always leave old ones behind.

Singleness Status?

@cczona


If you change the name, you shift the paradigm and possibilities. There's important
difference in these what these collect set out to measure. Naming ﬁelds -- With great
speciﬁcity, upfront. --Makes analyses more powerful, later.

Frustrating. I like to be truthful, and "It's
Complicated" is really deceiving. It is not
complicated. I am separated from my
husband, who I am still legally married to.
—Facebook user

@cczona


And some people really do have more than one relationship status at a time.

@cczona


Who's already spotted the fatal ﬂaw in this one?

This is what happens when we try to throw more labels at the problem instead of examining
the assumptions in the database schema. An open relationship is deﬁnitionally a 1:Many join
— WITH the usual engineering understanding, that is: that actual number of relationships
may be 0, 1, OR many. This is Facebook failing at modeling relationships in a _relational
database_. The schema forces the user to choose: LOOK evasive or BE inauthentic.

Modeling the real
world is complex and
that's okay.

@cczona


ONE

Assuming we know
who users are
surrenders
opportunity to
learn who they are.
@cczona


TWO [read] Early constraints in schema NET crappy, misleading data.
[beat]
So keep constraints out of user schema, at least at ﬁrst. Gather enough initial response to do
some data mining. Watch that data for a while. Keep an eye out for emergent trends. What
DISCOVERIES can you make?

More freeform
won't kill us.

@cczona


THREE

Quality
Speciﬁcity
Loyalty

@cczona


Data quality improves when lies are merely optional, not required.
Data that's rich and speciﬁc. So we can unearth patterns that are undetectable when data is
generic. We can discover. We can adapt.
When we show trust in people, they feel good about placing trust in us.
Their response to all this? Engagement. Passion. Loyalty.

@cczona


THOSE are foundations for great user experiences.

Image Credits
Postsecret Flickr

Facebook FetLife

OKCupid Kotangle

Google+ cutestpaw.com

Metaﬁlter hdwallpapers.in

Diaspora xkcd
@cczona


Resources
• “Disalienation: Why Gender is a Text Field on Diaspora” • "Recommendations for Inclusive Data Collection of Trans
People"

• “Gender & Drop Down Menus”
• "Gay Marriage: The Database Engineering Perspective"
• “Sex & Gender”
• "American Religious Identiﬁcation Survey, Summary Report
2009"
• “Bucket Gender”
• "Falsehoods Programmers Believe About Names"
• "Linguistic Potluck: Crowdsourcing Internationalization in Rails"

• "Your Last Name Contains Invalid Characters"
• "Does Facebook Hurt Relationships?"
• Spanish Names
• "Facebook Adds LGBT-Friendly Relationship Status Options"

• Chinese Names
• "Facebook Targeting by Relationship Status & Workplace"
• Database normalization
• "Your Facebook Relationship Status: It's Complicated"

• Sociological normalization
• "Redesigning the Country Selector"

@cczona


Many Thanks
DevChix
Yoz Grahame
Michele Titolo
Heather Rivers
Chiu-Ki Chan
Sarah Mei
SFSI

@cczona


Get in Touch

Carina C. Zona
@cczona
http://cczona.com
cczona@gmail.com
http://slideshare.net/cczona
http://linkedin.com/in/cczona
@cczona


Schemas for the Real World [RubyConf AU 2013]

More Related Content

Similar to Schemas for the Real World [RubyConf AU 2013]

More from Carina C. Zona

Recently uploaded

Schemas for the Real World [RubyConf AU 2013]