HLEG thematic workshop on Measuring Trust and Social Capital, 10 June 2016, Paris, France. More information at: www.oecd.org/statistics/measuring-economic-social-progress/hleg-workshop-on-measuring-trust-and-social-capital-2016.htm
2. • Trustlab: Overview
Goals and Rationale
Trustlab Overview
Trustlab Content
• Preliminary results:
A comparison of survey-based and experimental
measures of trust in others
Trust and other social norms
Individual determinants of trust in others
Outline
2
4. Trustlab: Goals (1)
• One of the major challenges with measuring
trust is validity (how do we know that the
measure works)?
• Ideally we would like a valid measure of trust
that we could compare to survey responses.
• Such a measure would be available for the same
people that we have survey data for so that it is
possible to see how the two measures of trust
relate to each other
4
5. Trustlab: Goals (2)
Joint project Sciences-Po Medialab and the OECD
Statistics Directorate, financed by voluntary
contributions and ERC (SOWELL Project)
Goals:
1. Produce new measures of trust and social
norms using a range of techniques
2. Compare trust and social norms across
countries (and different people in those)
and across techniques
3. Understand individual drivers of trust 5
6. • Build on the significant body of research from
experimental economics (e.g. Glaeser, Laibson,
Scheinkman and Soutter, 2000; Fehr, 2006; Johnson
and Mislin, 2006; Falk et al, 2015; Algan, 2016)
• These experimental approaches have potential to provide
insight into how people actually behave, but to date face
limitations:
– Largely based on very small sample sizes (<500)
– Samples generally not nationally representative
– Not linked to comparable survey data
Trustlab: Rationale
6
7. • An experimental programme funded by
voluntary contributions from
participating countries
• Integrated online platform
• Representative national sample of n=1000
• Combines traditional survey questions with
experimental games providing both
behavioural and self-reported information
• Games are played with real resources at
stake (mean value around 15 Euro)
Trustlab: Overview (1)
7
9. 1 2
3 4
Both
players
start with
€10
Player A’s
transfer
(trust) to
player B is
multiplied
by 3
Player B
transfers
back some
money
(trust-
worthiness)
Player A
payoff
depends on
decision
Player B
Trust game
9
10. 1 2
3 4
All players
start with
equal
endowment
Players
make
decision to
invest in a
joint
project
Investments
are
multiplied by
1.6
and equally
redistributed
Player A
payoff
depends on
own
investment
and that of
others
Public Goods Game
10
11. • IAT is a well-established method to investigate attitudes
towards race, gay, sexuality, gender (e.g. Greenwald et al.
2008)
• Measures the strength of association between
“categories” and their “evaluations”
• In Trustlab selected public institutions (government,
media, judicial system) are chosen as categories to be
evaluated on their trustworthiness
• The key idea is that the association between category and
evaluation depends on the participants’ response time
to allocate words to different categories
Trustlab Content:
Implicit Association Test (IAT)
11
12. Respondents are asked to sort stimuli, appearing in the
middle of the screen, as fast as they can to either the right
or left side of the screen.
This procedure is repeated across up to seven IAT blocks
Implicit Association Test (IAT)
Stimulus
(evaluation)
Target category
12
13. IAT Modules
Each respondent will take either the first three or the
second two IATs in the 15 minutes allocated to this section
IAT Category Attribute
1 Government Trustworthy//Untrustworthy
2 Judicial system Trustworthy//Untrustworthy
3 Media Trustworthy//Untrustworthy
4 Government Competent//Incompetent
5 Government Honest//Dishonest
Trust across
types of
institutions
Dimensions
driving trust
Trustlab Content:
Implicit Association Test (IAT)
13
14. Mod Theme Example questions
1 Trust and
trusting
behaviour
• Generally speaking, would you say that most people can be trusted, or that you can't be
too careful in dealing with people?
• If you lost a wallet or a purse that contained items of great value to you, and it was
found by a stranger, do you think it would be returned with its contents, or not?
2 Trust in
institutions
• How much confidence do you have in (list of institutions) to act in the best interest of
society?
• Do you agree with the following statements:
- Public institutions deliver public services in the best possible way.
- Public institutions pursue long term objectives
- People working in public institutions behave according to ethical standards aimed at
avoiding corruption
- Public institutions are transparent
- Public institutions treat all citizens fairly regardless of their gender, race, age or
economic condition
3 Demographics Questions on age, sex, nationality, HH income, educational attainment
Trustlab Content:
Survey and Demographic Module
14
15. • 2015
Survey development
• 2016
June: IT platform finalised and tested on M-Turk
(presented hereafter)
June/July: First wave (generalised trust +
institutional trust) implemented in Korea, France
Fall: 2 additional countries
Results of first wave published
• 2017/2018
Second wave of countries
Trustlab: Timeframes
15
17. • Trust in others: payment of player A in Trust game
• Trustworthiness: payment of player B divided by
amount available, averaged across all payments from
player A in conditional Trust game
• Altruism: payment from player A in Dictator game
• Cooperation: payment from player A in Public
Good Game
• Reciprocity: slope of the payment function of
player A depending on others’ contribution in
conditional Public Good Game
17
I - BILATERAL COOPERATION
Trustlab measures
18. Survey-based vs. experimental
measures of trust in others : M-Turk
• Very small sample (~220 obs./150 full survey) from M-Turk
used for demonstration: only for illustration!
18
No correlation between self-reported trust and behavioral-based
measures of trust and trustworthiness
(coherent with Johnson-Mislin, 2012 and Sapienza et al. 2012)
19. Why do experimental measures matter ? (1)
True contribution
19
• On micro data , experimental measures of trust do predict
observed individual contribution in the field, while self-declared
trust do not
• Exemple on Wikipedia (Algan et al. 2015)
20. 20
- Large set of different social motives can be distinguished, and are
not perfectly correlated: ex. difference between :
Altruistic motive (Andreoni, 1989) and Reciprocity motive (Rabin, 1993)
- Example from M-Turk and Wikipedia Samples:
Why do experimental measures matter ?(2)
Rich description of different social behaviors
21. 21
- Those different social behaviors do not have the same predictive
power for explaining true contribution in the fields
- Example from Wikipedia Sample: wikipedians contribution
Why do experimental measures matter ?(2)
Rich description of different social behaviors
Table: Number of Wikipedia contributions and altruism motive
22. 22
ALTERNATIVE MEASURES FROM Fischbacher et al. 2001
Categorization of four types from the conditional public good based on
preferences for reciprocity and or altruism
Why do experimental measures matter ?(2)
Rich description of different social behaviors
23. Distribution of social behaviors
23
Social behaviours among the French population (N=1000) and the
Wikipedia community
24. • Compute D-score of a dimension:
24
II - TRUST IN INSTITUTIONS
IAT MEASURES from M-Turk
𝑀𝑒𝑎𝑛(𝐿𝑎𝑡𝑒𝑛𝑐𝑦) 𝐷−
− 𝑀𝑒𝑎𝑛 (𝐿𝑎𝑡𝑒𝑛𝑐𝑦) 𝐷+
𝑆𝐷(𝐿𝑎𝑡𝑒𝑛𝑐𝑦) 𝐷
26. • Trustlab is an innovative platform aiming at
eliciting trust and other social norms through
experiments over large samples and across several
countries
• The platform is now ready
• Three key questions will be addressed:
– The comparison between survey-based and
experimental measures
– The links between various social norms (and in the
future the link with policy settings)
– The individual determinants of trust, social norms
and trust in institutions
26
Conclusion
Editor's Notes
Trustlab collects high-quality data in order to measure trust, to compare it across countries, and to understand its social and individual determinants.
It is likely that experimental settings can provide credible, high quality, measures of trust and social capital. However, to date, behavioral experiments have lacked the rigor and size that allows the use of this data for international comparison purposes. Trustlab aims to fill this gap by using quasi-experimental behavioural techniques, advancing our knowledge on trust in different countries as well as advancing an innovative research agenda.
The platform built by the Sciences Po Medialab involves participants into online behavioural experiments, with real monetary reward. The decisions that participants make determine directly the payoff they receive, and so they are stimulated to act as they would in real life.
The Trustlab surveys will be composed of 3 core question modules.
A core experimental module that will be repeated every wave and for every country and that focuses on measuring generalized trust through games
A core quasi-experimental module that focuses on measuring trust in institutions through a common psychological test to measure implicit attitudes
A core trust survey and demographic module containing basic information required for analytical purposes and survey questions to explore aspects of trust aligned with the core experimental module
In future rounds, flexible modules both relating to the experimental and the survey module can be added to explore new issues on a rotating or case-specific basis
The first game measures trust by simulating a situation where individual payoff is determined by an interaction between two players.
Here, a first player, player A, makes a decision to transfer part of his initial endowment of €10 to a second player, player B. The transfer is then tripled in amount, after which player B has ownership of the tripled investment, plus his or her own endowment.
Player B can then decide to reward player A for trusting him or her with their investment, by sending back an amount that is equal or bigger to the investment that A has made. In this case, both players profit from the interaction. Trust here is exhibited by player A’s investment decision, or whether A believes that he or she will benefit from the investment.
Player B also has the choice to keep the obtained investment from player A. The exhibited behaviour of player B is referred to as trustworthiness. In the Trustlab game, participants are randomly selected to the role of one of the players.
Following the Trust game, participants play a one-sided version of this game referred to as the Dictator game. In this game, there is no reciprocity. Player A has the opportunity to transfer money to a poorly endowed player B, but this transfer is final. There is no multiplication of the transfer and player B has no opportunity to reward player A. This game is used to extract the degree of altruism from player A’s decisions.
The final behavioural game is called the public goods game. It tests the extent to which participants are willing to cooperate with other players in order to collectively gain a higher return. Here, all players are asked to use part of their endowment as an investment towards a joined project. The inputs are multiplied by a factor of 1.6, and the returns are equally distributed. The game tests the degree of cooperation of participants, and is able to identify strong cooperators, weak cooperators, and free-riders.
The second part of Trustlab uses a quasi experimental technique to measure trust in institutions. Implicit Association Tests are used to tests the strength of association between specific categories, such as race, gender, and in this case, government institutions, with normative evaluations, such as good or bad, or in this case, trustworthiness.
By evaluating the time it takes for participants to make the association between a stimulus word and various combinations of government and trustworthiness, we can derive the implicit degree of trust toward government.
Please let me take two examples. On the left panel, respondents sort the word ‘moral’ to the left with the category trustworthy. If the respondent thinks that the association between the words trustworthy and government is weak, then it will take a longer time to sort ‘Moral’ to the left. Conversely, on the example on the right panel, it will take a shorter time to sort the word ‘inefficient’ to the right within the category ‘Untrustworthy’.
We measure associations between trust and three different institutions, namely government, judicial system, and media.
In addition, we measure different dimensions driving trust. These dimensions are competence, and honesty.
The final component is a survey that allows us to cross check participant’s behaviour in the trust games with their stated preferences. We collect the Global Values Survey question on trust, among others, to find people’s stated preferences. This section also allows us to collect important covariates, such as demographics, risk aversion, and controls for the survey environment.
Trustlab has advanced quickly over the past year and the platform has just undergone it’s first test on online users in the US through Amazon’s TURK platform. These results will be presented now. We are going to conduct the first nationally representative data collection in France and Korea in the months of June and August.
