Search engines like Google have a massive influence on what information users get to see, and on what search results users select. This leads to search engines having a significant impact on what information we as a society acquire. It has been often lamented that search engines are biased. I, however, argue that we have only scratched the surface because search engine bias is a multifaceted concept and the discussion usually solely focuses on some aspects. Search engine bias can be classified into four different areas. Firstly, there are biases on the side of the search engine, e.g., in their ranking functions. Secondly, there are biases through external influencing of search engine results, predominantly through “search engine optimization”. Thirdly, biases occur on the side of the user (e.g., position bias, confirmation bias, visual attraction bias). And fourthly, there are self-interests of search engine providers which influence the search results. Further to giving an overview of the topic, I will show how search engine providers (and regulators) can take steps towards making search fair. Whereas a bias-free search engine is impossible, a fair search is. Here, I will not only focus on the big web search engines but also on how developers and product owners can make their search systems fair. Or, to put it another way, I will show what can we learn from these “worst practices” in web search when designing our own systems.

1. IN A WORLD OF BIASED SEARCH ENGINES
Prof. Dr. Dirk Lewandowski
Hamburg University of Applied Sciences, Germany
Keynote at the 18th International Conference on Perspectives in Business
Informatics Research, Katowice, Poland, 25 September 2019

2. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
1
https://www.wsj.com/articles/amazon-changed-search-algorithm-in-ways-that-boost-its-own-products-11568645345
https://medium.com/global-editors-network/a-closer-look-at-googles-plan-to-spotlight-original-reporting-ad9b89899cd6
16 September 2019 19 September 2019

3. GOOGLE SERVES MORE THAN
2.000.000.000.000 QUERIES PER YEAR.

4. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
AGENDA
1. Introduction: Search engine result pages
2. Search engine bias: a multifaceted concept
3. Search engine bias in action
4. Search engine bias: Is this just a “Google Problem”?
5. Conclusion
3

5. INTRODUCTION: SEARCH ENGINE
RESULTS PAGES

6. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
SEARCH ENGINE RESULTS PAGES
5
1 2 3

7. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
SERPS ON THE DESKTOP VS. ON THE MOBILE
6
1 2

8. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
WHAT MAKES USERS CLICK?
Users’ visual attention and selection behaviour
are influenced by
• Result position
• Visible area “above the fold”
• The relevance of the results description
(“snippet”)
• Size and design of the snippet (attraction)
Users trust search engines:
• Results are seen as accurate and trustworthy
(Purcell, Brenner & Raine 2012)
• Search engine ranking even is seen a criterion
for trustworthiness (Westerwick 2013)
7

9. SEARCH ENGINE BIAS: A MULTIFACETED
CONCEPT
8

10. "Search is a reflection of the content that
exists on the web.“
(Google 2016)

11. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
10

12. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
11

13. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
12
https://searchengineland.com/googles-results-no-longer-in-denial-over-holocaust-265832
(2016)

14. "Search is a reflection of the content that
exists on the web. The fact that hate sites
appear in Search results in no way means that
Google endorses these views.“
(Google 2016)

15. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
WHAT IS SEARCH ENGINE BIAS?
„Search engine bias is the tendency of a search engine to prefer certain results
through the assumptions inherent in its algorithms.“ (Lewandowski, 2017)
Three “distinct, albeit sometimes overlapping“ concerns (Tavani, 2012):
“(1) search-engine technology is not neutral, but instead has embedded features in its
design that favor some values over others;
(2) major search engines systematically favor some sites (and some kinds of sites) over
others in the lists of results they return in response to user search queries; and
(3) search algorithms do not use objective criteria in generating their lists of results for
search queries.”
14

16. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
BIASES IN SEARCH ENGINE RESULTS
Biases in regards to…
• Race (Noble, 2018)
• Gender (Noble, 2018; Otterbacher et al., 2017)
• Confirmatory information to queries regarding conspiracy theories (Ballatore, 2015)
• Promotion of hate speech (Bar-Ilan, 2006)
• Health information (White and Horvitz, 2009)
15

17. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
BIAS ON THE WEB
16Baeza-Yates, R. (2018). Bias on the web. Communications of the ACM, 61(6), 54–61. https://doi.org/10.1145/3209581

18. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
BIAS IN ORGANIC RESULTS IS JUST THE TIP OF THE
ICEBERG
Facets of search engine bias
1. Search engine (engineering) – Ranking
2. User – Cognitive biases
3. Search engine providers’ self-interests – Design of search engine result pages
4. External influences – Search Engine Optimization (SEO)
17

19. SEARCH ENGINE BIAS IN ACTION (1):
RANKING
18

20. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
19

21. SEARCH ENGINE BIAS IN ACTION (2):
COGNITIVE BIASES
20

22. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
COGNITIVE BIASES
We use heuristics to make decision.
Some examples of cognitive biases:
• Confirmation bias
• Position Bias
• Domain Bias (popular sources)
• Attractiveness Bias (keywords found in snippet)
These biases contribute to users’ result selection (Liu et al., 2014).
21
Liu, Y., Wang, C., Zhou, K., Nie, J., Zhang, M., & Labs, Y. (2014). From Skimming to Reading : A Two-stage Examination Model for
Web Search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge
Management (pp. 849–858). https://doi.org/10.1145/2661829.2661907

23. SEARCH ENGINE BIAS IN ACTION (3):
SEARCH ENGINE PROVIDERS’ SELF-
INTERESTS
22

24. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
THE EUROPEAN COMMISSION‘S DECISIONS ON GOOGLE’S
ANTI-COMPETITIVE PRACTICES
2017: Google Shopping (2.4 billion Euros)
2018: Android (4.34 billion Euros)
2019: Online advertisements (1.49 billion Euros)
[probably to be continued]
23

25. SELF-INTEREST (1): VERTICAL RESULTS

26. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
GOOGLE SHOPPING
25

27. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
26
Lewandowski, D., & Sünkler, S. (2013). Representative online study to evaluate the revised commitments proposed by Google on
21 October 2013 as part of EU competition investigation AT.39740-Google Report for Germany.

28. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
27
Lewandowski, D., & Sünkler, S. (2013). Representative online study to evaluate the revised commitments proposed by Google on
21 October 2013 as part of EU competition investigation AT.39740-Google Report for Germany.

29. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
28
Lewandowski, D., & Sünkler, S. (2013). Representative online study to evaluate the revised commitments proposed by Google on
21 October 2013 as part of EU competition investigation AT.39740-Google Report for Germany.

30. SELF-INTEREST (2): ADS LABELLING

31. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
SNIPPETS: ORGANIC RESULTS AND ADS
30

32. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
SAMPLE TASK: MARK ALL THE ADVERTISEMENTS ON THIS
SEARCH ENGINE RESULTS PAGE
N=1,000
35% marked all ads correctly.
18% marked at least some organic results as ads.
Still, more than 90% of users regard themselves as
competent when it comes to using search engines.
31
Lewandowski, D., Kerkmann, F., Rümmele, S., & Sünkler, S. (2018). An empirical investigation on search engine ad disclosure.
Journal of the Association for Information Science and Technology, 69(3), 420–437. https://doi.org/10.1002/asi.23963

33. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
RESULTS FROM THE EXPERIMENT
N=1,000
Users not able to distinguish
ads from organic results clicked
on the first ad about twice as
often (40.3% vs. 21.6%).
32
Lewandowski, D. (2017). Users’ Understanding of Search Engine Advertisements. Journal of Information Science Theory and
Practice, 5(4), 6–25. https://doi.org/10.1633/JISTaP.2017.5.4.1

34. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
UNDERSTANDING OF ADS INFLUENCES USERS’ VISUAL
BEHAVIOUR
33
Schultheiß, S.; Lewandowski, D.: How users’ knowledge of advertisements influences their viewing and selection behaviour in
search engines [submitted]
knowledge of ads scrolled into the area “below the fold” less often.465
Q14: “Imagine you want to find information about Work-&-Travel programs in Australia.
A Google search returned the following results. Please click on a result.”
little knowledge of ads comprehensive knowledge of ads
Figure 4: Heatmaps of subjects with little vs comprehensive knowledge of ads
5.4 Differences between the PC and smartphone condition
Figure 5 shows the fixation rates on ads for both devices. For "Text ad top 1",
n=960 is the sum of all text ads in the first position that the subjects saw on the PC.470
Each of the 20 PC SERPs of the experiment contained a “Text ad top 1”, whereby
Textadsbelowthefoldabovethefold
Textadsbelowthefoldabovethefold

35. SEARCH ENGINE BIAS IN ACTION (4):
EXTERNAL INFLUENCES
34

36. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
A RESULT OF SEARCH ENGINE OPTIMIZATION?
35

37. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
2 SEARCH ENGINE OPTIMIZATION (SEO)
External influences
• Search Engine Optimization industry now worth more than 65 billion USD (Sullivan,
2016).
36
Sullivan, L. (2016). Report: Companies Will Spend $65 Billion On SEO In 2016.
http://www.mediapost.com/publications/article/273956/report-companies-will-spend-65-billion-on-seo-in.html
Search Engine
Providers
Search Engine
Result Page
Content
ProvidersUsers
Search Engine
Optimizers

38. SEARCH ENGINE BIAS: IS THIS JUST A
“GOOGLE PROBLEM”?

39. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
AMAZON’S RANKING
38

40. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
AMAZON’S RANKING
39

41. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
RANKING IN A LIBRARY SEARCH SYSTEM
40

42. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
WHO WILL FACE (POTENTIAL) PROBLEMS WITH SEARCH
ENGINE BIAS?
As soon as search systems become platforms (i.e., allowing third-party data into the
system),
• data providers will have in interest in promoting their data
• platform providers will have an interest in favouring certain results
Some examples:
• Marketplaces (e.g., Amazon – “Amazon SEO”)
• Content aggregators (e.g., Lexis-Nexis, Expedia)
• Library information systems (commercial like Primo, not-for-profit like Econbiz)
• Academic search systems (e.g., Google Scholar – ”Academic Search Engine
Optimization”)
41

43. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
WHAT CAN WE DO ABOUT IT?
Search system providers, developers and product owners
• Be aware of external influences that can affect the results
• De-bias search results
• Let users see contrasting views on a topic in search results
• Let users re-rank results
• Give users tools to explore and analyse search results sets
Politics and regulation
• Force search systems to be transparent and fair
• Foster alternatives to existing search engines by funding infrastructure (cf.
Lewandowski, 2019)
Users
• Understand that information literacy is key to being an informed citizen in modern
society
42
Lewandowski, D. (2019). The web is missing an essential part of infrastructure: an Open Web Index. Communications of the
ACM, 62(4), 24–27. https://doi.org/10.1145/3312479

44. CONCLUSION

45. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
KEY TAKEAWAYS
• Search engine bias is a severe problem, as search engines are a major means of
knowledge acquisition.
• Search engine bias – or, bias on the Web more generally – is a complex problem where
there probably is no one best solution solving everything.
• Search engine bias may occur with all search systems.
44

46. THANK YOU
Dirk Lewandowski
Hamburg University of Applied Sciences, Hamburg, Germany
dirk.lewandowski@haw-hamburg.de
www.searchstudies.org/dirk

47. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
REFERENCES (1)
Baeza-Yates, R. (2018). Bias on the web. Communications of the ACM, 61(6), 54–61. https://doi.org/10.1145/3209581
Ballatore, A. (2015). Google chemtrails: A methodology to analyze topic representation in search engine results. First
Monday, 20(7). Retrieved from http://www.firstmonday.org/ojs/index.php/fm/article/view/5597/4652
Bar-Ilan, J. (2006). Web links and search engine ranking: The case of Google and the query ‘Jew’. Journal of the
American Society for Information & Techology, 57(12), 1581–1589.
Lewandowski, D. (2017). Is Google Responsible for Providing Fair and Unbiased Results? In M. Taddeo & L. Floridi
(Eds.), The Responsibilities of Online Service Providers (Vol. 31, pp. 61–77). Berlin Heidelberg: Springer.
https://doi.org/10.1007/978-3-319-47852-4_4
Lewandowski, D. (2017). Users’ Understanding of Search Engine Advertisements. Journal of Information Science Theory
and Practice, 5(4), 6–25. https://doi.org/10.1633/JISTaP.2017.5.4.1
Lewandowski, D. (2019). The web is missing an essential part of infrastructure: an Open Web Index. Communications of
the ACM, 62(4), 24–27. https://doi.org/10.1145/3312479
Lewandowski, D., Kerkmann, F., Rümmele, S., & Sünkler, S. (2018). An empirical investigation on search engine ad
disclosure. Journal of the Association for Information Science and Technology, 69(3), 420–437.
https://doi.org/10.1002/asi.23963
Lewandowski, D., & Sünkler, S. (2013). Representative online study to evaluate the revised commitments proposed by
Google on 21 October 2013 as part of EU competition investigation AT.39740-Google Report for Germany.
Liu, Y., Wang, C., Zhou, K., Nie, J., Zhang, M., & Labs, Y. (2014). From Skimming to Reading : A Two-stage Examination
Model for Web Search. In Proceedings of the 23rd ACM International Conference on Conference on Information and
Knowledge Management (pp. 849–858). https://doi.org/10.1145/2661829.2661907
46

48. FAKULTÄT DMI, DEPARTMENT INFORMATION
Prof. Dr. Dirk Lewandowski
REFERENCES (2)
Noble, S. U. (2018). Algorithms of Oppression: How Search Engines Reinforce Racism. New York, USA: New York
University Press.
Otterbacher, J., Bates, J., & Clough, P. (2017). Competent Men and Warm Women. In Proceedings of the 2017 CHI
Conference on Human Factors in Computing Systems - CHI ’17 (pp. 6620–6631). New York, New York, USA: ACM Press.
https://doi.org/10.1145/3025453.3025727
Purcell, K., Brenner, J., & Raine, L. (2012). Search Engine Use 2012. Washington, DC. Retrieved from
http://pewinternet.org/~/media/Files/Reports/2012/PIP_Search_Engine_Use_2012.pdf
Sullivan, L. (2016). Report: Companies Will Spend $65 Billion On SEO In 2016.
http://www.mediapost.com/publications/article/273956/report-companies-will-spend-65-billion-on-seo-in.html
Tavani, H. (2012, August 27). Search Engines and Ethics. Retrieved from http://plato.stanford.edu/entries/ethics-search/
Westerwick, A. (2013). Effects of Sponsorship, Web Site Design, and Google Ranking on the Credibility of Online
Information. Journal of Computer-Mediated Communication, 18(2), 80–97. https://doi.org/10.1111/jcc4.12006
White, R. W., & Horvitz, E. (2009). Cyberchondria. ACM Transactions on Information Systems, 27(4), Article No. 23.
https://doi.org/10.1145/1629096.1629101
47