Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Build a Truth Engine

1,221 views

Published on

Civilizations live or die by the quality of their fundamental truths. Governments, legal systems, commerce, industry, engineering, science, and education dependent upon verifiable and hard-won knowledge that has been tested and continually honed as new discoveries and revelations based on tested evidence and experience have emerged. However, truth, facts and knowledge are now under deliberate attack and suffering accelerating rates of distortion and corruption.

Today; establishing ‘the truth’ and/or applying verifiable facts and knowledge is no longer easy or straightforward. Modern media, the internet, social networks, has given everyone a voice, and a right to express their opinion even if they are totally ignorant. So, plagiarism, errors, and deliberate falsifications have become a new tool for some political and commercial operations. In effect, it has become a new form of information warfare!

Truth is no longer something we can simply take for granted; it is something we have to continuously check and verify. Populations find it far easier to unthinkingly accept simple (and often crude) misrepresentations and lies from anonymous sources, bogus median other pernicious sources including criminals, rogue states and corrupt political groups. And at the click of a key, they can distribute the falsehoods to millions of screens where quantity and not quality define a new truth!

“A lie can travel halfway around the world while the truth is putting on its shoes”
Charles Spurgeon


With the leader for the free world leading the charge into this ‘Post Truth Dystopia’ it seems that every dictator and despot feels emboldened to leverage this new degree of freedom. Even some regular politicians, bankers and business leaders are straying from the ‘straight-and-narrow’ supported by portions of the media. This is not only dangerous it can only lead to an indeterminate transformation of societies and the destruction of democracies. It really is a big step toward dystopian futures that may be irreversible!

So what can be done? How can we counter this? Fight-fire-with-fire; use the same technologies employed by the liars and distorters of the truth would seem to be the obvious course. And this, in turn, leads to the notion of a ‘Truth Engine’ capable of purging the net, checking the veracity of every media report, blog and social posting in text, voice, video, pic, and any other format in near real-time….but is it even possible? This slide set suggests that it is and many of the essential components already exist!

Published in: Data & Analytics
  • Be the first to comment

How to Build a Truth Engine

  1. 1. HOW TO BUILD A ‘TRUTH ENGINE’ P e t e r C o c h r a n e c o c h r a n e . o r g . u k
  2. 2. WORLD Loss of freedom of speech Legal system collapse Societal Breakdown Loss of Liberty Dictatorship Police State Despotism ++++ Lies Half Truths Misinformation Perceptual Distortion Character Assassination Social Distortions Disillusionment Rumour Mills Dogma Distrust Fake News Censorship Media Control Societal Division Democratic Failure Government Intervention
  3. 3. S U P R E M U M? B e t t e r t h a n a n y o t h e r U S P re s i d e n t e ve r ! T h e ‘ l e a d e r ’ o f t h e f r e e w o r l d h a s s e t a n e w s t a n d a r d t h a t o t h e r s n o w f e e l f r e e t o f o l l o w ! We n o w h a v e a p a n d e m i c o f l i e s i n p o l i t i c s , m e d i a a n d b u s i n e s s + + +
  4. 4. USA LEADING THE WAY ? Social division, breakdown of law and order The Cult of Trump
  5. 5. USA LEADING THE WAY ? Social division, breakdown of law and order The Cult of Trump
  6. 6. E N E M Y W I T H I N Routine lies and info/data distortion Leaders Bankers Politicians Managers Activists Media
  7. 7. E N E M Y W I T H O U T Using a new; powerful weapon of war Ideological subversion: Changing the perceptions of populations to such an extent that despite an abundance of data/information no one can to come to sensible conclusions to defend themselves/families, their community/country”  Demoralisation (15 to 20 years) Re-educate one generation of students and change their perceptions and thinking Destabilisation (2 - 5 years) Target essential structural elements of a nation: economy, foreign relations, and defence systems Normalisation (4 weeks) Take over = people living under a new ideology and reality Four Basic Stages Crisis (6 weeks) Violent disruption to, and change of: power, structure, and economy
  8. 8. LANDSCAPE of liars Banks Media Moguls Groups Criminals Politicians Terrorists Individuals Institutions Industrialists Rogue States Governments Political Factions +++++ Hackers Despots Dictators Reporters Executives Extremists Companies Conspirators The Paranoid The Vulnerable The Disaffected The Mischievous The Disadvantaged +++++ Elusive Amoral Corrupt Dynamic Dedicated Unethical Disguised Nefarious Transient Relentless Dedicated Networked Distributed Untraceable Multi-Media Evolutionary Camouflaged Multi-Faceted Multi-Dimensional
  9. 9. T H E N AT U R E O F T RUT H Truth = A Dynamic Binary Truth = Can be Certain Truth = Can be Dynamic Truth = Can be Uncertain Concatenating 1/2 truths lends great credibility…
  10. 10. EXTREME W EAPONS S p o o f O b a m a s a y i n g a n y t h i n g !
  11. 11. COMPOUNDED
  12. 12. H U M A N I G N O RA N C E Apparently unlimited and unbounded It takes far less thought, effort and energy to dismiss wisdom and deny a truth than blindly accepting the ridiculous Truths are hard won: and often very difficult to establish and accept. They can demand deep thought, energy, and freedom of thought, an acceptance of debate, reason and education!
  13. 13. H U M A N I G N O RA N C E Apparently unlimited and unbounded It takes far less thought, effort and energy to dismiss wisdom and deny a truth than blindly accepting the ridiculous Truths are hard won: and often very difficult to establish and accept. They can demand deep thought, energy, and freedom of thought, an acceptance of debate, reason and education! Oh, and by-the-way: man d i d n o t l a n d o n t h e m o o n ! I t w a s a l l a government spoof and it was filmed in a studio!!
  14. 14. P r o u d t o b e i g n o r a n T Knowledge & expertise derided
  15. 15. Bigger and propagate faster, further, and deeper than the truth and validated information….and now growing in volume, extremism and span Sim ple & Easy to Grasp Plays to DispositionsLIES & FAKE NEWS Creates social division, social instability, unease, dissatisfaction, paranoia, extremism, rebellion, social instability leading to violence… No Thinking Required Reinforces Prejudices
  16. 16. Bigger and propagate faster, further, and deeper than the truth and validated information….and now growing in volume, extremism and span Sim ple & Easy to Grasp Plays to DispositionsLIES & FAKE NEWS Creates social division, social instability, unease, dissatisfaction, paranoia, extremism, rebellion, social instability leading to violence… No Thinking Required Reinforces Prejudices A lie can travel round the world and back again whilst truth is lacing up its boots MarkTwain 1921
  17. 17. FAKE NEWS
  18. 18. MergED
  19. 19. I G N O R A N C E & L I ES T h e b i g g e s t t h r e a t & c h a l l e n g e e v e r ?
  20. 20. Science, Discovery and Fundamental Truths Technology, Engineering and Infrastructures Farming, Food, Industries Institutions and Defence B a l a n c i n g A C T Government Media, Society No place for lies & dubious truths
  21. 21. Science, Discovery and Fundamental Truths Technology, Engineering and Infrastructures Farming, Food, Industries Institutions and Defence Lies Untruths Errors Guesses Distortions Cross Checks Evidence Base Understanding Knowledge Theories Models Tests B a l a n c i n g A C T Government Media, Society No place for lies & dubious truths
  22. 22. N ot h i n g i s i s o l at e d E v e r y t h i n g a n d e v e r y o n e i s c o n n e c t e d Ecology Economics Technology Society Care Markets Industry Education Policing Defence Commerce Security Logistics ing Co ies “In networked and fast moving world simple minded thinking, approximations, ignorance and lies are dangerous - they destroy companies and countries” Trade
  23. 23. W h at To D o ? Fight fire with fire: use the same technologies/techniques but on a fully automated and global scale Educating people to think may be fundamentally impossible! Filtering the web of non validated content is/ may be possible but not risk free!
  24. 24. E X P L O I TAT I O N A new; powerful weapon of war Continually scan the internet: - Purge errors - Flag lies & liars - Publish veracity ratings - Remove/Attenuate rogue users - Block/Remove/Attenuate extreme channels - Identify/Advertise regular/persistent offenders - Attenuate/Attack/Disable criminal/rogue states - Detect an disconnect postings by AI lie generators - Isolate and mute rogue states and criminals
  25. 25. E X P L O I TAT I O N A new; powerful weapon of war Continually scan the internet: - Purge errors - Flag lies & liars - Publish veracity ratings - Remove/Attenuate rogue users - Block/Remove/Attenuate extreme channels - Identify/Advertise regular/persistent offenders - Attenuate/Attack/Disable criminal/rogue states - Detect an disconnect postings by AI lie generators - Isolate and mute rogue states and criminals Key Enabler = TRUTH ENGINES
  26. 26. T H E B I G Q U ES T I O N S Can we accurately and reliably identify truths, half truths, distortions, falsehoods and lies ? Can we establish new framework and metrics ? Can we make judgement calls fast enough ? Can we block/attenuate the pernicious ? Can we continuously cleanse the net ? Can we educate society to the risks ?
  27. 27. T H E B I G Q U ES T I O N S Can we accurately and reliably identify truths, half truths, distortions, falsehoods and lies ? Can we establish new framework and metrics ? Can we make judgement calls fast enough ? Can we block/attenuate the pernicious ? Can we continuously cleanse the net ? Can we educate society to the risks ? ✔ ✔ ✔ ? ✔ ? ? ? ? ? ? ?
  28. 28. W H Y T r u t h E n g i n e s Easy to confirm facts - conjecture is harder Vital for: • Politics • Society • Science • Farming • Internet • Security • Industry • Education • Healthcare • Commerce • Government • ++++ “In most arenas truth is dynamic and changes with our ability to test & measure” “Accepted truths advance toward an absolute state with our ability to challenge them” Ignorance & Uncertainty Flat Spheroid Flat Spheroid Oblate Spheroid Dynamic Oblate Spheroid Precision Measurement & Truth
  29. 29. 1) Blatant: Knowingly contrived and delivered information 2) White: Knowingly contrived in order to be polite or to stop someone from being upset by the truth 3) Propagating: Repeat a stated lie without fact checking 4) Falsehood: To knowingly distort the truth or facts 5) Misinterpretation: Knowingly or unknowingly distort the truth or facts 6) Error: Innocently distort situations, events, truth or facts 7) Latent Error: Representing dynamic situations, occurrences, truth or facts on the basis of old information L I ES
  30. 30. E x p e r i m e n ts 8) Bayesian Inference: Detecting a lie from linguistic patterns and previous behaviours of individuals and populations? 9) Simple AI: Can we detect and/or identify linguistic patterns and/or previous behaviours of individuals and populations? 10) Correlations: Can we detect repeating patterns for known lies and liars? 11) Trending Analysis: Can we establish historical analysis of past reporting as a reliable measure of accuracy? 12) Lie Recognition: Can we devise metrics from (8-11) above to detect an lie and then categorise it aka (1-7) ?
  31. 31. TRUTH Engine Is it possible to build one?
  32. 32. TRUTH Engine What do we have & what do we have to develop ? How big is this problem ? Does it look tractable ? Might we crack it ? Is it really mission critical ? What is it going to cost ? How long will it take ? Truth is vital for our planet and the survival of all life forms including us ✔ Is it possible to build one?
  33. 33. W h at W e N e e d Print Radio TV On ScreenSpokes People Context - Meaning Extractor - Inference Engine Fact Checkers - Primary Source Identification/Proximity Style - Behaviours, History, Publications Analysis Trending Analysis - Veracity % Rating ✔ ✘ ✔ ✔
  34. 34. CO R E QU EST I O NS Veracity, accuracy, credibility testing Who is the author ? Why did they publish ? Who do they work for ? What is their reputation ? Are they paid/sponsored ? Do they have a following ? Do they cite trusted sources ? Do they have relevant expertise ? What is their purpose/motivation ? Is the material current/outdated ? What site/s is/are being used ? ++++++
  35. 35. CO R E QU EST I O NS Veracity, accuracy, credibility testing Who is the author ? Why did they publish ? Who do they work for ? What is their reputation ? Are they paid/sponsored ? Do they have a following ? Do they cite trusted sources ? Do they have relevant expertise ? What is their purpose/motivation ? Is the material current/outdated ? What site/s is/are being used ? ++++++ Corroboration ! Fact Checking ! Correlation ! Likelihood ! Confidence Trending ! History ! C O U L D W E A U T O M A T E A L L T H I S
  36. 36. S o u r c e S e a r c h Original accounts v repeats Identify primary sources and qualified reports… A very small number of sources can trigger a massive amount of activity!
  37. 37. T H E DY N A M I CS Information spreading world-wide Identify primary sources and all qualified reports… not always possible…especially when a story is developing rapidly
  38. 38. T H E DY N A M I CS Information spreading world-wide Errors, distortions, interpretations, air time availability, sensationalism, number of reports disconnected from any real significance, sees distorted perceptions Identify primary sources and all qualified reports… not always possible…especially when a story is developing rapidly
  39. 39. C h e c k i n g Easy To Do ? Snopes Politifact FactCheck Wikipedia ProPublica MBFCNews OpenSecrets NewsBusters MediaMatters FactCheck.org HoaxSlayer.net TruthOrFiction.com Washington Post Fact Checker +++++ People intensive
  40. 40. C h e c k i n g Easy To Do ? Snopes Politifact FactCheck Wikipedia ProPublica MBFCNews OpenSecrets NewsBusters MediaMatters FactCheck.org HoaxSlayer.net TruthOrFiction.com Washington Post Fact Checker +++++ People intensive AGGREGATION Fact Checking Organisations Now exceed 160 World-Wide Collaboration, aggregation, sharing, and networking between checker projects underway in >50 countries
  41. 41. C h e c k i n g Easy To Do ? Snopes Politifact FactCheck Wikipedia ProPublica MBFCNews OpenSecrets NewsBusters MediaMatters FactCheck.org HoaxSlayer.net TruthOrFiction.com Washington Post Fact Checker +++++ People intensive AGGREGATION Fact Checking Organisations Now exceed 160 World-Wide Collaboration, aggregation, sharing, and networking between checker projects underway in >50 countries R i p e f o r A u t o m a t i o n
  42. 42. TWO FoR you TO TRY ! Both are fast and powerful - primary tools for building an automated truth engine !
  43. 43. Across multiple sources - locations C o r r e l at i o n Weighted sources by reputation/history: BBC = 0.95 Bloomberg = 0.90 FoxNews = 0.55 BreitBart = 0.05 +++ +++ Twitter = 0.01 Early R&D phase but of great interest to Google, IBM, CIA, FBI, et al
  44. 44. TESTED resources ≈ ≈ Accurate 80% ~ 45% Accurate
  45. 45. TESTED resources ≈ ≈ Accurate 80% ~ 45% Accurate A F U N C T I O N O F T R U T H B E I N G D Y N A M I C
  46. 46. A comprehensive construct of every uttered, printed and recorded lie over all time is required as a reference source across all human and A I i n f o r m a t i o n sources… …does not exist as yet, but independents are starting to build the components…. L I B R A RY O F L I ES
  47. 47. A comprehensive construct of every uttered, printed and recorded lie over all time is required as a reference source across all human and A I i n f o r m a t i o n sources… …does not exist as yet, but independents are starting to build the components…. L I B R A RY O F L I ES www.thestar.com/news/world/uselection/2016/11/04/donald-trump-the-unauthorized-database-of-false-things.html http://www.davesweboflies.com/dwol.htm https://
  48. 48. A comprehensive construct of every uttered, printed and recorded lie over all time is required as a reference source across all human and A I i n f o r m a t i o n sources… …does not exist as yet, but independents are starting to build the components…. L I B R A RY O F L I ES www.thestar.com/news/world/uselection/2016/11/04/donald-trump-the-unauthorized-database-of-false-things.html http://www.davesweboflies.com/dwol.htm https:// Not hard tocompile butrequires acoordinatedAUTOMATED global efforT
  49. 49. A comprehensive construct of every uttered, printed and recorded lie over all time is required as a reference source across all human and A I i n f o r m a t i o n sources… …does not exist as yet, but independents are starting to build the components…. L I B R A RY O F L I ES www.thestar.com/news/world/uselection/2016/11/04/donald-trump-the-unauthorized-database-of-false-things.html http://www.davesweboflies.com/dwol.htm https:// Human LEGAL POLITIcS AND BEUROCRACY CONSTRAINTS ARE THE PRIMELIMITERS Not hard tocompile butrequires acoordinatedAUTOMATED global efforT
  50. 50. ANALYSIS Could we rank all sources, individuals and media in order of truthfulness and accuracy based on validated facts and information over a significant time by a u t o m a t i n g t h e process to give a confidence rating or uncertainty % ? 100%50%0% 100% 50% 0%
  51. 51. ANALYSIS Could we rank all sources, individuals and media in order of truthfulness and accuracy based on validated facts and information over a significant time by a u t o m a t i n g t h e process to give a confidence rating or uncertainty % ? 100%50%0% 100% 50% 0% C o u l d w e A U T O e v a l u a t e t h e E n t i r e w e b i n r e a l t i m e n o n s t o p
  52. 52. s e n s at i o n a l i s m F u k u s h i m a g r o s s o v e r e s t i m a t e s Feb 27, 2017 Fukushima prefecture tallied 2,129 “disaster-related deaths” 1368 have been listed as "related to the nuclear power plant” One worker is acknowledged to have died from radiation induced illness The Tsunami killed over 15,000 people
  53. 53. s e n s at i o n a l i s m F u k u s h i m a g r o s s o v e r e s t i m a t e s Feb 27, 2017 Fukushima prefecture tallied 2,129 “disaster-related deaths” 1368 have been listed as "related to the nuclear power plant” One worker is acknowledged to have died from radiation induced illness The Tsunami killed over 15,000 people Assessing cases like this is easy but perplexing as truth is not important to such sources
  54. 54. T H E DY N A M I CS Identify primary sources and qualified reports… Errors, distortions, interpretations, air time availability, sensationalism, number of reports disconnected from any real significance, sees distorted perceptions Information spreading world-wide at a server level - 2D only Identify primary sources and qualified reports… very difficult
  55. 55. T H E DY N A M I CS Identify primary sources and qualified reports… Errors, distortions, interpretations, air time availability, sensationalism, number of reports disconnected from any real significance, sees distorted perceptions Information spreading world-wide at a server level - 2D only Identify primary sources and qualified reports… very difficult B O T H c a s e s s a w s i g n i f i c a n t e r r o r p r o p a g a t i o n
  56. 56. 774 Deaths in 37 countries 457 Deaths ~11,000 Deaths ~300 Deaths Mole Hills & MOUNTAINS H5N1 = 10 Deaths NO - we are not all going to die! 3 DeathsZip 42 Deaths Not Proven
  57. 57. DANGEROUS POLITICAL AND MEDIA MANTRA Facts and Truths don't actually matter! People repeat falsehoods so often that you/they believe them This is the “illusory truth effect,” where people equate repetition with truth Exploited by advertisers, marketeers, politicians, despots et al to the nth degree
  58. 58. B e y o n d H u m a n C a p a b i l i t y D E L AY E D T I M E L I N E Applied across all sources continuously Reported Event
  59. 59. B e y o n d H u m a n C a p a b i l i t y D E L AY E D T I M E L I N E Applied across all sources continuously Reported Event Established Truth
  60. 60. B e y o n d H u m a n C a p a b i l i t y D E L AY E D T I M E L I N E Applied across all sources continuously Reported Event Established Truth Accuracy Rating 100% 85% 95%90%
  61. 61. B e y o n d H u m a n C a p a b i l i t y D E L AY E D T I M E L I N E Applied across all sources continuously Reported Event Established Truth Accuracy Rating 100% 85% 95%90% All Time Aggregated Trust Rating100% 95% 91.7% 92.5%
  62. 62. T R U t h Q u A l i t y Degree of Lie Error Satire Parody Falshood Distortion Manipulation Misrepresentation 100% True Time Report 1 Report 2 Report 3 Report 4 BBC, PBS Times, NY Times, Reuters, Economist, WSJ Washington Post et al Fact check, recant, recall restate, refine, make every effort to get to the truth
  63. 63. T R U t h Q u A l i t y Degree of Lie Error Satire Parody Falshood Distortion Manipulation Misrepresentation 100% True Time Report 1 Report 2 Report 3 Report 4 BBC, PBS Times, NY Times, Reuters, Economist, WSJ Washington Post et al Fact check, recant, recall restate, refine, make every effort to get to the truth Sun, Daily Mail F o x , B u z z f e e d , Breitbart, Trump, Infowars et al No fact checking, political agenda, truth has little or no value - perception distortion is the goalIsolated Erroneous Reports Beware Russia Today et al
  64. 64. M a n u a l R es u lts Degree of Lie Error Satire Parody Falshood Distortion Manipulation Misrepresentation 100% True Time Report 1 Report 2 Report 3 Report 4 Washington Post ~ 98.4% BBC, Economist ~ 98.2% Times, Reuters ~ 98.1% NY Times, WSJ ~ 97.5%
  65. 65. M a n u a l R es u lts Degree of Lie Error Satire Parody Falshood Distortion Manipulation Misrepresentation 100% True Time Report 1 Report 2 Report 3 Report 4 Washington Post ~ 98.4% BBC, Economist ~ 98.2% Times, Reuters ~ 98.1% NY Times, WSJ ~ 97.5% A u t o m a t i n g T h i s i s n o t g o i n g t o b e e a s y b u t i t l o o k s p o s s i b l e
  66. 66. T H E A M B I T I O N Facts Ethical History Honesty Pedigree Checked Rectitude Traceable Authentic Legitimate Actuality Genuine Veracity Verified Correct Certain Maxin Exact ++++ Misrepresentation Misinterpretation Disinformation Manipulation Subterfuge Vilification Ignorance Invention Mistakes Delusion Rumour Malice ++++ Myth Fable Fraud Errors Fiction Forgery Distortion Deception Falsehood Fabrication Inaccuracy Dishonesty Falsehoods Data Monitoring, Farming, Tracking, Measures, Metrics, Analysis, Decisions and Confidence Truth Lies A I ? Decisions to be automated & fast The only technology we have that might just do it is AI
  67. 67. SO W HERE ARE W E ? Many components exist or we can see how we might build them ! Context/ Meaning Extractor Inference Engine Primary Source Identifier
  68. 68. SO W HERE ARE W E ? Many components exist or we can see how we might build them ! Context/ Meaning Extractor Inference Engine Primary Source Identifier Fact Checkers
  69. 69. SO W HERE ARE W E ? Many components exist or we can see how we might build them ! Context/ Meaning Extractor Inference Engine Primary Source Identifier Fact Checkers Style History Publications Behaviours Library of Lies
  70. 70. SO W HERE ARE W E ? Many components exist or we can see how we might build them ! Context/ Meaning Extractor Inference Engine Primary Source Identifier Fact Checkers Style History Publications Behaviours Library of Lies Trending Analysis Veracity Rating
  71. 71. H U M A N B I A S Build multiple Truth Engines Different configurations, components, AI Run continuous comparisons on the same data/cases and to identify anomalies We are talking joint human-AI opinion, judgement and assessment !
  72. 72. Critical Thinking Is it now nullified ?? Dominant Mode ? “I’m right you are wrong, and so you are an idiot” What happened to the art of debate, tolerance and thinking ?
  73. 73. Critical Thinking Is it now nullified ?? Dominant Mode ? “I’m right you are wrong, and so you are an idiot” What happened to the art of debate, tolerance and thinking ? O p I n i o n h a s l i t t l e v a l u e
  74. 74. Critical Thinking Is it now nullified ?? Dominant Mode ? “I’m right you are wrong, and so you are an idiot” What happened to the art of debate, tolerance and thinking ? O p I n i o n h a s l i t t l e v a l u e Q u a l i f i e d A n a l y t i c s c a n n o w h a v e e v e n l e s s
  75. 75. Not an option for science, the law engineering, medicine, industry, banking, commerces and any society aspiring to prosperity, peace and longevity Bigotry: intolerance towards those who hold different opinions from oneself BELIEF SYSTEMS Damaging decisions based on false data can only lead to a societal collapse D i s c o u ra g e t h i n k i n g a n d d e b a t e
  76. 76. T H E U LT I M AT E F I X D o w e n e e d a R E D o r a B l u e p i l l ?
  77. 77. The Voight-Kampff Test - Replicant orHuman? Rachel Blade Runner 1982 Ridley Scott Thank You www. cochrane.org.uk www.slideshare.net/PeterCochrane

×