SlideShare a Scribd company logo
Accommodating the Deep Learning
Revolution by a Development Process
Methodology
Jochen L. Leidner
Coburg University of Applied Sciences and Arts, Coburg, Germany
KnowledgeSpaces®
UG (haftungsbeschränkt), Coburg, Germany
University of Sheffield, Sheffield, UK
2022-10-11
Overview
●
Introduction: Motivation, Pre-Trained Language Model Revolution
●
Quick Recap: Some Machine Learning Methodologies (CRISP-DM, KDD, SEMMA, Data-to-Value)
●
Before and After Pre-Trained LMs
●
Comparison: Where Project Work is Spent: Pre-BERT and Post-BERT
●
A Comment about Energy
●
Summary & Conclusion
A Step Change in NLP: Deep Learning and Pre-Trained Language Models
●
In recent years, Pre-Trained Language Models (PTLMs) like Google’s BERT have emerged (Devlin et
al., 2018/2019).
●
This led to enormous improvements in terms of accuracy in most NLP tasks.
●
PTLMs show that transfer learning is possible by splitting up training into two phases.
BERT: An Example Pretained Neural Language Model – Pre-Training versus Fine-Tuning
 Two Training Phases:
– Pre-training: train deep neural network with masked sentence pairs on generic language (billions of words
from books, Wikipedia)
– Fine-tuning: adapt generic LM to specific task (e.g. question answering) using supervised learning (extra
rounds on top of pre-trained LM)
Practical Questions
●
RQ 1. How do PTLMs change the way NLP projects are done?
●
RQ 2. In particular, How do PTLMs interact with existing methodologies?
Some Methodologies
●
KDD
●
CRISP-DM Azevedo and Santos (2008)
●
SEMMA
●
Data-to-Value (Leidner, 2013; Leidner 2022a,b)
The Data-to-Value Methodology (Leidner, 2013; Leidner 2022a,b) (1 of 2)
The Data-to-Value Methodology (Leidner, 2013; Leidner 2022a,b) (2 of 2)
Minor fine-tuning
sufficient
Before and After PTLMs
Before:
●
Any classifier/regressor a bespoke activity
(100% custom development from scratch)
●
Relatively slow and expensive to build
●
Knobs: more labelled training data, more
features
After:
●
Classifiers can be derived from PTLMs
(80% re-use and 20% custom dev. →fine-tuning)
●
Rapid/agile prototyping, cheap to get started
●
Knobs: more unlabelled training data, more
labelled training data, 3 training regimes:
– Zero-shot (apply PTLM as-is)
– Fine tuning only (take pre-trained LM and
add a few hundred training rounds using
annotated data)
– Pre-training (huge unlabelled data) and fine-
tuning (small labelled data)
Increasing effort
Comparison: How Time May Be Spent – Before and After PTLMs
Before:
●
Data Collection & Pre-Processing 70%
●
Annotation 10%
●
Feature Engineering 10%
●
Model Training 7%
●
Evaluation 3%
After:
●
Data Collection & Pre-Processing 50% - 70%
●
Annotation 2% - 10%
●
Feature Engineering 0%
●
Model Training 0% - 12%
●
Evaluation 3%
Percentages are estimates (an empirical
study is needed but hard to obtain); ranges
reflect training regimes
symbolizes
size of the
project
Deep Learning & Energy Consumption
●
Pre-training neural models is resource-intensive (Strubell,
Ganesh and McCallum, 2019).
●
Individual estimates vary, but cloud cost
and environmental footprint are issues.
●
While experiments show that “bigger is better“ (in terms of F1),
there is a research drive to “distill“ smaller models.
Summary & Conclusions
●
PTLMs have made NLP projects more agile.
– While more unlabelled data may be needed, less labelled data may be required (sufficient data is
sometimes unavailable in industrial practice).
– Most importantly, the feature engineering cycle is removed from projects.
– PTLMs offer 3 training regimes: zero-shot, tune-tuning and pre-training with increasing cost/effort.
●
As artifacts they are also more clunky and energy-inefficient.
●
Implications:
– Research: Increasingly bigger models means some academic teams excluded from research (requires
expensive GPU clusters) → research moves to industry (similar to semiconductor space).
– Business: Public availability of PTLMs generates more level playing-ground, makes competitive
differentiation harder and reduces barriers to entry.
References
●
Devlin, Jacob, Ming-Wei Chang, Kenton Lee and Kristina Toutanova (2018) BERT: Pre-training of Deep
Bidirectional Transformers for Language Understanding, Technical Report/Unpublished ArXiv Pre-print,
https://arxiv.org/abs/1810.04805.
●
Devlin, Jacob, Ming-Wei Chang, Kenton Lee and Kristina Toutanova (2019) "BERT: Pre-training of Deep
Bidirectional Transformers for Language Understanding" Proc. NAACL-HLT, Minneapolis.
●
Azevedo, A. and Santos, M. F. (2008) "KDD, SEMMA and CRISP-DM: a parallel overview", Proc. IADIS
European Conference on Data Mining, Amsterdam, 24-26 July 2000, 182–185.
●
Leidner, Jochen L. (2013) “Data-to-Value“, unpublished lecture notes Big Data and Language Technology,
University of Zurich, Zurich, Switzerland.
●
Leidner, Jochen L. (2022a) Data-to-Value: An Evaluation-First Methodology for Natural Language Projects,
Technical Report/Unpublished ArXiv Pre-print https://arxiv.org/abs/2201.07725.
●
Leidner, Jochen L. (2022b) "Data-to-Value: An Evaluation-First Methodology for Natural Language Projects",
Proceedings of the 27th International Conference on Natural Language & Information Systems (NLDB 2022),
Valencia, Spain, 15-17 June 2022, LNCS 13286, 517–523.
●
Strubell, Emma, Ananya Ganesh and Andrew McCallum (2019) "Energy and Policy Considerations for Deep
Learning in NLP", ArXiv pre-print, https://arxiv.org/pdf/1906.02243.pdf .
Accommodating the Deep Learning Revolution by a Development Process Methodology
Word embeddings, deep learning, transformer models and other pre-trained neural language models
(sometimes recently referred to as "foundational models") have fundamentally changed the way state-of-the-
art systems for natural language processing and information access are built today. The "Data-to-Value"
process methodology (Leidner 2013; Leidner 2022a,b) has been devised to embody best practices for the
construction of natural language engineering solutions; it can assist practitioners and has also been used to
transfer industrial insights into the university classroom.
This talk recaps how the methodology supports engineers in building systems more consistently and then
outlines the changes in the methodology to adapt it to the deep learning age. The cost and energy
implications will also be discussed.
Abstract
About the Presenter
Prof. Dr. Jochen L. Leidner, M.A. M.Phil. Ph.D. FRGS is Professor for Explainable and Responsible Artificial
Intelligence in Insurance at Coburg University of Applied Sciences, a Visiting Professor in the Department of
Computer Science, University of Sheffield. He is also founder and CEO of KnowledgeSpaces.
His experience includes positions as Director of Research at Thomson Reuters and Refinitiv in London,
where he headed its R&D team, which he founded He was also the Royal Academy of Engineering Visiting’
Professor of Data Analytics at the Department of Computer Science, University of Sheffield (2017-2020).
His background includes a Master's in computational linguistics, English and computer science (University of Erlangen-Nuremberg), a
Master's in Computer Speech, Text and Internet Technology (University of Cambridge) and a PhD in Informatics (University of Edinburgh),
which won the first ACM SIGIR Doctoral Consortium Award.
His scientific contributions include leading the teams that developed the QED and ALYSSA open-domain question answering Systems
(evaluated at US NIST/DARPA TREC), proposing a new algorithm and comparing existing algorithms for spatial resolution of named
entities, and information extraction of usual and unusual things (e.g. event extraction, company risk mining, sentiment analysis).
At Thomson Reuters he has led projects in the vertical comains of finance, regulatory/law enforcement, legal, pharmacology, and news. His
code and machine learning models have been transitioned into products deployed at institutions ranging from international banks to the
U.S. Supreme Court.
Prior to Thomson Reuters, he has worked for SAP and founded and co-founded a number of start-ups. He has lived and worked in
Germany, Scotland, the USA, Switzerland and the UK, and has taught at various universities (Erlangen, Saarbrücken, Frankfurt, Zurich and
now Coburg), and is a scientific expert for the European Commission (FP7, H2020, Horizon Europe) and other funding bodies. He is an
author or co-author of several dozen peer-reviewed publications (incl. one best paper award), has authored/co-edited two books and holds
several patents in the areas of information retrieval, natural language processing, and mobile computing.
He has been twice winner of the Thomson Reuters inventor of the year award for the best patent application.
About KnowledgeSpaces®
●
Contact for consulting:
E-Mail:
info@knowledgespaces.de
Phone:
+49 (172) 904 8908

More Related Content

Similar to AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Process Methodology Jochen Leidner (Coburg University, Germany)

siegelCV.doc.doc
siegelCV.doc.docsiegelCV.doc.doc
siegelCV.doc.doc
butest
 
siegelCV.doc.doc
siegelCV.doc.docsiegelCV.doc.doc
siegelCV.doc.doc
butest
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Data Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data ScienceData Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data Science
Pouria Amirian
 
Data Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data ScienceData Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data Science
Pouria Amirian
 
Industry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringIndustry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software Engineering
Per Runeson
 
Technology supported requirement handling an estimation
Technology supported requirement handling an estimationTechnology supported requirement handling an estimation
Technology supported requirement handling an estimation
Kjetil Moløkken-Østvold
 
Data-X-v3.1
Data-X-v3.1Data-X-v3.1
Data-X-v3.1
Ikhlaq Sidhu
 
DataScience SG | Undergrad Series | 26th Sep 19
DataScience SG | Undergrad Series | 26th Sep 19DataScience SG | Undergrad Series | 26th Sep 19
DataScience SG | Undergrad Series | 26th Sep 19
Yong Siang (Ivan) Tan
 
DRESD Project Presentation - December 2006
DRESD Project Presentation - December 2006DRESD Project Presentation - December 2006
DRESD Project Presentation - December 2006
santa
 
Data-X-Sparse-v2
Data-X-Sparse-v2Data-X-Sparse-v2
Data-X-Sparse-v2
Ikhlaq Sidhu
 
Full resume dr_russell_john_childs_2016
Full resume dr_russell_john_childs_2016Full resume dr_russell_john_childs_2016
Full resume dr_russell_john_childs_2016
Russell Childs
 
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
Jinho Choi
 
Guia 2-examen-de-ingles
Guia 2-examen-de-inglesGuia 2-examen-de-ingles
Guia 2-examen-de-ingles
Liz Castro B
 
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docxHorton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
wellesleyterresa
 
Cultivating Sustainable Software For Research
Cultivating Sustainable Software For ResearchCultivating Sustainable Software For Research
Cultivating Sustainable Software For Research
Neil Chue Hong
 
Presentation 2019 08-30
Presentation 2019 08-30Presentation 2019 08-30
Presentation 2019 08-30
Mahdi_Fahmideh
 
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
Billy Kid
 
Goal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
Goal Decomposition and Abductive Reasoning for Policy Analysis and RefinementGoal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
Goal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
Emil Lupu
 
Tds — big science dec 2021
Tds — big science dec 2021Tds — big science dec 2021
Tds — big science dec 2021
Gérard Dupont
 

Similar to AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Process Methodology Jochen Leidner (Coburg University, Germany) (20)

siegelCV.doc.doc
siegelCV.doc.docsiegelCV.doc.doc
siegelCV.doc.doc
 
siegelCV.doc.doc
siegelCV.doc.docsiegelCV.doc.doc
siegelCV.doc.doc
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Data Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data ScienceData Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data Science
 
Data Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data ScienceData Science as a Service: Intersection of Cloud Computing and Data Science
Data Science as a Service: Intersection of Cloud Computing and Data Science
 
Industry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringIndustry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software Engineering
 
Technology supported requirement handling an estimation
Technology supported requirement handling an estimationTechnology supported requirement handling an estimation
Technology supported requirement handling an estimation
 
Data-X-v3.1
Data-X-v3.1Data-X-v3.1
Data-X-v3.1
 
DataScience SG | Undergrad Series | 26th Sep 19
DataScience SG | Undergrad Series | 26th Sep 19DataScience SG | Undergrad Series | 26th Sep 19
DataScience SG | Undergrad Series | 26th Sep 19
 
DRESD Project Presentation - December 2006
DRESD Project Presentation - December 2006DRESD Project Presentation - December 2006
DRESD Project Presentation - December 2006
 
Data-X-Sparse-v2
Data-X-Sparse-v2Data-X-Sparse-v2
Data-X-Sparse-v2
 
Full resume dr_russell_john_childs_2016
Full resume dr_russell_john_childs_2016Full resume dr_russell_john_childs_2016
Full resume dr_russell_john_childs_2016
 
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
The Pupil Has Become the Master: Teacher-Student Model-Based Word Embedding D...
 
Guia 2-examen-de-ingles
Guia 2-examen-de-inglesGuia 2-examen-de-ingles
Guia 2-examen-de-ingles
 
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docxHorton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
Horton+Pruim+Kaplan_MOSAIC-StudentGuide.pdf Nicholas J. .docx
 
Cultivating Sustainable Software For Research
Cultivating Sustainable Software For ResearchCultivating Sustainable Software For Research
Cultivating Sustainable Software For Research
 
Presentation 2019 08-30
Presentation 2019 08-30Presentation 2019 08-30
Presentation 2019 08-30
 
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
ICT50715 – Assignment 01 – Task 1 – Platform Research Report – SSDM (APA).2600
 
Goal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
Goal Decomposition and Abductive Reasoning for Policy Analysis and RefinementGoal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
Goal Decomposition and Abductive Reasoning for Policy Analysis and Refinement
 
Tds — big science dec 2021
Tds — big science dec 2021Tds — big science dec 2021
Tds — big science dec 2021
 

More from Dr. Haxel Consult

AI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
AI-SDV 2022: Henry Chang Patent Intelligence and Engineering ManagementAI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
AI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
Dr. Haxel Consult
 
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
Dr. Haxel Consult
 
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
Dr. Haxel Consult
 
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
Dr. Haxel Consult
 
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
Dr. Haxel Consult
 
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
Dr. Haxel Consult
 
AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...
Dr. Haxel Consult
 
AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...
Dr. Haxel Consult
 
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
Dr. Haxel Consult
 
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
Dr. Haxel Consult
 
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
Dr. Haxel Consult
 
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
Dr. Haxel Consult
 
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
Dr. Haxel Consult
 
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
Dr. Haxel Consult
 
AI-SDV 2022: Copyright Clearance Center
AI-SDV 2022: Copyright Clearance CenterAI-SDV 2022: Copyright Clearance Center
AI-SDV 2022: Copyright Clearance Center
Dr. Haxel Consult
 
AI-SDV 2022: Lighthouse IP
AI-SDV 2022: Lighthouse IPAI-SDV 2022: Lighthouse IP
AI-SDV 2022: Lighthouse IP
Dr. Haxel Consult
 
AI-SDV 2022: New Product Introductions: CENTREDOC
AI-SDV 2022: New Product Introductions: CENTREDOCAI-SDV 2022: New Product Introductions: CENTREDOC
AI-SDV 2022: New Product Introductions: CENTREDOC
Dr. Haxel Consult
 
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
Dr. Haxel Consult
 
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
Dr. Haxel Consult
 
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
Dr. Haxel Consult
 

More from Dr. Haxel Consult (20)

AI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
AI-SDV 2022: Henry Chang Patent Intelligence and Engineering ManagementAI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
AI-SDV 2022: Henry Chang Patent Intelligence and Engineering Management
 
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
AI-SDV 2022: Creation and updating of large Knowledge Graphs through NLP Anal...
 
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
AI-SDV 2022: The race to net zero: Tracking the green industrial revolution t...
 
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
AI-SDV 2022: Domain Knowledge makes Artificial Intelligence Smart Linda Ander...
 
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
AI-SDV 2022: Embedding-based Search Vs. Relevancy Search: comparing the new w...
 
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
AI-SDV 2022: Rolling out web crawling at Boehringer Ingelheim - 10 years of e...
 
AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...
 
AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...AI-SDV 2022: Machine learning based patent categorization: A success story in...
AI-SDV 2022: Machine learning based patent categorization: A success story in...
 
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
AI-SDV 2022: Finding the WHAT – Will AI help? Nils Newman (Search Technology,...
 
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
AI-SDV 2022: New Insights from Trademarks with Natural Language Processing Al...
 
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
AI-SDV 2022: Extracting information from tables in documents Holger Keibel (K...
 
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
AI-SDV 2022: Scientific publishing in the age of data mining and artificial i...
 
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
AI-SDV 2022: AI developments and usability Linus Wretblad (IPscreener / Uppdr...
 
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
AI-SDV 2022: Where’s the one about…? Looney Tunes® Revisited Jay Ven Eman (CE...
 
AI-SDV 2022: Copyright Clearance Center
AI-SDV 2022: Copyright Clearance CenterAI-SDV 2022: Copyright Clearance Center
AI-SDV 2022: Copyright Clearance Center
 
AI-SDV 2022: Lighthouse IP
AI-SDV 2022: Lighthouse IPAI-SDV 2022: Lighthouse IP
AI-SDV 2022: Lighthouse IP
 
AI-SDV 2022: New Product Introductions: CENTREDOC
AI-SDV 2022: New Product Introductions: CENTREDOCAI-SDV 2022: New Product Introductions: CENTREDOC
AI-SDV 2022: New Product Introductions: CENTREDOC
 
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
AI-SDV 2022: Possibilities and limitations of AI-boosted multi-categorization...
 
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
AI-SDV 2022: Big data analytics platform at Bayer – Turning bits into insight...
 
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
The Artificial Intelligence Conference on Search, Data and Text Mining, Analy...
 

Recently uploaded

Gen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needsGen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needs
Laura Szabó
 
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaalmanuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
wolfsoftcompanyco
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
davidjhones387
 
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
fovkoyb
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
xjq03c34
 
[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024
hackersuli
 
Search Result Showing My Post is Now Buried
Search Result Showing My Post is Now BuriedSearch Result Showing My Post is Now Buried
Search Result Showing My Post is Now Buried
Trish Parr
 
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
uehowe
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
k4ncd0z
 
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
cuobya
 
Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!
Toptal Tech
 
Design Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptxDesign Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptx
saathvikreddy2003
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
Paul Walk
 
7 Best Cloud Hosting Services to Try Out in 2024
7 Best Cloud Hosting Services to Try Out in 20247 Best Cloud Hosting Services to Try Out in 2024
7 Best Cloud Hosting Services to Try Out in 2024
Danica Gill
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
Donato Onofri
 
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
bseovas
 
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
zoowe
 
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
cuobya
 
Azure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdfAzure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdf
AanSulistiyo
 
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
zyfovom
 

Recently uploaded (20)

Gen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needsGen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needs
 
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaalmanuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
 
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
 
[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024
 
Search Result Showing My Post is Now Buried
Search Result Showing My Post is Now BuriedSearch Result Showing My Post is Now Buried
Search Result Showing My Post is Now Buried
 
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
办理毕业证(NYU毕业证)纽约大学毕业证成绩单官方原版办理
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
 
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
 
Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!
 
Design Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptxDesign Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptx
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
 
7 Best Cloud Hosting Services to Try Out in 2024
7 Best Cloud Hosting Services to Try Out in 20247 Best Cloud Hosting Services to Try Out in 2024
7 Best Cloud Hosting Services to Try Out in 2024
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
 
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
 
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
 
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
 
Azure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdfAzure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdf
 
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
 

AI-SDV 2022: Accommodating the Deep Learning Revolution by a Development Process Methodology Jochen Leidner (Coburg University, Germany)

  • 1. Accommodating the Deep Learning Revolution by a Development Process Methodology Jochen L. Leidner Coburg University of Applied Sciences and Arts, Coburg, Germany KnowledgeSpaces® UG (haftungsbeschränkt), Coburg, Germany University of Sheffield, Sheffield, UK 2022-10-11
  • 2. Overview ● Introduction: Motivation, Pre-Trained Language Model Revolution ● Quick Recap: Some Machine Learning Methodologies (CRISP-DM, KDD, SEMMA, Data-to-Value) ● Before and After Pre-Trained LMs ● Comparison: Where Project Work is Spent: Pre-BERT and Post-BERT ● A Comment about Energy ● Summary & Conclusion
  • 3. A Step Change in NLP: Deep Learning and Pre-Trained Language Models ● In recent years, Pre-Trained Language Models (PTLMs) like Google’s BERT have emerged (Devlin et al., 2018/2019). ● This led to enormous improvements in terms of accuracy in most NLP tasks. ● PTLMs show that transfer learning is possible by splitting up training into two phases.
  • 4. BERT: An Example Pretained Neural Language Model – Pre-Training versus Fine-Tuning  Two Training Phases: – Pre-training: train deep neural network with masked sentence pairs on generic language (billions of words from books, Wikipedia) – Fine-tuning: adapt generic LM to specific task (e.g. question answering) using supervised learning (extra rounds on top of pre-trained LM)
  • 5. Practical Questions ● RQ 1. How do PTLMs change the way NLP projects are done? ● RQ 2. In particular, How do PTLMs interact with existing methodologies?
  • 6. Some Methodologies ● KDD ● CRISP-DM Azevedo and Santos (2008) ● SEMMA ● Data-to-Value (Leidner, 2013; Leidner 2022a,b)
  • 7. The Data-to-Value Methodology (Leidner, 2013; Leidner 2022a,b) (1 of 2)
  • 8. The Data-to-Value Methodology (Leidner, 2013; Leidner 2022a,b) (2 of 2) Minor fine-tuning sufficient
  • 9. Before and After PTLMs Before: ● Any classifier/regressor a bespoke activity (100% custom development from scratch) ● Relatively slow and expensive to build ● Knobs: more labelled training data, more features After: ● Classifiers can be derived from PTLMs (80% re-use and 20% custom dev. →fine-tuning) ● Rapid/agile prototyping, cheap to get started ● Knobs: more unlabelled training data, more labelled training data, 3 training regimes: – Zero-shot (apply PTLM as-is) – Fine tuning only (take pre-trained LM and add a few hundred training rounds using annotated data) – Pre-training (huge unlabelled data) and fine- tuning (small labelled data) Increasing effort
  • 10. Comparison: How Time May Be Spent – Before and After PTLMs Before: ● Data Collection & Pre-Processing 70% ● Annotation 10% ● Feature Engineering 10% ● Model Training 7% ● Evaluation 3% After: ● Data Collection & Pre-Processing 50% - 70% ● Annotation 2% - 10% ● Feature Engineering 0% ● Model Training 0% - 12% ● Evaluation 3% Percentages are estimates (an empirical study is needed but hard to obtain); ranges reflect training regimes symbolizes size of the project
  • 11. Deep Learning & Energy Consumption ● Pre-training neural models is resource-intensive (Strubell, Ganesh and McCallum, 2019). ● Individual estimates vary, but cloud cost and environmental footprint are issues. ● While experiments show that “bigger is better“ (in terms of F1), there is a research drive to “distill“ smaller models.
  • 12. Summary & Conclusions ● PTLMs have made NLP projects more agile. – While more unlabelled data may be needed, less labelled data may be required (sufficient data is sometimes unavailable in industrial practice). – Most importantly, the feature engineering cycle is removed from projects. – PTLMs offer 3 training regimes: zero-shot, tune-tuning and pre-training with increasing cost/effort. ● As artifacts they are also more clunky and energy-inefficient. ● Implications: – Research: Increasingly bigger models means some academic teams excluded from research (requires expensive GPU clusters) → research moves to industry (similar to semiconductor space). – Business: Public availability of PTLMs generates more level playing-ground, makes competitive differentiation harder and reduces barriers to entry.
  • 13. References ● Devlin, Jacob, Ming-Wei Chang, Kenton Lee and Kristina Toutanova (2018) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Technical Report/Unpublished ArXiv Pre-print, https://arxiv.org/abs/1810.04805. ● Devlin, Jacob, Ming-Wei Chang, Kenton Lee and Kristina Toutanova (2019) "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" Proc. NAACL-HLT, Minneapolis. ● Azevedo, A. and Santos, M. F. (2008) "KDD, SEMMA and CRISP-DM: a parallel overview", Proc. IADIS European Conference on Data Mining, Amsterdam, 24-26 July 2000, 182–185. ● Leidner, Jochen L. (2013) “Data-to-Value“, unpublished lecture notes Big Data and Language Technology, University of Zurich, Zurich, Switzerland. ● Leidner, Jochen L. (2022a) Data-to-Value: An Evaluation-First Methodology for Natural Language Projects, Technical Report/Unpublished ArXiv Pre-print https://arxiv.org/abs/2201.07725. ● Leidner, Jochen L. (2022b) "Data-to-Value: An Evaluation-First Methodology for Natural Language Projects", Proceedings of the 27th International Conference on Natural Language & Information Systems (NLDB 2022), Valencia, Spain, 15-17 June 2022, LNCS 13286, 517–523. ● Strubell, Emma, Ananya Ganesh and Andrew McCallum (2019) "Energy and Policy Considerations for Deep Learning in NLP", ArXiv pre-print, https://arxiv.org/pdf/1906.02243.pdf .
  • 14. Accommodating the Deep Learning Revolution by a Development Process Methodology Word embeddings, deep learning, transformer models and other pre-trained neural language models (sometimes recently referred to as "foundational models") have fundamentally changed the way state-of-the- art systems for natural language processing and information access are built today. The "Data-to-Value" process methodology (Leidner 2013; Leidner 2022a,b) has been devised to embody best practices for the construction of natural language engineering solutions; it can assist practitioners and has also been used to transfer industrial insights into the university classroom. This talk recaps how the methodology supports engineers in building systems more consistently and then outlines the changes in the methodology to adapt it to the deep learning age. The cost and energy implications will also be discussed. Abstract
  • 15. About the Presenter Prof. Dr. Jochen L. Leidner, M.A. M.Phil. Ph.D. FRGS is Professor for Explainable and Responsible Artificial Intelligence in Insurance at Coburg University of Applied Sciences, a Visiting Professor in the Department of Computer Science, University of Sheffield. He is also founder and CEO of KnowledgeSpaces. His experience includes positions as Director of Research at Thomson Reuters and Refinitiv in London, where he headed its R&D team, which he founded He was also the Royal Academy of Engineering Visiting’ Professor of Data Analytics at the Department of Computer Science, University of Sheffield (2017-2020). His background includes a Master's in computational linguistics, English and computer science (University of Erlangen-Nuremberg), a Master's in Computer Speech, Text and Internet Technology (University of Cambridge) and a PhD in Informatics (University of Edinburgh), which won the first ACM SIGIR Doctoral Consortium Award. His scientific contributions include leading the teams that developed the QED and ALYSSA open-domain question answering Systems (evaluated at US NIST/DARPA TREC), proposing a new algorithm and comparing existing algorithms for spatial resolution of named entities, and information extraction of usual and unusual things (e.g. event extraction, company risk mining, sentiment analysis). At Thomson Reuters he has led projects in the vertical comains of finance, regulatory/law enforcement, legal, pharmacology, and news. His code and machine learning models have been transitioned into products deployed at institutions ranging from international banks to the U.S. Supreme Court. Prior to Thomson Reuters, he has worked for SAP and founded and co-founded a number of start-ups. He has lived and worked in Germany, Scotland, the USA, Switzerland and the UK, and has taught at various universities (Erlangen, Saarbrücken, Frankfurt, Zurich and now Coburg), and is a scientific expert for the European Commission (FP7, H2020, Horizon Europe) and other funding bodies. He is an author or co-author of several dozen peer-reviewed publications (incl. one best paper award), has authored/co-edited two books and holds several patents in the areas of information retrieval, natural language processing, and mobile computing. He has been twice winner of the Thomson Reuters inventor of the year award for the best patent application.
  • 16. About KnowledgeSpaces® ● Contact for consulting: E-Mail: info@knowledgespaces.de Phone: +49 (172) 904 8908