SlideShare a Scribd company logo
1 of 18
Download to read offline
1
Big Data – a view
DBC
14 January 2016
Bjarne Kjær Ersbøll / bker@dtu.dk
2 DTU Compute, Technical University of Denmark
Acknowledgements
This slide deck is compiled from material from a lot of my colleagues and
people I collaborate with at DTU. The following list is incomplete:
• Jakob Eg Larsen
• Mark Riis
• Mads Odgaard
• Knut Conradsen
• Tage Thyrsted
• Lone Falsig Hansen
• Elena Guarneri
• And many more…
2
3 DTU Compute, Technical University of Denmark
So, what is Big Data anyway?
4 DTU Compute, Technical University of Denmark
The 4 V’s
3
5 DTU Compute, Technical University of Denmark
Data
explosion
6 DTU Compute, Technical University of Denmark
4
7 DTU Compute, Technical University of Denmark
Crowds, Bluetooth and Rock n’ Roll:
Understanding Music Festival Participant
Behavior
8 DTU Compute, Technical University of Denmark
5
9 DTU Compute, Technical University of Denmark
10 DTU Compute, Technical University of Denmark
6
BIG1
Den 3. december 2013
12 DTU Compute, Technical University of Denmark
BIG1 purpose
• Identify technological challenges associated with exploiting the
potential of Big Data / Data-driven business development - to
improve animal health and higher food quality and safety.
7
13 DTU Compute, Technical University of Denmark
BIG1 participants
• DTU Compute
• DTU National Food Institute
• DTU Veterinary Institute
• DTU Management
• DTU Biosys
• DTU Administration
14 DTU Compute, Technical University of Denmark
Big Data Value-chain
Data 
Origins
The Internet, 
sensors, 
machines, 
etc.
Data 
Collection 
Web log,
sensor data,  
images/au‐
dio, RFID and 
videos etc.
Data 
Storage
Technologies 
supporting 
data storage
Analytics 
Predictive 
analytics, 
patterns in 
data, 
decision 
making
Consumers
Business 
processes, 
humans, and 
applications
Sense Think Act
8
15 DTU Compute, Technical University of Denmark
Feed/plants Animals Processing Consumers
Value chain
Actors
Data
Feed producers
Plant producers
Equipm. producers
Farmers
Abbatoir
Dairy
Retail sector
Export
Eg feed quality Eg growth rate
of animals
Eg efficiency in
slaughtering
process
Consumer
patterns and
food quality
Big Data
Stakeholders in BIG1 value-chain
16 DTU Compute, Technical University of Denmark
Optimere/speede algoritmernes funktionalitet og gøre beregningerne billigere
GenericBigData
problemtopics
Domain / application areas
Cattle Pigs Nutritional
composition
… and other
applications
Collection of data, eg sensors on individuals (eg RFID or image analysis)
Storage, manipulation, real-time data
Establising a dynamic Big Data cloud
Structuring data, distributed data and data-sharing
Merging and integration of databases
Pattern recognition, machine learning, artificial intelligence, query-algorithms
Multivariat analsis and advanced statistics and data analysis
Privacy/ethics regarding data
Visualisation of data wrt descision support
Platform project
Targeted projects
Optimation/speed-up algorithm functionality and lower cost of calculation
BIG1: What can we do?
9
17 DTU Compute, Technical University of Denmark
18 DTU Compute, Technical University of Denmark
Sensors and data generation
10
19 DTU Compute, Technical University of Denmark
Hardware and software
DTU Compute, Technical University of Denmark
Big Data – 1991 – Economic Geology
20 18.01.2016
11
DTU Compute, Technical University of Denmark
Data
• Landsat satellite (common reference) – 4 scenes – 8 tapes
– Geometric rectification, mosaicking, ratios, factor scores,
• Geological – geological maps, topographic maps
– Structural information, lineaments converted to concentrations in 10
directions
• Geochemical – K, Rb, Sr, U, Nb, Y, Ga, Fe in stream sediments.
– Kriging to a 1 km grid, interpolation by bicubic spline to Landsat
pixels
• Radiometric – helicoptor-bourne gamma-spectrometric measurements,
U, Th, K, and Total concentration.
– Max in 1 km grid interpolated by minimum curvature and further by
bicubic spline
• Aeromagnetic data – 11 map sheets
– Manually digitized and interpolated
• Resulting in 40 variables on a pixel level (50.8m x 50.8m)
21 18.01.2016
DTU Compute, Technical University of Denmark
Data
• Converted to a 5km x 5km grid – trying to preserve information by
taking (when relevant):
– Min, max, 1%, 5%, median, 95%, 99%, mean, stddev, %land-cover
– 240 variables in all in 1084 squares
• Training set of
– 17 mineralized, central
– 21 mineralized, marginal
– 14 barren, central
– 5 barren, marginal
• Discriminant analysis using stepwise selection
– 1084 squares classified
22 18.01.2016
12
DTU Compute, Technical University of Denmark23 18.01.2016
DTU Compute, Technical University of Denmark24 18.01.2016
13
DTU Compute, Technical University of Denmark
Big Data ?
25 18.01.2016
DTU Compute, Technical University of Denmark
Other Big Data cases
ELIXIR Data describing the human
genetic variation
Development of personal
medical drugs which take
variation between patients
into account
Global Microbial Identifier Global system on genome-
sequence data from micro-
organismes to improve
national clinical diagnostics
and international
surveillance of diseases
CITIES IT-solutions for analysis,
operation and development
of integrated energy-
systems (electricity, gas,
district heating and bio-
masse) in cities to achieve
higher flexibility in eg
energy-storage
14
Data Science (Big Data)
Profile at DTU Compute
28 DTU Compute, Technical University of Denmark
Data Science – main elements
 Ambitious – courses: 45 ECTS (4/6
core) + thesis: A further 30-35 ECTS
 Pioneering – across the Big Data
value chain and competences
 Application oriented:
o Work with concrete data sets
o Collaboration with companies
15
29 DTU Compute, Technical University of Denmark
Entry via all 3 DTU Compute programs
• Computer Science and Engineering
• Mathematical Modelling and Computation
• Digital Media Engineering
• …and now also: IT & Health (combination education btw KU & DTU)
• Cross-educational skills
30 DTU Compute, Technical University of Denmark
Big Data Value chain
data BIG data model
analysis
Data Origins
The Internet, sensors, 
machines, etc.
Data Collection 
Web log, sensor data, 
images/audio, RFID and 
videos, etc.
Data Storage
Technologies 
supporting data storage
Analytics: 
Predictive analytics, 
patterns in data, 
decision making
Consumers: 
Business processes, 
humans, and 
applications
Sense Think Act
16
31 DTU Compute, Technical University of Denmark
Courses in Data Science specialization
Origin Collection Storage Analytics Consumers
01227 Graph theory (5) 1 3
01405 Error correcting codes 2 1 1
01617 Dynamical Systems 1 2
02170 Database systems (5) 4
02232 Applied Cryptography (5) 2 3 1 1
Core 02239 Data Security 1 4 1
02249 Computationally hard problems (7.5) 1 1 4
02266 User experience engineering 1 1 5
02281 Data Logic (5) 1 2 1 1
Core 02282 Algorithms for Massive Data Sets (7.5) 2 3 3
Core 02288 Missing a course on “Advanced databases/w arehouses”? 2
02407 Stochastic Processes (5) 3
02409 Multivariate Statistics (5) 4
02417 Time Series Analysis (5) 4
02443 Stochastic Simulation (5) 4 1
02450 Introduction to Machine Learning and Data Modeling (5) 3 1
02457 Non-linear signal processing 1 1
02458 Cognitive Modelling (5) 3 2
02460 Advanced Machine Learning (5) 1 3 1
02506 Advanced Image Analysis 3
02515 Health technology 1 2
Core 02582 Computational dataanalysis 3
02586 Statistical Genetics (5) 2
Core 02806 Social data analysis and visualization(5) 2 3
Core 02819 Data Mining using Python (5) 1 3 1
30530 Geographical information systems 1 1 1
25303 Mathematical Biology 1 1 1 1
27411 Biological data analysis and chemometrics 1
27625 Algorithms in bioinformatics 1 1
42112 Mathematical Programming w ith Modelling Softw are 1 1
32 DTU Compute, Technical University of Denmark
Big Data
Hackathon
65 students
 10 groups
 48 hours
 DTU's Skylab
 Funding
 1-2 start up companies
17
33 DTU Compute, Technical University of Denmark
Big Data solutions for Lyngby-Taarbæk
municipality
”Smart City app” to make it a better place to
live
34 DTU Compute, Technical University of Denmark
Projects!
 Energy utilization in buildings
 Optimization of Bus-routes
 Smart Traffic-regulation
 Smart Energy renovation
 Personalized Care for elderly
 Smart tests for the Schools
 Flexible collection of Waste
18
35 DTU Compute, Technical University of Denmark
36 DTU Compute, Technical University of Denmark
Implementation of first recommendation:
Big Data•DTU

More Related Content

What's hot

Satellite and Land Cover Image Classification using Deep Learning
Satellite and Land Cover Image Classification using Deep LearningSatellite and Land Cover Image Classification using Deep Learning
Satellite and Land Cover Image Classification using Deep Learningijtsrd
 
Copernicus and AI workshop 2020
Copernicus and AI workshop 2020Copernicus and AI workshop 2020
Copernicus and AI workshop 2020ExtremeEarth
 
An Introduction to Machine Learning and Genomics
An Introduction to Machine Learning and GenomicsAn Introduction to Machine Learning and Genomics
An Introduction to Machine Learning and GenomicsBrittany Lasseigne, Ph.D.
 
What can Machine Learning Do and What Does It Mean for the Economy?
What can Machine Learning Do and What Does It Mean for the Economy?What can Machine Learning Do and What Does It Mean for the Economy?
What can Machine Learning Do and What Does It Mean for the Economy?Structuralpolicyanalysis
 
Machine Learning for Chemistry: Representing and Intervening
Machine Learning for Chemistry: Representing and InterveningMachine Learning for Chemistry: Representing and Intervening
Machine Learning for Chemistry: Representing and InterveningIchigaku Takigawa
 
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...gerogepatton
 
Upavan Gupta
Upavan GuptaUpavan Gupta
Upavan Guptabutest
 
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITY
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITYA NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITY
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITYIJCSIS Research Publications
 
Top 10 neural networks
Top 10 neural networksTop 10 neural networks
Top 10 neural networksijsc
 
Cao report 2007-2012
Cao report 2007-2012Cao report 2007-2012
Cao report 2007-2012Elif Ceylan
 
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...Edge AI and Vision Alliance
 
Scientific data management (v2)
Scientific data management (v2)Scientific data management (v2)
Scientific data management (v2)Jian Qin
 
ISCRAM Overview
ISCRAM OverviewISCRAM Overview
ISCRAM Overviewglobal
 
2011 Big Data - Bigger Problems for Drug Discovery and Development
2011 Big Data - Bigger Problems for Drug Discovery and Development2011 Big Data - Bigger Problems for Drug Discovery and Development
2011 Big Data - Bigger Problems for Drug Discovery and DevelopmentAyasdi
 
June 2020: Most Downloaded Article in Soft Computing
June 2020: Most Downloaded Article in Soft Computing  June 2020: Most Downloaded Article in Soft Computing
June 2020: Most Downloaded Article in Soft Computing ijsc
 
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...cseij
 

What's hot (20)

Satellite and Land Cover Image Classification using Deep Learning
Satellite and Land Cover Image Classification using Deep LearningSatellite and Land Cover Image Classification using Deep Learning
Satellite and Land Cover Image Classification using Deep Learning
 
Copernicus and AI workshop 2020
Copernicus and AI workshop 2020Copernicus and AI workshop 2020
Copernicus and AI workshop 2020
 
An Introduction to Machine Learning and Genomics
An Introduction to Machine Learning and GenomicsAn Introduction to Machine Learning and Genomics
An Introduction to Machine Learning and Genomics
 
What can Machine Learning Do and What Does It Mean for the Economy?
What can Machine Learning Do and What Does It Mean for the Economy?What can Machine Learning Do and What Does It Mean for the Economy?
What can Machine Learning Do and What Does It Mean for the Economy?
 
AI that/for matters
AI that/for mattersAI that/for matters
AI that/for matters
 
Machine Learning for Chemistry: Representing and Intervening
Machine Learning for Chemistry: Representing and InterveningMachine Learning for Chemistry: Representing and Intervening
Machine Learning for Chemistry: Representing and Intervening
 
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...
MOST READ ARTICLES IN ARTIFICIAL INTELLIGENCE - International Journal of Arti...
 
GSU-RF-2013-Reddy-4
GSU-RF-2013-Reddy-4GSU-RF-2013-Reddy-4
GSU-RF-2013-Reddy-4
 
Hands-on Introduction to Machine Learning
Hands-on Introduction to Machine LearningHands-on Introduction to Machine Learning
Hands-on Introduction to Machine Learning
 
Upavan Gupta
Upavan GuptaUpavan Gupta
Upavan Gupta
 
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITY
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITYA NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITY
A NEW SELF-ADAPTIVE APPROACH FOR MEDICAL IMAGE SECURITY
 
Top 10 neural networks
Top 10 neural networksTop 10 neural networks
Top 10 neural networks
 
Cao report 2007-2012
Cao report 2007-2012Cao report 2007-2012
Cao report 2007-2012
 
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...
“Structures as Sensors: Smaller-Data Learning in the Physical World,” a Prese...
 
Scientific data management (v2)
Scientific data management (v2)Scientific data management (v2)
Scientific data management (v2)
 
ISCRAM Overview
ISCRAM OverviewISCRAM Overview
ISCRAM Overview
 
2011 Big Data - Bigger Problems for Drug Discovery and Development
2011 Big Data - Bigger Problems for Drug Discovery and Development2011 Big Data - Bigger Problems for Drug Discovery and Development
2011 Big Data - Bigger Problems for Drug Discovery and Development
 
June 2020: Most Downloaded Article in Soft Computing
June 2020: Most Downloaded Article in Soft Computing  June 2020: Most Downloaded Article in Soft Computing
June 2020: Most Downloaded Article in Soft Computing
 
project documentation
project documentationproject documentation
project documentation
 
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...
Top 1 cited papers - COMPUTER SCIENCE & ENGINEERING: AN INTERNATIONAL JOURNAL...
 

Similar to Big Data - A view

Ci2004-10.doc
Ci2004-10.docCi2004-10.doc
Ci2004-10.docbutest
 
BIMCV: The Perfect "Big Data" Storm.
BIMCV: The Perfect "Big Data" Storm. BIMCV: The Perfect "Big Data" Storm.
BIMCV: The Perfect "Big Data" Storm. maigva
 
Enabling Data-Intensive Science Through Data Infrastructures
Enabling Data-Intensive Science Through Data InfrastructuresEnabling Data-Intensive Science Through Data Infrastructures
Enabling Data-Intensive Science Through Data InfrastructuresLIBER Europe
 
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...Capacity Building Efforts & Data Infrastructure at Makerere University and UV...
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...African Open Science Platform
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science James Hendler
 
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la Iglesia
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaBIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la Iglesia
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaMaria de la Iglesia
 
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Research Data Alliance
 
Life science requirements from e-infrastructure: initial results from a joint...
Life science requirements from e-infrastructure:initial results from a joint...Life science requirements from e-infrastructure:initial results from a joint...
Life science requirements from e-infrastructure: initial results from a joint...Rafael C. Jimenez
 
Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53Mr.Sameer Kumar Das
 
Big Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science: The Technologies Shaping Our LivesBig Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science: The Technologies Shaping Our LivesRukshan Batuwita
 
Orbital presentation pt1_200112_v1
Orbital presentation pt1_200112_v1Orbital presentation pt1_200112_v1
Orbital presentation pt1_200112_v1ensmjd
 
Big Data and Computer Science Education
Big Data and Computer Science EducationBig Data and Computer Science Education
Big Data and Computer Science EducationJames Hendler
 
IoT 2014 Value Creation Workshop: SDIL
IoT 2014 Value Creation Workshop: SDILIoT 2014 Value Creation Workshop: SDIL
IoT 2014 Value Creation Workshop: SDILTill Riedel
 
Data Ecosystems for Geospatial Data
Data Ecosystems for Geospatial DataData Ecosystems for Geospatial Data
Data Ecosystems for Geospatial DataSlim Turki, Dr.
 
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...dkNET
 
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...AlmereDataCapital
 

Similar to Big Data - A view (20)

Ci2004-10.doc
Ci2004-10.docCi2004-10.doc
Ci2004-10.doc
 
BIMCV: The Perfect "Big Data" Storm.
BIMCV: The Perfect "Big Data" Storm. BIMCV: The Perfect "Big Data" Storm.
BIMCV: The Perfect "Big Data" Storm.
 
Enabling Data-Intensive Science Through Data Infrastructures
Enabling Data-Intensive Science Through Data InfrastructuresEnabling Data-Intensive Science Through Data Infrastructures
Enabling Data-Intensive Science Through Data Infrastructures
 
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...Capacity Building Efforts & Data Infrastructure at Makerere University and UV...
Capacity Building Efforts & Data Infrastructure at Makerere University and UV...
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science
 
DBMS
DBMSDBMS
DBMS
 
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la Iglesia
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaBIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la Iglesia
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la Iglesia
 
ppt_ids-data science.pdf
ppt_ids-data science.pdfppt_ids-data science.pdf
ppt_ids-data science.pdf
 
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...Towards a Community-driven Data Science Body of Knowledge – Data Management S...
Towards a Community-driven Data Science Body of Knowledge – Data Management S...
 
Life science requirements from e-infrastructure: initial results from a joint...
Life science requirements from e-infrastructure:initial results from a joint...Life science requirements from e-infrastructure:initial results from a joint...
Life science requirements from e-infrastructure: initial results from a joint...
 
50 Years of Data Science
50 Years of Data Science50 Years of Data Science
50 Years of Data Science
 
FDS_dept_ppt.pptx
FDS_dept_ppt.pptxFDS_dept_ppt.pptx
FDS_dept_ppt.pptx
 
Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53Sameer Kumar Das International Conference Paper 53
Sameer Kumar Das International Conference Paper 53
 
Big Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science: The Technologies Shaping Our LivesBig Data and Data Science: The Technologies Shaping Our Lives
Big Data and Data Science: The Technologies Shaping Our Lives
 
Orbital presentation pt1_200112_v1
Orbital presentation pt1_200112_v1Orbital presentation pt1_200112_v1
Orbital presentation pt1_200112_v1
 
Big Data and Computer Science Education
Big Data and Computer Science EducationBig Data and Computer Science Education
Big Data and Computer Science Education
 
IoT 2014 Value Creation Workshop: SDIL
IoT 2014 Value Creation Workshop: SDILIoT 2014 Value Creation Workshop: SDIL
IoT 2014 Value Creation Workshop: SDIL
 
Data Ecosystems for Geospatial Data
Data Ecosystems for Geospatial DataData Ecosystems for Geospatial Data
Data Ecosystems for Geospatial Data
 
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...
dkNET Webinar "Integrative Artificial Intelligence Approach to Predict T1D" 0...
 
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...
Maurice Bouwhuis (SARA/Vancis) - Hoe big data te begrijpen door ze te visuali...
 

More from Dansk BiblioteksCenter

Data science -Søgeforslag og anbefalinger
Data science  -Søgeforslag og anbefalingerData science  -Søgeforslag og anbefalinger
Data science -Søgeforslag og anbefalingerDansk BiblioteksCenter
 
DBC & Data Science - Where to go and why?
DBC & Data Science - Where to go and why? DBC & Data Science - Where to go and why?
DBC & Data Science - Where to go and why? Dansk BiblioteksCenter
 
Hvorfor skal vi arbejde med autoritetsdata
Hvorfor skal vi arbejde med autoritetsdataHvorfor skal vi arbejde med autoritetsdata
Hvorfor skal vi arbejde med autoritetsdataDansk BiblioteksCenter
 
DBC's arbejde med nationalbibliografiske autoritetsdata
DBC's arbejde med nationalbibliografiske autoritetsdataDBC's arbejde med nationalbibliografiske autoritetsdata
DBC's arbejde med nationalbibliografiske autoritetsdataDansk BiblioteksCenter
 
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...Dansk BiblioteksCenter
 

More from Dansk BiblioteksCenter (20)

Facebook-gruppen DBC Musik
Facebook-gruppen  DBC MusikFacebook-gruppen  DBC Musik
Facebook-gruppen DBC Musik
 
Nyt Netpunkt og søgning med CQL
Nyt Netpunkt og søgning med CQLNyt Netpunkt og søgning med CQL
Nyt Netpunkt og søgning med CQL
 
i2020 - Brugerdrevet innovation
i2020 - Brugerdrevet innovationi2020 - Brugerdrevet innovation
i2020 - Brugerdrevet innovation
 
Data science -Søgeforslag og anbefalinger
Data science  -Søgeforslag og anbefalingerData science  -Søgeforslag og anbefalinger
Data science -Søgeforslag og anbefalinger
 
Resource Description & Access (RDA)
Resource Description & Access (RDA)Resource Description & Access (RDA)
Resource Description & Access (RDA)
 
Søgning med CQL i netpunkt.dk
Søgning med CQL i netpunkt.dkSøgning med CQL i netpunkt.dk
Søgning med CQL i netpunkt.dk
 
Nyt om andre baser
Nyt om andre baserNyt om andre baser
Nyt om andre baser
 
Nyt Netpunkt og BoB
Nyt Netpunkt og BoBNyt Netpunkt og BoB
Nyt Netpunkt og BoB
 
RDA og brugernes navigering
RDA og brugernes navigeringRDA og brugernes navigering
RDA og brugernes navigering
 
Relatér dig til RDA
Relatér dig til RDARelatér dig til RDA
Relatér dig til RDA
 
RDA og brugernes navigering
RDA og brugernes navigering RDA og brugernes navigering
RDA og brugernes navigering
 
Den demokratiske søgefunktion
Den demokratiske søgefunktionDen demokratiske søgefunktion
Den demokratiske søgefunktion
 
Data science at DBC in 29 slides
Data science at DBC in 29 slidesData science at DBC in 29 slides
Data science at DBC in 29 slides
 
DBC & Data Science - Where to go and why?
DBC & Data Science - Where to go and why? DBC & Data Science - Where to go and why?
DBC & Data Science - Where to go and why?
 
Hvorfor skal vi arbejde med autoritetsdata
Hvorfor skal vi arbejde med autoritetsdataHvorfor skal vi arbejde med autoritetsdata
Hvorfor skal vi arbejde med autoritetsdata
 
DBC's arbejde med nationalbibliografiske autoritetsdata
DBC's arbejde med nationalbibliografiske autoritetsdataDBC's arbejde med nationalbibliografiske autoritetsdata
DBC's arbejde med nationalbibliografiske autoritetsdata
 
Identifikatorer
IdentifikatorerIdentifikatorer
Identifikatorer
 
Webservices på biblioteket
Webservices på biblioteketWebservices på biblioteket
Webservices på biblioteket
 
BIBFRAME - MARC-formaternes afløser?
BIBFRAME - MARC-formaternes afløser?BIBFRAME - MARC-formaternes afløser?
BIBFRAME - MARC-formaternes afløser?
 
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...
Det semantiske internet. Hvad er linked open data og hvad betyder de for bibl...
 

Recently uploaded

Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxNeo4j
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)IES VE
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4DianaGray10
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)codyslingerland1
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 

Recently uploaded (20)

Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 

Big Data - A view

  • 1. 1 Big Data – a view DBC 14 January 2016 Bjarne Kjær Ersbøll / bker@dtu.dk 2 DTU Compute, Technical University of Denmark Acknowledgements This slide deck is compiled from material from a lot of my colleagues and people I collaborate with at DTU. The following list is incomplete: • Jakob Eg Larsen • Mark Riis • Mads Odgaard • Knut Conradsen • Tage Thyrsted • Lone Falsig Hansen • Elena Guarneri • And many more…
  • 2. 2 3 DTU Compute, Technical University of Denmark So, what is Big Data anyway? 4 DTU Compute, Technical University of Denmark The 4 V’s
  • 3. 3 5 DTU Compute, Technical University of Denmark Data explosion 6 DTU Compute, Technical University of Denmark
  • 4. 4 7 DTU Compute, Technical University of Denmark Crowds, Bluetooth and Rock n’ Roll: Understanding Music Festival Participant Behavior 8 DTU Compute, Technical University of Denmark
  • 5. 5 9 DTU Compute, Technical University of Denmark 10 DTU Compute, Technical University of Denmark
  • 6. 6 BIG1 Den 3. december 2013 12 DTU Compute, Technical University of Denmark BIG1 purpose • Identify technological challenges associated with exploiting the potential of Big Data / Data-driven business development - to improve animal health and higher food quality and safety.
  • 7. 7 13 DTU Compute, Technical University of Denmark BIG1 participants • DTU Compute • DTU National Food Institute • DTU Veterinary Institute • DTU Management • DTU Biosys • DTU Administration 14 DTU Compute, Technical University of Denmark Big Data Value-chain Data  Origins The Internet,  sensors,  machines,  etc. Data  Collection  Web log, sensor data,   images/au‐ dio, RFID and  videos etc. Data  Storage Technologies  supporting  data storage Analytics  Predictive  analytics,  patterns in  data,  decision  making Consumers Business  processes,  humans, and  applications Sense Think Act
  • 8. 8 15 DTU Compute, Technical University of Denmark Feed/plants Animals Processing Consumers Value chain Actors Data Feed producers Plant producers Equipm. producers Farmers Abbatoir Dairy Retail sector Export Eg feed quality Eg growth rate of animals Eg efficiency in slaughtering process Consumer patterns and food quality Big Data Stakeholders in BIG1 value-chain 16 DTU Compute, Technical University of Denmark Optimere/speede algoritmernes funktionalitet og gøre beregningerne billigere GenericBigData problemtopics Domain / application areas Cattle Pigs Nutritional composition … and other applications Collection of data, eg sensors on individuals (eg RFID or image analysis) Storage, manipulation, real-time data Establising a dynamic Big Data cloud Structuring data, distributed data and data-sharing Merging and integration of databases Pattern recognition, machine learning, artificial intelligence, query-algorithms Multivariat analsis and advanced statistics and data analysis Privacy/ethics regarding data Visualisation of data wrt descision support Platform project Targeted projects Optimation/speed-up algorithm functionality and lower cost of calculation BIG1: What can we do?
  • 9. 9 17 DTU Compute, Technical University of Denmark 18 DTU Compute, Technical University of Denmark Sensors and data generation
  • 10. 10 19 DTU Compute, Technical University of Denmark Hardware and software DTU Compute, Technical University of Denmark Big Data – 1991 – Economic Geology 20 18.01.2016
  • 11. 11 DTU Compute, Technical University of Denmark Data • Landsat satellite (common reference) – 4 scenes – 8 tapes – Geometric rectification, mosaicking, ratios, factor scores, • Geological – geological maps, topographic maps – Structural information, lineaments converted to concentrations in 10 directions • Geochemical – K, Rb, Sr, U, Nb, Y, Ga, Fe in stream sediments. – Kriging to a 1 km grid, interpolation by bicubic spline to Landsat pixels • Radiometric – helicoptor-bourne gamma-spectrometric measurements, U, Th, K, and Total concentration. – Max in 1 km grid interpolated by minimum curvature and further by bicubic spline • Aeromagnetic data – 11 map sheets – Manually digitized and interpolated • Resulting in 40 variables on a pixel level (50.8m x 50.8m) 21 18.01.2016 DTU Compute, Technical University of Denmark Data • Converted to a 5km x 5km grid – trying to preserve information by taking (when relevant): – Min, max, 1%, 5%, median, 95%, 99%, mean, stddev, %land-cover – 240 variables in all in 1084 squares • Training set of – 17 mineralized, central – 21 mineralized, marginal – 14 barren, central – 5 barren, marginal • Discriminant analysis using stepwise selection – 1084 squares classified 22 18.01.2016
  • 12. 12 DTU Compute, Technical University of Denmark23 18.01.2016 DTU Compute, Technical University of Denmark24 18.01.2016
  • 13. 13 DTU Compute, Technical University of Denmark Big Data ? 25 18.01.2016 DTU Compute, Technical University of Denmark Other Big Data cases ELIXIR Data describing the human genetic variation Development of personal medical drugs which take variation between patients into account Global Microbial Identifier Global system on genome- sequence data from micro- organismes to improve national clinical diagnostics and international surveillance of diseases CITIES IT-solutions for analysis, operation and development of integrated energy- systems (electricity, gas, district heating and bio- masse) in cities to achieve higher flexibility in eg energy-storage
  • 14. 14 Data Science (Big Data) Profile at DTU Compute 28 DTU Compute, Technical University of Denmark Data Science – main elements  Ambitious – courses: 45 ECTS (4/6 core) + thesis: A further 30-35 ECTS  Pioneering – across the Big Data value chain and competences  Application oriented: o Work with concrete data sets o Collaboration with companies
  • 15. 15 29 DTU Compute, Technical University of Denmark Entry via all 3 DTU Compute programs • Computer Science and Engineering • Mathematical Modelling and Computation • Digital Media Engineering • …and now also: IT & Health (combination education btw KU & DTU) • Cross-educational skills 30 DTU Compute, Technical University of Denmark Big Data Value chain data BIG data model analysis Data Origins The Internet, sensors,  machines, etc. Data Collection  Web log, sensor data,  images/audio, RFID and  videos, etc. Data Storage Technologies  supporting data storage Analytics:  Predictive analytics,  patterns in data,  decision making Consumers:  Business processes,  humans, and  applications Sense Think Act
  • 16. 16 31 DTU Compute, Technical University of Denmark Courses in Data Science specialization Origin Collection Storage Analytics Consumers 01227 Graph theory (5) 1 3 01405 Error correcting codes 2 1 1 01617 Dynamical Systems 1 2 02170 Database systems (5) 4 02232 Applied Cryptography (5) 2 3 1 1 Core 02239 Data Security 1 4 1 02249 Computationally hard problems (7.5) 1 1 4 02266 User experience engineering 1 1 5 02281 Data Logic (5) 1 2 1 1 Core 02282 Algorithms for Massive Data Sets (7.5) 2 3 3 Core 02288 Missing a course on “Advanced databases/w arehouses”? 2 02407 Stochastic Processes (5) 3 02409 Multivariate Statistics (5) 4 02417 Time Series Analysis (5) 4 02443 Stochastic Simulation (5) 4 1 02450 Introduction to Machine Learning and Data Modeling (5) 3 1 02457 Non-linear signal processing 1 1 02458 Cognitive Modelling (5) 3 2 02460 Advanced Machine Learning (5) 1 3 1 02506 Advanced Image Analysis 3 02515 Health technology 1 2 Core 02582 Computational dataanalysis 3 02586 Statistical Genetics (5) 2 Core 02806 Social data analysis and visualization(5) 2 3 Core 02819 Data Mining using Python (5) 1 3 1 30530 Geographical information systems 1 1 1 25303 Mathematical Biology 1 1 1 1 27411 Biological data analysis and chemometrics 1 27625 Algorithms in bioinformatics 1 1 42112 Mathematical Programming w ith Modelling Softw are 1 1 32 DTU Compute, Technical University of Denmark Big Data Hackathon 65 students  10 groups  48 hours  DTU's Skylab  Funding  1-2 start up companies
  • 17. 17 33 DTU Compute, Technical University of Denmark Big Data solutions for Lyngby-Taarbæk municipality ”Smart City app” to make it a better place to live 34 DTU Compute, Technical University of Denmark Projects!  Energy utilization in buildings  Optimization of Bus-routes  Smart Traffic-regulation  Smart Energy renovation  Personalized Care for elderly  Smart tests for the Schools  Flexible collection of Waste
  • 18. 18 35 DTU Compute, Technical University of Denmark 36 DTU Compute, Technical University of Denmark Implementation of first recommendation: Big Data•DTU