Europe needs a clear strategy for leveraging Big Data
Economy in Europe. Our objectives are work at technical, business and policy levels, shaping the future through the positioning of Big Data in Horizon 2020. Bringing the necessary stakeholders into a sustainable industry-led initiative, which will greatly contribute to enhance the EU competitiveness taking full advantage of Big Data technologies.
Big data characteristics, value chain and challengesMusfiqur Rahman
Abstract—Recently the world is experiencing an deluge of
data from different domains such as telecom, healthcare
and supply chain systems. This growth of data has led to
an explosion, coining the term Big Data. In addition to the
growth in volume, Big Data also exhibits other unique
characteristics, such as velocity and variety. This large
volume, rapidly increasing and verities of data is becoming
the key basis of completion, underpinning new waves of
productivity growth, innovation and customer surplus. Big
Data is about to offer tremendous insight to the
organizations, but the traditional data analysis
architecture is not capable to handle Big Data. Therefore,
it calls for a sophisticated value chain and proper analytics
to unearth the opportunity it holds. This research
identifies the characteristics of Big Data and presents a
sophisticated Big Data value chain as finding of this
research. It also describes the typical challenges of Big
Data, which are required to be solved. As a part of this
research twenty experts from different industries and
academies of Finland were interviewed.
This document discusses steps towards a data value chain, including big data, public open data, and linked (open) data. It provides definitions and examples for each topic. For big data, it discusses the large volumes of data being created and challenges in working with such data. For public open data, it outlines principles like completeness and ease of access. It also shows examples of apps using open government data. For linked open data, it discusses moving from a web of documents to a web of interconnected data through using URIs and typed links. It also shows the growth of the linked open data cloud over time.
Big data - Key Enablers, Drivers & ChallengesShilpi Sharma
Big data is characterized by the 3 Vs - volume, velocity, and variety. The document discusses how big data is growing exponentially due to factors like the internet of things. Key enablers of big data include data storage, computation capacity, and data availability. Addressing big data requires technologies, techniques, and talent across the value chain of aggregating, analyzing, and consuming data to derive value. However, big data also presents management challenges around decision making, change management, technology clashes, and skills shortages. The document provides an example of how big data could help sales professionals better prepare for client meetings.
Big data refers to very large data sets that are analyzed computationally to reveal patterns, trends, and relationships. It is characterized by 3Vs - volume, velocity, and variety. Big data has many applications in recent scenarios including politics, weather, medicine, media, and manufacturing. It is used in politics to analyze voter data beyond basic demographics. In weather, sensor data from devices is used to create more detailed weather maps and forecasts. Medicine uses big data to identify patterns in symptoms that can help predict and prevent diseases like heart failure. Media analyzes data on user behaviors to tailor content instead of relying on traditional formats. Manufacturing leverages big data for increased transparency and insights into performance issues.
EDF2014: BIG - NESSI Networking Session: Edward Curry, National University of...European Data Forum
BIG - NESSI Networking Session, Talk by Edward Curry, National University of Ireland Galway at the European Data Forum 2014, 20 March 2014 in Athens, Greece: The Big Data Value Chain.
This document provides an overview of big data and commonly used methodologies. It defines big data as large volumes of complex data from various sources that is difficult to process using traditional data management tools. The key aspects of big data are volume, variety, and velocity. Hadoop is discussed as a popular framework for processing big data using the MapReduce programming model. HDFS is summarized as a distributed file system used with Hadoop to store and manage large datasets across clusters of computers. Challenges of big data such as storage capacity, processing large and complex datasets, and real-time analytics are also mentioned.
This document defines big data and discusses its importance. It describes big data as large datasets that are difficult to manage with traditional tools due to their volume, variety, and velocity. Examples are provided of how governments and private sector organizations like Walmart and Facebook generate and use big data. The phases of big data analysis and challenges like heterogeneity are outlined. Technologies used for big data from companies like Oracle, Microsoft and IBM are listed. Opportunities big data provides for driving revenue, understanding customers, and improving operations are discussed. The document concludes with other aspects of big data like its impact on knowledge and transparency.
Big data characteristics, value chain and challengesMusfiqur Rahman
Abstract—Recently the world is experiencing an deluge of
data from different domains such as telecom, healthcare
and supply chain systems. This growth of data has led to
an explosion, coining the term Big Data. In addition to the
growth in volume, Big Data also exhibits other unique
characteristics, such as velocity and variety. This large
volume, rapidly increasing and verities of data is becoming
the key basis of completion, underpinning new waves of
productivity growth, innovation and customer surplus. Big
Data is about to offer tremendous insight to the
organizations, but the traditional data analysis
architecture is not capable to handle Big Data. Therefore,
it calls for a sophisticated value chain and proper analytics
to unearth the opportunity it holds. This research
identifies the characteristics of Big Data and presents a
sophisticated Big Data value chain as finding of this
research. It also describes the typical challenges of Big
Data, which are required to be solved. As a part of this
research twenty experts from different industries and
academies of Finland were interviewed.
This document discusses steps towards a data value chain, including big data, public open data, and linked (open) data. It provides definitions and examples for each topic. For big data, it discusses the large volumes of data being created and challenges in working with such data. For public open data, it outlines principles like completeness and ease of access. It also shows examples of apps using open government data. For linked open data, it discusses moving from a web of documents to a web of interconnected data through using URIs and typed links. It also shows the growth of the linked open data cloud over time.
Big data - Key Enablers, Drivers & ChallengesShilpi Sharma
Big data is characterized by the 3 Vs - volume, velocity, and variety. The document discusses how big data is growing exponentially due to factors like the internet of things. Key enablers of big data include data storage, computation capacity, and data availability. Addressing big data requires technologies, techniques, and talent across the value chain of aggregating, analyzing, and consuming data to derive value. However, big data also presents management challenges around decision making, change management, technology clashes, and skills shortages. The document provides an example of how big data could help sales professionals better prepare for client meetings.
Big data refers to very large data sets that are analyzed computationally to reveal patterns, trends, and relationships. It is characterized by 3Vs - volume, velocity, and variety. Big data has many applications in recent scenarios including politics, weather, medicine, media, and manufacturing. It is used in politics to analyze voter data beyond basic demographics. In weather, sensor data from devices is used to create more detailed weather maps and forecasts. Medicine uses big data to identify patterns in symptoms that can help predict and prevent diseases like heart failure. Media analyzes data on user behaviors to tailor content instead of relying on traditional formats. Manufacturing leverages big data for increased transparency and insights into performance issues.
EDF2014: BIG - NESSI Networking Session: Edward Curry, National University of...European Data Forum
BIG - NESSI Networking Session, Talk by Edward Curry, National University of Ireland Galway at the European Data Forum 2014, 20 March 2014 in Athens, Greece: The Big Data Value Chain.
This document provides an overview of big data and commonly used methodologies. It defines big data as large volumes of complex data from various sources that is difficult to process using traditional data management tools. The key aspects of big data are volume, variety, and velocity. Hadoop is discussed as a popular framework for processing big data using the MapReduce programming model. HDFS is summarized as a distributed file system used with Hadoop to store and manage large datasets across clusters of computers. Challenges of big data such as storage capacity, processing large and complex datasets, and real-time analytics are also mentioned.
This document defines big data and discusses its importance. It describes big data as large datasets that are difficult to manage with traditional tools due to their volume, variety, and velocity. Examples are provided of how governments and private sector organizations like Walmart and Facebook generate and use big data. The phases of big data analysis and challenges like heterogeneity are outlined. Technologies used for big data from companies like Oracle, Microsoft and IBM are listed. Opportunities big data provides for driving revenue, understanding customers, and improving operations are discussed. The document concludes with other aspects of big data like its impact on knowledge and transparency.
This document discusses cloud computing and big data. It notes that while cloud computing standards have been developed since 2010 through organizations like ISO and ITU, big data standards are still being investigated. The relationship between cloud computing and big data is that big data will increasingly be stored and processed in the cloud due to the large data volumes. The document raises questions about what should be priorities for big data tools and services, and concludes that standards development is still needed to guide big data and cloud computing.
Presentation: Study: #Big Data in #Austria, Mario Meir-Huber, Big Data Leader Eastern Europe, Teradata GmbH & Martin Köhler, Austrian Institute of Technology, AIT (AT), at the European Data Economy Workshop taking place back to back to SEMANTiCS2015 on 15 September 2015 in Vienna.
This document discusses how big data can be used for smart cities. It provides three use cases: 1) social media and sentiment analysis to analyze opinions on topics, 2) retail analytics and recommendation engines to predict customer product preferences, and 3) security and risk management using machine learning to detect fraud patterns. It also discusses optimizing data warehouses to improve data services while reducing costs. Big data systems like Hadoop and Spark are introduced for large-scale storage and processing.
This document proposes a theme on big data analytics research. It notes that the world's data storage capacity doubles every 40 months and discusses how big data can provide value across many areas like health, policymaking, education and more. The proposal recommends that Hong Kong develop a state-of-the-art big data platform to make a difference in areas like smart cities and support aging populations. It outlines objectives like large-scale machine learning from big data and discusses how Hong Kong is well-positioned for this research with experts across universities and potential collaborators in industry. The expected outcomes include new methodologies, applications impacting society and industry, and educational programs to cultivate big data leaders.
Data 4 AI: For European Economic Competitiveness and Societal ProgressEdward Curry
1) Data is a key resource for developing artificial intelligence systems, but difficulties accessing data can reduce innovation and competition.
2) Data platforms and data sharing spaces will fuel the development of AI-driven decision-making by facilitating access to and portability of data.
3) The Big Data Value Association advocates for policies and technologies that create trusted frameworks for sharing data across sectors and borders to advance AI for European economic competitiveness and societal progress.
This document contains information about a group project on big data. It lists the group members and their student IDs. It then provides a table of contents and summaries various topics related to big data, including what big data is, data sources, characteristics of big data like volume, variety and velocity, storing and processing big data using Hadoop, where big data is used, risks and benefits of big data, and the future of big data.
Big Data Trends - WorldFuture 2015 ConferenceDavid Feinleib
David Feinleib's Big Data Trends presentation from the World Future Society's Annual Conference, WorldFuture 2015, held at the Hilton Union Square, San Francisco, California July 25, 2015.
A top-down look at current industry and technology trends for Big Data, Data Analytics and Machine Learning (cognitive technologies, AI etc.). New slides added for Ark Group presentation on 1st December 2016.
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Centerp...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Big Data & Analytics (Conceptual and Practical Introduction)Yaman Hajja, Ph.D.
A 3-day interactive workshop for startups involve in Big Data & Analytics in Asia. Introduction to Big Data & Analytics concepts, and case studies in R Programming, Excel, Web APIs, and many more.
DOI: 10.13140/RG.2.2.10638.36162
On Big Data Analytics - opportunities and challengesPetteri Alahuhta
This document discusses big data analytics and its opportunities and challenges. It defines big data and explains the increasing number of "V's" that characterize big data, such as volume, velocity, variety, and veracity. It also outlines some common uses of big data analytics including customer insights, security and risk analysis, and resource optimization. Additionally, it discusses challenges of big data adoption like skills shortages and infrastructure limitations, as well as trends in big data and areas of expertise related to big data that VTT focuses on.
Big Data may well be the Next Big Thing in the IT world
Big data burst upon the scene in the first decade of the 21st century
The first organizations to embrace it were online and startup firms. Firms like Google , eBay , LinkedIn , and Facebook were built around big data from the beginning.
Like many new information technologies , bigdata can bring about dramatic cost reductions , substantial improvements in the time required to perform to computing task , or new product and service offerings.
This document discusses various applications of big data across different domains. It begins by defining big data and its key characteristics of volume, variety and velocity. It then discusses how big data is being used in social media for recommendation systems, marketing, electioneering and influence analysis. Applications in healthcare discussed include personalized medicine, clinical trials, electronic health records, and genomics. Uses of big data in smart cities are also summarized, such as for smart transport, traffic management, smart energy, and smart governance. Specific examples and case studies are provided to illustrate the benefits and savings achieved from leveraging big data across these various sectors.
People are sometimes intimidated by big data because it seems overwhelming and they’re much more familiar with using statistics on survey data or analyzing opinions from focus group data. But here are nine examples from companies like Netflix, Ceasars Entertainment, Walmart, eBay, and UPS, that could have conducted survey or focus group research have instead used big data to accomplish big things.
This document discusses big data and Hadoop. It defines big data and Hadoop, and explains how big data can transform businesses through predictive analytics, understanding markets and customers, and optimizing business processes. It also outlines the challenges of utilizing big data, including data, process, security, and privacy challenges. Hadoop is introduced as an open source framework for storing and processing big data across clustered systems, and some of the challenges in implementing Hadoop are discussed.
The web-conference hosted by CRISIL Global Research & Analytics on “Big Data’s Big Impact on Businesses” on January 29, 2013, saw participation from senior officials of global multinationals from 9 countries. The presentation described how data analytics is helping businesses make “evidence-based” decisions, thereby creating a positive impact. It also spoke about the opportunities opening up in the Big Data space in India and across the globe.
Hosted by:
Sanjeev Sinha, President, CRISIL Global Research & Analytics
Gaurav Dua, Director & Practice Leader (Technology, Media & Telecom), CRISIL Global Research & Analytics
Viet-Trung Tran presents information on big data and cloud computing. The document discusses key concepts like what constitutes big data, popular big data management systems like Hadoop and NoSQL databases, and how cloud computing can enable big data processing by providing scalable infrastructure. Some benefits of running big data analytics on the cloud include cost reduction, rapid provisioning, and flexibility/scalability. However, big data may not always be suitable for the cloud due to issues like data security, latency requirements, and multi-tenancy overhead.
SC4 Workshop 1: Logistics and big data German herreroBigData_Europe
This document discusses the potential of big data in logistics. It notes that big data in logistics is characterized by large volumes of diverse data from both structured and unstructured sources. Applying advanced analytics to big data can provide greater insights across supply chains, enabling more efficient routing, inventory control, and issue resolution. Key challenges to realizing big data's potential in logistics include identifying appropriate business cases, gaining data sharing between stakeholders, and developing data science expertise in the logistics field.
Big Data Public-Private Forum_General PresentationBIG Project
The BIG project aims to establish a public-private forum on big data in Europe. It will bring together stakeholders from industry and research to develop a shared vision for big data in Europe over the next 5-10 years. The project will identify existing big data technologies, understand how big data can be applied across sectors, and develop roadmaps for specific sectors. It seeks to increase European competitiveness and position big data research priorities in the EU framework program Horizon 2020. The project is coordinated across four work packages covering management, strategy and operations, dissemination, and establishing the public-private forum.
Kanwal Prakash Singh gave a presentation at the India Analytics and Big Data Summit 2015 in Mumbai on February 3rd. He discussed how Housing uses data science and analytics to optimize operations like reducing flat duplication, forecasting supply and demand, and route optimization. Specifically, he described how Housing analyzed data to identify issues with low data collector utilization and many travelers between jobs. This led to developing a new branch location identification model and scheduling algorithm that optimally allocated listing requests to data collectors, reducing operational costs by 30% while making the solutions transferable to other delivery systems.
This document discusses cloud computing and big data. It notes that while cloud computing standards have been developed since 2010 through organizations like ISO and ITU, big data standards are still being investigated. The relationship between cloud computing and big data is that big data will increasingly be stored and processed in the cloud due to the large data volumes. The document raises questions about what should be priorities for big data tools and services, and concludes that standards development is still needed to guide big data and cloud computing.
Presentation: Study: #Big Data in #Austria, Mario Meir-Huber, Big Data Leader Eastern Europe, Teradata GmbH & Martin Köhler, Austrian Institute of Technology, AIT (AT), at the European Data Economy Workshop taking place back to back to SEMANTiCS2015 on 15 September 2015 in Vienna.
This document discusses how big data can be used for smart cities. It provides three use cases: 1) social media and sentiment analysis to analyze opinions on topics, 2) retail analytics and recommendation engines to predict customer product preferences, and 3) security and risk management using machine learning to detect fraud patterns. It also discusses optimizing data warehouses to improve data services while reducing costs. Big data systems like Hadoop and Spark are introduced for large-scale storage and processing.
This document proposes a theme on big data analytics research. It notes that the world's data storage capacity doubles every 40 months and discusses how big data can provide value across many areas like health, policymaking, education and more. The proposal recommends that Hong Kong develop a state-of-the-art big data platform to make a difference in areas like smart cities and support aging populations. It outlines objectives like large-scale machine learning from big data and discusses how Hong Kong is well-positioned for this research with experts across universities and potential collaborators in industry. The expected outcomes include new methodologies, applications impacting society and industry, and educational programs to cultivate big data leaders.
Data 4 AI: For European Economic Competitiveness and Societal ProgressEdward Curry
1) Data is a key resource for developing artificial intelligence systems, but difficulties accessing data can reduce innovation and competition.
2) Data platforms and data sharing spaces will fuel the development of AI-driven decision-making by facilitating access to and portability of data.
3) The Big Data Value Association advocates for policies and technologies that create trusted frameworks for sharing data across sectors and borders to advance AI for European economic competitiveness and societal progress.
This document contains information about a group project on big data. It lists the group members and their student IDs. It then provides a table of contents and summaries various topics related to big data, including what big data is, data sources, characteristics of big data like volume, variety and velocity, storing and processing big data using Hadoop, where big data is used, risks and benefits of big data, and the future of big data.
Big Data Trends - WorldFuture 2015 ConferenceDavid Feinleib
David Feinleib's Big Data Trends presentation from the World Future Society's Annual Conference, WorldFuture 2015, held at the Hilton Union Square, San Francisco, California July 25, 2015.
A top-down look at current industry and technology trends for Big Data, Data Analytics and Machine Learning (cognitive technologies, AI etc.). New slides added for Ark Group presentation on 1st December 2016.
Big Data Applications & Analytics Motivation: Big Data and the Cloud; Centerp...Geoffrey Fox
Motivating Introduction to MOOC on Big Data from an applications point of view https://bigdatacoursespring2014.appspot.com/course
Course says:
Geoffrey motivates the study of X-informatics by describing data science and clouds. He starts with striking examples of the data deluge with examples from research, business and the consumer. The growing number of jobs in data science is highlighted. He describes industry trend in both clouds and big data.
He introduces the cloud computing model developed at amazing speed by industry. The 4 paradigms of scientific research are described with growing importance of data oriented version. He covers 3 major X-informatics areas: Physics, e-Commerce and Web Search followed by a broad discussion of cloud applications. Parallel computing in general and particular features of MapReduce are described. He comments on a data science education and the benefits of using MOOC's.
Big Data & Analytics (Conceptual and Practical Introduction)Yaman Hajja, Ph.D.
A 3-day interactive workshop for startups involve in Big Data & Analytics in Asia. Introduction to Big Data & Analytics concepts, and case studies in R Programming, Excel, Web APIs, and many more.
DOI: 10.13140/RG.2.2.10638.36162
On Big Data Analytics - opportunities and challengesPetteri Alahuhta
This document discusses big data analytics and its opportunities and challenges. It defines big data and explains the increasing number of "V's" that characterize big data, such as volume, velocity, variety, and veracity. It also outlines some common uses of big data analytics including customer insights, security and risk analysis, and resource optimization. Additionally, it discusses challenges of big data adoption like skills shortages and infrastructure limitations, as well as trends in big data and areas of expertise related to big data that VTT focuses on.
Big Data may well be the Next Big Thing in the IT world
Big data burst upon the scene in the first decade of the 21st century
The first organizations to embrace it were online and startup firms. Firms like Google , eBay , LinkedIn , and Facebook were built around big data from the beginning.
Like many new information technologies , bigdata can bring about dramatic cost reductions , substantial improvements in the time required to perform to computing task , or new product and service offerings.
This document discusses various applications of big data across different domains. It begins by defining big data and its key characteristics of volume, variety and velocity. It then discusses how big data is being used in social media for recommendation systems, marketing, electioneering and influence analysis. Applications in healthcare discussed include personalized medicine, clinical trials, electronic health records, and genomics. Uses of big data in smart cities are also summarized, such as for smart transport, traffic management, smart energy, and smart governance. Specific examples and case studies are provided to illustrate the benefits and savings achieved from leveraging big data across these various sectors.
People are sometimes intimidated by big data because it seems overwhelming and they’re much more familiar with using statistics on survey data or analyzing opinions from focus group data. But here are nine examples from companies like Netflix, Ceasars Entertainment, Walmart, eBay, and UPS, that could have conducted survey or focus group research have instead used big data to accomplish big things.
This document discusses big data and Hadoop. It defines big data and Hadoop, and explains how big data can transform businesses through predictive analytics, understanding markets and customers, and optimizing business processes. It also outlines the challenges of utilizing big data, including data, process, security, and privacy challenges. Hadoop is introduced as an open source framework for storing and processing big data across clustered systems, and some of the challenges in implementing Hadoop are discussed.
The web-conference hosted by CRISIL Global Research & Analytics on “Big Data’s Big Impact on Businesses” on January 29, 2013, saw participation from senior officials of global multinationals from 9 countries. The presentation described how data analytics is helping businesses make “evidence-based” decisions, thereby creating a positive impact. It also spoke about the opportunities opening up in the Big Data space in India and across the globe.
Hosted by:
Sanjeev Sinha, President, CRISIL Global Research & Analytics
Gaurav Dua, Director & Practice Leader (Technology, Media & Telecom), CRISIL Global Research & Analytics
Viet-Trung Tran presents information on big data and cloud computing. The document discusses key concepts like what constitutes big data, popular big data management systems like Hadoop and NoSQL databases, and how cloud computing can enable big data processing by providing scalable infrastructure. Some benefits of running big data analytics on the cloud include cost reduction, rapid provisioning, and flexibility/scalability. However, big data may not always be suitable for the cloud due to issues like data security, latency requirements, and multi-tenancy overhead.
SC4 Workshop 1: Logistics and big data German herreroBigData_Europe
This document discusses the potential of big data in logistics. It notes that big data in logistics is characterized by large volumes of diverse data from both structured and unstructured sources. Applying advanced analytics to big data can provide greater insights across supply chains, enabling more efficient routing, inventory control, and issue resolution. Key challenges to realizing big data's potential in logistics include identifying appropriate business cases, gaining data sharing between stakeholders, and developing data science expertise in the logistics field.
Big Data Public-Private Forum_General PresentationBIG Project
The BIG project aims to establish a public-private forum on big data in Europe. It will bring together stakeholders from industry and research to develop a shared vision for big data in Europe over the next 5-10 years. The project will identify existing big data technologies, understand how big data can be applied across sectors, and develop roadmaps for specific sectors. It seeks to increase European competitiveness and position big data research priorities in the EU framework program Horizon 2020. The project is coordinated across four work packages covering management, strategy and operations, dissemination, and establishing the public-private forum.
Kanwal Prakash Singh gave a presentation at the India Analytics and Big Data Summit 2015 in Mumbai on February 3rd. He discussed how Housing uses data science and analytics to optimize operations like reducing flat duplication, forecasting supply and demand, and route optimization. Specifically, he described how Housing analyzed data to identify issues with low data collector utilization and many travelers between jobs. This led to developing a new branch location identification model and scheduling algorithm that optimally allocated listing requests to data collectors, reducing operational costs by 30% while making the solutions transferable to other delivery systems.
Currently, India accounts for 2.5 per cent of the global Big Data market of $8 billion and is expected to grow to $1 billion by 2015. The average amount Indian companies spent on Big Data in 2012 was $ 9.5 million (about Rs 52 crore), according to a 2013 TCS study.
A presentation from Gavin Freeguard (senior researcher at the Institute for Government) opening the Young Policy Professionals event, ‘Public policy in the “big data” age’, held on 9 March 2016 at the National Audit Office, London.
Apache Hadoop India Summit 2011 talk "Informatica and Big Data" by Snajeev KumarYahoo Developer Network
This document discusses big data and Informatica's role in addressing big data challenges. It begins by explaining the rapid growth of data volumes from sources like the internet, social media, mobile devices and IoT. This has led to new big data applications in areas like sentiment analysis, operational efficiency, recommendations and prediction. The key big data challenges are around storage, processing and regulatory compliance of both structured and unstructured data. Hadoop has emerged as a popular solution, with technologies like HDFS, MapReduce, Pig and HBase. The document outlines several enterprise case studies using Hadoop. It positions Informatica as providing a comprehensive platform to enable data integration, quality and management for both traditional and big data sources, including enabling
A presentation from Roeland Beerten (Director of Policy and Public Affairs at the Royal Statistical Society) as part of the Young Policy Professionals event, ‘Public policy in the “big data” age’, held on 9 March 2016 at the National Audit Office, London.
Hitachi kokusai electric inc. financial and strategic swot analysis reviewCompanyProfile123
Companyprofilesandconferences.com glad to promote a new report on "Hitachi Kokusai Electric Inc. - Financial and Strategic SWOT Analysis Review" which provides you an in-depth strategic analysis of the company’s businesses and operations.
This document discusses big data and analytics opportunities in India. It provides an overview of India's growing digital footprint through increasing internet and social media penetration. It presents several case studies of big data and analytics applications in e-commerce, telecom, retail, sports/entertainment and governance. It also outlines some implementation challenges around data silos, talent availability and infrastructure readiness. Finally, it predicts that the Indian big data and analytics market will double in size to $375 million by 2018 with key growth areas including education, healthcare, governance and agriculture.
A presentation from Martin Ralphs (Office for National Statistics, head of the Good Practice team across the Government Statistical Service) as part of the Young Policy Professionals event, ‘Public policy in the “big data” age’, held on 9 March 2016 at the National Audit Office, London.
Barbara Ryan @OECD - 21 Sept 2015 - Water Policy in the Age of Big DataOECD Governance
Presentation of Dr. Barbara Ryan [Secretariat Director of the Intergovernmental Group on Earth Observations] at the OECD event "Water Policy in the Age of Big Data on 21 September 2015.
Big Data, analytics, and behavioral analysis can help combat financial crime by:
1) Identifying hidden relationships and detecting anomalies across large volumes of structured and unstructured data.
2) Analyzing both financial and non-financial transactions to better understand customer behavior.
3) Using predictive models, social network analysis, and other techniques to surface emerging threats and reduce false positives.
This document discusses how Hadoop is used at Aadhaar to store and analyze large volumes of identity data. Key points:
1) Aadhaar handles 600-800 million identity records with 1 million new enrollments daily, totaling over 30TB of data stored across various Hadoop systems like HDFS, HBase, Hive and MySQL.
2) Authentication involves 100 million requests daily, generating over 4TB of encrypted audit logs every 10 days.
3) A complex architecture is used to distribute data across systems like HDFS, HBase and MySQL while ensuring low latency for queries, high throughput for analytics and data replication across data centers.
This document summarizes how the role of data has grown exponentially since 2010 when the term "big data" was introduced. Some key points made are:
- The amount of data in the world doubles every two years and is predicted to reach 44 zettabytes by 2020.
- 80% of data is now unstructured, presenting new data management challenges.
- Many governments have made more data openly accessible, and countries with more open data tend to be less corrupt.
- Advances in data and artificial intelligence are automating many jobs but also improving recruitment, project management, and other business functions for companies that master how to leverage data.
These slides by the OECD Competition Division introduce the OECD background note presented during the discussion on "Big Data: Bringing competition policy to the digital era" held during the 126th meeting of the OECD Competition Committee on 29 November 2016. More papers and presentations on the topic can be found out at www.oecd.org/daf/competition/big-data-bringing-competition-policy-to-the-digital-era.htm
As more companies grow their business in global markets, they discover the need to capture new opportunities in a matter of days rather than months to have competitive advantage and to capture new market share. Their machines are producing terabytes of various data types — video, audio, Microsoft® SharePoint®, sensor data, Microsoft Excel® files — and leaders are searching for the right technologies to capture this data and help provide a better understanding of their business. The HDS big data product roadmap will help customers build a big data enterprise plan that ingests data faster and correlate meaningful data sets to create intelligence that’s easy to consume and helps leaders make the right business decisions. View this webcast to learn about Hitachi’s product roadmap to big data. For more information on HDS Big Data Solutions please visit: http://www.hds.com/solutions/it-strategies/big-data/?WT.ac=us_mg_sol_bigdat
This presentation by Maurice E. Stucke from the Konkurrenz Group was made during the discussion on "Big Data: Bringing competition policy to the digital era" held during the 126th meeting of the OECD Competition Committee on 29 November 2016. More papers and presentations on the topic can be found out at www.oecd.org/daf/competition/big-data-bringing-competition-policy-to-the-digital-era.htm
Key Technology Trends for Big Data in EuropeEdward Curry
In this presentation we will discuss some of the results of the BIG project including analysis of foundational Big Data research technologies, technology and strategy roadmaps to enable business to understand the potential of Big Data technologies across different sectors, and the necessary collaboration and dissemination infrastructure to link technology suppliers, integrators and leading user organizations.
Edward Curry is leading the Technical Working Group of the BIG Project with over 30 committed experts along the big data value chain (Acquisition, Analysis, Curation, Storage, Usage). With the help of the other technical leads, he will elaborate on the key technology trends identified in the BIG Project and how they bring data-driven value to industrial sectors.
New Horizons for a Data-Driven Economy – A Roadmap for Big Data in Europe inside-BigData.com
In this video from the ISC Big Data'14 Conference, Edward Curry from the NUI Galway & Nuria de Lama Sanchez from Atos present: New Horizons for a Data-Driven Economy – A Roadmap for Big Data in Europe.
"In this talk we summarize the results of the BIG project including analysis of foundational Big Data research technologies, technology and strategy roadmaps to enable business to understand the potential of Big Data technologies across different sectors, together with the necessary collaboration and dissemination infrastructure to link technology suppliers, integrators and leading user organizations."
Learn more:
http://www.isc-events.com/bigdata14/schedule.html
and
http://big-project.eu/
Watch the video presentation: http://wp.me/p3RLEV-37G
The document discusses the collision of big data in biomedical imaging. Specifically, it notes that population image data from millions of hardware devices and thousands of software tools creates the perfect storm for big data in computational neuroimaging and digital pathology. It provides examples of how terabytes of raw imaging data and petabytes of derived analytical results are being generated from sources like digital pathology and neuroimaging studies. Managing and analyzing this large, multi-modal medical imaging data requires scalable big data techniques and architectures.
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaMaria de la Iglesia
Según Hal Varian (experto en microeconomía y economía de la información y, desde el año 2002, Chief Economist de Google) “En los próximos años, el trabajo más atractivo será el de los estadísticos: La capacidad de recoger datos, comprenderlos, procesarlos, extraer su valor, visualizarlos, comunicarlos serán todas habilidades importantes en las próximas décadas. Ahora disponemos de datos gratuitos y omnipresentes. Lo que aún falta es la capacidad de comprender estos datos“.
This document proposes a theme on big data analytics research. It motivates the importance of big data due to the exponential growth of digital data and limitations of traditional databases. The power of big data analytics is discussed through its wide applications in health, policymaking, smart cities, education and robotics. The objectives are outlined as large-scale machine learning, distributed computing, theory development, and multi-disciplinary analytics. Hong Kong is well positioned for this research due to its institutions, industries and potential collaborators. A multi-university and interdisciplinary approach is advocated to tackle big data challenges and transform society through new technologies, applications, insights and knowledge.
This document proposes a theme on big data analytics to be conducted by researchers from multiple universities in Hong Kong. It motivates the theme by discussing the rapid growth of data and potential of big data to impact many aspects of life. The objectives are to develop state-of-the-art big data platforms and analytics to benefit Hong Kong in areas like smart cities, health, finance and education. The researchers have strengths in relevant areas and Hong Kong has data and potential partners in industry. The theme aims to produce new methodologies, applications and digital economies through interdisciplinary collaboration between academia, government and industry.
Towards a BIG Data Public Private PartnershipEdward Curry
Building an industrial community around Big Data in Europe is the priority of the BIG: Big Data Public Private Forum project. In this workshop we will present the work of the project including analysis of foundational Big Data research technologies, technology and strategy roadmaps to enable business to understand the potential of Big Data technologies, and the necessary collaboration and dissemination infrastructure to link technology suppliers, integrators and leading user organizations. BIG is working towards the definition and implementation of a clear strategy that tackles the necessary efforts in terms of Big Data research and innovation, while also providing a major boost for technology adoption and supporting actions for the successful implementation of the Big Data economy.
The document discusses big data analytics and creating a big data-enabled organization. It begins with an introduction to big data, defining it and explaining its four V's: volume, variety, velocity, and veracity. It then discusses big data analytics, explaining that it involves more than just data and requires methods like machine learning. The document provides examples of big data analytics in various industries and development contexts. It concludes by outlining three steps to creating a big data-enabled organization: 1) be clear on the specific questions and needs big data can address, 2) build an integrated foundation of data, tools, and skills, and 3) establish a culture of experimentation and learning from failures.
L'economia europea dei dati. Politiche europee e opportunità di finanziamento...Data Driven Innovation
L'economia europea dei dati: soluzioni politiche e giuridiche per realizzare un'economia dei dati a livello di Unione Europea, nell'ambito della strategia per il mercato unico digitale. La consultazione pubblica 'Building the European Data Economy'. Il paternariato pubblico privato (PPP) Big Data Value ed opportunità di finanziamento in Horizon 2020. L'incubatore Data Pitch: opportunità per Start-up e Piccole e Medie Imprese.
Borys Pratsiuk is the Head of R&D at an unnamed company. He has over 15 years of experience in engineering roles related to Android development, embedded systems, and solid state electronics. He holds a PhD in Solid State Electronics from Kiev Polytechnic Institute and has worked in both academic and industry roles in South Korea and Ukraine. The presentation discusses big data, analytics, artificial intelligence and machine learning applications across various industries. It provides examples of deep learning solutions developed for clients in areas like computer vision, natural language processing, predictive analytics and process automation. The presentation emphasizes Ciklum's full-service approach to developing and deploying deep learning solutions from data collection and modeling to deployment and ongoing support.
In the third part of the workshop series Smart Policies for Data, we will focus on two central building blocks – interoperability and balanced data sharing.
The presentations of the event:
- Szymon Lewandowski, DG CONNECT, European Commission
- Marko Turpeinen, CEO, 1001 Lakes
- Lars Nagel, CEO, International Data Spaces Association
The document discusses big data and its potential applications for payment cards. It summarizes the European Commission's concerns about Europe falling behind the US and China in embracing big data. The Commission is calling for initiatives to develop enabling technologies, share public data, and ensure legal frameworks support innovation. The document defines big data and outlines typical benefits like improved marketing, pricing, and supply chain optimization. It also discusses how payment card schemes could be sources and users of big data to enhance fraud prevention, customer segmentation, merchant support, and new product development.
Structuring Big Data results to create new information: Smart Data. These Smart Data can be used to advance knowledge and support decision-making processes.
A close cooperation between industry and science creates better conditions for cutting-edge research in Data Engineering/Smart Data.
Big data promises to make the world more intelligent by identifying patterns and allowing decisions to be made based on large datasets rather than limited expertise. However, individual privacy must be protected and public interests considered. Economically, big data allows high-velocity analysis of large, diverse datasets to extract value. This changes business models and relationships as firms optimize operations and networks using data. Policymakers must ensure big data is used responsibly while creating opportunities for stakeholders.
Cross-Disciplinary Insights on Big Data Challenges and SolutionsBYTE Project
This document summarizes a cross-disciplinary workshop on big data challenges and solutions. It began with an introduction to the BYTE project, which aims to address societal externalities of big data through a multi-disciplinary community and roadmap. Presentations then covered big data applications and externalities in smart cities, oil and gas, and crisis management. Workshop participants discussed the key externalities identified for each sector and potential solutions. The session concluded by inviting attendees to join the BYTE community to further address big data challenges through a cross-disciplinary approach.
This document provides an overview of big data in Catalonia. It defines big data as large volumes of complex data that require new technologies to extract value. Big data is characterized by its volume, variety, and velocity. The document discusses open data and data ethics. It also reviews the global big data market and how big data is important for different industries. Finally, it examines big data applications, the big data ecosystem in Catalonia, business cases, and the role of big data in addressing COVID-19.
This document provides an overview of big data in Catalonia. It defines big data as large volumes of data that require new technologies to process and extract value due to issues with volume, variety, and velocity. It discusses related concepts like open data, data ethics, artificial intelligence, machine learning, and more. The document also examines the global big data market and applications across different industry sectors and UN Sustainable Development Goals.
The event presents real-life examples from European organisations that have used the Rulebook for Fair Data Economy to develop data-driven business. The online event was organised on 3 March 2021 by Sitra.
Presentations:
- Jaana Sinipuro, Sitra
- Olli Pitkänen, 1001 Lakes
- Marko Turpeinen, 1001 Lakes
- Lars Nagel, International Data Spaces Association
- Cátia Pinto, Serviços Partilhados do Ministério da Saúde
- Matthias De Bièvre, aNewGovernance
1. Determine if a Big Data approach is suitable based on factors like volume, variety and velocity of data as well as the need for iterative, exploratory analysis.
2. Use techniques like Hadoop, MapReduce and NoSQL databases that can analyze large, diverse, unstructured datasets in a distributed, parallel manner.
3. Follow data management best practices like data governance, quality checks, and master data management to ensure clean, well-organized data.
Similar to Towards a big data roadmap for europe (20)
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/how-axelera-ai-uses-digital-compute-in-memory-to-deliver-fast-and-energy-efficient-computer-vision-a-presentation-from-axelera-ai/
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit.
As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability and privacy. This migration, however, introduces the challenge of operating within the stringent confines of resource constraints typical at the edge, including small form factors, low energy budgets and diminished memory and computational capacities. Axelera AI addresses these challenges through an innovative approach of performing digital computations within memory itself. This technique facilitates the realization of high-performance, energy-efficient and cost-effective computer vision capabilities at the thin and thick edge, extending the frontier of what is achievable with current technologies.
In this presentation, Verhoef unveils his company’s pioneering chip technology and demonstrates its capacity to deliver exceptional frames-per-second performance across a range of standard computer vision networks typical of applications in security, surveillance and the industrial sector. This shows that advanced computer vision can be accessible and efficient, even at the very edge of our technological ecosystem.
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.
The Microsoft 365 Migration Tutorial For Beginner.pptxoperationspcvita
This presentation will help you understand the power of Microsoft 365. However, we have mentioned every productivity app included in Office 365. Additionally, we have suggested the migration situation related to Office 365 and how we can help you.
You can also read: https://www.systoolsgroup.com/updates/office-365-tenant-to-tenant-migration-step-by-step-complete-guide/
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
1. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
TOWARDS A BIG DATA ROADMAP
FOR EUROPE
Martin Strohbach, AGT International
Tilman Becker, Edward Curry, John Domnique, Ricard
Munné, Sebnem Rusitschka, Sonja Zillner
2. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
2
OVERVIEW
• Big Data Public Private Forum and Partnership
• 3 Data Sharing Use Cases
• Methodology for creating a community based roadmap
• Findings from the Use Cases, Sectors and Data Value Chain Analysis
• Impact of the findings towards a European roadmap
• Conclusions
3. 2nd JTC 1 SGBD Meeting - Workshop
THE EU PROJECT BIG
BIG DATA PUBLIC PRIVATE FORUM
Europe needs a clear strategy for leveraging Big Data
Economy in Europe
Work at technical, business and policy levels, shaping the
future through the positioning of Big Data in Horizon 2020.
Bringing the necessary stakeholders into a sustainable
industry-led initiative, which will greatly contribute to
enhance the EU competitiveness taking full advantage of
Big Data technologies.
Objectives
Trigger
Type of project: Coordination & Support Action
Project start date: September 2012
Duration: 26 months
Call: FP7-ICT-2011-8
Budget: 3,038 M€
Consortium: 11 partners
Facts
4. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
4
BIG DATA PRIVATE PUBLIC PARTNERSHIP
• Objectives: „The goal […] is to increase the amount
of productive European economic activities and the
number of European jobs that depend on the
availability of high quality data assets and the
technologies needed to derive value from them.”
(Source. Strategic Research and Innovation Agenda, SRIA)
• Neelie Kroes, EU Commissioner for the Digital
Agenda: „This is a revolution and I want the EU
to be right at the front of it“
• 29 European organisations from
industry and research
• Results of public consultattion to
be presented at NESSI Summit
Brussels, 27th May
5. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
SELECTED USE CASES
6. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
6
OVERCOMING THE DATA SILO CULTURE IN
PUBLIC SECTOR
ID data
Tax data
Civil
Registry
Home Office
Tax Agency
Civil Registry
Education
Ministry
Local
Administration
Data
aggregation
process
Citizen
Scholarship
Registry
7. 2nd JTC 1 SGBD Meeting - Workshop
USE CASE HEALTHCARE
SECONDARY USAGE OF HEALTH DATA
▶ Secondary usage of health data is defined as the
aggregation, analysis and concise presentation of
clinical, financial, administrative as well as other
related health data
▶ in order to discover new valuable knowledge, for
instance to identify trends, predict outcomes or
influence patient care, drug development, or
therapy choices.
Description
Example Use Cases
▶ Example 1: Patient recruiting and profiling suitable
for conducting clinical studies. Today, often
clinical studies, in particular studies investigating
rare diseases, fail due to the fact that not enough
patients are available for conducting clinical
studies (Berliner Forschungsplattform Gesundheit
(BFG), Astra Zeneca)
▶ Example 2: Social Media Based Medication
Intelligence Mining of health data on the discover
insights about side effects of medications (e.g.
Treato)
8. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
8
PEER ENERGY CLOUD
Smart grid pilot in Saarlouis
100 households
Berlin
SaarlouisInnovation award
Engage consumers to optimally
use local solar energy
Understand consumption and
save
Trade solar energy in the
neighborhood to balance
the grid
9. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
9
DEVICE LEVEL ENERGY MONITORING
Monitored/controlled grid today
Monitored/controlled grid tomorrow
Germany aims at 30%
clean/renewable energy by
2020, seeking to build a smart
grid
Sensors
today
Sensors
tomorrow
(consumer
level)
Energy
Consumption
Temperature
Movement,...
10. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
10
GETTING READY FOR DATA VOLUMES IN
FUTURE GRIDS
PeerEnergyCloud Pilots allows us to get ready for future data
volumes today
How much data is really needed
for what?
1 value per
year
35.040 values
per year
today smart
metering
540 million
values per year
PeerEnergy-
Cloud
? Billion values
per year
Future
possibilities
Optimum?
7 devices per
household every
2 seconds , 4-5
measurements
per devices
every 15
minutes
real-time analytics
on mass data (grouped
aggregation)
Scalable statistics
over hundreds of millions
of measurements
Automatic detection
of load anomalies
(spotting inefficiencies
and defects)
Household activity
state inference and
prediction
11. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
BIG METHODOLOGY
12. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
12
SECTORIAL FORUMS AND TECHNICAL
WORKING GROUPS
Health Public Sector
Finance &
Insurance
Telco, Media&
Entertainment
Manufacturing,
Retail, Energy,
Transport
Needs Offerings
Big Data Value Chain
Technical Working Groups
Industry Driven Sectorial Forums
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
• Structured data
• Unstructured data
• Event processing
• Sensor networks
• Protocols
• Real-time
• Data streams
• Multimodality
• Structured data
• Unstructured data
• Event processing
• Sensor networks
• Protocols
• Real-time
• Data streams
• Multimodality
• Stream mining
• Semantic analysis
• Machine learning
• Information
extraction
• Linked Data
• Data discovery
• ‘Whole world’
semantics
• Ecosystems
• Community data
analysis
• Cross-sectorial data
analysis
• Stream mining
• Semantic analysis
• Machine learning
• Information
extraction
• Linked Data
• Data discovery
• ‘Whole world’
semantics
• Ecosystems
• Community data
analysis
• Cross-sectorial data
analysis
• Data Quality
• Trust / Provenance
• Annotation
• Data validation
• Human-Data
Interaction
• Top-down/Bottom-up
• Community / Crowd
• Human Computation
• Curation at scale
• Incentivisation
• Automation
• Interoperability
• Data Quality
• Trust / Provenance
• Annotation
• Data validation
• Human-Data
Interaction
• Top-down/Bottom-up
• Community / Crowd
• Human Computation
• Curation at scale
• Incentivisation
• Automation
• Interoperability
• In-Memory DBs
• NoSQL DBs
• NewSQL DBs
• Cloud storage
• Query Interfaces
• Scalability and
Performance
• Data Models
• Consistency,
Availability, Partition-
tolerance
• Security and Privacy
• Standardization
• In-Memory DBs
• NoSQL DBs
• NewSQL DBs
• Cloud storage
• Query Interfaces
• Scalability and
Performance
• Data Models
• Consistency,
Availability, Partition-
tolerance
• Security and Privacy
• Standardization
• Decision support
• Prediction
• In-use analytics
• Simulation
• Exploration
• Visualisation
• Modeling
• Control
• Domain-specific
usage
• Decision support
• Prediction
• In-use analytics
• Simulation
• Exploration
• Visualisation
• Modeling
• Control
• Domain-specific
usage
13. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
13
SECTOR ANALYSIS METHODOLOGY
14. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
14
TECHNICAL WORKGROUP APPROACH
Senior
Academic
Senior
Management
Middle
Researcher
Middle
Management
Position in Organisation
University
MNC
SME
Other
Types of Organisations
1. Literature & Technical Survey
2. Subject Matter Expert Interviews
3. Stakeholder Workshops
4. Online Questionnaire (with
NESSI)
• Early adopters
• Business enablement
• Technical maturity
• Key Opinion Leaders
Methodology
Interviewee Breakdown
Target Interviewee
15. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
TWG FINDINGS
16. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
16
KEY INSIGHTS
Key Trends
▶ Lower usability barrier for data tools
▶ Blended human and algorithmic data processing for coping with
for data quality
▶ Leveraging large communities (crowds)
▶ Need for semantic standardized data representation
▶ Significant increase in use of new data models (i.e. graph)
(expressivity and flexibility)
▶ Much of (Big Data) technology
is evolving evolutionary
▶ But business processes change
must be revolutionary
▶ Data variety and verifiability
are key opportunities
▶ Long tail of data variety is a
major shift in the data landscape
The Data Landscape
▶ Lack of Business-driven Big Data
strategies
▶ Need for format and data storage
technology standards
▶ Data exchange between
companies, institutions, individuals,
etc.
▶ Regulations & markets for data
access
▶ Human resources: Lack of skilled
data scientists and data
engineers
Biggest Blockers
17. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
17
DATA VALUE CHAIN - ANALYSIS
Key Insights
Old technologies applied in a new context
(Volume, Variety, Velocity)
Need for
Stream data mining
‘Good’ data discovery
Techniques to deal with dealing with both
very broad and very specific data
Features to Increase Take-up
Simplicity including the ‘democratisation of
semantic technologies’
Ecosystems of tools
Communities and Big Data will be involved
in new and interesting relationships
In collection, improving data accuracy,
analysis and usage
Improves community engagement
Cross-sectorial uses of Big Data will open
up new business opportunities
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Social & Economic Impacts
• Cross-sectorial businesses
Data-cleaning today requires a lot of work –
area for new business
eHealth transformed from push (patient
visits doctor) to pull (continuous
monitoring)
Also large scale trend analysis
Conduit between research and individual
patient care
E.g. creation of patient avatars based
on combination of specific data and
generic research data
Proactive fraud detection even before it
happens
Engaged citizens create and have access to
all data related to local, regional and
national social and policy issues
18. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
18
DATA VALUE CHAIN - CURATION
State of the Art
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Use Case
Master Data Management (MDM)
Centralized, single point of reference for
data representation
Collaboration platforms & Web 2.0 tools
Wikis, Content Management Systems (CMS)
Early-stage crowdsourcing services
E.g. Amazon Mechanical Turk
Health and Life Sciences
ChemSpider: Collaborative platform for data
curation of chemical structures
Protein Data Bank: Data curation platform
for 3D protein structures
FoldIt: Crowdsourcing game platform for
protein folding
Telco, Media, Entertainment
Press Association, Thomson Reuters, The
New York Times
Use of semantic technologies to structure
and categorize unstructured data (texts,
images and videos), improving content
accessibility and reuse
Retail
Ebay: use of crowdsourcing services to
improve product categorization
Unilever: use of crowdsourcing services for
product feedback and sentiment analysis
19. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
19
DATA VALUE CHAIN - CURATION
Future Requirements
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Emerging Trends
Creation of incentives mechanisms for the
maintenance and publication of curated
datasets
Definition of models for the data economy
Understanding of social engagement
mechanisms
Reduction of the cost associated with the data
curation task (scalability)
Improvement of the Human-Data interaction
aspects. Enabling domain experts and casual
users to query, explore, transform and curate
data
Inclusion of trustworthiness mechanisms in
data curation
Integration and interoperability between data
curation platforms / Standardization
Investigation of theoretical and domain specific
models for data curation
Better integration between unstructured and
structured data and tools
Incentives and social engagement
Better recognition of the data curation role
Understanding of social engagement mechanisms
Economic Models
Pre-competitive and public-private partnerships
Curation at Scale
Evolution human computation and crowdsourcing
Instrumenting popular apps for data curation
General-purpose data curation pipelines
Human-data interaction
Trust
Capture of data curation decisions & provenance
management
Fine-grained permission management models
and tools
Standardization & interoperability
Standardized data model and vocabularies
Better integration between data curation tools
Data Curation Models
Minimum information models
Nanopulications
Theoretical principles and domain-specific model
20. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
20
DATA VALUE CHAIN - STORAGE
Key Insights
From state-of-the art
Capability to store and manage virtually
unbounded volumes of data has huge potential
to transform society and business
Better Scalability at Lower Operational Complexity
and Costs
Big Data Storage Has Become a Commodity
Business.
Maturity Has Reached Enterprise-Grade Level
Open Challenges
Unclear Adoption Paths for Non-IT Based Sectors
Lack of standards and best practices is major
barrier for adoption
Open Scalability Challenges
Privacy and Security is Lacking Behind
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Social & Economic
Impacts
Enables a truly data-driven economy that is
able to manage a variety of data sets in an
integrated way
Privacy and Security becomes more important
Energy: high resolution smart meter data
contributes to the stable integration of renewable
energies
Health Care: storage is key enabling technology
to solve integration challenges
Media: enables using the voice of the crowd
Transport: facilitates personalized and multi-
modal on large scale
21. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
21
DATA VALUE CHAIN - STORAGE
Future Requirements
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Emerging Trends
Standardization of Query Interfaces
Legal Frameworks for data management
Data Tracing and provenance in order to assess
trust in data and ensure compliance
Better support and scalability for storing
semantic data models, e.g. in health care
Graph databases will become more important
Columnar stores experience wider adoptions as
they are often faster in most practical
applications
Convergence with analytical frameworks
Analytical databases for better performance and
lower development complexity (Mahout, Spark,
Hadoop/R, rasdaman, SciDB)
Data Hubs and Markets: Hadoop-based
solutions tend to become a central integration
point for all enterprise, sectorial and cross-
sectorial data
Smart Data: only use relevant data in network
and at the edge processessing
22. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
22
DATA VALUE CHAIN - USAGE
Key Insights
Data
Acquisition
Data
Analysis
Data
Curation
Data
Storage
Data
Usage
Social & Economic Impacts
Key task of Data Usage is to support business
decisions
There is great variety of Big Data Applications.
Some key areas are:
Industry 4.0 (industrial internet)
Predictive maintenance
Smart data and service integration
Interactive exploration: Big Data generates
insights beyond existing models, new analysis
interfaces must support browsing and modeling
(visual analytics)
Opportunities for data markets: integration with
customer and supplier data, services from
infrastructure (Iaas) to software (SaaS) to
business processes (BPaaS) to knowledge (KaaS)
Regulatory issues:
Data protection and privacy
Ownership of derived data
Integration of business processes and data
Beyond single company; data market places
Transparency through Data Usage impacts:
Economy
Society
Privacy
Current drivers are big companies
Opportunities for SMEs though services
(integration) and data market places
23. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
USE CASE ANALYSIS
24. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
24
PUBLIC SECTOR
• What the Public sector is facing…
– Internal data silos → no integration of systems across public bodies
– Raising pressure on productivity → lacks productivity compared to the private sector
– Older workforce → compared to the private sector, relies on a far older workforce
– Aging population → increasing demand for medical and social services
• …and in what can Big data help:
– Improving many areas in public sector services:
Strengthen collaboration among PS bodies
Social welfare
Citizen services
Improve healthcare
Government transparency and accountability
Public safety
– Open Data initiatives → act as catalysts in the development of a data ecosystem through
the opening of their own datasets, and actively managing their dissemination and use
– Analysis of sensor data: Real time data processing to search for patterns and
relationships and present real-time views on Smart Cities for better urban management
and citizen engagement
25. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
25
Data Privacy and
Security
Legal framework supporting data
access and usage of citizen’s
information
PUBLIC SECTOR REQUIREMENTS
OVERCOMING THE DATA SILO CULTURE
Data ownership
Fragmentation of data ownership
that leads to the data silo and
interoperability issues
Data Sharing
Overcome data silos
because of legacy
technologies
Common strategy at
all levels of
administration
So much energy is lost and will
remain so until a common
strategy is realized for the reuse
of cross technology platforms
Not-Technology-related Regulation & Technology Technology-related
Legislative and
political willingness
To promote legislation that allows
the reuse of data for other
purposes than those for what it
was originally collected
Openness of Data
Leadership to promote common
standards for Government Open
Data: APIs, format and schemas,
as well as for open data licensing
26. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
26
Data Security
Legal processes for data sharing
& communication are needed
Data Quality
Reliable insights for health-related
decisions require high data quality
HEALTHCARE REQUIREMENTS
High Investment
Long-term investments require
conjoint engagement of several
partners
Data Digitalization
only small percentage of data is
documented (lack of time) with
low quality
Semantic Annotation
transform unstructured data into
structured format
Data Sharing
Overcome data silos
and inflexible
interfaces
Business Cases
Undiscovered und unclaimed
potential business values
Value-based system
incentives
Current incentives enforce “high
number” instead of “high
quality” of care services
Not-Technology-related Regulation & Technology Technology-related
27. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
27
DATA POOLS IN HEALTHCARE
MAIN IMPACT BY INTEGRATING VARIOUS AND
HETEROGENEOUS DATA SOURCES
Clinical Data
Owned by providers (such as
hospitals, care centers, physicians,
etc.)
Encompass any information stored
within the classical hospital
information systems or EHR, such as
medical records, medical images, lab
results, genetic data, etc.
Claims, Cost &
Administrative Data
Owned by providers and payors
Encompass any data sets relevant for
reimbursement issues, such as
utilization of care, cost estimates,
claims, etc.
Pharmaceutical &
R&D Data
Owned by the pharmaceutical
companies, research
labs/academia, government
Encompass clinical trials,
clinical studies, population and
disease data, etc.
Patient Behaviour &
Sentiment Data
Owned by consumers
or monitoring device
producer
Encompass any
information related to
the patient behaviours
and preferences
Health data on the
web
Mainly open source
Examples are
websites such as
PatientLikeMe,
Linked Open Data,
etc.
Highest Impact
on integrated data sets
28. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
28
ENERGY SECTOR REQUISITES
STAKEHOLDERS, USE CASES, DATA SOURCES
TSO1 TSO2
DSO11
DSO12
DSO21
price signals
renewables
parks
prosumer
weather
e-car roaming
sustainable connected cities
industrial DSM
New business requires the combined value creation on mass data coming from a
variety of data sources, once the volume and velocity of energy data is mastered
Flexible energy tariffs for prosumers and eCars
• Actual energy usage instead standardized profiles
Efficient portfolio management
• detailed data on market prices,
• actual energy usage, feed-in
• weather conditions
Increased situational awareness
• intermittent, decentralized supply
• new responsive demand
• network asset and weather conditions
TSO – Transmission System Operator
DSO – Distribution System Operator
NETWORK OPERATORS RETAIL/WHOLESALE
END USER ENERGY MANAGEMENT
29. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
29
A digitally united
European Union
European stakeholders require reliable
minimally consistent rules and
regulation regarding digital rights and
regulations, whilst making use of all
technological enhancements
Data Access
Dynamic configurability of data
access of which data can be
collected for what purpose in
what granularity and time span
and location
Privacy and confidentiality
preserving data analytics are
required to enable the service
provider to retrieve the
knowledge without violating the
agreed upon granularity
ENERGY SECTOR REQUIREMENTS
Investment in
communication and
connectedness
Broadband communication or ICT in
general, needs to be widely available
alongside energy and transportation
infrastructure such that real-time data
access is given
Abstraction
from the actual big data
infrastructure is required to enable
(a) ease of use and (b) extensibility
and flexibility
Adaptive models of data
and system model
new knowledge extracted from
domain analytics or the ever changing
circumstances of the system can be
redeployed into the data analytics
framework
Data interpretability &
analytics
Expert and domain know-how must
be blended into the data
management and analytics. Data
analytics is required as part of every
step from data acquisition to data
management to data usage. Fast and
even real-time analytics is required to
support decisions, which need to be
made in ever shorter time spans
Skilled people
programming, statistical tools for
example needs to be part of engineering
education until big data technology
becomes more business user-friendly or
in case this becomes the “new normal”
Standardization &
Open Data
Open data is a great opportunity,
but, standardization is required.
Regarding data models,
representation, as well as protocols
practical migration paths
Not-Technology-related Regulation & Technology Technology-related
30. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
CONCLUSIONS
31. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
31
(SOME) COMMON REQUIREMENTS
Data Privacy and
Security
Legal frameowrks for data
sharing & communication are
needed
High Investment
Long-term investments require
conjoint engagement of several
partners
Not-Technology-related
Data Digitalization
only small percentage of data is
documented (lack of time) with
low quality
Semantic Annotation
transform unstructured data into
structured format
Data Sharing
Overcome data silos
and inflexible
interfaces
Business Cases
Undiscovered und unclaimed
potential business values
Regulation & Technology Technology-related
Data Quality
Reliable insights for health-related
decisions require high data quality
Openness of Data
Leadership to promote common
standards for Open Data: APIs,
format and schemas, as well as
covering licensing and legal
aspects
Data ownership
Fragmentation of data ownership
that leads to the data silo and
interoperability issues
32. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
32
TECHNOLOGY ROADMAP DEVELOPMENT
▶ Sector-specific data management technologies that
need to be in place before any big data scenario
can be realized
▶ Ensure that the relevant data of the sector is
available (e.g. EHR, IED)
▶ Driven by user/need driven (sector) analysis
▶ Focus on domain-specific requirements of data
management and related research questions
Enabling Technologies
▶ Large Gap between needs mentioned by the stakeholder and users of the (health) sector and the
technological opportunities envisioned by the technical groups
▶ Technical requirements mentioned within sectorial interviews mainly relate to efficient data
management approaches
▶ Technical opportunities mentioned by the technical groups highlight opportunities within a world
of open data access, such a the web
▶ Challenge: How to align the two perspectives?
▶ First Step “Focus on big data readiness”: Any technological requirements, such as efficient data
management, need to be addressed /solved before big data capabilities can be implemented
(enabling technologies)
▶ Second Step: “Elaborate big data opportunities”: Develop transitional scenarios that could be
realized assuming that the sector has achieved big data readiness (big data technologies), scenarios
should generate value.
Observations & Learnings from sector and technical investigations
▶ advanced/big IT capabilities that help to improve
healthcare delivery
▶ Data-driven: Investigate in public available
health data sources as basis for use case
brainstorming
▶ Technology-Driven: Investigate to which
extent applications /technologies from other
domains can be transferred to the healthcare
sector
Big data opportunities
We need to distinguish between
33. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
33
Domain Specific
Services
TOWARDS A BIG DATA ROADMAP
Secure Data
Sharing
Semantic
enrichment
Anonymisation
Scalable
Storage
Distributed Data
Acquistion
Scalable Storage
Analytical
Platforms
Analytical
Databases
Distributed Processing
Complex Event
Processing
Semantic Processing
Explorative Analysis
Energy
Public Sector
Health
Encryption
Manufacturing
Retail
Telco
Media & Entertainment
34. 2nd JTC 1 SGBD Meeting - Workshop
BIG
Big Data Public Private Forum
34
SUMMARY
• BIG is contributing to the
creation of an industry-lead Big Data community in Europe
– Sector and Data Value Chain Analysis
– Initial results in SRIA
– Next steps: roadmap development, stakeholder platform
• 3 Data Sharing Use Cases Introduced
• Data Sharing is a key requirement for driving data driven-business on a
larger scale and evolve technologies
– Requires addressing issues across business (value), regulatory (society) and technical
(principled engineering and development of new capabilities)
– Initiatives such as PPP in combination with national will help address these issues
http://www.bigdatavalue.eu
35. 2nd JTC 1 SGBD Meeting - Workshop
Thank you
http://www.bigdatavalue.eu http://www.big-project.eu
Dr. Martin Strohbach
Senior Researcher, AGT International
Technical Lead Storage WG, BIG
mstrohbach@agtinternaitonal.com
Tilman Becker (DFKI, Data Usage), Edward Curry (NUI Galway, Data Curation), John
Domnique (STI, Data Analysis), Ricard Munné (ATOS, Public Sector), Sebnem
Rusitschka (Siemens, Energy and Transport), Holger Ziekow (AGT, PEC), Sonja Zillner
(Siemens, Health)