First online hangout SC5 - Big Data Europe first pilot-presentation-hangoutBigData_Europe
This document describes a pilot project to support data-intensive climate research. The pilot aims to provide researchers with an intuitive interface to search, download, and dynamically downscale climate model and observational data. It will orchestrate the downscaling process on institutional computational resources while managing data products and lineage. Currently, acquiring and preprocessing climate data for downscaling is an ad-hoc process. The pilot seeks to improve research productivity by facilitating efficient data access, model execution, and reuse of experiments through the Big Data Europe platform. It may help climate impact assessments in other societal challenge areas like energy, food, and agriculture.
Societal Challnge 5 and Big Data Europe 1st hangout BigData_Europe
SC5 focuses on addressing the effects of climate change through climate modeling, impact assessment, and developing climate services. The Big Data Europe (BDE) project aims to help users access and utilize large climate datasets through the development of an integrated data platform. BDE will engage climate researchers and stakeholders to design, build, and evaluate a prototype platform to facilitate big data use in climate science. Workshops will gather requirements, review architectures, and evaluate the platform and its pilots. BDE seeks to provide climate experts and technical specialists tools to better handle, publish, and apply big data resources.
SC6 Workshop 1: Big Data Europe platform requirements and draft architecture:...BigData_Europe
Presentation by Martin Kaltenböck, Semantic Web Company, at the first workshop of Societal Challlenge 6 in the BigDataEurope project, taking place in Luxembourg on 18 November 2015.
http://www.big-data-europe.eu/social-sciences/
BDE Technical Webinar 1 : Pilot InstantiationBigData_Europe
This document describes 7 pilots for addressing societal challenges through big data projects. The first pilot focuses on linking and integrating pharmaceutical research data for life sciences. The second pilot aims to support advanced crop data discovery, processing, and visualization for food and agriculture. The third pilot focuses on real-time turbine monitoring, stream processing and analytics for energy production monitoring. Further pilots address data integration for transport planning, supporting climate data research, integrating budget data for social sciences, and detecting man-made changes using remote sensing and social data for security applications.
Big Data technology for systems monitoring in Energy – Big Data Europe BigData_Europe
The document summarizes a workshop on using big data technology for energy system monitoring. It discusses the Big Data Europe project, which aims to develop an open-source big data management platform to address challenges across different domains. Big data can benefit the energy sector through applications like monitoring power infrastructure, forecasting renewable energy production, and managing smart grids. The platform will integrate various data analytics tools to extract insights from large and complex energy datasets.
SC4 Hangout 1: Big data europe transport webinar Philippe CristBigData_Europe
BigDataEurope organized its first webinar on the 21st September 10h00-11h00 (CET) to introduce the BigDataEurope project, in particular the domain of Smart, Green, and Integrated Transport.
The presentation was created and presented by Philippe Crist from OECD.
First online hangout SC5 - Big Data Europe first pilot-presentation-hangoutBigData_Europe
This document describes a pilot project to support data-intensive climate research. The pilot aims to provide researchers with an intuitive interface to search, download, and dynamically downscale climate model and observational data. It will orchestrate the downscaling process on institutional computational resources while managing data products and lineage. Currently, acquiring and preprocessing climate data for downscaling is an ad-hoc process. The pilot seeks to improve research productivity by facilitating efficient data access, model execution, and reuse of experiments through the Big Data Europe platform. It may help climate impact assessments in other societal challenge areas like energy, food, and agriculture.
Societal Challnge 5 and Big Data Europe 1st hangout BigData_Europe
SC5 focuses on addressing the effects of climate change through climate modeling, impact assessment, and developing climate services. The Big Data Europe (BDE) project aims to help users access and utilize large climate datasets through the development of an integrated data platform. BDE will engage climate researchers and stakeholders to design, build, and evaluate a prototype platform to facilitate big data use in climate science. Workshops will gather requirements, review architectures, and evaluate the platform and its pilots. BDE seeks to provide climate experts and technical specialists tools to better handle, publish, and apply big data resources.
SC6 Workshop 1: Big Data Europe platform requirements and draft architecture:...BigData_Europe
Presentation by Martin Kaltenböck, Semantic Web Company, at the first workshop of Societal Challlenge 6 in the BigDataEurope project, taking place in Luxembourg on 18 November 2015.
http://www.big-data-europe.eu/social-sciences/
BDE Technical Webinar 1 : Pilot InstantiationBigData_Europe
This document describes 7 pilots for addressing societal challenges through big data projects. The first pilot focuses on linking and integrating pharmaceutical research data for life sciences. The second pilot aims to support advanced crop data discovery, processing, and visualization for food and agriculture. The third pilot focuses on real-time turbine monitoring, stream processing and analytics for energy production monitoring. Further pilots address data integration for transport planning, supporting climate data research, integrating budget data for social sciences, and detecting man-made changes using remote sensing and social data for security applications.
Big Data technology for systems monitoring in Energy – Big Data Europe BigData_Europe
The document summarizes a workshop on using big data technology for energy system monitoring. It discusses the Big Data Europe project, which aims to develop an open-source big data management platform to address challenges across different domains. Big data can benefit the energy sector through applications like monitoring power infrastructure, forecasting renewable energy production, and managing smart grids. The platform will integrate various data analytics tools to extract insights from large and complex energy datasets.
SC4 Hangout 1: Big data europe transport webinar Philippe CristBigData_Europe
BigDataEurope organized its first webinar on the 21st September 10h00-11h00 (CET) to introduce the BigDataEurope project, in particular the domain of Smart, Green, and Integrated Transport.
The presentation was created and presented by Philippe Crist from OECD.
European Open Data Portal and Policy Compass: from national Open Data reposit...OW2
In November 2015 the European Commission officially lunched the European Data Portal http://www.europeandataportal.eu . The mission of the portal is to become the catalogue of all European public data providing them in all official languages of the European Union. The portal is harvesting metadata from heterogeneous open data portals of 28 EU and other 11 European countries. It lists over 580 000 datasets and it is the biggest Open Data portal worldwide. From the techincal perspective, it is the first official Open Data portal implementing the new DCAT Application Profile specification.
The portal is the place to find European public data and it is a basis for other innovative services. One of them is Policy Compass https://policycompass.eu. It brings together open public data, social media, e-participation platforms, causal models, and argumentation technology for constructing, sharing, visualizing and debating progress metrics and impacts of policies.
Both portals are Open Source. They provide rich APIs and may become a data source for other applications.
SC4 Hangout 1: BDE-Transport Webinar Simon ScerriBigData_Europe
BigDataEurope organized its first webinar on the 21st September 10h00-11h00 (CET) to introduce the BigDataEurope project, in particular the domain of Smart, Green, and Integrated Transport.
The presentation was created and presented by Simon Scerri from the University of Bonn.
This document identifies 9 elements for the development of open data marketplaces:
1) Bringing stakeholders together to match supply and demand
2) Providing rich metadata
3) Enabling data quality assessment
4) Ensuring trust, security and critical mass
5) Having an appropriate revenue model
6) Providing use cases, training and support
7) Providing technical support like open data processing tools
8) Providing a full API for machine-to-machine operation
9) Targeting multiple nationalities
General Introduction to the Oxford e-Research CentreDavid Wallom
Digital Oxford is a collaborative research hub that is transforming research through innovative digital methods. It focuses on key areas like data capture, curation, publication and standards to create technologies that disseminate and reuse research data. The hub works with various communities including curators, data producers, consumers, developers, researchers, and policy makers through its network of high performance computing, infrastructure, and communications systems.
Presentation from CNI Spring Membership Meeting 2018, describing four-part series of research reports examining university research data management services. Further information about this project can be found at oc.lc/rdm This joint presentation included a slide deck (not included here) describing research data services at the University of Illinois, Urbana-Champaign, presented by Heidi Imker.
SC6 Workshop 1: What can big data do for you? BigData_Europe
Presentation by Sören Auer, Fraunhofer IAIS, Coordinator of Big Data Europe, at the first workshop of Societal Challlenge 6 in the BigDataEurope project, taking place in Luxembourg on 18 November 2015.
http://www.big-data-europe.eu/social-sciences/
EDF2014: Talk of Krzysztof Wecel, Assistant professor, Poznan University of E...European Data Forum
Selected Talk of Krzysztof Wecel, Assistant professor, Poznan University of Economics, Poland at the European Data Forum 2014, 19 March 2014 in Athens, Greece: Advanced Exploration of Public Procurement Data in Linked Data Paradigm
SC4 Workshop 1: Simon Scerri: Existing tools and technologiesBigData_Europe
This document discusses technologies for addressing big data challenges in Europe. It describes the 3Vs of big data - volume, velocity, and variety. It outlines requirements for batch processing of historical data, real-time queries of online data, and low-latency analysis of streaming data. The document proposes a lambda architecture that combines batch and real-time processing layers with a data storage layer to enable both batch and real-time views of the data. Semantic technologies are proposed to preserve semantics and metadata in big data systems. The data aggregator platform would integrate semantic and non-semantic data using mapping techniques and exploit semantics for analysis.
SC7 Hangout 1: Community Building and user requirements for Big Data in Secur...BigData_Europe
SatCen's role is to develop pilots using the BigDataEurope stack for secure societies scenarios and build a secure societies community. To gather user requirements, SatCen conducted stakeholder interviews through customized questionnaires, held a workshop with 45 participants from space, security, data and cybersecurity domains, and discussed needs with other entities through events and projects. The goal is to inform Big Data applications that meet the needs of the secure societies domain.
The document discusses challenges around data management and analytics for smart grids with distributed generation. It notes that smart grids are aimed at improving grid resilience, facilitating new energy markets, and better integrating renewable energy. However, a lack of unified data models presents a challenge for analyzing the large volumes of diverse data from smart meters, weather sensors, and other sources. The author proposes collaborating with Big Data Europe to define use cases around technical grid management and market forecasting that leverage big data analytics to help decarbonize energy systems with high renewable penetration.
SemIoT is a project funded by The Ministry of Education and Science of Russian Federation which aims to provide an access to sensor networks using unified data models and interfaces that hide heterogeneity of the network and facilitate effective data access, interoperability, resource search and discovery
Leveraging Big Data to Manage Transport Operations (LeMO) project will address these issues by investigating the implications of the utilisation of such big data to enhance the economic sustainability and competitiveness of European transport sector. The project will study and analyse big data in the European transport domain in particular with respect to five transport dimensions: mode, sector, technology, policy and evaluation. LeMO will accomplish this by conducting a series of case studies, in order to provide recommendations on the prerequisites of effective big data implementation in the transport field.
This document provides an overview of an EU code of conduct workshop on advanced research computing. It includes background information on high performance computing and datacenter challenges. Several facts are presented about data center energy usage and growth. The agenda for the workshop is also outlined, covering topics like the EU code of conduct background, monitoring and metrics, and datacenters' role in the UK economy.
1) The document discusses Linked Data and semantic application development. It provides examples of semantic technologies like the Google Knowledge Graph, Freebase, and DBpedia.
2) It explains key concepts of Linked Data including URIs, HTTP URIs, RDF, SPARQL, ontologies, and the semantic web stack. The four Linked Data principles are also summarized.
3) Architectures for Linked Data applications are covered including crawling, on-the-fly dereferencing, and federated query patterns. Components like wrappers, mediators, caches and triple stores are also discussed.
SC4 Workshop 1: Dave Marples: Role of social media in transport BigData_Europe
Social media allow people to create, share, and exchange information and media online. When used for transportation, social media data can be acquired, integrated, analyzed, and interpreted to provide insights but also present challenges regarding quality, reliability, privacy, and potential spoofing. Key questions remain around deciding what social media data can be relied on, obtaining consent to use electively published data, and preventing spoofed information from confusing users or encouraging inappropriate behavior.
This document provides an overview of a geospatial metadata and spatial data workshop held at the University of Oxford. The workshop covered topics such as metadata standards, application profiles, geospatial metadata tools and portals for sharing spatial data and metadata. Hands-on sessions demonstrated how to create metadata using the Geodoc Metadata Editor tool and access spatial data repositories through the Go-Geo portal and ShareGeo open data portal.
This document summarizes the Leveraging Big Data to Manage Transport Operations (LeMO) project. The 3-year project, funded by the EU, aims to (1) produce a research roadmap for using big data in transport; (2) involve stakeholders to identify opportunities and barriers; and (3) disseminate findings. It will conduct 7 case studies on topics like rail transport, open data, and logistics. The project aims to enhance sustainability and competitiveness in transport through big data analysis of modes, sectors, technologies, policies, and evaluations. It will provide a framework for a consistent European big data strategy in transport.
The document discusses how big data and digital transformation can help address climate change challenges through the energy sector. It provides examples of digital use cases for power generation, transmission and distribution networks, retailers and aggregators, consumers and prosumers, and new market platforms. These use cases leverage technologies like predictive analytics, asset intelligence networks, demand response programs, and real-time energy visibility to improve grid reliability and efficiency, increase renewable energy integration, empower customers, and reduce costs.
This document discusses requirements engineering and its key processes. It describes how requirements engineering involves eliciting requirements through activities like interviews and prototyping. It also involves analyzing requirements by documenting them, resolving conflicts between stakeholder needs, and validating requirements. The document stresses the importance of requirements engineering in defining a system's functionality and ensuring project success.
The document describes the requirement engineering process. It involves conducting a feasibility study, eliciting and analyzing requirements, modeling the system, specifying user and system requirements, and validating requirements. This leads to the creation of a software requirements specification document. Key activities include gathering requirements through stakeholder interviews, modeling system data, functions, and behaviors, and documenting all requirements and models.
European Open Data Portal and Policy Compass: from national Open Data reposit...OW2
In November 2015 the European Commission officially lunched the European Data Portal http://www.europeandataportal.eu . The mission of the portal is to become the catalogue of all European public data providing them in all official languages of the European Union. The portal is harvesting metadata from heterogeneous open data portals of 28 EU and other 11 European countries. It lists over 580 000 datasets and it is the biggest Open Data portal worldwide. From the techincal perspective, it is the first official Open Data portal implementing the new DCAT Application Profile specification.
The portal is the place to find European public data and it is a basis for other innovative services. One of them is Policy Compass https://policycompass.eu. It brings together open public data, social media, e-participation platforms, causal models, and argumentation technology for constructing, sharing, visualizing and debating progress metrics and impacts of policies.
Both portals are Open Source. They provide rich APIs and may become a data source for other applications.
SC4 Hangout 1: BDE-Transport Webinar Simon ScerriBigData_Europe
BigDataEurope organized its first webinar on the 21st September 10h00-11h00 (CET) to introduce the BigDataEurope project, in particular the domain of Smart, Green, and Integrated Transport.
The presentation was created and presented by Simon Scerri from the University of Bonn.
This document identifies 9 elements for the development of open data marketplaces:
1) Bringing stakeholders together to match supply and demand
2) Providing rich metadata
3) Enabling data quality assessment
4) Ensuring trust, security and critical mass
5) Having an appropriate revenue model
6) Providing use cases, training and support
7) Providing technical support like open data processing tools
8) Providing a full API for machine-to-machine operation
9) Targeting multiple nationalities
General Introduction to the Oxford e-Research CentreDavid Wallom
Digital Oxford is a collaborative research hub that is transforming research through innovative digital methods. It focuses on key areas like data capture, curation, publication and standards to create technologies that disseminate and reuse research data. The hub works with various communities including curators, data producers, consumers, developers, researchers, and policy makers through its network of high performance computing, infrastructure, and communications systems.
Presentation from CNI Spring Membership Meeting 2018, describing four-part series of research reports examining university research data management services. Further information about this project can be found at oc.lc/rdm This joint presentation included a slide deck (not included here) describing research data services at the University of Illinois, Urbana-Champaign, presented by Heidi Imker.
SC6 Workshop 1: What can big data do for you? BigData_Europe
Presentation by Sören Auer, Fraunhofer IAIS, Coordinator of Big Data Europe, at the first workshop of Societal Challlenge 6 in the BigDataEurope project, taking place in Luxembourg on 18 November 2015.
http://www.big-data-europe.eu/social-sciences/
EDF2014: Talk of Krzysztof Wecel, Assistant professor, Poznan University of E...European Data Forum
Selected Talk of Krzysztof Wecel, Assistant professor, Poznan University of Economics, Poland at the European Data Forum 2014, 19 March 2014 in Athens, Greece: Advanced Exploration of Public Procurement Data in Linked Data Paradigm
SC4 Workshop 1: Simon Scerri: Existing tools and technologiesBigData_Europe
This document discusses technologies for addressing big data challenges in Europe. It describes the 3Vs of big data - volume, velocity, and variety. It outlines requirements for batch processing of historical data, real-time queries of online data, and low-latency analysis of streaming data. The document proposes a lambda architecture that combines batch and real-time processing layers with a data storage layer to enable both batch and real-time views of the data. Semantic technologies are proposed to preserve semantics and metadata in big data systems. The data aggregator platform would integrate semantic and non-semantic data using mapping techniques and exploit semantics for analysis.
SC7 Hangout 1: Community Building and user requirements for Big Data in Secur...BigData_Europe
SatCen's role is to develop pilots using the BigDataEurope stack for secure societies scenarios and build a secure societies community. To gather user requirements, SatCen conducted stakeholder interviews through customized questionnaires, held a workshop with 45 participants from space, security, data and cybersecurity domains, and discussed needs with other entities through events and projects. The goal is to inform Big Data applications that meet the needs of the secure societies domain.
The document discusses challenges around data management and analytics for smart grids with distributed generation. It notes that smart grids are aimed at improving grid resilience, facilitating new energy markets, and better integrating renewable energy. However, a lack of unified data models presents a challenge for analyzing the large volumes of diverse data from smart meters, weather sensors, and other sources. The author proposes collaborating with Big Data Europe to define use cases around technical grid management and market forecasting that leverage big data analytics to help decarbonize energy systems with high renewable penetration.
SemIoT is a project funded by The Ministry of Education and Science of Russian Federation which aims to provide an access to sensor networks using unified data models and interfaces that hide heterogeneity of the network and facilitate effective data access, interoperability, resource search and discovery
Leveraging Big Data to Manage Transport Operations (LeMO) project will address these issues by investigating the implications of the utilisation of such big data to enhance the economic sustainability and competitiveness of European transport sector. The project will study and analyse big data in the European transport domain in particular with respect to five transport dimensions: mode, sector, technology, policy and evaluation. LeMO will accomplish this by conducting a series of case studies, in order to provide recommendations on the prerequisites of effective big data implementation in the transport field.
This document provides an overview of an EU code of conduct workshop on advanced research computing. It includes background information on high performance computing and datacenter challenges. Several facts are presented about data center energy usage and growth. The agenda for the workshop is also outlined, covering topics like the EU code of conduct background, monitoring and metrics, and datacenters' role in the UK economy.
1) The document discusses Linked Data and semantic application development. It provides examples of semantic technologies like the Google Knowledge Graph, Freebase, and DBpedia.
2) It explains key concepts of Linked Data including URIs, HTTP URIs, RDF, SPARQL, ontologies, and the semantic web stack. The four Linked Data principles are also summarized.
3) Architectures for Linked Data applications are covered including crawling, on-the-fly dereferencing, and federated query patterns. Components like wrappers, mediators, caches and triple stores are also discussed.
SC4 Workshop 1: Dave Marples: Role of social media in transport BigData_Europe
Social media allow people to create, share, and exchange information and media online. When used for transportation, social media data can be acquired, integrated, analyzed, and interpreted to provide insights but also present challenges regarding quality, reliability, privacy, and potential spoofing. Key questions remain around deciding what social media data can be relied on, obtaining consent to use electively published data, and preventing spoofed information from confusing users or encouraging inappropriate behavior.
This document provides an overview of a geospatial metadata and spatial data workshop held at the University of Oxford. The workshop covered topics such as metadata standards, application profiles, geospatial metadata tools and portals for sharing spatial data and metadata. Hands-on sessions demonstrated how to create metadata using the Geodoc Metadata Editor tool and access spatial data repositories through the Go-Geo portal and ShareGeo open data portal.
This document summarizes the Leveraging Big Data to Manage Transport Operations (LeMO) project. The 3-year project, funded by the EU, aims to (1) produce a research roadmap for using big data in transport; (2) involve stakeholders to identify opportunities and barriers; and (3) disseminate findings. It will conduct 7 case studies on topics like rail transport, open data, and logistics. The project aims to enhance sustainability and competitiveness in transport through big data analysis of modes, sectors, technologies, policies, and evaluations. It will provide a framework for a consistent European big data strategy in transport.
The document discusses how big data and digital transformation can help address climate change challenges through the energy sector. It provides examples of digital use cases for power generation, transmission and distribution networks, retailers and aggregators, consumers and prosumers, and new market platforms. These use cases leverage technologies like predictive analytics, asset intelligence networks, demand response programs, and real-time energy visibility to improve grid reliability and efficiency, increase renewable energy integration, empower customers, and reduce costs.
This document discusses requirements engineering and its key processes. It describes how requirements engineering involves eliciting requirements through activities like interviews and prototyping. It also involves analyzing requirements by documenting them, resolving conflicts between stakeholder needs, and validating requirements. The document stresses the importance of requirements engineering in defining a system's functionality and ensuring project success.
The document describes the requirement engineering process. It involves conducting a feasibility study, eliciting and analyzing requirements, modeling the system, specifying user and system requirements, and validating requirements. This leads to the creation of a software requirements specification document. Key activities include gathering requirements through stakeholder interviews, modeling system data, functions, and behaviors, and documenting all requirements and models.
Software Requirement Elicitation by Aime - Pankamol Srikaew
- What is Requirement Elicitation?
- Why? - Importance of Requirement Elicitation
- Challenges of Requirement Elicitation
- Types of Requirement
- 5 Steps to Extract Requirement
- Applying with Agile
- Requirement Management and Tools
This presentation is related to Object Oriented Software Engineering book by David C. Kung
This document discusses elicitation techniques used in language research, including production tasks, interviews, and questionnaires. Production tasks aim to elicit natural language samples but are time-consuming. Interviews can be structured, semi-structured, or unstructured and allow flexibility but introduce bias. Questionnaires use closed and open questions and must be carefully designed and piloted to avoid confusion and bias. Responses require categorization and quantification to analyze qualitative data. Proper planning and conduct of interviews and questionnaires is important to obtain valid results.
This affects the quality of software and increases the production cost of ... effectiveness of every method, it is useful to select the particular elicitation
http://www.imran.xyz
The document provides requirements for an Ambulance Dispatch System (ADS). It describes 9 key requirements:
1) Allow operators to input 911 call details
2) Help determine if calls are unique
3) Prioritize calls based on severity
4) Locate the three nearest available ambulances
5) Allow dispatchers to update ambulance statuses
6) Calculate ambulance arrival times
7) Store all information in a secure database
8) Provide management reports on ambulance service metrics
9) Allow users to access past call information
This document discusses requirement elicitation techniques used in systems analysis and design. It describes requirement elicitation as identifying stakeholder needs through interviews, meetings, ethnography and other techniques. It outlines best practices for elicitation including preparing for interviews and meetings, using scenarios, questionnaires, and observation to understand user needs and ensure requirements are unambiguous, complete, verifiable and consistent. The goal of elicitation is to gather requirements that accurately reflect stakeholder needs.
Software Engineering- Requirement Elicitation and SpecificationNishu Rastogi
The document discusses the process of requirements engineering for software development. It involves four main steps:
1) Feasibility study to determine if the project is possible.
2) Requirements gathering by communicating with clients and users to understand what the software should do.
3) Creating a software requirements specification (SRS) document that defines system functions and constraints.
4) Validating requirements to ensure they are clear, consistent, and can be implemented.
This document discusses fundamentals, techniques, and assistance tools for validating requirements. It outlines the context and goals of requirements validation, including quality criteria to evaluate requirements against and risks of insufficient validation. The document presents principles of validation, such as involving stakeholders, separating defect detection from correction, leveraging multiple independent views, and using appropriate documentation formats. Validation techniques and assistance tools are also covered.
This document discusses project planning and feasibility studies. It provides details on the importance of project planning, the basic components of a project plan, and the project planning process which involves 20 steps such as developing the project management plan, collecting requirements, defining the scope, and planning risk management. It also discusses what a feasibility study entails, including examining the market, organizational/technical, and financial aspects of a proposed project to determine its viability before significant resources are invested. A feasibility study aims to identify any issues that could prevent a project from being successful in the marketplace.
Automated Discovery of Performance Regressions in Enterprise ApplicationsSAIL_QU
This document summarizes the author's research on automated discovery of performance regressions in enterprise applications. It discusses challenges with current performance verification practices, and proposes approaches at the design and implementation levels. At the design level, it suggests using layered simulation models to evaluate design changes early. At the implementation level, it presents techniques to analyze large performance datasets, detect regressions while limiting subjectivity, and deal with tests in heterogeneous environments. Case studies show the approaches achieve 75-100% precision and 52-80% recall. The research aims to help analysts efficiently identify performance regressions.
The document discusses Southwest Power Pool's initial steps towards creating a data lake. It describes:
- Storing historical and real-time data that exceeded initial expectations, with around 50% being less frequently used
- Conducting a proof-of-concept evaluation of three vendors to offload less frequently used data and allow SQL query access with minimal changes to existing queries
- Choosing BigInsights based on its ability to do this along with supporting existing Netezza functions and allowing federated queries between Netezza and BigInsights
- The multi-phase vision to eventually incorporate more data types and workloads while improving performance, security, and governance
Looking for expertise or support on Data Integrity? Contact us today.
Recently, the pharmaceutical industry has been challenged with the regulatory requirements to provide complete, consistent and accurate data, throughout all GMP regulated processes.
Moreover, during audits the regulatory bodies have observed a level of inconsistency in the application of the predicate rules in GMP processes. This has become a growing concern and has led to a set of new (draft) guidances from different market authorities.
Index:
Data Integrity – Why / What
Data life cycle
Core Data Integrity concepts & building blocks
Short & mid-term actions enabling a focused road to compliance
Need for System Analysis
Stages in System Analysis
Structured SAD and tools :
DFD
Context Diagram
Decision Table
Structured Diagram.
System Development Models:
Water Flow
Prototype
Spiral
RAD
Roles and responsibilities of
System Analyst,
Database Administrator
Database Designer
(ATS6-APP01) Unleashing the Power of Your Data with DiscoverantBIOVIA
In the fast-paced, high demand environment of manufacturing, it’s almost impossible to find the time to gather large amounts of data and organize it into a common context. This session presents best practices for using Discoverant hierarchies and Direct Connects to provide your end-users with on-demand and scheduled self-service access to their contextualized data, freeing their time for the more important trending and analysis activities that enable improved process performance and predictability.
This document summarizes the implementation of Demantra as a replacement forecasting tool for an education technology company's legacy Manugistics system. Key points:
- The legacy system was outdated, at end of life, and presented business risks. Demantra was selected after an evaluation process to provide an integrated supply chain planning solution.
- The project involved upgrading the existing Oracle Advanced Supply Chain Planning (ASCP) infrastructure for stability. Phase I implemented Demantra for sales forecasting. Phase II added real-time sales and operations planning capabilities.
- Challenges included convincing users to change and introducing the new software. The implementation enabled more accurate forecasting, reduced manual work, and integrated supply chain
Following are high level tasks will be performed as part of consolidation process or data migration activity from existing customer’s systems into newly consolidated Database.
The document outlines a multi-month implementation plan for a BI project with the following key stages:
1) Preparation and Planning in Month 1 involving prioritization, hardware installation, staffing, and software procurement.
2) ETL development from Month 1-3 involving requirement analysis, design, development and testing of the ETL processes.
3) Initial deployment from Month 2-3 setting up the metadata framework and data governance with report reductions.
4) Ongoing development from Month 4-10 involving further report reductions, incremental deployments, building the data library and dashboards. Headcount savings also take effect during this stage.
5) Long term operations starting from Month 11 involving targeting
Industrial IoT to Predictive Analytics: A Reverse Engineering Approach from S...Lokukaluge Prasad Perera
A novel mathematical framework to support industrial digitization of shipping is presented in this study. The framework supports a data flow path, i.e. from Industrial IoT (i.e. with Big Data) to Predictive Analytics, where digital models with advanced data analytics are introduced. The digital models are derived from ship performance and navigation data sets and a combination of such models facilitates towards the proposed Predictive Analytics. Since the respective data sets are used to derive the Predictive Analytics, this mathematical framework is also categorized as a reverse engineering approach. Furthermore, a data anomaly detection and recover procedure that is associated with the same framework to improve the respective data quality are also described in this study.
Test Data, Information, Knowledge, Wisdom: past, present & future of standing...Neil Thompson
The document discusses test data for software testing from both past and present perspectives. It begins by covering definitions and concepts related to test data from ISO 29119 standards. It then discusses the basics of test data and how test data requirements and information are developed through the test planning, design and implementation processes. The document also discusses different sources of test data and how test data relates to different testing levels and techniques like unit testing, integration testing and system testing.
Advanced Analytics and Machine Learning with Data VirtualizationDenodo
Watch full webinar here: https://bit.ly/32c6TnG
Advanced data science techniques, like machine learning, have proven an extremely useful tool to derive valuable insights from existing data. Platforms like Spark, and complex libraries for R, Python and Scala put advanced techniques at the fingertips of the data scientists. However, these data scientists spent most of their time looking for the right data and massaging it into a usable format. Data virtualization offers a new alternative to address these issues in a more efficient and agile way.
Attend this webinar and learn:
- How data virtualization can accelerate data acquisition and massaging, providing the data scientist with a powerful tool to complement their practice
- How popular tools from the data science ecosystem: Spark, Python, Zeppelin, Jupyter, etc. integrate with Denodo
- How you can use the Denodo Platform with large data volumes in an efficient way
- About the success McCormick has had as a result of seasoning the Machine Learning and Blockchain Landscape with data virtualization
This document discusses a project that aims to develop graphical data workflows and cross-platform optimization. It allows users to design data processing workflows with minimal programming. The project also optimizes workflows to run efficiently across multiple computing clusters. Example use cases discussed are studying drug synergies in cancer research, predicting financial market behaviors, and maritime situational awareness. The document outlines the graphical editor, streaming operations, and how an optimization component can split workflows across clusters for better performance.
Exploring Neo4j Graph Database as a Fast Data Access LayerSambit Banerjee
This article describes the findings of an extensive investigative work conducted to explore the feasibility of using a Neo4j Graph Database to build a Fast Data Access Layer with near-real time data ingestion from the underlying source systems.
Presentation given to the BCS Data Management Specialist Group by Steve Higgins of CSC on healthcare data management
A video of the presentation is available at http://youtu.be/Fqm4XDyA6fI
Introduction of streaming data, difference between batch processing and stream processing, Research issues in streaming data processing, Performance evaluation metrics , tools for stream processing.
The document discusses Enterprise Resource Planning (ERP) systems. It describes the ERP architecture as using a client-server model with a relational database to store and process data. The ERP lifecycle involves definition, construction, implementation, and operation phases. Core ERP components manage accounting, production, human resources and other internal functions, while extended components provide external capabilities like CRM, SCM, and e-business. Proper implementation requires screening software, evaluating packages, analyzing process gaps, reengineering workflows, training staff, testing, and post-implementation support.
The document proposes a Rapid Prototyping Capability (RPC) system to efficiently evaluate integrating Earth observation data from NASA satellites and models. The RPC would:
1) Integrate tools to access, process, and analyze data and model outputs to support experiments.
2) Reduce the time typically required to evaluate new data streams in models through a simulated operational environment.
3) Be accessible to various user groups, including specialists responsible for data/models and domain experts performing analyses.
This presentation on batch process analytics was given at Emerson Exchange, 2010. A overview of batch data analytics is presented and information provided on a field trail of on-line batch data analytics at the Lubrizol, Rouen, France plant.
Similar to BDE Technical Webinar 1 : Requirements elicitation (20)
Luigi Selmi - The Big Data Integrator PlatformBigData_Europe
The document discusses the SC4 pilot project, which aims to build a scalable and fault-tolerant platform for processing large datasets using open source frameworks. The platform utilizes a microservices architecture and processes real-time floating car data for tasks like map matching and short-term traffic forecasting using algorithms like feedforward artificial neural networks. It also discusses how semantic technologies from projects like SANSA-Stack and LinkedGeoData could enable additional use cases for the SC4 platform.
Josep Maria Salanova - Introduction to BDE+SC4BigData_Europe
The BigDataEurope project aims to empower mobility management with big data. It has developed a modular platform that allows end-users to easily deploy functionality in their own systems using Docker containers. The platform is being tested through 7 pilot projects aligned with European societal challenges related to data-driven solutions. One pilot focuses on transport in Thessaloniki, Greece, using GPS, Bluetooth, and other probe data to improve map matching and mobility pattern recognition and forecasting to better manage traffic. The project coordinators are available for any questions.
The LeMO project aims to leverage big data to manage transport operations. It will identify issues around effective data mining and exploitation in transportation. The project will analyze barriers and opportunities for using big data in transport. It will also design recommendations for research and policy regarding big data in transport. The project involves 5 partners from 5 countries and will run from 2017 to 2020. It seeks to produce a roadmap for data collection, sharing, and exploitation to support European transport stakeholders. The project will study big data in transport through case studies focusing on issues like infrastructure innovation, transport efficiency, and data protection. It expects its recommendations and roadmap to help policymakers and industry better utilize big data.
Big Data Europe SC6 WS #3: PILOT SC6: CITIZEN BUDGET ON MUNICIPAL LEVEL, Mart...BigData_Europe
This document describes a pilot project that aims to create an online dashboard of municipal budget and spending data from multiple European cities. The project will harvest, link, analyze and visualize open budget data to make it more useful for citizens, researchers and decision makers. Data will be integrated from Athens, Thessaloniki and Kalamaria in Greece initially, and potentially Vienna, Linz and Barcelona. The technical architecture uses Apache tools like Flume, Kafka and Spark to ingest, store and analyze the heterogeneous data sources in real-time. The goal is to evaluate how a big data approach can provide new insights into public finances from an integrated, multilingual and longitudinal perspective.
Big Data Europe SC6 WS #3: Big Data Europe Platform: Apps, challenges, goals ...BigData_Europe
This document discusses Big Data Europe (BDE), an open source big data platform. It provides an overview of BDE's goals, architecture, and applications. The key points are:
1) BDE's goals are to make it easy to install, develop for, deploy, and integrate big data applications. It aims to unlock the value of data through an open platform.
2) BDE supports a variety of frameworks and uses Docker to package components. Its architecture includes layers for resources, data, processing, and applications.
3) BDE is being applied to challenges in domains like health, transport, energy, and security. Examples analyze traffic patterns, perform predictive maintenance, and detect changes in infrastructure
Big Data Europe SC6 WS 3: Where we are and are going for Big Data in OpenScie...BigData_Europe
Where we are and are going for Big Data in OpenScience
Keynote talk at the Big Data Europe SC6 Workshop on 11.9.2017 in Amsterdam co-located with SEMANTiCS2017: The perspective of European official statistics by Fernando Reis, Task-Force Big Data, European Commission (Eurostat).
Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...BigData_Europe
Slides for keynote talk at the Big Data Europe workshop nr 3 on 11.9.2017 in Amsterdam co-located with SEMANTiCS2017 conference by Ron Dekker, Director CESSDA: European Open Science Agenda: where we are and where we are going?
Big Data Europe: SC6 Workshop 3: The European Research Data Landscape: Opport...BigData_Europe
Slides of the keynote at the 3rd Big Data Europe SC6 Workshop co-located at SEMANTiCS2018 in Amsterdam (NL) on: The European Research Data Landscape: Opportunities for CESSDA by Peter Doorn, Director DANS, Chair, Science Europe W.G. on Research Data. Chair, CESSDA ERIC General Assembly
The BigDataEurope 3rd Workshop took place on November 28, 2017 in Amsterdam. The workshop focused on learning about current and future data management challenges, Big Data tools, and the Open Big Data Platform. The agenda included talks on EU priorities on energy and digitalization, asset management and Big Data in wind energy, the BDE Open Platform, and system monitoring case studies. There was also a roundtable discussion on future data needs, the BDE platform, and research opportunities.
BDE SC3.3 Workshop - Data management in WT testing and monitoring BigData_Europe
Aresse Engineering is a company that provides engineering services including data acquisition, analysis, and monitoring for industries like wind energy. They have grown from 9 engineers in 2007 to over 20 staff members today. To address challenges of big data, Aresse developed tools like XDAS for acquisition, Hivex for data transfer, and a cloud platform called Hive for storage, analysis, and visualization of data. These tools allow customers to access massive amounts of synchronized measurement data. Machine learning and physical models are areas of ongoing development to enhance data quality.
BDE SC3.3 Workshop - Options for Wind Farm performance assessment and Power f...BigData_Europe
Options for Wind Farm performance assessment and Power forecasting (Mr. A. Kyritsis, ALTSOL/TERNA) at the BigDataEurope Workshop, Amsterdam, Novermber 2017.
Big Data Europe: Workshop 3 SC6 Social Science: THE IMPORTANCE OF METADATA & ...BigData_Europe
Big Data Europe: Workshop 3 SC6 Social Science - 11.09.2017 in Amsterdam, co-located with SEMANTiCS2017 titled: THE IMPORTANCE OF METADATA & BIG DATA IN OPEN SCIENCE. Slides by Ivana Versic (Cessda) and Martin Kaltenböck (SWC)
This document summarizes the MIDAS project, which is funded by the European Union to develop a platform using big data to support public health policies. The key points are:
1) MIDAS has received top funding scores and will receive €4.5 million over 40 months. It involves universities, technical partners, and policy boards across Europe.
2) The project aims to improve public health policies across Europe by facilitating analysis of diverse health datasets using its platform. This could lead to more effective policies and citizen benefits.
3) MIDAS will develop a secure data integration and analysis platform, with visualization tools to help policymakers. It will gather data from various sources to inform policy creation and evaluation
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...alexjohnson7307
Predictive maintenance is a proactive approach that anticipates equipment failures before they happen. At the forefront of this innovative strategy is Artificial Intelligence (AI), which brings unprecedented precision and efficiency. AI in predictive maintenance is transforming industries by reducing downtime, minimizing costs, and enhancing productivity.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Tatiana Kojar
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
3. Big Data -4Vs
Big
Data
Volume
Velocity
Variety
Veracity
• Must be considered at: data acquisition, data
processing and data display (“fresh” results) level
• A need to find a solution to accommodate all 3
levels
• It is an important concern to most SC
• Common feeling “better integration solution of
wider variety of data leads to better statistics”
• Most help in this direction is needed by SC1 and SC5
and it remains an important aspect for all SCs
• Decisions depend on results of statistics which are as
good as the data quality which is used
5. Functional requirements
Societal Challenges have specific requirement
needs and we did not see a way to cluster them
Next, we started looking at common functional
requirements over all Societal Challenges
These functional blocks will be reflected in
components which are:
o base platform components that are prepared in WP4;
o pilot-specific components that are already available;
o pilot-specific components that will be developed for the
pilot.
6. Functional Blocks–Data acquisition
Data ingestion Real time traffic data ingestion SC4
Data ingestion via ftp, or otherwise from an
intermediate (local) processing level
SC3
Monitor multiple text services and stored
together with provenance and any other
metadata
SC7
Download satellite images SC7
Data
integration
Integrate multiple metadata services, including
Strabon for geodata
SC5
Datasets are aligned and linked at data
ingestion time
SC1,
SC2
7. Functional Blocks – Data analysis
Data
type
Multimedia
data
Pattern recognition algorithms SC2,
SC7
Detection of changes in areas of interest using
satellite image
SC7
Automatic video coding data SC4
Web data Event detection over text SC7
Sensor
data
Signal processing SC3
Stored data Analysis of historical data SC7
8. Functional Blocks –Data analysis 2
Modelling Complex agricultural and environmental modelling SC2
Model parameterization SC3
Prediction and
forecasting
Demand forecasting SC4
Traffic flow prediction SC4
Support the incremental, data-oriented carrying out
of climate-related experiments
SC5
Assessment Power production and operation assessment SC3
Planning Power production prognostics SC3
Maintenance planning SC3
Annotation tools Text annotation SC2
Statistical
analysis
Integration with IBM SPSS Statistics SC6
Integration with Matlab SC6
9. Functional Blocks – Data curation
Variance Measure the quality of the information SC4,
SC7
Track provenance during the chain of executions SC1
Usage of Open
PHACTS
platform
functionalities
Aligning and linking various datasets SC1
Data update Periodical updates of the datasets SC1
Cleaning Spike removal, integrity check, faulty data labelling SC3
10. Functional Blocks – Data storage
Store a set of geocoded areas SC7
Storing intermediate data products and associated data lineage SC5
Storing basic data provenance SC5,
SC2
RDF data storage SC1
11. Functional Blocks – Data usage
Condition
monitoring
system
assessment
Logging functionality of the whole process for
debugging
SC1, SC7
Monitoring of the operational status of units,
considering the cluster operation in total
SC3
Incorporating third party systems (condition
monitoring systems, experimental research modules)
into the monitoring process and performing
correlated assessment
SC3, SC7
Other View results on a map SC5, SC7
Query a set of geocoded areas, map based GUI SC7, SC2
Publishing Aggregated data must be publicly available at least
in RDF/XML, JSON, CSV formats [through API]
SC1, SC2
Alert system Receives as input the areas with detected changes
and presents them to the user as an alert
SC7
Visualization Present data in a dashboard SC3
A GUI that exposes search over data SC2