This document provides an overview of Mundi, an Atos DIAS (Data and Information Access Service) platform. It discusses how Mundi provides simple access to Copernicus and other satellite data through cloud and big data technologies. The document outlines Mundi's offerings, including Jupyter notebooks, APIs, data formats, historical data access, on-demand processing, and its growing marketplace. Users are invited to create an account, explore the marketplace and data, and try Mundi.
This work is about how both private enterprise and government wish to improve their data value and how they deal with this issue. The talk summarizes the way of thinking about Big Data, Open Data and their use by organizations or individuals. Big Data is explained from collecting, storing, analyzing and put in value. This data is collected from numerous sources including sensor networks, government data holdings, company market databases, and public profiles on social networking sites. Organizations use many data analytical techniques to study both structured and unstructured data. Due to the volume, velocity and variety of data, some specific techniques have been developed. MapReduce, Hadoop and other related as RHadoop are trending topic nowadays.
Data which come from government must be open. Every day more and more cities and countries are opening their data. Open Data is then presented as a specific case of public data with a special role in Smartcity. The main goal of Big and Open Data in Smartcity is to develop systems which can be useful for citizens. In this sense RMap (Mapa de Recursos) is shown as an Open Data application, an open system for Madrid City Council, avalaible for smarthphones and totally developed by the researching group G-TeC (www.tecnologiaUCM.es).
The Seagate Kinetic Storage platform represents an opportunity to substantially address the inefficiencies of traditional data centers whose legacy architectures are not well-adapted to highly distributed and capacity-optimized workloads of exploding unstructured data and applications.
The Open Data Center Alliance is working actively to shape the future of cloud computing — a future based on open, interoperable standards. Our membership includes more than 300 companies spanning multiple continents and industries.
FinTech and InsuranceTech case studies digitally transforming Europe's future with BigData and AI
The new data-driven industrial revolution highlights the need for big data technologies to unlock the potential in various application domains. The insurance and finance services industry is rapidly transformed by data-intensive operations and applications. FinTech and InsuranceTech combine very large datasets from legacy banking systems with other data sources such as financial markets data, regulatory datasets, real-time retail transactions, and more, improving financial services and activities for customers.
This work is about how both private enterprise and government wish to improve their data value and how they deal with this issue. The talk summarizes the way of thinking about Big Data, Open Data and their use by organizations or individuals. Big Data is explained from collecting, storing, analyzing and put in value. This data is collected from numerous sources including sensor networks, government data holdings, company market databases, and public profiles on social networking sites. Organizations use many data analytical techniques to study both structured and unstructured data. Due to the volume, velocity and variety of data, some specific techniques have been developed. MapReduce, Hadoop and other related as RHadoop are trending topic nowadays.
Data which come from government must be open. Every day more and more cities and countries are opening their data. Open Data is then presented as a specific case of public data with a special role in Smartcity. The main goal of Big and Open Data in Smartcity is to develop systems which can be useful for citizens. In this sense RMap (Mapa de Recursos) is shown as an Open Data application, an open system for Madrid City Council, avalaible for smarthphones and totally developed by the researching group G-TeC (www.tecnologiaUCM.es).
The Seagate Kinetic Storage platform represents an opportunity to substantially address the inefficiencies of traditional data centers whose legacy architectures are not well-adapted to highly distributed and capacity-optimized workloads of exploding unstructured data and applications.
The Open Data Center Alliance is working actively to shape the future of cloud computing — a future based on open, interoperable standards. Our membership includes more than 300 companies spanning multiple continents and industries.
FinTech and InsuranceTech case studies digitally transforming Europe's future with BigData and AI
The new data-driven industrial revolution highlights the need for big data technologies to unlock the potential in various application domains. The insurance and finance services industry is rapidly transformed by data-intensive operations and applications. FinTech and InsuranceTech combine very large datasets from legacy banking systems with other data sources such as financial markets data, regulatory datasets, real-time retail transactions, and more, improving financial services and activities for customers.
Hajira Jabeen introduces the Big Data Europe Integrator Platform. The deck also includes the slides use to summarise the other presentations in the launch webinar.
The Pandemic Changes Everything, the Need for Speed and ResiliencyAlluxio, Inc.
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
The Pandemic Changes Everything, the Need for Speed and Resiliency
Parviz Peiravi, Global CTO of Financial Services Solutions, Intel
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
OpenStack and Red Hat: How we learned to adapt with our customers in a maturi...OpenStack
Audience Level
All levels
Synopsis
Peter has been involved in OpenStack community since its B-release, and he has been enabling and helping customers across various industries adopt OpenStack in strategic ways. In this session, you will learn from his experience what Red Hat’s perspective is on the current state of affairs in the OpenStack community and the path we see ahead that Red Hat is putting its efforts in. OpenStack is not a product that tries to solve any one business problem in particular, but a technology that aims to be usable for many – what are the required steps to make sure that your organisation is ready for the OpenStack-based cloudification and transformation.
Speaker Bio:
Peter Jung is a Senior Business Development Manager at Red Hat where he leads the practice in the areas of Cloud, SDN/NFV and IoT across Australia and New Zealand. He is passionate about open innovation and open source software development model as the foundation for next generation society and ICT systems. Prior to Red Hat, he had various roles at Cisco and Dell for 15 years. He holds a BSEE and an MBA.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Presentation at the European Grid Infrastructure (EGI) Conference 2015, Business Track "Examples of SMEs as consumers and providers".
EGI is focused on supporting SMEs and the full innovation chain between business and academia to create opportunities of economic impact through open data generated and the technical services both offered and required to support research and innovation.
Terradue Cloud Platform delivers Cloud bursting for Earth Science Applications and Services, illustrated in this presentation by many use cases and collaborations fostering the reuse of Earth Observation open data and open services.
What we Learned About Application Resiliency When the Data Center Burned DownScyllaDB
Is your data infrastructure architected to withstand anything —including a disaster suddenly taking down an entire data center? When a cloud provider’s data center burned to the ground, 3.6 million websites went dark. But one leading travel service kept running without a hitch, thanks to the design of their environment-aware distributed database. Learn how they architected their data infrastructure for extreme resiliency, how their strategy held up and what lessons they learned.
DSD-INT 2018 Earth Science Through Datacubes - MerticariuDeltares
Presentation by Vlad Merticariu (Rasdaman) at the Data Science Symposium 2018, during Delft Software Days - Edition 2018. Thursday 15 November 2018, Delft.
Presentation by Bruno Schulze, Senior Researcher / Professor at Laboratório Nacional de Computação Científica (LNCC) at Cloudscape Brazil 2017 & WCN 2017
Project Natick-Datacenters under sea, a project by microsoft.
Project Natick reflects Microsoft’s ongoing quest for cloud datacenter solutions that offer rapid provisioning, lower costs, high responsiveness, and are more environmentally sustainable.
In order to meet the compute infrastructure requirements for the broad range of service consumers, a common framework is required around which infrastructure as a service can be defined, provisioned, monitored and managed. A common set of principles, metrics and architectural frameworks can be defined, resulting in consistent capabilities, service levels and service attributes across multiple providers, while still allowing the individual providers to innovate and differentiate. In this session ODCA representatives will outline the framework and some usage scenarios.
Presentation of the future of cloud storage at Scality SDS Day in London (The Shard) in September 2017. Topics covered include private/public cloud and software-defined storage.
OpenNebulaConf 2013 - Keynote: Opening the Path to Technical Excellence by Jo...OpenNebula Project
Bio:
Jordi Farrés is Service Manager at the European Space Agency (ESA), being responsible for ESA’s grid processing infrastructure and related SciOps and for technology evolution projects in the Earth Observation Ground Segment Engineering Division. He has been also responsible for ESA corporation business applications in Finance, Procurement and Human Resources and Facility Management. Dr. Farrés received his M.S. in Computer Science from UPC and his Ph.D. in Computer Science from the University of Edinburgh.
Hajira Jabeen introduces the Big Data Europe Integrator Platform. The deck also includes the slides use to summarise the other presentations in the launch webinar.
The Pandemic Changes Everything, the Need for Speed and ResiliencyAlluxio, Inc.
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
The Pandemic Changes Everything, the Need for Speed and Resiliency
Parviz Peiravi, Global CTO of Financial Services Solutions, Intel
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
OpenStack and Red Hat: How we learned to adapt with our customers in a maturi...OpenStack
Audience Level
All levels
Synopsis
Peter has been involved in OpenStack community since its B-release, and he has been enabling and helping customers across various industries adopt OpenStack in strategic ways. In this session, you will learn from his experience what Red Hat’s perspective is on the current state of affairs in the OpenStack community and the path we see ahead that Red Hat is putting its efforts in. OpenStack is not a product that tries to solve any one business problem in particular, but a technology that aims to be usable for many – what are the required steps to make sure that your organisation is ready for the OpenStack-based cloudification and transformation.
Speaker Bio:
Peter Jung is a Senior Business Development Manager at Red Hat where he leads the practice in the areas of Cloud, SDN/NFV and IoT across Australia and New Zealand. He is passionate about open innovation and open source software development model as the foundation for next generation society and ICT systems. Prior to Red Hat, he had various roles at Cisco and Dell for 15 years. He holds a BSEE and an MBA.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Presentation at the European Grid Infrastructure (EGI) Conference 2015, Business Track "Examples of SMEs as consumers and providers".
EGI is focused on supporting SMEs and the full innovation chain between business and academia to create opportunities of economic impact through open data generated and the technical services both offered and required to support research and innovation.
Terradue Cloud Platform delivers Cloud bursting for Earth Science Applications and Services, illustrated in this presentation by many use cases and collaborations fostering the reuse of Earth Observation open data and open services.
What we Learned About Application Resiliency When the Data Center Burned DownScyllaDB
Is your data infrastructure architected to withstand anything —including a disaster suddenly taking down an entire data center? When a cloud provider’s data center burned to the ground, 3.6 million websites went dark. But one leading travel service kept running without a hitch, thanks to the design of their environment-aware distributed database. Learn how they architected their data infrastructure for extreme resiliency, how their strategy held up and what lessons they learned.
DSD-INT 2018 Earth Science Through Datacubes - MerticariuDeltares
Presentation by Vlad Merticariu (Rasdaman) at the Data Science Symposium 2018, during Delft Software Days - Edition 2018. Thursday 15 November 2018, Delft.
Presentation by Bruno Schulze, Senior Researcher / Professor at Laboratório Nacional de Computação Científica (LNCC) at Cloudscape Brazil 2017 & WCN 2017
Project Natick-Datacenters under sea, a project by microsoft.
Project Natick reflects Microsoft’s ongoing quest for cloud datacenter solutions that offer rapid provisioning, lower costs, high responsiveness, and are more environmentally sustainable.
In order to meet the compute infrastructure requirements for the broad range of service consumers, a common framework is required around which infrastructure as a service can be defined, provisioned, monitored and managed. A common set of principles, metrics and architectural frameworks can be defined, resulting in consistent capabilities, service levels and service attributes across multiple providers, while still allowing the individual providers to innovate and differentiate. In this session ODCA representatives will outline the framework and some usage scenarios.
Presentation of the future of cloud storage at Scality SDS Day in London (The Shard) in September 2017. Topics covered include private/public cloud and software-defined storage.
OpenNebulaConf 2013 - Keynote: Opening the Path to Technical Excellence by Jo...OpenNebula Project
Bio:
Jordi Farrés is Service Manager at the European Space Agency (ESA), being responsible for ESA’s grid processing infrastructure and related SciOps and for technology evolution projects in the Earth Observation Ground Segment Engineering Division. He has been also responsible for ESA corporation business applications in Finance, Procurement and Human Resources and Facility Management. Dr. Farrés received his M.S. in Computer Science from UPC and his Ph.D. in Computer Science from the University of Edinburgh.
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
Following the popularity of "Cloud Revolution: Exploring the New Wave of Serverless Spatial Data," we're thrilled to announce this much-anticipated encore webinar.
In this sequel, we'll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR.
Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios.
Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects.
Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you're building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.
Cloud computing is a type of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. It is a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage, applications and services),
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar.
In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR.
Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios.
Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects.
Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.
What is expected from Chief Cloud Officers?Bernard Paques
The new CxO is taking care of cloud computing for his company. Among his responsabilities: brand experience, go-to-market and business agility. What do these mean in terms of capabilities?
Similar to Mundi Presentation - A Space of New Opportunities (20)
Agrihub INSPIRE Hackathon 2021: Challenge #7: Analysis, processing and standa...plan4all
This is a presentation of results of Challenge #7: Analysis, processing and standardisation of data from agriculture machinery for easier utilization by farmers of the Agrihub INSPIRE Hackathon 2021.
Calculation of agro climatic factors from global climatic dataplan4all
Authors: Pavel Hájek,
Raitis Berzins , Jiří Valeš, Martin Pitoňák , Vincent
Onckelet , Tomáš Andrš, Veronika Osmiková , Ronald
Ssembajwe , Amit Kirschenbaum , Jörg Schliesser , Michal Kepka & Karel Jedlička
Digitalization of indigenous knowledge in African agriculture for fostering f...plan4all
Authors:
Antoine Kantiza, AKANTIZA CONSULT, Burundi
Didier Muyiramye, Swedish University of Agricultural Sciences, Rwanda
Elias Cherenet Weldemariam, HARAMAYA UNIVERSITY, Ethiopia
Petr Horak, WIRELESSINFO, Czech Republic
Robert Sabimana, Frutus Fresco Ltd, Uganda
Pavel Hajek, West Bohemia University, Czech Republic
Tuula Löytty, Smart & Lean Hub Oy, Finland
Demet Osmancelebioglu, Smart & Lean Hub Oy, Finland
Karel charvat map-compositions-format-intro-presentation-by-karel (1)plan4all
Karel Charvat on behalf of Plan4all, Lesprojekt, BOSC and Asplan Viak gave a presentation about the project to create a Google Docs-like map application and map composition format.
Karel charvat map-whiteboard-collaborative-map-making-breakout-sessionplan4all
Karel Charvat on behalf of Plan4all, Lesprojekt, BOSC and Asplan Viak gave a presentation about the project to create a Google Docs-like map application and map composition format.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
4. Combining simple access to Copernicus, other satellite and third
party data.
Using cloud and big data technologies,
Mundi accelerates the time to market for innovative services based
on earth observation data.
Mundi allows to deliver a service that is distinctive, reliable, and
immediately monetizable.
Mundi, the Atos DIAS
Photo: ESA4
5. Use Earth observation data:
a technological & human
challenge
• Combine satellite data with in situ
or enterprise data
• Understand, select, download,
conserve and process each data
• Harness a range of scientific and
technical skills and manpower
• Load and store petabytes of data
• Deploy high performance
processing capabilities
Photo: ESA55
6. Robust Cloud Platform Curated Data Collection
Selected Tools Collection
Adapted Support
Technical - Business – Functional
Key elements
6
10. Mundi Offer
Jupyter Notebook
10
• Mundi Jupyter Notebook (open for all Mundi users)
• How-to use Mundi APIs
• Catalogue access : OpenSearch/CSW
• OGC interfaces : WMS/WMTS/WCS
• Access to CMEMS and CAMS products (C3S to come)
• Direct access to products
• Allowing data fusion
• From prototyping to high-processing
• Upgrading Mundi Jupyter Notebook
• JupyterHub for a community
• Jupyter Notebook powered by Spark
11. Mundi Offer
Mundi Explore: example of Front Office
11
• Based on Mundi API
• Display images
• Search for products
• Source code
12. Mundi offering
Basic format
12
• Mundi basic offering = ESA offering
• Users access products with the same formats as available today by ESA.
• The format has the drawback that they are complex and designed for file by file
usage.
• Example: The S1 compressed file formats makes it necessary to copy the
whole file locally, un-compress it and then read it as a file.
• Example: The S2 JPEG2000 format is complex and requires commercial
readers which in most cases are slow.
Sentinel 1
Zipped TIFF
Sentinel 2
JPEG2000
Sentinel 3
NET CDF
Landsat 8
Zipped TIFF
Sentinel 5P
NET CDF
13. Mundi offering
Recommended Mundi format
13
• Establish a Mundi format as a COG-layer for each dataset (Cloud Optimized
GeoTIFF).
Sentinel 1
Zipped TIFF
Sentinel 2
JPEG2000
Sentinel 3
NET CDF
Landsat 8
Zipped TIFF
Sentinel 5P
NET CDF
COG COG COG COG COG
YES
simple
and fast
to read
YES
simple
and fast
to read
YES
simple
and fast
to read
YES
simple
and fast
to read
YES
simple
and fast
to read
14. Mundi offering
Access to historical data
14
• Historical data are accessible for free on Mundi
• More than 300 products can be reloaded per month by a single user
• For specific needs, Mundi can work on a dedicated plan
• Be aware that ESA hubs have rolling policies
Mission Rolling start date
Sentinel-1 12 September 2018
Sentinel-2 Q3 2019
Sentinel-3 Q3 2019
Mission dataset Retention period
Sentinel-1 12 months
Sentinel-2 L1c 12 months
Sentinel-2 L2a 18 months
Sentinel-3 12 months
15. Mundi processing
On-demand
15
• As user of Mundi, a processing service is available
• All processing is possible, few examples of available processings:
• Sentinel-1 GRD Orthoimages
• Cloud mask (Fmask for example) computation
• Mosaicking images
• Conversion to COG (out of last 3 month)
• Your own processing
• Billing of this service depends on the usage
• Lower cost using Mundi processing
16. Mundi Offer
Growing Mundi Marketplace
16
• Mundi Advanced Access Datacube
• Joint wok of Atos and MEEO
• Trial access for Mundi users
• API and GUI access
• Sentinel Hub (coming next)
• Full access to Mundi data
• Bring your own data feature
• Trial access for Mundi users
