Metadata For Humans and Machines

•Download as PPTX, PDF•

1 like•1,132 views

New data access paradigms support a variety of human and machine access paths with data servers (THREDDS, https://www.unidata.ucar.edu/software/thredds/current/tds/ and Hyrax, http://opendap.org) that support multiple services for a given dataset. We need metadata that can describe those services and unambiguously differentiate between access paths for humans and for machines. The ISO 19115 metadata standard includes service metadata and allows data and services for that data to be described in the same record. I propose that we use the service metadata for machine access and the more traditional distribution information for human access. This talk was presented at the ESIP (espied.org) meeting during January 2014.

Metadata For
Humans
and Machines
Ted Habermann
Director of Earth Science
The HDF Group
thabermann@hdfgroup.org
January 8-10, 2014

ESIP Winter 2014

1

Evolving Data Access
HDF

HDF

FTP
HTTP

Data Center
January 8-10, 2014

ESIP Winter 2014

2

Evolving Data Access
HDF

Data Center
January 8-10, 2014

HDF

FTP
HTTP

<metadata>
<distinfo>
<stdorder>
<digform>
<digtinfo>
<formname> ASCII </formname>
<transize> 208.000000 </transize>
</digtinfo>
<digtopt>
<onlinopt>
<computer>
<networka>
<networkr> http://www.ngdc.noaa.gov/geomag/aromag.shtml </networkr>
</networka>
</computer>
</onlinopt>
</digtopt>
</digform>
<fees> Depends on the Data Set </fees>
<ordering> Price information is available upon request. </ordering>
<turnarnd> 4 Days </turnarnd>
</stdorder>
</distinfo>
</metadata>

ESIP Winter 2014

3

Evolving Data Access
T
H
H
R
Y
E or R
D
A
D
X
S

OPeNDAP
HTTP
WMS/WCS
Metadata

Data Center
January 10, 2014

5

THREDDS = Data + Services

January 8-10, 2014

THREDDS = Data + Services

Data Services

ESIP Winter 2014

January 8-10, 2014

8

THREDDS = Data + Services

Metadata Services

ESIP Winter 2014

January 8-10, 2014

9

THREDDS = Data + Services

Tools

January 8-10, 2014

Multiple Services = Good Thing

January 8-10, 2014

ESIP Winter 2014

11

Service Metadata (nciSO NcML)
<group name="THREDDSMetadata">
<attribute name="id" value="hdf4DatasetScan/NASAHDF/SW_S3E_2003100.20053531923.hdf" />
<attribute name="full_name" value="NASAHDF/SW_S3E_2003100.20053531923.hdf" />
<attribute name="data_type" value="GRID" />
<group name="services">
<attribute name="opendap_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/dodsC/mnt/ftp/
pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf" />

<attribute name="httpserver_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/fileServer/
mnt/ftp/pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf" />
<attribute name="wcs_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/wcs/mnt/ftp/pub/
outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf?
service=WCS&version=1.0.0&request=GetCapabilities" />

<attribute name="wms_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/wms/mnt/ftp/pub/
outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf?
service=WMS&version=1.3.0&request=GetCapabilities" />
<attribute name="nccs_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/ncss/grid/mnt/ftp/
pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf/dataset.html" />
</group>

January 8-10, 2014

ESIP Winter 2014

12

ISO Online Resource

January 8-10, 2014

ESIP Winter 2014

13

ISO Online Resource

January 8-10, 2014

ESIP Winter 2014

14

ISO Online Resource

January 8-10, 2014

ESIP Winter 2014

15

Metadata (ISO)

1 data identification
4 service identification

2 transfer options

January 8-10, 2014

ESIP Winter 2014

16

Service Information

January 8-10, 2014

ESIP Winter 2014

17

Service Information

January 8-10, 2014

ESIP Winter 2014

18

Distribution Information

January 8-10, 2014

ESIP Winter 2014

19

Distribution Information

January 8-10, 2014

ESIP Winter 2014

20

Questions?
tedhabermann@hdfgroup.org

21

Acknowledgements

This work was partially supported by contract number NNG10HP02C from NASA.
Any opinions, findings, conclusions, or recommendations expressed in this material are
those of the author and do not necessarily reflect the views of NASA or The HDF Group.

July 24, 2013

BESSIG

22

NASA's Earth Observing System (EOS) archive includes data collected over many years by many satellite instruments. These data are stored in the HDF format that includes data and metadata. The content of the metadata was examined for compliance with a set of conventions developed by the NASA science community at the beginning of the EOS Project (the HDF-EOS conventions). The initial results show that ~50% of the data files and 76% of the datasets have metadata that allows them to be used easily in standard tools. This talk was presented at the ESIP (espied.org) meeting during January 2014.

HDF Group Support for NPP/NPOESS/JPSSThe HDF-EOS Tools and Information Center

Data Are from Mars, Tools Are from Venus

The HDF-EOS Tools and Information Center

HDF5 Performance Enhancements with the Elimination of Unlimited Dimension

The HDF-EOS Tools and Information Center

SPD and KEA: HDF5 based file formats for Earth Observation

The HDF-EOS Tools and Information Center

NEON HDF5

The HDF-EOS Tools and Information Center

HDF Update 2016

The HDF-EOS Tools and Information Center

Incorporating ISO Metadata Using HDF Product Designer

The HDF-EOS Tools and Information Center

This is an introductory slide for accessing NASA HDF/HDF-EOS data for beginners. NASA distributes many Earth Science data in HDF/HDF-EOS file format and new users struggle to understand the file format and use the NASA HDF/HDF-EOS data properly. This brief presentation will help new users to understand the basic concepts about the HDF/HDF-EOS and to know the available tools that can access the NASA data easily.

Presentation sreenu dwh-services

Sreenu Musham

GES DISC Eexperiences with HDF Formats for MEaSUREs ProjectsThe HDF-EOS Tools and Information Center

Visualizing and Analyzing HDF-EOS5 and HDF5 data with NCL

The HDF-EOS Tools and Information Center

NCAR Command Language (NCL) is an interpreted language designed for sceintific data analysis and visualization with high quality graphics, espeially for atmospherice scince. NCL has been support NetCDF 3/4, GRIB 1/2, HDF-SDS, HDF_EOS, shapefiles, binary, and ASCII files for years. Now HDF-EOS5 is the released version, and HDF5 in beta-test stage. Now NCL team are developing NCL to write HDF5 files, and to read HDF-EOS5 data with OPeNDAP. NCL team will share with people their experience to visualize and analyze HDF-EOS5 and HDF5 data.

HDFCloud Workshop: HDF5 in the Cloud

The HDF-EOS Tools and Information Center

Summary of HDF-EOS5 Files, Data Model and File Format

The HDF-EOS Tools and Information Center

Visualising Research Graph using Neo4j and Gephi

amiraryani

Pilot Project for HDF5 Metadata Structures for SWOT

The HDF-EOS Tools and Information Center

Putting some Spark into HDF5

The HDF-EOS Tools and Information Center

Guided Tour of Pythonian Museum

The HDF-EOS Tools and Information Center

ArcGIS and Multi-D: Tools & Roadmap

The HDF-EOS Tools and Information Center

Scientific Computing and Visualization using HDF

The HDF-EOS Tools and Information Center

Citations in ISO Metadata

Ted Habermann

Metadata Evaluation and Improvement

Ted Habermann

For many years metadata development activities have focused on developing and sharing metadata for discovering data. This is important. Once data are discovered, metadata supporting use and understanding become important. Efforts to encourage scientists and data providers to create those metadata have had limited success. This talk describes some approaches and tools for supporting the organizational change efforts required to integrate use and understanding metadata into organizational cultures. These approaches are described in terms of the ideas presented in Switch: How to Change Things When Change is Hard.

What's hot

Efficiently serving HDF5 via OPeNDAP

The HDF-EOS Tools and Information Center

ICESat-2 Metadata and Status

The HDF-EOS Tools and Information Center

Using IDL with Suomi NPP VIIRS DataThe HDF-EOS Tools and Information Center

Moving form HDF4 to HDF5/netCDF-4

The HDF-EOS Tools and Information Center

Open-source Scientific Computing and Data Analytics using HDF

The HDF-EOS Tools and Information Center

Bridging ICESat and ICESat-2 Standard Data ProductsThe HDF-EOS Tools and Information Center

Improved Methods for Accessing Scientific Data for the Masses

The HDF-EOS Tools and Information Center

Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 productsThe HDF-EOS Tools and Information Center

NASA HDF/HDF-EOS Data for Dummies (and Developers)

The HDF-EOS Tools and Information Center

Presentation sreenu dwh-services

Sreenu Musham

GES DISC Eexperiences with HDF Formats for MEaSUREs ProjectsThe HDF-EOS Tools and Information Center

Visualizing and Analyzing HDF-EOS5 and HDF5 data with NCL

The HDF-EOS Tools and Information Center

HDFCloud Workshop: HDF5 in the Cloud

The HDF-EOS Tools and Information Center

Summary of HDF-EOS5 Files, Data Model and File Format

The HDF-EOS Tools and Information Center

Visualising Research Graph using Neo4j and Gephi

amiraryani

Pilot Project for HDF5 Metadata Structures for SWOT

The HDF-EOS Tools and Information Center

Putting some Spark into HDF5

The HDF-EOS Tools and Information Center

Guided Tour of Pythonian Museum

The HDF-EOS Tools and Information Center

ArcGIS and Multi-D: Tools & Roadmap

The HDF-EOS Tools and Information Center

Scientific Computing and Visualization using HDF

The HDF-EOS Tools and Information Center

What's hot (20)

Efficiently serving HDF5 via OPeNDAP

ICESat-2 Metadata and Status

Using IDL with Suomi NPP VIIRS Data

Moving form HDF4 to HDF5/netCDF-4

Open-source Scientific Computing and Data Analytics using HDF

Bridging ICESat and ICESat-2 Standard Data Products

Improved Methods for Accessing Scientific Data for the Masses

Interoperability with netCDF-4 - Experience with NPP and HDF-EOS5 products

NASA HDF/HDF-EOS Data for Dummies (and Developers)

Presentation sreenu dwh-services

GES DISC Eexperiences with HDF Formats for MEaSUREs Projects

Visualizing and Analyzing HDF-EOS5 and HDF5 data with NCL

HDFCloud Workshop: HDF5 in the Cloud

Summary of HDF-EOS5 Files, Data Model and File Format

Visualising Research Graph using Neo4j and Gephi

Pilot Project for HDF5 Metadata Structures for SWOT

Putting some Spark into HDF5

Guided Tour of Pythonian Museum

ArcGIS and Multi-D: Tools & Roadmap

Scientific Computing and Visualization using HDF

Viewers also liked

Citations in ISO Metadata

Ted Habermann

Metadata Evaluation and Improvement

Ted Habermann

Translation proofing

Ted Habermann

Hdf Augmentation: Interoperability in the Last Mile

Ted Habermann

Science data files are generally written to serve well-defined purposes for a small science teams. In many cases, the organization of the data and the metadata are designed for custom tools developed and maintained by and for the team. Using these data outside of this context many times involves restructuring, re-documenting, or reformatting the data. This expensive and time-consuming process usually prevents data reuse and thus decreases the total life-cycle value of the data considerably. If the data are unique or critically important to solving a particular problem, they can be modified into a more generally usable form or metadata can be added in order to enable reuse. This augmentation process can be done to enhance data for the intended purpose or for a new purpose, to make the data available to new tools and applications, to make the data more conventional or standard, or to simplify preservation of the data. The HDF Group has addressed augmentation needs in many ways: by adding extra information, by renaming objects or moving them around in the file, by reducing complexity of the organization, and sometimes by hiding data objects that are not understood by specific applications. In some cases these approaches require re-writing the data into new files and in some cases it can be done externally, without affecting the original file. We will describe and compare several examples of each approach.

Wikis, Rubrics and Views: An Integrated Approach to Improving Documentation

Ted Habermann

For many years scientists and data managers have focused on creating metadata that supports the discovery of available data. This is important, but once data sets are discovered, users need metadata that supports use and understanding of those data. This talk describes a system developed to support the required metadata improvements using wikis, rubrics, and metadata views. The wikis provide a mechanism for the community to record experiences and lessons learned and provide high-quality examples. Rubrics provide a mechanism for consistent and clear quantitative evaluation of the completeness of metadata records. The results displays include integrated links to the wiki. Views provide views with connections to the wiki and on-going interactive learning. These tools can be used with metadata from any standard and can facilitate translation of the metadata between multiple standards.

Hdf Inside

Ted Habermann

Earth Science Platform

Ted Habermann

Science platforms are made up of (at least) four planks: data formats, services, tools and conventions. I focus here on formats and conventions, specifically the HDF5 format, already used in many disciplines, and the Climate-Forecast and HDF-EOS Conventions. Many science disciplines have already agreed on HDF as the preferred format for storing and sharing data. It is well established in high performance computing and supports arbitrary grouping and annotation. Community conventions are critical for useful data on top of the format. The Climate-Forecast (CF) conventions were created for relatively simple gridded data types while the HDF-EOS conventions originally considered more complex data (swaths). Making simple conventions more complex makes adoption more difficult. Community input and the need for stable data processing systems must be balanced in governance of conventions.

ESDIS and International Standards

Ted Habermann

The NASA Earth Science Data and Information System (ESDIS) is migrating documentation for their data and products towards International Standards developed by ISO Technical Committee 211 (ISO/TC211). In order to do this effectively, NASA must understand and participate in the ISO process. This presentation was given at a NASA ISO Seminar during November 2012. It outlines the ISO standards process and describes some extensions to the ISO standards that are being proposed to address ESDIS requirements not addressed in the original standard.

ISO Metadata in HDF Data Files

Ted Habermann

The HDF Product Designer – Interoperability in the First Mile

Ted Habermann

Interoperable data have been a long-time goal in many scientific communities. The recent growth in analysis, visualization and mash-up applications that expect data stored in a standardized manner has brought the interoperability issue to the fore. On the other hand, producing interoperable data is often regarded as a sideline task in a typical research team for which resources are not readily available. The HDF Group is developing a software tool aimed at lessening the burden of creating data in standards-compliant, interoperable HDF5 files. The tool, named HDF Product Designer, lowers the threshold needed to design such files by providing a user interface that combines the rich HDF5 feature set with applicable metadata conventions. Users can quickly devise new HDF5 files while at the same time seamlessly incorporating the latest best practices and conventions from their community. That is what the term interoperability in the first mile means: enabling generation of interoperable data in HDF5 files from the onset of their production. The tool also incorporates collaborative features, allowing team approach in the file design, as well as easy transfer of best practices as they are being developed. The current state of the tool and the plans for future development will be presented. Constructive input from interested parties is always welcome.

ISO Metadata Improvements - Questions and Answers

Ted Habermann

Granules and ISO Metadata

Ted Habermann

19157 Questions and Answers

Ted Habermann

Can ISO 19157 support current NASA data quality metadata?

Ted Habermann

ISO 19157 provides a powerful framework for describing quality of Earth science datasets. As NASA migrates towards using that standard, it is important to understand whether and how existing data quality content fits into the ISO 19157 model. This talk demonstrates that fit and concludes that ISO 19157 can include all existing content and also includes new capabilities that can be very useful for all kinds of NASA data users.

Viewers also liked (14)

Citations in ISO Metadata

Metadata Evaluation and Improvement

Translation proofing

Hdf Augmentation: Interoperability in the Last Mile

Wikis, Rubrics and Views: An Integrated Approach to Improving Documentation

Hdf Inside

Earth Science Platform

ESDIS and International Standards

ISO Metadata in HDF Data Files

The HDF Product Designer – Interoperability in the First Mile

ISO Metadata Improvements - Questions and Answers

Granules and ISO Metadata

19157 Questions and Answers

Can ISO 19157 support current NASA data quality metadata?

Similar to Metadata For Humans and Machines

WellData and WellData Real-Time systems FlyerMohammad Saiful Islam

My other computer_is_a_datacentre

Steve Loughran

20150314 sahara intro and the future plan for open stack meetup

Wei Ting Chen

Fluentd and Embulk Game Server 4

N Masahiro

PHP Continuous Data ProcessingMichael Peacock

PPTVideoguy

SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...

Chester Chen

Building highly efficient data lakes using Apache Hudi (Incubating) Even with the exponential growth in data volumes, ingesting/storing/managing big data remains unstandardized & in-efficient. Data lakes are a common architectural pattern to organize big data and democratize access to the organization. In this talk, we will discuss different aspects of building honest data lake architectures, pin pointing technical challenges and areas of inefficiency. We will then re-architect the data lake using Apache Hudi (Incubating), which provides streaming primitives right on top of big data. We will show how upserts & incremental change streams provided by Hudi help optimize data ingestion and ETL processing. Further, Apache Hudi manages growth, sizes files of the resulting data lake using purely open-source file formats, also providing for optimized query performance & file system listing. We will also provide hands-on tools and guides for trying this out on your own data lake. Speaker: Vinoth Chandar (Uber) Vinoth is Technical Lead at Uber Data Infrastructure Team

Evolution of a big data projectMichael Peacock

Help your Enterprise Implement Big Data with Control-M for Hadoop

BMC Software

Inroduction to Big Data

Omnia Safaan

Data science for infrastructure dev week 2022

ZainAsgar1

Dynamic Components using Single-Page-Application Concepts in AEM/CQ

Andon Sikavica

Dynamic components display content dependable on context, hence they cannot be cached. Out of the box, Adobe Experience Manager doesn't give us many options for granular caching on a component level. When faced with this problem, we usually resort to developing components that are leveraging Server-Side Includes or AJAX to get the HTML with dynamic data. As an alternative solution, we have also developed dynamic components that use Single Page Application concepts, by using templates and JSON-responses, to provide the same dynamic behavior. In this presentation we will cover all of the dynamic components types, compare the benefits and drawbacks of each, and state the use-cases where each can be effectively applied. We will take a deeper look at the dynamic components done with SPA concepts, as they are rarity in the AEM world, and also provide a walk-through of the technologies used, how some common problems were solved, as well as the benefits that have been gained by their usage.

Dynamic Components using Single-Page-Application Concepts in AEM/CQ

Netcetera

Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015

Rajit Saha

First in Class: Optimizing the Data Lake for Tighter Integration

Inside Analysis

The Briefing Room with Dr. Robin Bloor and Teradata RainStor Live Webcast October 13, 2015 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=012bb2c290097165911872b1f241531d Hadoop data lakes are emerging as peers to corporate data warehouses. However, successful data management solutions require a fusion of all relevant data, new and old, which has proven challenging for many companies. With a data lake that’s been optimized for fast queries, solid governance and lifecycle management, users can take data management to a whole new level. Register for this episode of The Briefing Room to learn from veteran Analyst Dr. Robin Bloor as he discusses the relevance of data lakes in today’s information landscape. He’ll be briefed by Mark Cusack of Teradata, who will explain how his company’s archiving solution has developed into a storage point for raw data. He’ll show how the proven compression, scalability and governance of Teradata RainStor combined with Hadoop can enable an optimized data lake that serves as both reservoir for historical data and as a "system of record” for the enterprise. Visit InsideAnalysis.com for more information.

Hadoop crashcourse v3

Hortonworks

Dynamic components using SPA concepts in AEM

Bojana Popovska

Subsetting

The HDF-EOS Tools and Information Center

Partners 2013 LinkedIn Use Cases for Teradata Connectors for HadoopEric Sun

SplunkLive! Customer Presentation – Harris

Splunk

Similar to Metadata For Humans and Machines (20)

WellData and WellData Real-Time systems Flyer

My other computer_is_a_datacentre

20150314 sahara intro and the future plan for open stack meetup

Fluentd and Embulk Game Server 4

PHP Continuous Data Processing

PPT

SF Big Analytics 20190612: Building highly efficient data lakes using Apache ...

Evolution of a big data project

Help your Enterprise Implement Big Data with Control-M for Hadoop

Inroduction to Big Data

Data science for infrastructure dev week 2022

Dynamic Components using Single-Page-Application Concepts in AEM/CQ

Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015

First in Class: Optimizing the Data Lake for Tighter Integration

Hadoop crashcourse v3

Dynamic components using SPA concepts in AEM

Subsetting

Partners 2013 LinkedIn Use Cases for Teradata Connectors for Hadoop

SplunkLive! Customer Presentation – Harris

Recently uploaded

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

DevOps and Testing slides at DASA Connect

Kari Kakkonen

Generating a custom Ruby SDK for your web service or Rails API using Smithy

g2nightmarescribd

Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Product School

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

Paul Groth

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Tobias Schneck

As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other? Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.

Knowledge engineering: from people to machines and back

Elena Simperl

Bits & Pixels using AI for Good.........

Alison B. Lowndes

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

Recently uploaded (20)

The Art of the Pitch: WordPress Relationships and Sales

Assuring Contact Center Experiences for Your Customers With ThousandEyes

PCI PIN Basics Webinar from the Controlcase Team

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Epistemic Interaction - tuning interfaces to provide information for AI support

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

DevOps and Testing slides at DASA Connect

Generating a custom Ruby SDK for your web service or Rails API using Smithy

Neuro-symbolic is not enough, we need neuro-*semantic*

UiPath Test Automation using UiPath Test Suite series, part 3

Mission to Decommission: Importance of Decommissioning Products to Increase E...

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Knowledge engineering: from people to machines and back

Bits & Pixels using AI for Good.........

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Monitoring Java Application Security with JDK Tools and JFR Events

Accelerate your Kubernetes clusters with Varnish Caching

Metadata For Humans and Machines

1. Metadata For Humans and Machines Ted Habermann Director of Earth Science The HDF Group thabermann@hdfgroup.org January 8-10, 2014 ESIP Winter 2014 1

2. Evolving Data Access HDF HDF FTP HTTP Data Center January 8-10, 2014 ESIP Winter 2014 2

3. Evolving Data Access HDF Data Center January 8-10, 2014 HDF FTP HTTP <metadata> <distinfo> <stdorder> <digform> <digtinfo> <formname> ASCII </formname> <transize> 208.000000 </transize> </digtinfo> <digtopt> <onlinopt> <computer> <networka> <networkr> http://www.ngdc.noaa.gov/geomag/aromag.shtml </networkr> </networka> </computer> </onlinopt> </digtopt> </digform> <fees> Depends on the Data Set </fees> <ordering> Price information is available upon request. </ordering> <turnarnd> 4 Days </turnarnd> </stdorder> </distinfo> </metadata> ESIP Winter 2014 3

4. Evolving Data Access HDF Data Center January 8-10, 2014 HDF FTP HTTP <metadata> <distinfo> <stdorder> <digform> <digtinfo> <formname> ASCII </formname> <transize> 208.000000 </transize> </digtinfo> <digtopt> <onlinopt> <computer> <networka> <networkr> http://www.ngdc.noaa.gov/geomag/aromag.shtml </networkr> </networka> </computer> </onlinopt> </digtopt> </digform> <fees> Depends on the Data Set </fees> <ordering> Price information is available upon request. </ordering> <turnarnd> 4 Days </turnarnd> </stdorder> </distinfo> </metadata> ESIP Winter 2014 4

5. Evolving Data Access T H H R Y E or R D A D X S OPeNDAP HTTP WMS/WCS Metadata Data Center January 10, 2014 5

6. THREDDS = Data + Services January 8-10, 2014

7. THREDDS = Data + Services January 8-10, 2014

8. THREDDS = Data + Services Data Services ESIP Winter 2014 January 8-10, 2014 8

9. THREDDS = Data + Services Metadata Services ESIP Winter 2014 January 8-10, 2014 9

10. THREDDS = Data + Services Tools January 8-10, 2014

11. Multiple Services = Good Thing January 8-10, 2014 ESIP Winter 2014 11

12. Service Metadata (nciSO NcML) <group name="THREDDSMetadata"> <attribute name="id" value="hdf4DatasetScan/NASAHDF/SW_S3E_2003100.20053531923.hdf" /> <attribute name="full_name" value="NASAHDF/SW_S3E_2003100.20053531923.hdf" /> <attribute name="data_type" value="GRID" /> <group name="services"> <attribute name="opendap_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/dodsC/mnt/ftp/ pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf" /> <attribute name="httpserver_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/fileServer/ mnt/ftp/pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf" /> <attribute name="wcs_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/wcs/mnt/ftp/pub/ outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf? service=WCS&version=1.0.0&request=GetCapabilities" /> <attribute name="wms_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/wms/mnt/ftp/pub/ outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf? service=WMS&version=1.3.0&request=GetCapabilities" /> <attribute name="nccs_service" value="http://eosdap.hdfgroup.uiuc.edu:8887/thredds/ncss/grid/mnt/ftp/ pub/outgoing/NASAHDF/SW_S3E_2003100.20053531923.hdf/dataset.html" /> </group> January 8-10, 2014 ESIP Winter 2014 12

13. ISO Online Resource January 8-10, 2014 ESIP Winter 2014 13

14. ISO Online Resource January 8-10, 2014 ESIP Winter 2014 14

15. ISO Online Resource January 8-10, 2014 ESIP Winter 2014 15

16. Metadata (ISO) 1 data identification 4 service identification 2 transfer options January 8-10, 2014 ESIP Winter 2014 16

17. Service Information January 8-10, 2014 ESIP Winter 2014 17

18. Service Information January 8-10, 2014 ESIP Winter 2014 18

19. Distribution Information January 8-10, 2014 ESIP Winter 2014 19

20. Distribution Information January 8-10, 2014 ESIP Winter 2014 20

21. Questions? tedhabermann@hdfgroup.org 21

22. Acknowledgements This work was partially supported by contract number NNG10HP02C from NASA. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author and do not necessarily reflect the views of NASA or The HDF Group. July 24, 2013 BESSIG 22

Editor's Notes

Historic access patterns: download entire file do analysis on desktop.
FGDC metadata included URLs without much supporting information
Those links pointed to html pages that needed to be interpreted by humans in order to access the data.
Today we have more diverse users and serve them with multiple services connected to the data with THREDDS Data Server or Hyrax
The THREDDS Data Server connects data sets to multiple services
The close-up of the Access options shows links to nine different services.
Some are data services
Some are metadata services implemented as part of the ncISO and ncWMS packages.
Links to tools / viewers are also supported (with data download)
Multiple data services for a single dataset are a good thing – like motherhood and apple pie.
The service metadata is added to the dataset metadataibNcML by ncISO. This dataset has five services.
The FGDC online resource included just a URL. This worked well for simple URLs, but became less useful as web usage and diversity increasde.
The ISO 19115 standard added more elements to the URL’s to make them more useful for humans.
This made humans happy!
The ISO metadata standard allows multiple identificationInfo objects in the same record. The ISO metadata can include both data and service identification. The distribution information section includes two transfer options.
The identificationInfo section can be implemented as MD_DataIdentification or SV_ServiceIdentification. Note the XML ids used so that users can easily identify the services.
This service metadata is for machines
The distributionInfo section has a transfer option (onlineResource) for the human OPenDAP interface and for the viewer.
These distribution mechanisms are for humans

Metadata For Humans and Machines

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to Metadata For Humans and Machines

Similar to Metadata For Humans and Machines (20)

Recently uploaded

Recently uploaded (20)

Metadata For Humans and Machines

Editor's Notes