2 the conceptual model

BEXIS Tech Talk Series
#2: The Conceptual Model
Javad Chamanara
January 2016

Recall from the first talk
• Requirements
– Data Lifecycle Management
– Generic
– Extensible
– Portable
– Scalable
2BEXIS Tech Talk #1: The Big Picture

Requirements -> DLM
• Flexible Data Structures
• Data Submission
• Validation
• Preserving
• Metadata Management
• Versioning
3BEXIS Tech Talk #1: The Big Picture

Conceptual Overview
BEXIS Tech Talk #1: The Big Picture 4
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
«use»
«use»
«use» «use»
«use»

Extension Method
SearchPublishing
CMLand Use
Reservation
Data
Submission

Data Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Datasets
• A set of tuples
• Data container for
observations,
measurements,
simulations, and other
supported forms of data
• has one Data Structure
(later)
Dataset

Versions
• Each dataset can have
multiple versions.
• processing and
citations, independent
of the following
changes.
• Check-Out, Edit, Check-
In procedure
• Blocking check-outs
• Same user check-ins
Dataset
Dataset Version
1

Tuples
• Tuple as a collection of Data
Cells containing the Data Items
• The data tuples belong to the
versions.
• Edits, deletion, additions are
preserved
• Previous versions are
reproducible
• Differential versioning
Dataset
Dataset Version
Tuple
1
{Delta Association,
Only Structured Data}

Data Cells
• The value of an
observation, simulation,
etc
• Single vs Multiple Value
Cell
• Auxilliary Infomation
– sampling time
– result time
– description
Dataset Version
Tuple
DataValue
Data Cell
+Variable Values

Metadata
• Each version has its own
metadata
• Dataset’s metadata is
the metadata of its
latest version
Dataset Version
Metadata::Metadata
1
1

Satging
• Stages indicate quality,
status, state in
workflows, etc.
• Versions have stages
• The stage of the latest
version is the dataset’s
current stage.
Dataset
Dataset VersionDataset Stage
1
10..1
1
+Current Stage 1

Extensions: Amendments
• Special kinds of data
cells which can be
attached to specific
tuples
• Capturing exceptional/
occasional observations
• Diff. tuples may have
diff. amendments
Dataset Version
Tuple
Amendment

Extensions: Extended Properties
• User defined, dataset
specific attribute whose
value applies to a single
column in a single
dataset
• Sampe usage :
– Storing the error margin
of the instrument used
to measure the values of
a variable
Dataset
Dataset Version
Extended Property
Value
1

Extensions: Views
• Subset of a dataset
obtained by selection or
projection
• Purpose
– Further processing, sharing
or sampling
– Security /Digital rights
management
• Spanning view
– View across multiple
dataset using the same
Data Structure
Dataset
View
Criteria
0..1
1
+Content Selection Criteria
0..1
1
+Variable Selection Criteria
0..1

Example of Extensions
Views
S.N. Tmp Time S.M. Depth Pos. Hu.
14 22 1 12 -10 A 46 78 Green
13 23 2 10 -10 B 45
16 21 3 12 -11 C 30 0.11
16 18 5 15 -10 A 25
18 14 6 17 -9 D 25 Yes 100 EP
Variable 1 Variable 2 Amendments
±0.10%Error
YesRounded
1 Sec.Interval
Tmp Time Hu.
22 1 46
23 2 45
21 3 30
18 5 25
14 6 25
S.N. S.M. Depth
14 12 -10
13 10 -10
16 15 -10
18 17 -9
Extended Properties ‫خاک‬ ‫رطوبت‬Persian
BodenfeuchteGerman
Soil MoistureEnglish
Globalization Info
Data Structure
Observation (Tuple)

Data Package All Together
Dataset
Dataset Version
Dataset Stage
Tuple
DataValue
Data Cell
Extended Property
Extended Property
Value
Data Structure Content DescriptorView MetadataAmendment
+Variable Values1
1
1
1
10..1
1
+Current Stage
1
{Delta Association,
Only Structured Data}
1
1
{ordered}
{At least one for Unstructured}
0..*
0..1

Data Structure Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Data Structure
• Defines the organization
& meaning of the data
•
Data Structure

Types of Data Structures
• Structured data has a
header information and
is in tabular form
• Unstructured data can
be of any format
Data Structure
Structured DataUnstructured Data

Tabular data headers
• Variables act as table
headers
• There are parameters
too, auxiliary data
about a variable
• Data attributes are the
shared/ reusable parts
of the variables
Data Structure
Data Container
Data Attribute
Base Usage
Variable
+Variables
1..*

Dataset and Data Structure
Dataset
Data Structure
Dataset Version
1
1

Data Container
• Factors out reusable
elements of variables (such
as UoM, data types, and
data validation rules)
• Re-use Data Containers in
different data structures
used in different datasets
• Automatic unit conversion
functionality
• Benefits
– Cross dataset query
– Easier data integration
– Enhanced data discovery
Data Attribute
Data Container
Metadata Attribute
{No Extended Property}

Data Container in Detial
Data Container
Parameter VariableMetadata Attribute
Data Container
Data Type Unit
0..1
+Applies To
1
Data Container
Data Type Unit
0..1
+Applies To
1
Data Container
Methodology
Aggregate Function
0..1
Data Container
Methodology
Aggregate Function
0..1
Data Container Constraint
Default Value
Domain Value
Validator
Data Container
Extended Property
Globalization Info«enumeration»
Measurement Scale
0..1
1
{No Duplicate}
Data Container Constraint
Default Value
Domain Value
Validator
Data Container
Extended Property
Globalization Info«enumeration»
Measurement Scale
0..1
1
{No Duplicate}
Data Container
Semantic
Description
Data Container
Semantic
Description

Data Cells to Hold Data
DataValue
Data Cell
Dataset
Dataset Version Amendment
Tuple
Base Usage
Variable
Data Structure
Structured Data
Data Container
Data Attribute
1
+Variable Values
1
+Variables
1..*

Data Structure Package All
Together
Data Attribute
Data Container
Aggregate Function
Globalization Info
Structured Data
Semantic
Description
Data::Dataset
Value Type
Container
Reference Type
Container
Domain Value
«enumeration»
Measurement Scale
Constraint
Extended PropertyUnstructured Data
Data Structure
View
Default Value
Validator
Methodology
Classifier
Unit Conversion Method
Data Type
ParameterVariable
Spanning View
Common::Base
Usage
1
{No Duplicate}
0..1
0..1
{Structure Type}
+Indexer
0..1
0..1
1
+Variables
1..*
{Duplicate Control}
+Parameters
+Usage Context
1
{Data Structure Only}
1
+Applies To
+To
1
+From
1
0..1
* *
0..1
0..1

Metadata Package
DataMetadata
«use»
«use»
«use» «use»
«use»

The Metadata Entity
• Metadata is all the data
gathered about a
dataset
• Belongs to a version
• Is checked-out/in with
the version
Metadata
Dataset Version
1
1

Metadata Values
• Metadata consists of
values, which have
definitions.
• Definitions are captured
using the same
technique as Data Cells;
“Data Containers”
Metadata
Metadata Attribute
Value
Dataset Version
Metadata Simple
Atribute
1
1..*
11
1

Metadata has structure, too
• What attributes should
be present in a
metadata and how they
are arranaged
Metadata
Dataset Version
MetadataStructure
11
1

Metadata Package All Together
Dataset
MetadataMetadata Attribute
Value
Dataset Version
Metadata Simple
Atribute
MetadataStructure
1
11..*
1
1
1 1

Metadata Structure Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Metadata Structure
• Metadata structure
determines what should
be captured by the
metadata
• Different datasets may
choose to have different
metadata structures
MetadataStructure
Metadata Package
Metadata Attribute
1..*
+Parent
+Children

Metadata Structure
• Structures are
hierarchical making it
possible to have any sub
tree as a structure
• Structures at any level
have a collection of
packages, which are a
bunch of attributes
• Attributes can be shared
among various packages,
the same for the
packages
MetadataStructure
Metadata Package
Metadata Attribute
1..*
+Parent
+Children

Simple & Compound Attributes
• Simple attributes are
metadata value
specifiers
• Compound attributes
are collection of simple
and compound
attributes to build
coarser grain attributes
Metadata Attribute
Metadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
1
2..*

Metadata Structure Composition
MetadataStructureMetadata PackageMetadata Attribute
Metadata::MetadataMetadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
1
+Parent
+Children
11..*
1
2..*
1..*

How a package is used in a
structure
• Packages may have
their
– Roles, e.g., an EML Party
can be the author, the
owner, etc. of a dataset.
– Cardinalities: Min/Max
occurrences
– Optional/Mandatory
– …
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Package
Usage
1
+Parent
+Children
11..*
1
2..*
1..*

Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Attribute
Usage
1
+Parent
+Children
11..*
1
2..*
1..*
How an attribute is used in a
package
• Attributes may have their
– Roles, e.g., a Date attribute
can be the project start
date, publication date, etc.
– Cardinalities: Min/Max
occurrences
– Optional/Mandatory
– …
• In their associated
packages

Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Compound
Usage
1
+Parent
+Children
11..*
1
2..*
1..*
How an attribute is used in a
package
• A compound attribute
may contain other
compound and/or
simple attributes. E.g.,
Person can have name
and Address
• At least two attributes
are needed
• Cardinality and Role
playing are available.

Mapping to external metadata
formats
• Proper for metadata
format conversion
• Import/export from/to
standard metadata
schemas
MetadataStructure
Metadata PackageMetadata Attribute
Mapping Info
1..*
+Parent
+Children
1
0..*

Metadata Structure Package All
Together
MetadataMetadata Attribute
Value
Mapping Info
Dataset Version
Dataset
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Package
Usage
Data Container
Metdata Attribute
Usage
Metdata Compound
Usage
Base Usage
1
+Parent
+Children
11
11..*
1
10..*
1
2..*
1..*

Administration Package
DataMetadata
«use»
«use»
«use» «use»
«use»

The Party Entity
• Represents individuals,
institutes, projects,
consortiums, etc.
Party
Person Organization

Parties & Datasets
Party
Person Organization
Data::Dataset
**

Parties have various contacts
Person Organization
Party Locator

Parties have statuses
Person Organization
Party Status
1
+CurrentStatus
1
1
+History
*

Statuses are limited
Person Organization
Party Status
StatusType
1
+CurrentStatus
1
1
+History
*
*
1

Statuses are Specific to a Party
Type
Person Organization
Party Status
StatusTypePartyType
1
+CurrentStatus
1
1
+History
*
*
1
1 1..*

Parties are Party Type specific,
too.
Person Organization
Party Status
StatusTypePartyType
1
+CurrentStatus
1
1
+History
*
*
1
1 1..*
+type 1
*

Party subclasses may not be
needed
Party
PartyType
Status
StatusType
1 1..*
1
+CurrentStatus
1
*
1
1
+History
*
+type 1
*

Parties have profiles
Party
PartyType
CustomAttribute
CustomAttributeValue
+type 1
*
1
*
*
1
1
*

Parties can be in relationships
PartyPartyRelationship
PartyType
+1st
+2nd
+type 1
*

Relationship types are known,
too
PartyTypePartyRelationshipType
+1st
+2nd
+type 1
*
+type 1
*

Pairing is controlled
Party
PartyRelationship
PartyRelationshipType PartyTypePair PartyType
+1st
+2nd
+type 1
*
1
*
+AllowedSource
+AllowedTarget
+type 1
*

Time!
• Parties have lifetime (start/end)
• Relationships have lifetime
• Relationship’s lifetime can’t exceed neither
parties’ lifetimes

Party Sub-package All Together
PartyRelationshipType PartyTypePartyTypePair
Status
StatusType
Locator
CustomAttribute
CustomAttributeValue
Dataset Data Plan
1 1..*
+type 1
*
+type 1
*
1
+History
*
1
*
+AllowedSource
+AllowedTarget
+1st
+2nd
1
+CurrentStatus
1
1
*
1
*
*
1
*
*
*
1
0..1
*
*

Data Plan
• Data Plan
– Enforces policies
– Determines chosen data
structures available to
the data plan users
• Each party may have a
set of plans, e.g.,
Project A uses Plan P1
to enforce open access
policy
Data Structure::Data
Structure
Data::Dataset
Data Plan
Accessibility Policy
Ownership PolicyPublishing Policy Access Policy
Party::Party
*
+Proposed Data Structures
1..*
1
0..1
*
*
**

Semantics Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Basic Definitions
• An ontology:
– is a formal specification of a shared
conceptualization (Tom Gruber)
– is the study of entities and their relations in an
area of concern

The Purpose
• Semantic Annotation of Variables, Attributes,
metadata, and data to enhance:
– Data discovery
– Data integration

Ontology
• Ontology is a collection
of relationships
between some terms
• Can be hierarchical to
build sub ontologies
Ontology
TermRelationship
+Sub Ontologies

• Relationships follow this
pattern:
Subject -> Predicate-> Object
• Terms can be anything,
but controlled by the
“TermType”
Ontology Term
TermRelationship
«enumeration»
Term Type
+Sub Ontologies
1
+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1

Data Containers Get Annotated
• Data Containers may
get annotated as a unit,
an entity, or a
characteristic, but can
be relaxed
• Data and metadata
attributes inherit the
annotation feature
• Variables, data cells,
and metadata values
are included, too.
Data Container
Semantic
Description
Ontology Term
TermRelationship
+Sub Ontologies
+Chracteristic
0..1
+Entity
0..1+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1
+Unit
0..1

Semantic Package All Together
Data Attribute
Data Container
Semantic
Description
Ontology Term
TermRelationship
«enumeration»
Term Type
Metadata Attribute
+Sub Ontologies
1
+Chracteristic
0..1
+Entity
0..1+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1
+Unit
0..1
Its just the conceptual model;
implementation may differ

Security Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Package Responsibilities
• Authentication
– Internal user base
– Single Sign-On
• Authorization
– Access to functionalities
– Access to data objects
– Attribute & Expression based Az
• Auditing
– Who, what, when, on what, etc.

Authorization
• Permission indicates that
whether a subject is
granted a right over an
object.
• Grant is effective only
during "From" and "To"
date/ times.
• No permission means
DENY, which can be also
determined by a global
security policy.
• The Az can be completely
turned OFF!
Subject
Permission
Object
1
0..*
1
0..*

What is a Subject
• Can be user, a group, or
a role
• Various memberships in
place
• Effective subject is
derived from the
membership graph,
knowing the user
Subject
User
Role
Group
Security Role
Security User
Membership
«trace»
Membership
«trace»

Permission
• A Right limits the
permission to a aspect,
e.g., Read, Create,
Execute, etc.
Permission
«enumeratio...
Right
0..*
+Operation 1..*

What can be an Object
• Actions are system
functionalities
• They can be hierarchical
• Permission on parents
propagates to children
• Data are the single or
partial entities, e.g.,
dataset, metadata,
view, etc.
Object
Data
Action
*
0..1

Complex Authorization Rules
• A logical expression of:
– Attributes of designated
data item
– Operators
– Precedence
• That is evaluated at
runtime to determine
whether the
“Permission” is granted
Permission
Expression
1
0..*

Security Package All Together
Subject
User
Role
Group Permission
Expression
«enumeratio...
Right
Object
«enumeratio...
Action Type
Data
Action
{XOR}
Security Role
Security User
1
1 0..*
1
0..*
0..*
+Operation 1..*
Membership
«trace»
Membership
*
0..1
«trace»
10..*

Geographical Information
Package
DataMetadata
«use»
«use»
«use» «use»
«use»

Geo Package
• Outsourced to a third party system that
provides:
– Features
– Geometry
– Feature Attributes
– Security Integration
– API Access
• Querying
• Visualizing
Geographic
Description
Geometry
Feature AttributeParty::Party
Feature
Feature Attribute
Value
1
1 0..1 1
1

References
• Journal paper
• Datasets paper
• Brazil Presentation
• CM Model URL
• Anything more

Outlook
Whats next in the talk series?
• The Overal Architecture
• The Database Design

7777
Thanks!
Questions?
Contact:
javad.chamanara@uni-jena.de
http://fusion.cs.uni-jena.de/bexis
BEXIS Tech Talk #1: The Big Picture
Acknowledgment

2 the conceptual model

More Related Content

What's hot

Viewers also liked

Similar to 2 the conceptual model

More from javadch

Recently uploaded

2 the conceptual model

Editor's Notes