• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Data Tactics Unified Dataspace Architecture and Description
 

Data Tactics Unified Dataspace Architecture and Description

on

  • 985 views

 

Statistics

Views

Total Views
985
Views on SlideShare
984
Embed Views
1

Actions

Likes
0
Downloads
11
Comments
0

1 Embed 1

https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Data Tactics Unified Dataspace Architecture and Description Data Tactics Unified Dataspace Architecture and Description Presentation Transcript

    • Data Tactics Unified DataSpaceWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • CloudWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Systems Engineering & Integration SYSTEMS ENGINEERING • Data Ingestion Frameworks (structured, unstructured, semi-structured) • Semantic DataSpace Enrichment SYSTEM INTEGRATION • Cloud Management Systems (CMS) • Ingestion • Cloudbase/Accumulo – Generalized Ingest / – Pig (Big Data) Plug-in NiagraFiles • Dissemination and Reporting Tools • Geospatial Capabilities • Data Mining, Exploitation, and Correlation Tools • Biometric CapabilitiesWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Cloud Experience 17 Enclaves at SECRET//NOFORN 4 Enclaves for NATO ISAF • 3 in Tyson’s • 2 in Afghanistan • 1 at GISA, Ft. Bragg • 1 at GISA, Fort Bragg • 2 in Hawaii • 1 in Germany • 2 in Germany US BICES Cloud in Germany • 7 at Aberdeen Over a dozen at UNCLASS//FOUO • 2 in Afghanistan • Supporting real-world missions on 6 Enclaves at TS//SCI contract • AF TENCAP • At various levels of complexity • NRL • DARPA • INSCOM • DCGS-A • DHS OI&A Cloud Domains is where we live Data, is the Hard ProblemWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Data – The Hard PartWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • BigData Architecture Data Tactics has delivered solutions that manage PETABYTES of data and provide mission relevant analytics, metrics and user interfaces • DESIGN, DEVELOPMENT AND INTEGRATION OF REFERENCE ARCHITECTURES – Ghost Machine – Stratus • SECURE DATABASE ARCHITECTURES – Secure Entity Database (SED) – Defense Cross-Domain Analytic Capability (DCAC) • DATA MIGRATION, EXTRACTION, TRANSFORM AND PARSING • FEDERATED DATA MANAGEMENT – Federated Search, Multi-Source / Multi-Vendor Integration – Storage Cluster Management • DATA MINING AND FORENSIC ANALYSIS • SPATIAL, MULTI-DOMAIN, AND CLOUD DATA SERVICESWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Unified DataSpace The Wild • Data sources with rich data & Segment 3 - Model Description semantic context locked in domain Data Rich semantic silos Models context • Data tightly coupled to data-models • Data-models Segment 2 - Data Description tightly coupled to Structured Integration Enrichment storage models Data Exploitation Exploration Silos isolated by Across all sources • Implementation Segment 1 - Artifact Description technology • Storage structure Unstructured Rich data • Data Data context representation • Data modalityWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Unified DataSpace High-Level Conceptual Model of the DataSpace and Ingest/Extraction Flows Segment 3 - Model Semantics . 2 . 2 CONCEPT CONCEPT_ASSOCIATION PREDICATE PREDICATE_ASSOCIATION . . . Uses Uses Segment 1 - Artifact Semantics Segment 2 - Data Semantics Semantics SOURCE . + 2 . 2 . Metadata ARTIFACT ARTIFACT_ASSOCIATION TERM . STATEMENT . . . . Data Uses + Metadata Metadata Segment 0 - Artifacts Ingest ExtractionWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Unified DataSpace •Segment 0 is an artifact store (i.e., binary representation of artifacts). High-Level Conceptual Model of the DataSpace and Ingest/Extraction Flows Segment 3 - Model Semantics •Segment 1 represents artifact semantics . 2 . 2 CONCEPT CONCEPT_ASSOCIATION PREDICATE PREDICATE_ASSOCIATION . . . and includes artifact metadata and Uses Uses associations between the artifacts. Indexing Segment 1 - Artifact Semantics Segment 2 - Data Semantics Semantics of Segment 1 supports search on text SOURCE . + 2 . 2 . Metadata ARTIFACT ARTIFACT_ASSOCIATION TERM . STATEMENT . . . content, geospatial, and artifact meta data. . Data Uses + Metadata Metadata Segment 0 - Artifacts •Segment 2 represents data and semantics Ingest Extraction of structured data elements extracted from artifacts. Indexing of Segment 2 supports search on properties of entities (e.g., Person, Location) based on their properties and relationships. •Segment 3 represents data-models extracted from artifacts and models used for aligning, disambiguating, and enriching the elements of Segments 1 and 2.WWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Data Description Framework • DDF – looks at data in the following ways – Mention: A chunk of data, either physically located within a tangible artifact, or contained within an analyst’s mind • “Washington” at offset x in file Y – Sign: A representation of all disambiguated mentions that are identical except for their indexicality • E.g., “Washington” – Concept: An abstract idea, defined explicitly or implicitly by a source data- model • E.g., City, Person, Name, Address, Photo – Predicate: An abstract idea used to express a relationship between “things” • E.g., isCity, isPerson, hasName, hasAddress, hasPhoto – Term: A disambiguated sign abstracted from the source artifact or asserting analyst • E.g., Washington Person; Washington Location – Statement: Encodes a binary relationship between a subject (term) and an object mediated by a predicate • E.g.,[Washington, Person] hasPhoto [GeorgeWashingtonImage.jpg]WWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Unified DataSpaceWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Data Model ExampleWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • DataSpace WorkbenchWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
    • Elastic Data Ingest Java Messaging Service Artifact Processor Persistence Index Manager Error Manager Queue Manager Queue Queue Queue Queue Artifact Persistence Index Error Loader Processor Manager Manager Manager UDS Components Lucene File System Custom Components Hadoop DFS Artifact Processor Persistence Modules Manager Modules BigTableWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS