Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Data Tactics             Unified DataSpaceWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – ...
CloudWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
Systems Engineering & Integration    SYSTEMS ENGINEERING    •   Data Ingestion Frameworks (structured, unstructured, semi-...
Cloud Experience 17 Enclaves at SECRET//NOFORN                  4 Enclaves for NATO ISAF      • 3 in Tyson’s              ...
Data – The Hard PartWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
BigData Architecture    Data Tactics has delivered solutions that manage PETABYTES of data    and provide mission relevant...
Unified DataSpace  The Wild  • Data sources  with rich data &                      Segment 3 - Model Description  semantic...
Unified DataSpace                                    High-Level Conceptual Model of the DataSpace                         ...
Unified DataSpace  •Segment 0 is an artifact store (i.e., binary  representation of artifacts).                           ...
Data Description Framework  •   DDF – looks at data in the following ways       – Mention: A chunk of data, either physica...
Unified DataSpaceWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
Data Model ExampleWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
DataSpace WorkbenchWWW.DATA–TACTICS.COM   © 2012 Data Tactics   ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
Elastic Data Ingest                                        Java Messaging Service       Artifact Processor          Persis...
Upcoming SlideShare
Loading in …5
×

Data Tactics Unified Dataspace Architecture and Description

1,209 views

Published on

  • Be the first to comment

  • Be the first to like this

Data Tactics Unified Dataspace Architecture and Description

  1. 1. Data Tactics Unified DataSpaceWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  2. 2. CloudWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  3. 3. Systems Engineering & Integration SYSTEMS ENGINEERING • Data Ingestion Frameworks (structured, unstructured, semi-structured) • Semantic DataSpace Enrichment SYSTEM INTEGRATION • Cloud Management Systems (CMS) • Ingestion • Cloudbase/Accumulo – Generalized Ingest / – Pig (Big Data) Plug-in NiagraFiles • Dissemination and Reporting Tools • Geospatial Capabilities • Data Mining, Exploitation, and Correlation Tools • Biometric CapabilitiesWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  4. 4. Cloud Experience 17 Enclaves at SECRET//NOFORN 4 Enclaves for NATO ISAF • 3 in Tyson’s • 2 in Afghanistan • 1 at GISA, Ft. Bragg • 1 at GISA, Fort Bragg • 2 in Hawaii • 1 in Germany • 2 in Germany US BICES Cloud in Germany • 7 at Aberdeen Over a dozen at UNCLASS//FOUO • 2 in Afghanistan • Supporting real-world missions on 6 Enclaves at TS//SCI contract • AF TENCAP • At various levels of complexity • NRL • DARPA • INSCOM • DCGS-A • DHS OI&A Cloud Domains is where we live Data, is the Hard ProblemWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  5. 5. Data – The Hard PartWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  6. 6. BigData Architecture Data Tactics has delivered solutions that manage PETABYTES of data and provide mission relevant analytics, metrics and user interfaces • DESIGN, DEVELOPMENT AND INTEGRATION OF REFERENCE ARCHITECTURES – Ghost Machine – Stratus • SECURE DATABASE ARCHITECTURES – Secure Entity Database (SED) – Defense Cross-Domain Analytic Capability (DCAC) • DATA MIGRATION, EXTRACTION, TRANSFORM AND PARSING • FEDERATED DATA MANAGEMENT – Federated Search, Multi-Source / Multi-Vendor Integration – Storage Cluster Management • DATA MINING AND FORENSIC ANALYSIS • SPATIAL, MULTI-DOMAIN, AND CLOUD DATA SERVICESWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  7. 7. Unified DataSpace The Wild • Data sources with rich data & Segment 3 - Model Description semantic context locked in domain Data Rich semantic silos Models context • Data tightly coupled to data-models • Data-models Segment 2 - Data Description tightly coupled to Structured Integration Enrichment storage models Data Exploitation Exploration Silos isolated by Across all sources • Implementation Segment 1 - Artifact Description technology • Storage structure Unstructured Rich data • Data Data context representation • Data modalityWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  8. 8. Unified DataSpace High-Level Conceptual Model of the DataSpace and Ingest/Extraction Flows Segment 3 - Model Semantics . 2 . 2 CONCEPT CONCEPT_ASSOCIATION PREDICATE PREDICATE_ASSOCIATION . . . Uses Uses Segment 1 - Artifact Semantics Segment 2 - Data Semantics Semantics SOURCE . + 2 . 2 . Metadata ARTIFACT ARTIFACT_ASSOCIATION TERM . STATEMENT . . . . Data Uses + Metadata Metadata Segment 0 - Artifacts Ingest ExtractionWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  9. 9. Unified DataSpace •Segment 0 is an artifact store (i.e., binary representation of artifacts). High-Level Conceptual Model of the DataSpace and Ingest/Extraction Flows Segment 3 - Model Semantics •Segment 1 represents artifact semantics . 2 . 2 CONCEPT CONCEPT_ASSOCIATION PREDICATE PREDICATE_ASSOCIATION . . . and includes artifact metadata and Uses Uses associations between the artifacts. Indexing Segment 1 - Artifact Semantics Segment 2 - Data Semantics Semantics of Segment 1 supports search on text SOURCE . + 2 . 2 . Metadata ARTIFACT ARTIFACT_ASSOCIATION TERM . STATEMENT . . . content, geospatial, and artifact meta data. . Data Uses + Metadata Metadata Segment 0 - Artifacts •Segment 2 represents data and semantics Ingest Extraction of structured data elements extracted from artifacts. Indexing of Segment 2 supports search on properties of entities (e.g., Person, Location) based on their properties and relationships. •Segment 3 represents data-models extracted from artifacts and models used for aligning, disambiguating, and enriching the elements of Segments 1 and 2.WWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  10. 10. Data Description Framework • DDF – looks at data in the following ways – Mention: A chunk of data, either physically located within a tangible artifact, or contained within an analyst’s mind • “Washington” at offset x in file Y – Sign: A representation of all disambiguated mentions that are identical except for their indexicality • E.g., “Washington” – Concept: An abstract idea, defined explicitly or implicitly by a source data- model • E.g., City, Person, Name, Address, Photo – Predicate: An abstract idea used to express a relationship between “things” • E.g., isCity, isPerson, hasName, hasAddress, hasPhoto – Term: A disambiguated sign abstracted from the source artifact or asserting analyst • E.g., Washington Person; Washington Location – Statement: Encodes a binary relationship between a subject (term) and an object mediated by a predicate • E.g.,[Washington, Person] hasPhoto [GeorgeWashingtonImage.jpg]WWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  11. 11. Unified DataSpaceWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  12. 12. Data Model ExampleWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  13. 13. DataSpace WorkbenchWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS
  14. 14. Elastic Data Ingest Java Messaging Service Artifact Processor Persistence Index Manager Error Manager Queue Manager Queue Queue Queue Queue Artifact Persistence Index Error Loader Processor Manager Manager Manager UDS Components Lucene File System Custom Components Hadoop DFS Artifact Processor Persistence Modules Manager Modules BigTableWWW.DATA–TACTICS.COM © 2012 Data Tactics ARCHITECT – ENGINEER – INTEGRATE – SOLUTIONS

×