• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
GRAU DataSpace Architecture
 

GRAU DataSpace Architecture

on

  • 303 views

GRAU DataSpace provides FileShare & Sync for enterprises and managed service providers

GRAU DataSpace provides FileShare & Sync for enterprises and managed service providers

Statistics

Views

Total Views
303
Views on SlideShare
298
Embed Views
5

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 5

https://twitter.com 5

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    GRAU DataSpace Architecture GRAU DataSpace Architecture Presentation Transcript

    • GRAU Data Space 2.0 – The Secure Communication Platform for Businesses and Organizations YOUR DATA. YOUR CONTROL 7. Dez 2013
    • Architectural Overview ● ● ● The GDS is based on a very robust core which is available since years The architecture scales from SMB (<100 user) to large enterprises and service providers (>100.000 users) The key features for scalability are: – – – – – – – – – – – Separation between data and meta data (optional) Transactional scalable storage backend Versioning of all file objects (UUID) Chunking of large objects (chunksize can be different for each object) Hashing of chunked objects (offloading to object store is possible) Chunk level deduplication based on hash (under development) Bidirectional master/master replication of all data and meta data on folder level Session director allows redirection of sessions to another node RESTful APIs CMIS (getContentChanges) Distributable in-memory cache for meta data
    • Open interfaces ● Open standard interfaces – – JSON/SOAP core API – ● WebDAV CIFS Gateways – – ● OwnCloud CMIS 1.1 (SOAP, AtomPub, JSON) Identity Management – Provisioning Gateway (LDAP, AD,SQL) – Authentication Gateway (LDAP, AD, RADIUS)
    • Architecture Admin GUI WebGUI ownCloudGW Adm GW GDS2 API (JSON) CMIS GW CIFS WebDAV GDS core Storage Backend Object-Store Caringo S3 SWIFT FS/CIFS NAS GAM Metadata SQL DB/2 Oracle MySQL Postgres SQL DB/2 Oracle MySQL Postgres
    • Storage Backend (1) ● Storage backends: – Filesystem (ext4, XFS) – – RDBMS (MySQL, Oracle, Postgres, MSSQL, DB2) – ● NAS / CIFS Object stores (Caringo, S3, SWIFT) Plugins: – Object chunking (size definable on object level, 512k default) – Hashing (MD5, SHA-1, SHA-256) – Dedup on chunk-level [under development] – Mirroring (one or many backends) [planed] – Crypto (symmetrical) [planed] – HSM [planed]
    • Storage Backend (2) GDS core Storage Backend Hashing (optional) Chunking (512kB) Crypto (sym.) Mirroring Object store Filesystem CIFS SQL NAS GAM/Archive DB/2 Oracle MySQL Postgres Caringo RADOS SWIFT/S3 ext4 XFS
    • Storage Backend (3) GDS2 API (JSON) GDS2 API (JSON) GDS core GDS core Metadata Object Store Replication Metadata Object Store SWIFT SWIFT RADOS GW RADOS GW librados librados RADOS OSD RADOS OSD RADOS OSD Metadata
    • Scalability / High availibility ● Master/master replication on folder level – – Users, groups – ● Data, metadata Access lists Shared nothing architecture – – High availability – Users that share a lot of folders can be relocated to the same node – Adding or removing nodes dynamically – ● Horizontal scalability Software updates on deactivated nodes Distributed metadata cache – ● CMIS gateway allows session and metadata caching Session redirector (reverse proxy) – Redirects session to the home node of the user – If the home node is down, one of the backup nodes will be used
    • High availibility Load Balancer Load Balancer GDS (Session) Director GDS (Session) Director GDS2 API (JSON) GDS2 API (JSON) GDS core GDS core Storage Metadata Replication Data Metadata Storage Metadata
    • Scalability (1) Load Balancer Load Balancer GDS (Session) Director GDS (Session) Director GDS2 API (JSON) GDS2 API (JSON) GDS core GDS core Metadata Data Master/Master Replication Metadata Objectstore / Cluster filesystem Data Metadata
    • Scalability (2) Load Balancer Load Balancer GDS (Session) Director GDS (Session) Director CMIS Cache CMIS Cache CMIS Cache CMIS Cache GDS2 API (JSON) GDS2 API (JSON) GDS2 API (JSON) GDS core GDS core GDS core MD Data Metadata Replication MD Data Metadata Replication Objectstore / Cluster filesystem MD Data
    • Multiple Sites - Roaming (1) ● ● ● ● ● ● ● Every user has a home node which is stored in the account data Redundancy of file objects is provided by objects store at each site Users, groups and ACLs are synchronized between all sites File objects are not synchronized between sites Synchronization takes place asynchronously Load balancer directs client request to session director Session director redirects request based on user account to – – Node which hosts shared data room [shared] – ● Home node of the user [my] Any node [global] Session director analyzes the request and forwards to – CMIS caching layer – JSON API layer
    • Multiple Sites - Roaming (2) CMIS JSON LB LB LB LB GDS Director GDS Director GDS Director GDS Director CMIS Cache CMIS Cache CMIS Cache CMIS Cache GDS2 API GDS2 API GDS2 API GDS2 API GDS core GDS core GDS core GDS core MD Data Data Site A MD MD Data Data Site B MD
    • Identity Management (1) ● ● ● ● Separation between user provisioning and authentication Multiple instances of gateways are possible Multiple directories can be connected in parallel Provisioning gateway – LDAP/AD/SQL crawler – Users that match a regular expression are created in the GDS – Users that got deleted in the directory get deactivated in the GDS – SCIM/SAML module [planed]
    • Identity Management (2) ● Authentication gateway – – Multilevel authentication – Google authenticator [planed] – RADIUS module [planned] – ● LDAP/AD/SQL module MTAN/OTP module [planed] Single Sign-On [planned] – Kerberos module – OAUTH2 module
    • Identity Management (3) WebGUI Admin GUI Admin GW GDS2 API (JSON) Provisioning Gateway LDAP/AD SQL SAML GDS core Storage Backend Metadata Authentication Gateway LDAP/AD SAML RADIUS
    • Multi Tenancy ● Dedicated Hardware – – ● Highest level of separation and security No performance impact of virtualization layer Full virtualization (KVM, HyperV, Vmware, XEN) – – Similar static memory pages can be shared between instances – ● Highest level of separation and security in virtualized environment GDS version can be different for each tenant Linux Containers (LXC) – – ● Lightweight virtualization Memory and program files on disk can be shared between instances Single instance – Same GDS version for all tenants – Everything gets shared – Software bugs or operational problems affect all tenants
    • Distributed Data Space GDS CIFS FW Site B GDS CIFS JSON LAN Site A CIFS JSON HT T S TP HT CIFS FW LAN FW LAN GDS CIFS PS Internet HT T LAN CIFS FW GDS CIFS JSON PS Site C HT Site D PS T CIFS JSON
    • CMIS Site B1 Site B2 WebDAV GDS HT T Site B CIFS GDS CMIS GDS WebDAV PS OS OS Site C CIFS CMIS WebDAV GDS HT T PS CMIS Cache HT S TP SD WebDAV CMIS GDS GDS CMIS CIFS HT WebDAV PS T HTTPS CIFS GDS SD CIFS CMIS Cache Site A CMIS Cache Corporate CDN OS GDS
    • Cloud attached Data Space Site A GDS CIFS LAN CIFS JSON GDS FW HT LB PS T GDS Internet HT T LAN CIFS FW GDS CIFS JSON FW GDS PS Site B LB GDS
    • YOUR DATA. YOUR CONTROL. WWW: HTTP://WWW.GRAUDATA.COM/DATASPACE E-MAIL: THOMAS.UHL@GRAUDATA.COM CEL: +49 151 54354373 TWITTER: @graudataspace