Copyright©2017 NTT Corp. All Rights Reserved.
Lightning Talk:
OpenStack Swiftの最新機能と
Storlets
Kota Tsuyuzaki [IRC: kota_]
tsuyuzaki.kota@lab.ntt.co.jp
NTT Software Innovation Center
OpenStack Storlets Project Team Lead
OpenStack Swift Core Team
Copyright(c)2009-2017 NTT CORPORATION. All Rights Reserved.
2Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
OpenStack Swift and OpenStack Storlets
OpenStack Swift: Distributed Object Storage
OpenStack Storlets: Computable Object Storage
Swift Cluster
Storlets Environment User
Application
Stolrets is computable environment on Swift Object Storage
APP APP APP
3Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
本日のLightning Talk
4Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
本日のLightning Talk
受かった
落ちた
⇒ こちらを中心にLT
5Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Global Erasure Coding
6Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Erasure Coding
REPLICATION
DATA DATA DATA
ERASURE CODING (10:4)
3x
1.4x
DATA
Part 2
DATA
Part 3
DATA
Part 4
DATA
Part 5
PARITY
Part 1
PARITY
Part 2
DATA
Part 7
DATA
Part 8
DATA
Part 9
DATA
Part 10
PARITY
Part 3
PARITY
Part 4
DATA
Part 1
DATA
Part 6
7Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Problem with Data Centers
Network
DATA
Part 5
PARITY
Part 1
PARITY
Part 2
DATA
Part 9
DATA
Part 10
PARITY
Part 3
PARITY
Part 4
DATA
Part 2
DATA
Part 3
DATA
Part 4
DATA
Part 7
DATA
Part 8
DATA
Part 1
DATA
Part 6
8Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
1.EC Fragment Duplication
2.Composite Ring
Global Erasure Coding
9Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Global Erasure Coding
DATA
Part 2
DATA
Part 3
DATA
Part 4
DATA
Part 5
PARITY
Part 1
PARITY
Part 2
DATA
Part 7
DATA
Part 8
DATA
Part 9
DATA
Part 10
PARITY
Part 3
PARITY
Part 4
DATA
Part 1
DATA
Part 6
DATA
Part 2
DATA
Part 3
DATA
Part 4
DATA
Part 5
PARITY
Part 1
PARITY
Part 2
DATA
Part 7
DATA
Part 8
DATA
Part 9
DATA
Part 10
PARITY
Part 3
PARITY
Part 4
DATA
Part 1
DATA
Part 6
2.8x
10Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
• Multi-Worker Reconstructor
• Increase Part Power
Other Great Staffs on Swift Pike
Reconstructor
worker worker worker
11Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Storlets: Computing on Swift
https://www.openstack.org/videos/sydney-2017/using-storletsdocker-for-
large-scale-image-processing
12Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
Thank you
Repository:
https://github.com/openstack/swift
https://github.com/openstack/storlets
IRC:
#openstack-swift @ freenode
#openstack-storlets @ freenode
Wiki:
https://github.com/openstack/swift
https://github.com/openstack/storlets
Presenter:
tsuyuzaki.kota@lab.ntt.co.jp
IRC: kota_
13Copyright©2017 NTT Corp. All Rights Reserved.
NTT Confidential
• Global EC Cluster
• https://specs.openstack.org/openstack/swift-
specs/specs/in_progress/global_ec_cluster.html
• https://www.slideshare.net/tsuyuzaki/global-ec-
cluster-updates-openstack-mitaka-swift-design-summit
• Storlets
• https://www.slideshare.net/tsuyuzaki/using-
storletsdocker-for-large-scale-image-processing
参考資料など:

OpenStack Swiftの最新機能とStorlets

  • 1.
    Copyright©2017 NTT Corp.All Rights Reserved. Lightning Talk: OpenStack Swiftの最新機能と Storlets Kota Tsuyuzaki [IRC: kota_] tsuyuzaki.kota@lab.ntt.co.jp NTT Software Innovation Center OpenStack Storlets Project Team Lead OpenStack Swift Core Team Copyright(c)2009-2017 NTT CORPORATION. All Rights Reserved.
  • 2.
    2Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential OpenStack Swift and OpenStack Storlets OpenStack Swift: Distributed Object Storage OpenStack Storlets: Computable Object Storage Swift Cluster Storlets Environment User Application Stolrets is computable environment on Swift Object Storage APP APP APP
  • 3.
    3Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential 本日のLightning Talk
  • 4.
    4Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential 本日のLightning Talk 受かった 落ちた ⇒ こちらを中心にLT
  • 5.
    5Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Global Erasure Coding
  • 6.
    6Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Erasure Coding REPLICATION DATA DATA DATA ERASURE CODING (10:4) 3x 1.4x DATA Part 2 DATA Part 3 DATA Part 4 DATA Part 5 PARITY Part 1 PARITY Part 2 DATA Part 7 DATA Part 8 DATA Part 9 DATA Part 10 PARITY Part 3 PARITY Part 4 DATA Part 1 DATA Part 6
  • 7.
    7Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Problem with Data Centers Network DATA Part 5 PARITY Part 1 PARITY Part 2 DATA Part 9 DATA Part 10 PARITY Part 3 PARITY Part 4 DATA Part 2 DATA Part 3 DATA Part 4 DATA Part 7 DATA Part 8 DATA Part 1 DATA Part 6
  • 8.
    8Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential 1.EC Fragment Duplication 2.Composite Ring Global Erasure Coding
  • 9.
    9Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Global Erasure Coding DATA Part 2 DATA Part 3 DATA Part 4 DATA Part 5 PARITY Part 1 PARITY Part 2 DATA Part 7 DATA Part 8 DATA Part 9 DATA Part 10 PARITY Part 3 PARITY Part 4 DATA Part 1 DATA Part 6 DATA Part 2 DATA Part 3 DATA Part 4 DATA Part 5 PARITY Part 1 PARITY Part 2 DATA Part 7 DATA Part 8 DATA Part 9 DATA Part 10 PARITY Part 3 PARITY Part 4 DATA Part 1 DATA Part 6 2.8x
  • 10.
    10Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential • Multi-Worker Reconstructor • Increase Part Power Other Great Staffs on Swift Pike Reconstructor worker worker worker
  • 11.
    11Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Storlets: Computing on Swift https://www.openstack.org/videos/sydney-2017/using-storletsdocker-for- large-scale-image-processing
  • 12.
    12Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential Thank you Repository: https://github.com/openstack/swift https://github.com/openstack/storlets IRC: #openstack-swift @ freenode #openstack-storlets @ freenode Wiki: https://github.com/openstack/swift https://github.com/openstack/storlets Presenter: tsuyuzaki.kota@lab.ntt.co.jp IRC: kota_
  • 13.
    13Copyright©2017 NTT Corp.All Rights Reserved. NTT Confidential • Global EC Cluster • https://specs.openstack.org/openstack/swift- specs/specs/in_progress/global_ec_cluster.html • https://www.slideshare.net/tsuyuzaki/global-ec- cluster-updates-openstack-mitaka-swift-design-summit • Storlets • https://www.slideshare.net/tsuyuzaki/using- storletsdocker-for-large-scale-image-processing 参考資料など: