OPENSTACK @
BLOOMBERG
Openstack East // August 24, 2016
Jacob Rosenberg // @jrosenberg // Head of Infrastructure Engineering
Copyright 2016 Bloomberg Finance L.P.
Licensed under the CC BY-ND 4.0 license. See https://creativecommons.org/licenses/by-nd/4.0/legalcode
SESSION PROMISES:
1
• Gratuitous use of cloud imagery
• You can use this, it’s all OSS
• Yes, we’re hiring!
• Ceph, Neutron, Containers & more!
• DM @jrosenberg for more info!
WHY DO WE CLOUD?
WHY A CLOUD?
• Freedom & agility for our developers
• Flexable operational models
• Efficient use of resources
WHY PRIVATE?
• Proximity to Bloomberg data
• Security and compliance obligations
• Ability to tailor capabilities to our
market
• Unique network design and
protocols
WHY OPENSTACK?
COMMUNITY AND CONTROL
• Openstack clearly had the most
established community
• “Distributions” were many and often
very opinionated
• We wanted to make our own
technology choices
BUILDING OUR OWN PATH
• Bloomberg Clustered Private Cloud (BCPC)
• First commit circa Folsom era (2013)
• Currently migrating Kilo (5.x) to Liberty (6.x)
• Open Source: https://github.com/bloomberg/chef-bcpc
HOW DID WE DO?
SIGNIFICANT ADOPTION
• 11 clusters across 3 data centers
• 3000+ instances, 8000+ vCPUs
• Monthly growth rate of ~8%
SIGNIFICANT CHALLENGES
• Bloomberg standard application
frameworks needed modification to
operate with Cloud
• Ephemerality of instances challenged for
management, monitoring, and deployment
tooling assumptions
• Infrastructure-as-a-Service is good, but
most engineering groups wanted more
WHAT’S NEXT?
DRIVING ADOPTION
• Finish updating process and tooling to
be Cloud-aware and friendly
• Focused training and evangelism for
production application use cases
• Pre-built images which minimize tool
chain and workflow changes
TECHNOLOGY GOODNESS
• Neutron (gulp) and Calico / Layer 3
• Container hosting
• Platform as a Service
• Fancy new hardware
TIME TO FLY…
THANKS!
14
PHOTO CREDITS
Slide 1: “New York City, by day” CC-BY-2.0 by Kenny Louie Link
Slide 3: “Sunrise” CC-BY-2.0 by Sean MacEntee Link
Slide 4: “Sunrise” CC-BY-2.0 by Bhaskara Rao S Link
Slide 6: “Maverick Moon” CC-BY-2.0 by Steve Jurvetson Link
Slide 7: “The Open Road” CC-BY-2.0 by Ron Cogswell Link
Slide 9: “Golden Sunset” CC-BY-2.0 by lady_lbrty Link
Slide 10: “crepuscular rays” CC-BY-2.0 by Mark Freeth Link
Slide 12: “Little Fluffy Cloud” CC-BY-2.0 by Steve Garry Link
Slide 13: “Fluffy” CC-BY-2.0 by rjp Link
Slide 14: “Airplane” CC-BY-2.0 by Sean MacEntee Link
15

OpenStack at Bloomberg

  • 1.
    OPENSTACK @ BLOOMBERG Openstack East// August 24, 2016 Jacob Rosenberg // @jrosenberg // Head of Infrastructure Engineering Copyright 2016 Bloomberg Finance L.P. Licensed under the CC BY-ND 4.0 license. See https://creativecommons.org/licenses/by-nd/4.0/legalcode
  • 2.
    SESSION PROMISES: 1 • Gratuitoususe of cloud imagery • You can use this, it’s all OSS • Yes, we’re hiring! • Ceph, Neutron, Containers & more! • DM @jrosenberg for more info!
  • 3.
    WHY DO WECLOUD?
  • 4.
    WHY A CLOUD? •Freedom & agility for our developers • Flexable operational models • Efficient use of resources
  • 5.
    WHY PRIVATE? • Proximityto Bloomberg data • Security and compliance obligations • Ability to tailor capabilities to our market • Unique network design and protocols
  • 6.
  • 7.
    COMMUNITY AND CONTROL •Openstack clearly had the most established community • “Distributions” were many and often very opinionated • We wanted to make our own technology choices
  • 8.
    BUILDING OUR OWNPATH • Bloomberg Clustered Private Cloud (BCPC) • First commit circa Folsom era (2013) • Currently migrating Kilo (5.x) to Liberty (6.x) • Open Source: https://github.com/bloomberg/chef-bcpc
  • 9.
  • 10.
    SIGNIFICANT ADOPTION • 11clusters across 3 data centers • 3000+ instances, 8000+ vCPUs • Monthly growth rate of ~8%
  • 11.
    SIGNIFICANT CHALLENGES • Bloombergstandard application frameworks needed modification to operate with Cloud • Ephemerality of instances challenged for management, monitoring, and deployment tooling assumptions • Infrastructure-as-a-Service is good, but most engineering groups wanted more
  • 12.
  • 13.
    DRIVING ADOPTION • Finishupdating process and tooling to be Cloud-aware and friendly • Focused training and evangelism for production application use cases • Pre-built images which minimize tool chain and workflow changes
  • 14.
    TECHNOLOGY GOODNESS • Neutron(gulp) and Calico / Layer 3 • Container hosting • Platform as a Service • Fancy new hardware
  • 15.
  • 16.
    PHOTO CREDITS Slide 1:“New York City, by day” CC-BY-2.0 by Kenny Louie Link Slide 3: “Sunrise” CC-BY-2.0 by Sean MacEntee Link Slide 4: “Sunrise” CC-BY-2.0 by Bhaskara Rao S Link Slide 6: “Maverick Moon” CC-BY-2.0 by Steve Jurvetson Link Slide 7: “The Open Road” CC-BY-2.0 by Ron Cogswell Link Slide 9: “Golden Sunset” CC-BY-2.0 by lady_lbrty Link Slide 10: “crepuscular rays” CC-BY-2.0 by Mark Freeth Link Slide 12: “Little Fluffy Cloud” CC-BY-2.0 by Steve Garry Link Slide 13: “Fluffy” CC-BY-2.0 by rjp Link Slide 14: “Airplane” CC-BY-2.0 by Sean MacEntee Link 15