Watson developer cloud delivers Watson Cognitive services as micro services on the cloud that are being used by many IBM Watson customers. The micro services were packaged in ova at the first release. There were some drawbacks in ova deployment in the cloud. We gradually switched to use docker. As a result, the service deployment time and start up time are significantly improved. It also greatly simplified our continuous delivery process since our services run on both Intel and Power platform and we have offerings on our public cloud, dedicated cloud as well as customers’ on premise cloud. With minimal deployment time and quick startup time, Docker makes our dynamic creation of service instance on the fly per customer request possible.

2. Use Docker to Deliver Cognitive
Services in Multi Cloud Environments
- The Watson Developer Cloud Use Case
Susan Diamond
Senior Software Engineer/Continuous Delivery Leader
6/20/2016

3. Agenda
• Watson/Watson Services/Watson Developer Cloud (WDC)
Introduction
• WDC Transition from VM to Docker
• WDC Continuous Delivery Process
• Things to Consider to Run Private Docker Registries
• Watson Service Demo

4. What do we do …

5. Watson Cognitive Services

6. © 2015 International Business Machines Corporation
Watson Developer Cloud Architecture Goals
• Deliver open platform to host WDC services and provide operational
excellence
➢ Cloud Native Enablement
➢ High Availability / Auto Recovery
➢ Microservices Architecture
➢ Continuous Delivery
➢ Elasticity
➢ Operational Visibility
➢ Security / Compliance
➢ Productivity Increase for Service Developers
▪ Quick onboarding, common components, tools, process
improvements
5

7. 6
DevOps
[Imaginator, Artifactory,
Asgard]
Data Services
[Zookeeper,Cassandra
WS3,Priam]
Remote Log Archival/
Analytics
[ELK,Cloudsight]
Dynamic Routing
[DataPower,Zuul]
Service Registry/
Discovery
[Eureka]
DevOps Config
Management
[Archaius]
Operational Visibility
[Grafana, Graphite]
Testing
[Jenkins,Chaos]
Metrics and Event
Sources
[Servo,Crawler]
Internal Service Load
Balancing
[Ribbon,Litelinks,Eureka]
Service Health Checks
[Eureka]
Service Discovery/
Registration
[Eureka, Karyon,]Microservice
Architecture
Failure Resilient
Doocker Repo
Container Management
[Mesos,Marathon]
Image Management Baremetal
VM
Availability Zones/Pods
NetworkingDNS
ObjectStorage
Provisioning
[CSB]
Authorization
Authentication
[LDAP,Apache Shiro]
Alerting
[Seyren,Uptime,
PagerDuty]
Metering
Watson
Services
Patterns
Watson
Foundation
Services
Softlayer
IaaS
Watson
Security
ID Management
[USAM]
Vulnerability Scanning
[AppScan,Nesus]
Security Logging
[QRadar]
Watson Developer Cloud Architecture
* Netflix Components

8. Watson Developer Cloud and Docker
• Started investigate Docker in mid 2014
• Lack of monitoring/management tooling
• Picked up Docker again in beginning of 2015 and use docker in production
environment since mid 2015
Some characteristics of Cognitive
services:
• Self-learning
• Training data/models
• Data security
• One service instance per customer
Dock Deployment
• Deployment time significantly improved
• Start up time significantly improved
• Enable dynamic service instance creation on the fly
• Enable seamless DevOps experience across multi-cloud environments in multi-GEOs
VM deployment Pain Points:
• Long deployment time
• Slow start up time
• Problematic in dynamic deployment
automation

9. Transitioning from VM to Docker
1. VM/docker image building,
bake the image once and
use for all environments.
2. Service deployment
3. Service registration
4. Service request routing
5. Full stack in private
network except DataPower

10. Canary & Red/Black Deployments/Updates
▪Design Philosophy: No destructive or in-place upgrades
▪New ASG deployed alongside existing ASG
▪#instances determine load (point in time)
▪Disable old ASG after sufficient confidence in new
– Can flip to old with a couple of clicks in case of an issue

11. Ubuntu VM
Docker
engine
Docker Registry
on Docker Container
Ubuntu VM
Docker
engine
Docker Registry
on Docker Container
Docker
container
1. Java 7
2. Tomcat 7
3. Side-car
4. Eureka
5. Zookeeper
6. Zuul
…
Imaginator
Container
1 base image
deb files
2. Services
debfiles
3. Dreamfiles
4. dockerfiles
…
SL object storage
Docker AAS
SL image
template
repository v3 VM Image template
v1 Deb files + dream file
d1 Deb files + dock file
d2 Store/retrieve deb files
v2 Store/retrieve
deb files
d3 Docker image
SL firewall
Jenkins
Git Enterprise
Ubuntu VM
Haproxy
source code
Imaginator server/Ubuntu VM
Docker
engine
Imaginator
VM AAS
• Design Philosophy: Every change should
be done via images
• Base images available to service teams
➢ Ubuntu 14.04 LTS
+ security hardening + agents (logging,
metrics)
+ JRE + Tomcat
+ Sidecar for non-Java services
• Service teams build their image on top of
base image
➢ “Dream” or Docker file describes
dependencies and build targets
➢ Gradle fpm plugin for Debian
package generation
• Output is Softlayer CCI image or Docker
image for deployment
Continuous Delivery - Image Baking Process

12. © 2015 International Business Machines Corporation
VM DevOps Console: Asgard
• From Images to Running Servers – Continuous Delivery
• Few button clicks to deploy one or more service
instances into Auto Scaling Group (ASG)
11
• Console allows you to:
➢ Provide parameters for HA and
instance recovery
➢ Deploy one or more service
versions at a time
➢ Canary Testing / Red-Black
Deployments
➢ Increase/Decrease number of
instances in ASG
➢ Configure auto scaling policies

13. Docker DevOps: Active Deploy (Investigating)

14. Service build Server
9.x
Docker deamon
SL object
storage
Docker images
Docker Registry
9.x.x.x:5000
SL object
storage
Docker images
Dev
Staging
Private Docker Registries in Public Cloud
Docker Registry
9.x.x.x:5001
haproxy
haproxy
Docker
deamon
Softlayer IBM intranet
Docker
deamon
push
push
pull
pull
dockerrepo-v2-01:5000 dockerrepo-v2-02:5000
dockerregistry1-v2:5000 dockerregistry2-v2:5000
SL object
storage
Docker images
Production
haproxy
Docker
deamon
pull
dockerregistry1-v2:5000 dockerregistry2-v2:5000

15. • Security
• Proprietary source code needs to be stored in a private registry
• Customer’s security requirement: fully isolated stack
• Network accessible to all docker agent in the environment.
• docker agents are in private network that is not accessible to
public network
• Docker registries need to available in each GEO for good
performance.
• Maintenance
• Maintaining the multiple private docker registries up to date
and operational needs resources.
Things to Consider to Run Private Docker Registries

16. Watson Services Demo

17. Reference Links
http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/starter-kits.html
http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/
https://www.ibmchefwatson.com/community