Cloud Deployment of Data Harmony Jeffrey Gordon, Lead Developer, Access Innovations, Inc. Jeffrey will describe the cloud deployment of the Data Harmony software.

1. DATA HARMONY
D H U G 2 0 2 1
CLOUD VERSION

2. 2
M e e t i n g t h e g r o w i n g d e m a n d s f o r
d a t a a n a l y s i s
2
• What is the purpose of Data Harmony Cloud?
• To help meet the growing needs of our customers to index and process
increasingly large amounts of data
• Allow our customers to easily deploy scalable, responsive, and self-managing
services utilizing Data Harmony
• How do we achieve this?
• Centralized project management
• GIT driven project storage and version control
• Docker containers
• For ease of deployment and project loading
• Kubernetes clusters
• Highly scalable and self-managing clusters capable of mostly
administrating itself
DATA HARMONY CLOUD

3. 3
S i m p l i f y i n g t h e p r o j e c t m a n a g e m e n t
w o r k f l o w
3
• Utilizing GIT style repositories for project storage and versioning
• BitBucket, GitHub, etc…
• What are the benefits of this approach?
• Allows for project development to be more agile
• Thesaurus features can be developed in sprints
• Development versions of the thesaurus can be deployed for
testing before deployment to production
• Thesauri can be versioned and branched for different portions
of the application
• Project management is greatly simplified at all levels
• Native integration with Jira, Trello, etc…
CENTRALIZED GIT PROJECT STORE

4. 4
I n s t a n t D a t a H a r m o n y D e p l o y m e n t s
4
• Data Harmony is can now scale alongside our client’s
infrastructure
• Implementing Platform as a Service alongside our Software as a
Service offerings
• The amount of data being processed is scaling at a dramatic pace
• Our cloud implementation allows us to scale alongside our
customers seamlessly, through Docker containers
• Allow for development around Data Harmony to be streamlined
and integrate with agile practices
• Resources are available as needed
• Create production and staging environments when updating
service versions or implementing thesaurus changes
DOCKER CONTAINERS

5. DOCKER
BASICS
Image
The basis of a Docker container. The content atrest.
Container
The image when it is ‘running.’ The standard unit for app service
Engine
The software that executes commands for containers. Networking and volumes are part of
Engine. Can be clustered together.
Registry
Stores, distributes and manages Dockerimages
Control Plane
Management plane for container and clusterorchestration

6. • Kubernetes is an open-source system for automating deployment,
scaling, and management of containerized applications.
• Improves reliability
- Continuously monitors and manages your containers
- Will scale your application to handle changes in load
• Better use of infrastructure resources
- Helps reduce infrastructure requirements by gracefully scaling up
and down your entire platform via autoscaling
• Coordinates what containers run where and when across your system
• How do all the different types of containers in a system talk to each
other?
• Easily coordinate deployments of your system
- Which containers/projects need to be deployed
- Where should the containers be deployed
o Mars, AWS, Azure, SWCP, …
WHAT DOES KUBERNETES DO?

7. THE POD IS THE CORE KUBERNETES
COMPONENT
• The Pod is the core component of Kubernetes
• Collection of 1 or more containers
• Each pod should focus on one container, however sidecar containers
can be added to enhance features of the core container
spec:
template:
spec:
containers:
- name: accessinnovations/dataharmonyrepo
APIKEY: 384c1bb9-b539-420e-a647-a368e86b47b7

8. PODS CAN HANDLE SCALING AND
DEPLOYMENTS
• Once Kubernetes understands what is in a pod, multiple
management features are available:
• System Performance
- Scale up/down the number of pods based on CPU load or
other criteria
• System Monitoring
- Probes to check the health of each pod
- Any unhealthy ones get killed and new pod is put into service
• Deployments
- Deploy new versions of the container
- Control traffic to the new pods to test the new version
o Blue/Green deployments
o Rolling deployments

9. 9
C o n t a i n e d H e r e i n
9
• Deployment Workflows
• Deploying, Upgrading, and maintaining
• Centralization of projects in BitBucket
• Porting Projects, Production/Development
• Agile platform development
• On-demand production environments
• Self-managing service with scaling logic provided
• Scalability of Data Harmony
• Cloud scalability
• AWS, Azure, Google, and On-Premise
HOW DOES THIS BENEFIT YOU

10. 10
Deploying, Upgrading, and maintaining
10
• Seamless service deployments and upgrades
• Server upgrades are facilitated through the Docker service
• Reduces or eliminates any need for downtimes
• Project migration is handled automatically through the container’s
logic
• All server/project relationships are managed by the API keys
• Utilizing Docker to automate the deployment reduces deployment
time
• Rollover time to new versions or deployments reduced to
seconds
• Maintaining the service is handled automatically through the
container logic
DEPLOYMENT WORKFLOWS

11. 11
Centralization of projects in BitBucket
11
• Projects are now maintained through centralized repositories
• BitBucket, GitHub, etc…
• Simplifies how projects are stored and tracked
• Data Harmony projects are now centralized in BitBucket
• Makes deploying projects through Docker extremely easy
• In fact, this process is automated through the deployment
process
• For new pipelines, workflows, or applications
• Single command line allows for immediate deployment
• Projects are now strictly versioned and branched
• Allows for app dependent version of thesauri
DEPLOYMENT WORKFLOWS

12. 12
Porting Projects, Production/Development
12
• Simplified and efficient deployment/updates of your service
• Improved development/production cycle for your projects
• Implementing BitBucket and version control into the workflow
allows for a streamlined experience when working with
development versions of your thesauri.
• Centralizing the projects in BitBucket alleviates this issue
• Development projects are clones of a project on different a
different branch
• When all changes are finished, the changes can
merged/pushed to production
• Production version of the project on BitBucket can be
seamlessly pushed the containers
HOW YOUR PROJECT IS DEPLOYED

13. 13
On-demand production environments
13
• Project deployment, upgrading, or migration is now simplified to
allow for swift platform changes
• DevOP’s, IT, and management have complete control over Data
Harmony services are deployed for resources that utilize it
• Allows for agile platform deployment
• Infrastructure planning minimized
• Staging environments for updates
• Development teams have access to deploy identical or development
versions of their Data Harmony stack
• Moving from staging to live is also streamlined once the system is
deemed ready for production
AGILE PLATFORM DEVELOPMENT

14. 14
Self monitoring and disaster recovery
14
• Utilizing built in monitoring tools in our container, the Kubernetes
cluster or container can self-govern and heal itself
• With little to no intervention from DevOp’s the Data Harmony service
is capable of monitoring itself and taking steps to prevent service
interruptions.
• The logic for governing these behaviors is provided
• In the case of disaster on any infrastructure
• On boot of the server, the docker service will restore all known
containers of Data Harmony currently running
• Downtime reduced from potentially a day to a matter of minutes,
mitigating the worst-case scenario
SELF-MANAGING SERVICE

15. 15
AWS, Azure, Google, and On-Premise
15
• Whether your use-case is to use on premise hardware, lease
hardware, or deploy in the cloud
• Data Harmony Cloud is platform agnostic and can be deployed
anywhere whether it’s AWS, Azure, Google Cloud, your own private
cloud, or a standard server.
• Combining the power of Docker and Kubernetes, the service can
be scaled to meet any indexing need
• Services can be configured to scale “infinitely” or finitely depending
on the use-case
• Kubernetes handles day-to-day management of the service
• Monitoring, roll-over, and automatic container deployment
depending on current system load
SCALABILITY OF DATA HARMONY

16. DEPLOYING A CONTAINER
DEMO

17. 17
17
• When will this be ready?
• Data Harmony Cloud is available now
• How can I try it out
• Ask us for a trial!
• How complex is this to work with?
• For most people, just a single command line!
CONCLUSION