SlideShare a Scribd company logo
TDWI LAS VEGAS STRATEGY SUMMIT
tdwi.org/StrategySummitVegas/Presentations
Analytics at Cloud Scale
AVOID MIGRATION PAIN
Disclaimer/Goals
 I have no allegiance to any tech or tech company
 When it comes to our tech choices, good or bad, YMMV
 This was our journey, there are many *right* paths
 My goal is to highlight our wins and losses to share learnings
2
Our Business
Case
 Inability to scale quickly
 Restrictive innovation environment
 Difficulty managing budget
 Shrink the growing DevOps chasm
 Re-focus our tech ops talent pool
 Attraction/Retention of talented people
3
What We Wanted
 Elasticity – think pay-per-use
 Scalability – think quick upgrade/downgrade
 Eliminate Pets / Create Cattle
 Increase profitability and smooth spend
4
Phase 1: Lift and shift
GET OUT OF YOUR DATA CENTER
5
Our Tech Stack - Then
Prod, Stage, Test, Dev
 Compute: Racks of Dell R910’s 32 x 1TB x
16TB
 All Pets
 NLB’s, switches, SAN’s,
 Database: SQL Server
 Reporting: Microstrategy
 Frameworks: Angular, .NET (v4.6+)
6
Preparing for Lift
and Shift
 Secure your budget
 Choosing a cloud provider
 Design communications infrastructure
 Understanding the EC2/Series instance
types
 Server inventory/collapsing server
environment
 Identify Pets and Cattle and build out
server templates
7
Preparing for
Lift and Shift
 Zones/Regions
 Create backout plan
 Time it right: Contracts,
penalties, CapEx
 Phased the approach
over months
 AWS Well Architected
Framework and Azure
Architecture Center
8
Wins & Losses
9
Did Great!
 Server discovery
 Combining instances
 Tried many instance types
 Supportive executive team
 Timed it well
Not Awesome
 Hiring ahead
 Getting in front of training
Execution
10
 Test migration with most complex system
 Redeploy, don’t move
 Leave a couple of weeks between phases
 Continually look for cattle
 Consider ethical destruction of servers
Wins & Losses
11
Did Great!
 Set up BDP early -
 Reduced our footprint
 Dramatically destressed our System
Engineering team
 Excitement → engagement
Not Awesome
 Cloud engineering and Software
engineering weren’t working together
all the way through
 Didn’t embrace DevOps early
 Didn’t set up monitoring early
 Inexperience created tech debt
Phase 2: Rearchitect in
place
USING CLOUD PROVIDER CLOUD NATIVE SERVICES, TOOLS,
AND PLATFORMS
12
What did we
build/update/r
eplace?
 Introduce DevOps to teams – everything is
code
 Convert .NET to .NET Core
 Built data lake to burst load
 Replace ETL with S3, Step Functions,
Lambda, and ECS
 Replace SQL Server with Aurora, EMR,
DynamoDB
 Tune our infrastructure – Size & HoO
13
Wins & Losses
14
Did Great!
 moved away from using SQL Server/SSIS
for ETL; switched to using Step Functions,
Lambda, ECS, RedShift, and S3
 Re:Invent and MS Build for your technical
teams
 DevOps
Not Awesome
 Adopting new tech when it’s first
announced like RedShift or SQL DW – do
your due diligence
 Load balancer costs add up, use ALB’s not
ELB’s
 Didn’t port everything fully
 No approach to tagging
Monitoring
 SumoLogic
 LogicMonitor
 CloudWatch
 CloudHealth
 Sentry
15
Wins & Losses
16
Did Great!
 Defining metrics
 Getting our tooling in place
 Displaying it everywhere
 Using graphs to share stories
Not Awesome
 Not having our monitoring in place fast
enough
Phase 3: Rearchitect
again
LOOKING AT WHAT’S AVAILABLE IN THE MARKET
17
Cloud Scale
Tooling and
Processes
 Fix all of our tagging
 Set HOO for all instances
 Readjust instance types after Re:Invent
 HOOPA
 Snowflake
18
HOOPA
19
Snowflake Performance
20
Snowflake Financials
21
Wins & Losses
22
Did Great!
 Snowflake was evolutionary for us.
Faster execution times, lower costs, cost
directly correlated to usage
 put multiple endpoints on a single alb
Not Awesome
 Tools that didn’t work out the way we
hoped – RedShift, EMR, SWF
Our Tech Stack - Now
 Compute: EC2, ECS (Elastic Container Service) / docker,
Lambda
 Workflows: Step Functions, SWF*
 Database: Snowflake, SQL Server, Aurora PostgreSQL,
Aurora MySQL, Redshift*, DynamoDB, EMR (PySpark)*
 Eventing/Streams: SQS, SNS, Kinesis
 Reporting: MicroStrategy
 Frameworks: Angular, .NET (v4.6+, core, standard)
 Conf Mgmt/Orchestration: Chef, Ansible, CloudFormation
23
Contact
Information
Paul Birkbeck, VP, Software Engineering
pbirkbeck@spscommerce.com
linkedin.com/in/paulbirkbeck/
24

More Related Content

What's hot

Cloud Migration Strategy and Best Practices
Cloud Migration Strategy and Best PracticesCloud Migration Strategy and Best Practices
Cloud Migration Strategy and Best Practices
QBurst
 
SyncHPC: A Multi-Cloud HPC Hosting Platform
SyncHPC: A Multi-Cloud HPC Hosting PlatformSyncHPC: A Multi-Cloud HPC Hosting Platform
SyncHPC: A Multi-Cloud HPC Hosting Platform
Syncious
 
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Pooyan Jamshidi
 
Ten key steps for on prem to azure cloud migration
Ten key steps for on prem to azure cloud migrationTen key steps for on prem to azure cloud migration
Ten key steps for on prem to azure cloud migration
Nuvento Systems Pvt Ltd
 
OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
OSDC 2016 - Hybrid Cloud - A Cloud Migration StrategyOSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
Schlomo Schapiro
 
Hybrid Cloud : Database-as-a-Service: OOW 16
Hybrid Cloud : Database-as-a-Service: OOW 16 Hybrid Cloud : Database-as-a-Service: OOW 16
Hybrid Cloud : Database-as-a-Service: OOW 16
Bala Kuchibhotla
 
Identifying Workloads to Move to the Cloud
Identifying Workloads to Move to the CloudIdentifying Workloads to Move to the Cloud
Identifying Workloads to Move to the Cloud
RightScale
 
Cloud Migration: Moving Data and Infrastructure to the Cloud
Cloud Migration: Moving Data and Infrastructure to the CloudCloud Migration: Moving Data and Infrastructure to the Cloud
Cloud Migration: Moving Data and Infrastructure to the Cloud
Safe Software
 
CRM Trilogix; Migrating Legacy Systems to the Cloud
CRM Trilogix; Migrating Legacy Systems to the CloudCRM Trilogix; Migrating Legacy Systems to the Cloud
CRM Trilogix; Migrating Legacy Systems to the Cloud
Craig F.R Read
 
Cloud Migration: Tales from the Trenches
Cloud Migration: Tales from the TrenchesCloud Migration: Tales from the Trenches
Cloud Migration: Tales from the Trenches
Hostway|HOSTING
 
Cloud migration strategies
Cloud migration strategiesCloud migration strategies
Cloud migration strategies
SogetiLabs
 
From on premise to the hybrid cloud with microsoft azure
From on premise to the hybrid cloud with microsoft azureFrom on premise to the hybrid cloud with microsoft azure
From on premise to the hybrid cloud with microsoft azure
DotNetCampus
 
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
Amazon Web Services
 
soCloud: a multi-cloud paas
soCloud: a multi-cloud paassoCloud: a multi-cloud paas
soCloud: a multi-cloud paas
Fawaz Fernand PARAISO
 
Cloud migration services
Cloud migration services Cloud migration services
Cloud migration services
harrissmith5
 
Four ways to digitally transform with HPC in the cloud
Four ways to digitally transform with HPC in the cloudFour ways to digitally transform with HPC in the cloud
Four ways to digitally transform with HPC in the cloud
Tyrone Systems
 
AWS Partner Webcast - Data Center Migration to the AWS Cloud
AWS Partner Webcast - Data Center Migration to the AWS CloudAWS Partner Webcast - Data Center Migration to the AWS Cloud
AWS Partner Webcast - Data Center Migration to the AWS Cloud
Amazon Web Services
 
Cloud migration
Cloud migration Cloud migration
Cloud migration
Anirban Kundu
 
Modest scale HPC on Azure using CGYRO
Modest scale HPC on Azure using CGYROModest scale HPC on Azure using CGYRO
Modest scale HPC on Azure using CGYRO
Igor Sfiligoi
 
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid CloudMigrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Rackspace
 

What's hot (20)

Cloud Migration Strategy and Best Practices
Cloud Migration Strategy and Best PracticesCloud Migration Strategy and Best Practices
Cloud Migration Strategy and Best Practices
 
SyncHPC: A Multi-Cloud HPC Hosting Platform
SyncHPC: A Multi-Cloud HPC Hosting PlatformSyncHPC: A Multi-Cloud HPC Hosting Platform
SyncHPC: A Multi-Cloud HPC Hosting Platform
 
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
Microservices Architecture Enables DevOps: Migration to a Cloud-Native Archit...
 
Ten key steps for on prem to azure cloud migration
Ten key steps for on prem to azure cloud migrationTen key steps for on prem to azure cloud migration
Ten key steps for on prem to azure cloud migration
 
OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
OSDC 2016 - Hybrid Cloud - A Cloud Migration StrategyOSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy
 
Hybrid Cloud : Database-as-a-Service: OOW 16
Hybrid Cloud : Database-as-a-Service: OOW 16 Hybrid Cloud : Database-as-a-Service: OOW 16
Hybrid Cloud : Database-as-a-Service: OOW 16
 
Identifying Workloads to Move to the Cloud
Identifying Workloads to Move to the CloudIdentifying Workloads to Move to the Cloud
Identifying Workloads to Move to the Cloud
 
Cloud Migration: Moving Data and Infrastructure to the Cloud
Cloud Migration: Moving Data and Infrastructure to the CloudCloud Migration: Moving Data and Infrastructure to the Cloud
Cloud Migration: Moving Data and Infrastructure to the Cloud
 
CRM Trilogix; Migrating Legacy Systems to the Cloud
CRM Trilogix; Migrating Legacy Systems to the CloudCRM Trilogix; Migrating Legacy Systems to the Cloud
CRM Trilogix; Migrating Legacy Systems to the Cloud
 
Cloud Migration: Tales from the Trenches
Cloud Migration: Tales from the TrenchesCloud Migration: Tales from the Trenches
Cloud Migration: Tales from the Trenches
 
Cloud migration strategies
Cloud migration strategiesCloud migration strategies
Cloud migration strategies
 
From on premise to the hybrid cloud with microsoft azure
From on premise to the hybrid cloud with microsoft azureFrom on premise to the hybrid cloud with microsoft azure
From on premise to the hybrid cloud with microsoft azure
 
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
Best Practices for Data Center Migration Planning - August 2016 Monthly Webin...
 
soCloud: a multi-cloud paas
soCloud: a multi-cloud paassoCloud: a multi-cloud paas
soCloud: a multi-cloud paas
 
Cloud migration services
Cloud migration services Cloud migration services
Cloud migration services
 
Four ways to digitally transform with HPC in the cloud
Four ways to digitally transform with HPC in the cloudFour ways to digitally transform with HPC in the cloud
Four ways to digitally transform with HPC in the cloud
 
AWS Partner Webcast - Data Center Migration to the AWS Cloud
AWS Partner Webcast - Data Center Migration to the AWS CloudAWS Partner Webcast - Data Center Migration to the AWS Cloud
AWS Partner Webcast - Data Center Migration to the AWS Cloud
 
Cloud migration
Cloud migration Cloud migration
Cloud migration
 
Modest scale HPC on Azure using CGYRO
Modest scale HPC on Azure using CGYROModest scale HPC on Azure using CGYRO
Modest scale HPC on Azure using CGYRO
 
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid CloudMigrating Traditional Apps from On-Premises to the Hybrid Cloud
Migrating Traditional Apps from On-Premises to the Hybrid Cloud
 

Similar to Cloud Migration journey

Accelerate Application Migration - August 5, 2020
Accelerate Application Migration - August 5, 2020Accelerate Application Migration - August 5, 2020
Accelerate Application Migration - August 5, 2020
VMware Tanzu
 
A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
XebiaLabs
 
Accelerate DevOps Transformation with App Migration to the Cloud
Accelerate DevOps Transformation with App Migration to the CloudAccelerate DevOps Transformation with App Migration to the Cloud
Accelerate DevOps Transformation with App Migration to the Cloud
XebiaLabs
 
DevOps With Chef and Azure
DevOps With Chef and AzureDevOps With Chef and Azure
DevOps With Chef and Azure
Matt Stratton
 
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM TeamEVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
Evolve The Adobe Digital Marketing Community
 
Size as an Advantage: How Big Enterprises Can Move Fast
Size as an Advantage: How Big Enterprises Can Move FastSize as an Advantage: How Big Enterprises Can Move Fast
Size as an Advantage: How Big Enterprises Can Move Fast
Chetan Desai
 
Best Practices and Lessons Learned on Our IBM Rational Insight Deployment
Best Practices and Lessons Learned on Our IBM Rational Insight DeploymentBest Practices and Lessons Learned on Our IBM Rational Insight Deployment
Best Practices and Lessons Learned on Our IBM Rational Insight Deployment
Marc Nehme
 
DevOps Offerings at WhiteHedge
DevOps Offerings at WhiteHedgeDevOps Offerings at WhiteHedge
DevOps Offerings at WhiteHedge
WhiteHedge Technologies Inc.
 
Zero to 12 Million
Zero to 12 MillionZero to 12 Million
Zero to 12 Million
VMware Tanzu
 
OpenDevOps 2019 - Disconnected pipelines the missing link
OpenDevOps 2019 - Disconnected pipelines the missing linkOpenDevOps 2019 - Disconnected pipelines the missing link
OpenDevOps 2019 - Disconnected pipelines the missing link
Emerasoft, solutions to collaborate
 
Intro to Cloud Native _ v1.0en (2021/01)
Intro to Cloud Native _ v1.0en (2021/01)Intro to Cloud Native _ v1.0en (2021/01)
Intro to Cloud Native _ v1.0en (2021/01)
Young Suk Ahn Park
 
A real-life account of moving 100% to a public cloud
A real-life account of moving 100% to a public cloudA real-life account of moving 100% to a public cloud
A real-life account of moving 100% to a public cloud
Julien SIMON
 
Docker Containers in the Enterprise DevOps Journey
Docker Containers in the Enterprise DevOps JourneyDocker Containers in the Enterprise DevOps Journey
Docker Containers in the Enterprise DevOps Journey
TechWell
 
Deploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragilityDeploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragility
Spyros Lambrinidis
 
DevOps Case Studies
DevOps Case StudiesDevOps Case Studies
DevOps Case Studies
WhiteHedge Technologies Inc.
 
Net3 Technology: 5 step guide to DevOps in the Cloud
Net3 Technology: 5 step guide to DevOps in the CloudNet3 Technology: 5 step guide to DevOps in the Cloud
Net3 Technology: 5 step guide to DevOps in the Cloud
Kate Bissinger
 
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
apidays
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
MongoDB
 
Moving Applications to the cloud
Moving Applications to the cloudMoving Applications to the cloud
Moving Applications to the cloud
Aarno Aukia
 
04 accelerating businessvaluewithdevops
04 accelerating businessvaluewithdevops04 accelerating businessvaluewithdevops
04 accelerating businessvaluewithdevops
Khairul Zebua
 

Similar to Cloud Migration journey (20)

Accelerate Application Migration - August 5, 2020
Accelerate Application Migration - August 5, 2020Accelerate Application Migration - August 5, 2020
Accelerate Application Migration - August 5, 2020
 
A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
 
Accelerate DevOps Transformation with App Migration to the Cloud
Accelerate DevOps Transformation with App Migration to the CloudAccelerate DevOps Transformation with App Migration to the Cloud
Accelerate DevOps Transformation with App Migration to the Cloud
 
DevOps With Chef and Azure
DevOps With Chef and AzureDevOps With Chef and Azure
DevOps With Chef and Azure
 
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM TeamEVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
EVOLVE'14 | Keynote | Cat Reusswig | Building a Great AEM Team
 
Size as an Advantage: How Big Enterprises Can Move Fast
Size as an Advantage: How Big Enterprises Can Move FastSize as an Advantage: How Big Enterprises Can Move Fast
Size as an Advantage: How Big Enterprises Can Move Fast
 
Best Practices and Lessons Learned on Our IBM Rational Insight Deployment
Best Practices and Lessons Learned on Our IBM Rational Insight DeploymentBest Practices and Lessons Learned on Our IBM Rational Insight Deployment
Best Practices and Lessons Learned on Our IBM Rational Insight Deployment
 
DevOps Offerings at WhiteHedge
DevOps Offerings at WhiteHedgeDevOps Offerings at WhiteHedge
DevOps Offerings at WhiteHedge
 
Zero to 12 Million
Zero to 12 MillionZero to 12 Million
Zero to 12 Million
 
OpenDevOps 2019 - Disconnected pipelines the missing link
OpenDevOps 2019 - Disconnected pipelines the missing linkOpenDevOps 2019 - Disconnected pipelines the missing link
OpenDevOps 2019 - Disconnected pipelines the missing link
 
Intro to Cloud Native _ v1.0en (2021/01)
Intro to Cloud Native _ v1.0en (2021/01)Intro to Cloud Native _ v1.0en (2021/01)
Intro to Cloud Native _ v1.0en (2021/01)
 
A real-life account of moving 100% to a public cloud
A real-life account of moving 100% to a public cloudA real-life account of moving 100% to a public cloud
A real-life account of moving 100% to a public cloud
 
Docker Containers in the Enterprise DevOps Journey
Docker Containers in the Enterprise DevOps JourneyDocker Containers in the Enterprise DevOps Journey
Docker Containers in the Enterprise DevOps Journey
 
Deploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragilityDeploying more technology to shift from agility to anti-fragility
Deploying more technology to shift from agility to anti-fragility
 
DevOps Case Studies
DevOps Case StudiesDevOps Case Studies
DevOps Case Studies
 
Net3 Technology: 5 step guide to DevOps in the Cloud
Net3 Technology: 5 step guide to DevOps in the CloudNet3 Technology: 5 step guide to DevOps in the Cloud
Net3 Technology: 5 step guide to DevOps in the Cloud
 
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
APIdays Paris 2018 - Cloud computing - we went through every steps of the Gar...
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
 
Moving Applications to the cloud
Moving Applications to the cloudMoving Applications to the cloud
Moving Applications to the cloud
 
04 accelerating businessvaluewithdevops
04 accelerating businessvaluewithdevops04 accelerating businessvaluewithdevops
04 accelerating businessvaluewithdevops
 

Recently uploaded

Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Tatiana Kojar
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Jeffrey Haguewood
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 

Recently uploaded (20)

Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 

Cloud Migration journey

  • 1. TDWI LAS VEGAS STRATEGY SUMMIT tdwi.org/StrategySummitVegas/Presentations Analytics at Cloud Scale AVOID MIGRATION PAIN
  • 2. Disclaimer/Goals  I have no allegiance to any tech or tech company  When it comes to our tech choices, good or bad, YMMV  This was our journey, there are many *right* paths  My goal is to highlight our wins and losses to share learnings 2
  • 3. Our Business Case  Inability to scale quickly  Restrictive innovation environment  Difficulty managing budget  Shrink the growing DevOps chasm  Re-focus our tech ops talent pool  Attraction/Retention of talented people 3
  • 4. What We Wanted  Elasticity – think pay-per-use  Scalability – think quick upgrade/downgrade  Eliminate Pets / Create Cattle  Increase profitability and smooth spend 4
  • 5. Phase 1: Lift and shift GET OUT OF YOUR DATA CENTER 5
  • 6. Our Tech Stack - Then Prod, Stage, Test, Dev  Compute: Racks of Dell R910’s 32 x 1TB x 16TB  All Pets  NLB’s, switches, SAN’s,  Database: SQL Server  Reporting: Microstrategy  Frameworks: Angular, .NET (v4.6+) 6
  • 7. Preparing for Lift and Shift  Secure your budget  Choosing a cloud provider  Design communications infrastructure  Understanding the EC2/Series instance types  Server inventory/collapsing server environment  Identify Pets and Cattle and build out server templates 7
  • 8. Preparing for Lift and Shift  Zones/Regions  Create backout plan  Time it right: Contracts, penalties, CapEx  Phased the approach over months  AWS Well Architected Framework and Azure Architecture Center 8
  • 9. Wins & Losses 9 Did Great!  Server discovery  Combining instances  Tried many instance types  Supportive executive team  Timed it well Not Awesome  Hiring ahead  Getting in front of training
  • 10. Execution 10  Test migration with most complex system  Redeploy, don’t move  Leave a couple of weeks between phases  Continually look for cattle  Consider ethical destruction of servers
  • 11. Wins & Losses 11 Did Great!  Set up BDP early -  Reduced our footprint  Dramatically destressed our System Engineering team  Excitement → engagement Not Awesome  Cloud engineering and Software engineering weren’t working together all the way through  Didn’t embrace DevOps early  Didn’t set up monitoring early  Inexperience created tech debt
  • 12. Phase 2: Rearchitect in place USING CLOUD PROVIDER CLOUD NATIVE SERVICES, TOOLS, AND PLATFORMS 12
  • 13. What did we build/update/r eplace?  Introduce DevOps to teams – everything is code  Convert .NET to .NET Core  Built data lake to burst load  Replace ETL with S3, Step Functions, Lambda, and ECS  Replace SQL Server with Aurora, EMR, DynamoDB  Tune our infrastructure – Size & HoO 13
  • 14. Wins & Losses 14 Did Great!  moved away from using SQL Server/SSIS for ETL; switched to using Step Functions, Lambda, ECS, RedShift, and S3  Re:Invent and MS Build for your technical teams  DevOps Not Awesome  Adopting new tech when it’s first announced like RedShift or SQL DW – do your due diligence  Load balancer costs add up, use ALB’s not ELB’s  Didn’t port everything fully  No approach to tagging
  • 15. Monitoring  SumoLogic  LogicMonitor  CloudWatch  CloudHealth  Sentry 15
  • 16. Wins & Losses 16 Did Great!  Defining metrics  Getting our tooling in place  Displaying it everywhere  Using graphs to share stories Not Awesome  Not having our monitoring in place fast enough
  • 17. Phase 3: Rearchitect again LOOKING AT WHAT’S AVAILABLE IN THE MARKET 17
  • 18. Cloud Scale Tooling and Processes  Fix all of our tagging  Set HOO for all instances  Readjust instance types after Re:Invent  HOOPA  Snowflake 18
  • 22. Wins & Losses 22 Did Great!  Snowflake was evolutionary for us. Faster execution times, lower costs, cost directly correlated to usage  put multiple endpoints on a single alb Not Awesome  Tools that didn’t work out the way we hoped – RedShift, EMR, SWF
  • 23. Our Tech Stack - Now  Compute: EC2, ECS (Elastic Container Service) / docker, Lambda  Workflows: Step Functions, SWF*  Database: Snowflake, SQL Server, Aurora PostgreSQL, Aurora MySQL, Redshift*, DynamoDB, EMR (PySpark)*  Eventing/Streams: SQS, SNS, Kinesis  Reporting: MicroStrategy  Frameworks: Angular, .NET (v4.6+, core, standard)  Conf Mgmt/Orchestration: Chef, Ansible, CloudFormation 23
  • 24. Contact Information Paul Birkbeck, VP, Software Engineering pbirkbeck@spscommerce.com linkedin.com/in/paulbirkbeck/ 24