Scaling MongoDB on Amazon Web Services (DAT209) | AWS re:Invent 2013Amazon Web Services
Over the past year, mobile in-app feedback provider Apptentive has scaled MongoDB on AWS from a single machine to a sharded, thousands-of-operations-per-second, several hundred gigabyte cluster. This session—packed with demos, code, and actual performance numbers—shares the lessons learned along the way. Topics include picking the right tools for the job (instance sizing and selection, I/O choices, and topological choices); using chef/AWS OpsWorks and AWS CloudFormation to deploy and scale; monitoring with Amazon CloudWatch and MMS; managing backups with Amazon EBS snapshots; and using Amazon Elastic MapReduce alongside MongoDB instances.
Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!ScyllaDB
As a real time Big Data database, there are few things more important than keeping latencies low and bounded. Scylla has been delivering great tail latencies from our day one, but the job of making them better never ends and there is always more to do. In this talk we will explore some of the changes made to Scylla in the past few releases to help keep latencies down.
With AWS, you can choose the right storage service like including Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Block Storage (Amazon EBS) for the right use case. This session shows the range of AWS choices—from object storage to block storage—that are available to you. The sessions will also include specifics about real-world deployments from customers who are using Amazon S3, Amazon EBS, Amazon Glacier, and AWS Storage Gateway.
Reasons to attend:
Learn how to select which storage options to use, based your requirements for cost, access pattern and use case.
Understand why AWS is a perfect platform for the storage of digital assets, data, media and backups.
Discover how Glacier can revolutionize your long term archive management by removing the need for costly and fragile media types.
Hear about customer use cases and a rich partner ecosystem of services built on AWS storage services.
Scaling MongoDB on Amazon Web Services (DAT209) | AWS re:Invent 2013Amazon Web Services
Over the past year, mobile in-app feedback provider Apptentive has scaled MongoDB on AWS from a single machine to a sharded, thousands-of-operations-per-second, several hundred gigabyte cluster. This session—packed with demos, code, and actual performance numbers—shares the lessons learned along the way. Topics include picking the right tools for the job (instance sizing and selection, I/O choices, and topological choices); using chef/AWS OpsWorks and AWS CloudFormation to deploy and scale; monitoring with Amazon CloudWatch and MMS; managing backups with Amazon EBS snapshots; and using Amazon Elastic MapReduce alongside MongoDB instances.
Scylla Summit 2018: Keeping Your Latency SLAs No Matter What!ScyllaDB
As a real time Big Data database, there are few things more important than keeping latencies low and bounded. Scylla has been delivering great tail latencies from our day one, but the job of making them better never ends and there is always more to do. In this talk we will explore some of the changes made to Scylla in the past few releases to help keep latencies down.
With AWS, you can choose the right storage service like including Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Block Storage (Amazon EBS) for the right use case. This session shows the range of AWS choices—from object storage to block storage—that are available to you. The sessions will also include specifics about real-world deployments from customers who are using Amazon S3, Amazon EBS, Amazon Glacier, and AWS Storage Gateway.
Reasons to attend:
Learn how to select which storage options to use, based your requirements for cost, access pattern and use case.
Understand why AWS is a perfect platform for the storage of digital assets, data, media and backups.
Discover how Glacier can revolutionize your long term archive management by removing the need for costly and fragile media types.
Hear about customer use cases and a rich partner ecosystem of services built on AWS storage services.
This presentation covers how to use CloudFormation for deploying customized services on AWS. It goes through the background and advantages, as well as some commonly-used functions. Presented by Joseph Maxwell: lead developer at SwiftOtter Studios.
AWS re:Invent 2016: Case Study: Librato's Experience Running Cassandra Using ...Amazon Web Services
At Librato, a Solarwinds company, we run hundreds of Cassandra instances across multiple rings and use it as our primary data store. In the past year, we embarked on a process to upgrade our fleet of Cassandra Amazon EC2 instances from instance store to instances using Amazon EBS and attached elastic network interfaces (ENIs). We find running Cassandra on EBS gives us the flexibility to choose the best instances for the best performance of our workload while saving us significant costs on infrastructure. In this session, we discuss how Librato operates Cassandra on EBS. Topics include how we chose the right instance for our workload, use detached EBS volumes and ENI mobility to reduce MTTR, use mixed EBS storage types for the best cost/performance tradeoff, debug performance issues, and continuously monitor Cassandra to get the most from AWS. We also look at performance tradeoffs made in the implementation of storage engines of large data systems like Cassandra.
Laine Campbell, CEO of Blackbird, will explain the options for running MySQL at high volumes at Amazon Web Services, exploring options around database as a service, hosted instances/storages and all appropriate availability, performance and provisioning considerations using real-world examples from Call of Duty, Obama for America and many more. Laine will show how to build highly available, manageable and performant MySQL environments that scale in AWS—how to maintain then, grow them and deal with failure. Some of the specific topics covered are:
* Overview of RDS and EC2 – pros, cons and usage patterns/antipatterns.
* Implementation choices in both offerings: instance sizing, ephemeral SSDs, EBS, provisioned IOPS and advanced techniques (RAID, mixed storage environments, etc…)
* Leveraging regions and availability zones for availability, business continuity and disaster recovery.
* Scaling patterns including read/write splitting, read distribution, functional dataset partitioning and horizontal dataset partitioning (aka sharding)
* Common failure modes – AZ and Region failures, EBS corruption, EBS performance inconsistencies and more.
* Managing and mitigating cost with various instance and storage options
Cloud storage is one of the primary service offered by almost all the leading cloud service providers. This presentation looks into the options of Cloud storage in Azure, AWS and Google Cloud platform.
Colombo Cloud User Meetup
Kubernetes and AWS Lambda can play nicely togetherEdward Wilde
Vendor lock-in can be a worry for many engineers . A new innovative approach, will for the first time, allow open-source serverless to run on AWS Lambda or Kubernetes using the same deployment artefact, packaged using the tools we love: containers.
OpenFaaS is an open-source function as a service (FaaS) platform on the [CNCF serverless landscape](https://landscape.cncf.io/format=serverless).
With OpenFaaS you can package anything as a serverless function and deploy to Kubernetes using containers. Due to UNIX-like primitives in the core architecture, it was possible to extend the system to run functions on both Kubernetes and AWS Lambda depending on user preference. The core components of OpenFaaS still run on Kubernetes but the functions are deployed and invoked on AWS Lambda
AWS Webcast - How to Migrate On-premise NAS Storage to Cloud NAS StorageAmazon Web Services
In this webinar, Amazon Web Services Solutions Architect Kyle Lichtenberg and SoftNAS Solutions Architect Mark Bichlmeier will discuss moving SaaS applications from on-premise to the AWS cloud using NAS storage. This webinar will also feature an in-depth case study on Recommind. Ranked among the fastest growing companies on Deloitte’s 2014 Technology Fast 500(tm), Recommind was faced with driving greater scale, agility, and cost savings out of its hosting operations for its SaaS-based business. Should Recommind maximize operational efficiencies and costs for its brick and mortar data centers or go all-in and provide its SaaS applications to thousands of customers from the cloud? In this webinar, you will learn: • Alternatives considered in moving SaaS applications from on-premise to the cloud • How to migrate on-premise applications to the AWS cloud and use cloud NAS storage • How to build high-availability cloud NAS storage on AWS for multi-tenant environments • How to configure cloud NAS storage on AWS for IOPS requirements • How to configure iSCSI for use through AWS VPCs • How to archive to S3 cloud disks
This "how-to" session will cover the basics to get started with AWS. After a brief overview, this session will dive into discussions of core AWS services and provide demonstrations of how to set up and utilize those services. Demonstrations and discussions will include: - Setting up and connecting to your first Elastic Compute Cloud (EC2) virtual machine - How to backup and restore your virtual machine instance - How to set an email alert for changes in your virtual machine instance - How to upload files to Amazon's Simple Storage Service (S3) and make them publicly available on the Internet
Scylla Summit 2018: Meshify - A Case Study, or Petshop SeamonstersScyllaDB
Meshify is the IOT platform focused on wireless sensor technology for industrial/insurance IOT. This talk will provide an overview of how Meshify is using Scylla. It will also explain why, when everything else in Meshify’s platform is moving to a managed cloud service or a container based microservice, why and how the Scylla nodes are the only pet “seamonsters” in Meshify’s platform.
This presentation covers how to use CloudFormation for deploying customized services on AWS. It goes through the background and advantages, as well as some commonly-used functions. Presented by Joseph Maxwell: lead developer at SwiftOtter Studios.
AWS re:Invent 2016: Case Study: Librato's Experience Running Cassandra Using ...Amazon Web Services
At Librato, a Solarwinds company, we run hundreds of Cassandra instances across multiple rings and use it as our primary data store. In the past year, we embarked on a process to upgrade our fleet of Cassandra Amazon EC2 instances from instance store to instances using Amazon EBS and attached elastic network interfaces (ENIs). We find running Cassandra on EBS gives us the flexibility to choose the best instances for the best performance of our workload while saving us significant costs on infrastructure. In this session, we discuss how Librato operates Cassandra on EBS. Topics include how we chose the right instance for our workload, use detached EBS volumes and ENI mobility to reduce MTTR, use mixed EBS storage types for the best cost/performance tradeoff, debug performance issues, and continuously monitor Cassandra to get the most from AWS. We also look at performance tradeoffs made in the implementation of storage engines of large data systems like Cassandra.
Laine Campbell, CEO of Blackbird, will explain the options for running MySQL at high volumes at Amazon Web Services, exploring options around database as a service, hosted instances/storages and all appropriate availability, performance and provisioning considerations using real-world examples from Call of Duty, Obama for America and many more. Laine will show how to build highly available, manageable and performant MySQL environments that scale in AWS—how to maintain then, grow them and deal with failure. Some of the specific topics covered are:
* Overview of RDS and EC2 – pros, cons and usage patterns/antipatterns.
* Implementation choices in both offerings: instance sizing, ephemeral SSDs, EBS, provisioned IOPS and advanced techniques (RAID, mixed storage environments, etc…)
* Leveraging regions and availability zones for availability, business continuity and disaster recovery.
* Scaling patterns including read/write splitting, read distribution, functional dataset partitioning and horizontal dataset partitioning (aka sharding)
* Common failure modes – AZ and Region failures, EBS corruption, EBS performance inconsistencies and more.
* Managing and mitigating cost with various instance and storage options
Cloud storage is one of the primary service offered by almost all the leading cloud service providers. This presentation looks into the options of Cloud storage in Azure, AWS and Google Cloud platform.
Colombo Cloud User Meetup
Kubernetes and AWS Lambda can play nicely togetherEdward Wilde
Vendor lock-in can be a worry for many engineers . A new innovative approach, will for the first time, allow open-source serverless to run on AWS Lambda or Kubernetes using the same deployment artefact, packaged using the tools we love: containers.
OpenFaaS is an open-source function as a service (FaaS) platform on the [CNCF serverless landscape](https://landscape.cncf.io/format=serverless).
With OpenFaaS you can package anything as a serverless function and deploy to Kubernetes using containers. Due to UNIX-like primitives in the core architecture, it was possible to extend the system to run functions on both Kubernetes and AWS Lambda depending on user preference. The core components of OpenFaaS still run on Kubernetes but the functions are deployed and invoked on AWS Lambda
AWS Webcast - How to Migrate On-premise NAS Storage to Cloud NAS StorageAmazon Web Services
In this webinar, Amazon Web Services Solutions Architect Kyle Lichtenberg and SoftNAS Solutions Architect Mark Bichlmeier will discuss moving SaaS applications from on-premise to the AWS cloud using NAS storage. This webinar will also feature an in-depth case study on Recommind. Ranked among the fastest growing companies on Deloitte’s 2014 Technology Fast 500(tm), Recommind was faced with driving greater scale, agility, and cost savings out of its hosting operations for its SaaS-based business. Should Recommind maximize operational efficiencies and costs for its brick and mortar data centers or go all-in and provide its SaaS applications to thousands of customers from the cloud? In this webinar, you will learn: • Alternatives considered in moving SaaS applications from on-premise to the cloud • How to migrate on-premise applications to the AWS cloud and use cloud NAS storage • How to build high-availability cloud NAS storage on AWS for multi-tenant environments • How to configure cloud NAS storage on AWS for IOPS requirements • How to configure iSCSI for use through AWS VPCs • How to archive to S3 cloud disks
This "how-to" session will cover the basics to get started with AWS. After a brief overview, this session will dive into discussions of core AWS services and provide demonstrations of how to set up and utilize those services. Demonstrations and discussions will include: - Setting up and connecting to your first Elastic Compute Cloud (EC2) virtual machine - How to backup and restore your virtual machine instance - How to set an email alert for changes in your virtual machine instance - How to upload files to Amazon's Simple Storage Service (S3) and make them publicly available on the Internet
Scylla Summit 2018: Meshify - A Case Study, or Petshop SeamonstersScyllaDB
Meshify is the IOT platform focused on wireless sensor technology for industrial/insurance IOT. This talk will provide an overview of how Meshify is using Scylla. It will also explain why, when everything else in Meshify’s platform is moving to a managed cloud service or a container based microservice, why and how the Scylla nodes are the only pet “seamonsters” in Meshify’s platform.
AWS Webcast - Introduction to RDS Low Admin High Perf DBSAmazon Web Services
Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business. In this webinar we will discuss how to move your existing databases to RDS with minimum disruption. We will also cover how to deploy very high performance databases on the cloud. Finally, we will provide examples of how customers have successfully deployed high performance databases using RDS.
Design, Deploy, and Optimize SQL Server on AWS - AWS Online Tech TalksAmazon Web Services
Enterprises are quickly moving database workloads like SQL Server to the cloud, but with so many options, the best approach isn’t always obvious. You exercise full control of your SQL Server workloads by running them on Amazon EC2 instances, or leverage Amazon RDS for a fully managed database experience. This session will go deep on best practices and considerations for running SQL Server on AWS. We will cover best practices for deploying SQL Server, how to choose between Amazon EC2 and Amazon RDS, ways to optimize the performance of your SQL Server deployment for different applications types. We review in detail how to provision and monitor your SQL Server databases, and how to manage scalability, performance, availability, security, and backup and recovery, in both Amazon RDS and Amazon EC2.
Design, Deploy, and Optimize SQL Server on AWS - June 2017 AWS Online Tech TalksAmazon Web Services
Learning Objectives:
- Learn how to build applications on AWS from a strong foundation on SQL Server
- Learn when to deploy SQL Server on Amazon EC2 versus Amazon RDS
- Learn how to take advantage of the latest features in SQL Server 2016 when running on AWS
Enterprises are quickly moving database workloads like SQL Server to the cloud, but with so many options, the best approach isn’t always obvious. You exercise full control of your SQL Server workloads by running them on Amazon EC2 instances, or leverage Amazon RDS for a fully managed database experience. This session will go deep on best practices and considerations for running SQL Server on AWS. We will cover best practices for deploying SQL Server, how to choose between Amazon EC2 and Amazon RDS, ways to optimize the performance of your SQL Server deployment for different applications types. We review in detail how to provision and monitor your SQL Server databases, and how to manage scalability, performance, availability, security, and backup and recovery, in both Amazon RDS and Amazon EC2.
AWS Certified Solutions Architect Professional Course S10-S14Neal Davis
This deck contains the slides from our AWS Certified Solutions Architect Professional video course. It covers:
Section 10 AWS Database Services
Section 11 Serverless Applications
Section 12 Docker Containers and PaaS
Section 13 Deployment and Management
Section 14 Migration and Transfer Services
Full course can be found here: https://digitalcloud.training/courses/aws-certified-solutions-architect-professional-video-course/
Running Oracle EBS in the cloud (OAUG Collaborate 18 edition)Andrejs Prokopjevs
This presentation is based on a real-life experience migrating Oracle E-Business Suite R12.1 production to Amazon AWS, and additional proof-of-concept effort done getting various client systems upgraded to R12.2 and migrated to main cloud vendor platforms on the market. We are going to cover here various areas, like:
- Certification basics. Overview look into supported configurations.
- How to architect. Basic recommendations based on migration and 2+ year production runtime experience. We will mainly cover Amazon AWS use case.
- Advanced configurations outline.
- R12.2 and features / nuances coming with it.
- Microsoft Azure and Oracle Cloud review. Quick comparison outline of main alternative platforms.
- Cloud deployment automation and the most common scenario - auto-scaling.
This is a very client demanding topic and many are looking into cloud migration options and how they can optimize the cost comparing to the on-premise hardware hosting. And many are still misunderstanding the complexity of Oracle EBS stack being capable for cloud deployment.
This presentation is based on a real life experience migrating Oracle E-Business Suite production to AWS.
We will talk about:
- Certification basics. Overview on supported configurations.
- How to build. Recommendations based on migration and 2 year production runtime experience.
- Advanced configurations.
- R12.2.
- Microsoft Azure and Oracle Cloud review. Quick comparison outline of main alternative platforms. How ready is Oracle's own cloud service.
- Scaling.
This is a very client demanding topic. Many are looking into cloud migration options and how they can optimize the cost compared to the on-premise hosting, and many misunderstand the complexity of Oracle EBS stack being capable for cloud deployment.
Running Oracle EBS in the cloud (UKOUG APPS16 edition)Andrejs Prokopjevs
This presentation is based on a real life experience migrating Oracle E-Business Suite R12.1 production to AWS.
We will talk about:
- Certification basics.
- How to architect. Recommendations.
- Advanced configurations.
- R12.2.
- Microsoft Azure and Oracle Cloud review.
- Horizontal auto-scaling. Is this a supported configuration?
Day 2 - Amazon RDS - Letting AWS run your Low Admin, High Performance DatabaseAmazon Web Services
Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and re-sizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business. In this webinar we review the different types of Amazon RDS available and how to move your existing databases to Amazon RDS with minimum disruption.
Reasons to attend:
- Learn how Amazon RDS can reduce the overhead of running high performance mission critical databases.
- Learn how to migrate your existing database workloads into Amazon RDS running on the AWS Cloud.
- Learn how to scale up and scale down your Amazon RDS instance and save money with reserved instances.
Day 3 - AWS MySQL Relational Database Service Best Practices for Performance ...Amazon Web Services
Amazon RDS makes it easy to set up, operate, and scale, relational databases in the cloud. Amazon RDS for MySQL supports applications that require up to tens of thousands of IOPS, and allows you to scale on demand without administrative complexity. In this webinar, we will discuss best practices for getting the most out of Amazon RDS for MySQL, as well as techniques for migrating data to and from the service.
Reasons to attend:
- Learn the details of Master Slave dual AZ configuration.
- Learn about cross region replication.
- Learn about Provisioned IOPS and tips on getting the most from your Amazon RDS MySQL Service.
In addition to running databases in Amazon EC2, AWS customers can choose among a variety of managed database services. These services save effort, save time, and unlock new capabilities and economies. In this session, we make it easy to understand how they differ, what they have in common, and how to choose one or more. We explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service; Amazon RDS, a relational database service in the cloud; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution that can be surprisingly economical. We’ll cover how each service might help support your application, how much each service costs, and how to get started.
In addition to running databases in Amazon EC2, AWS customers can choose among a variety of managed database services. These services save effort, save time, and unlock new capabilities and economies. In this session, we make it easy to understand how they differ, what they have in common, and how to choose one or more. We explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service; Amazon RDS, a relational database service in the cloud; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution that can be surprisingly economical. We’ll cover how each service might help support your application, how much each service costs, and how to get started.
AWS Webcast - Introduction to Amazon RDS: Low Admin, High Performance Databas...Amazon Web Services
Amazon Relational Database Service (Amazon RDS) makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business.
In this webinar we review how to move your existing databases to RDS with minimum disruption. We will also cover how to deploy very high performance databases on the cloud. And finally, we will provide examples of how customers have successfully deployed high performance databases using RDS.
In addition to running databases in Amazon EC2, AWS customers can choose among a variety of managed database services. These services save effort, save time, and unlock new capabilities and economies. In this session, we make it easy to understand how they differ, what they have in common, and how to choose one or more. We explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service; Amazon RDS, a relational database service in the cloud; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution that can be surprisingly economical. We will cover how each service might help support your application, how much each service costs, and how to get started. We will also have with us Jeongsang Baek, the VP of Engineering from IGAWorks, Korea’s No.1 mobile business platform, who will walk us through their architecture and share with us the key insights that they gained from using the various AWS database technologies to deliver a reliable, efficient and cost-effective experience.
In addition to running databases in Amazon EC2, AWS customers can choose among a variety of managed database services. These services save effort, save time, and unlock new capabilities and economies. In this session, we make it easy to understand how they differ, what they have in common, and how to choose one or more. We explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service; Amazon RDS, a relational database service in the cloud; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution that can be surprisingly economical. We’ll cover how each service might help support your application, how much each service costs, and how to get started.
Similar to Data Scotland 2019: You can run SQL Server on AWS (20)
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
1. You can run SQL Server
on AWS?
John McCormack | DATA:Scotland | Sep 2019
2. Hello!
▪ John McCormack
▪ DBA since 2012
▪ MCSE Data Management and Analytics
▪ AWS Certified Cloud Solutions Architect
▪ https://johnmccormack.it
▪ @actualjohn
2
3. Terminology
▪ AWS – Amazon Web Services (Cloud computing company)
▪ Region – A geographic area comprising of 2 or more AZs
▪ Availability Zone (AZ) – A data centre within a region, providing
options for HA/DR
▪ EC2 – VM within AWS (Windows or Linux)
▪ EBS – Block storage in AWS
▪ S3 – Object storage based in AWS (think dropbox)
▪ Bucket – Folder within S3. Name is globally unique
▪ RDS – Relational Database Service.
e.g. SQL Server, MySQL, Postgressql, Oracle, MariaDB and
Aurora
3
4. ▪ RDS
□ Managed instance
□ Less admin, less control
▪ EC2
□ VM in the cloud
□ More admin, more control
2 ways to run SQL Server in AWS
4
Administration Effort
Control
RDS
Azure MI
Azure SQL
DB
EC2
Azure VM
5. RDS vs EC2
FEATURE EC2 RDS
Use SSMS ✔ ✔
Use database as source or
target for SSIS
✔ ✔
Full control over the instance ✔
Automated Backups ✔
Automated Patching ✔
Always On Availability
Groups
✔ (Kind of)
5
7. Shared responsibility model
AWS EC2 RDS
HA/DR YOU 👉 AWS
Database Backups YOU 👉 AWS
SQL Server Patching YOU 👉 AWS
Installing SQL Server YOU 👉 AWS
OS Patching YOU 👉 AWS
Network AWS AWS
Power AWS AWS
Physical Security AWS AWS
7
8. Lets talk licensing
8
RDS
• Express edition is the only license free edition
• License included in instance cost
- Standard, Enterprise, Web
EC2
• You can use developer edition for non prod workloads
• BYOL or License included
• No software assurance = Dedicated host
9. Backup / Restore - RDS
9
• Automated backups
– Set your own retention period
• Take ad-hoc snapshots
– Kept in your S3 bucket until deleted by user
• Snapshots can be shared between accounts (prod to dev)
– Accounts is similar to subscriptions in Azure
• Point in time restores (within 5 minutes)
• You can restore to a new RDS instance
10. Backup / Restore – EC2
10
S3 offers cross-region bucket
replication and versioning !
You pay for all versions so look at setting a ‘lifecycle
policy’
Back up to S3 to save money
Directly with 3rd party tools
Copy from EBS to S3
Backup directly to file system EBS offers automatic redundancy and encryption
Responsible for your own backups
and restores
Just like on-premises
11. Monitoring
CloudWatch
▪ Near real time info
▪ View multiple instances at once
▪ Trigger alerts
3rd party tools
▪ sp_whoisactive
▪ Sentry One
▪ Grafana
11
12. Deep dive into
RDS
Versions and Editions
Instance types
Storage types
HA/DR
Security
Encryption
Parameter groups
13. RDS – Versions & Editions
13
▪ Express
▪ Web
▪ Standard
▪ Enterprise
▪ 2012 – 2017
▪ SQL Server 2008 R2 was removed from options on 1st June 2019.
▪ Customers who didn’t act were automatically upgraded 🤦♂️
15. RDS – Storage
15
ALL EDITIONS
Magnetic *
Min: 200 GiB Enterprise/Standard editions, 20 GiB
Web/Express
Max: 1 TiB all editions
General Purpose
(SSD)
Min: 200 GiB Enterprise/Standard editions, 20 GiB
Web/Express
Max: 16 TiB all editions
Provisioned IOPS
(SSD)
Min: 200 GiB Enterprise/Standard editions, 100 GiB
Web/Express
Max: 16 TiB all editions
16. • No manual intervention needed to fix original primary in
case of failover
• Automatically chosen depending on version / edition
• No readable secondary with AGs
Automatic Multi-AZ
deployment
Manual failovers
Any fault with original
primary is fixed for you
Mirroring or
Always On Availability
Groups
• Just tell AWS you want to go Multi-AZ
RDS – Multi AZ
16
• Pick a time to test that suits your business
17. RDS – Security
17
▪ Network Security
□ VPC
□ Security Groups
□ Choose your port
▪ Database security
□ SQL Logins
▫ Master user & others
□ Active Directory
18. RDS – Parameter Groups
18
▪ Parameter groups allow you to change settings above and
beyond your level of access to the instance.
□ i.e Something requiring sysadmin or securityadmin role
▪ Examples
□ Some trace flags e.g. 1204, 1222
□ Cost threshold for parallelism
□ Optimize for ad hoc workloads
▪ Changes to parameter groups apply to all instances using the
group
19. RDS – CLI & PowerShell
19
▪ Cloud computing at scale requires automation
▪ AWS supports CLI and Powershell to save you from too many clunky
manual process
▪ Examples
□ aws rds restore-db-instance-from-db-snapshot
□ Restore-RDSDBInstanceFromDBSnapshot
We will cover scripting in detail during the demo – learn at least 1
27. Pre Migration Planning
TASK
Create full inventory
Identify features used (What version / edition do we need)
Do you have licenses to take to the cloud?
What are your high availability / disaster recovery requirements
Can you purge unwanted data
What are your team’s capabilities
27
30. Credits
Special thanks to all the people who made
and released these awesome resources for
free:
▪ Presentation template by SlidesCarnival
▪ Photographs by Unsplash
30