Glidewell Laboratories Moves Data to Amazon
Redshift with Attunity CloudBeam
October 20, 2015
Agenda
• Overview of Amazon Redshift by Mike
Ruiz, Solution Architect, AWS Emerging
Partner Team
• Overview of Attunity CloudBeam by
Rodan Zadeh, Director of Product
Management, Attunity
• Glidewell Customer Story by Mike Selberis,
CIO, Glidewell Dental Laboratories
Agenda
• Overview of Amazon Redshift by Mike
Ruiz, Solution Architect, AWS Emerging
Partner Team
Amazon Redshift
Fast, simple, petabyte-scale data warehousing for less than $1,000/TB/Year
Data warehousing done the AWS way
• Easy to provision
• Pay as you go, no up front costs
• Fast, cheap, easy to use
• SQL
Constant Innovation: 95+ new features since launch…
• Regions – N. Virginia, Oregon, Dublin, Tokyo, Singapore, Sydney, Frankfurt
• Certifications – PCI, SOC 1/2/3, HIPAA and FedRamp
• Security – Load/unload encrypted files, Resource-level IAM, Temporary credentials,
HSM/CloudHSM, Audit Logging
• Manageability – Snapshot sharing, backup/restore progress indicators, SNS Alerts, faster cluster
creation, cross-region backups, faster resize, WLM resource management
• Query – Regex, Cursors, MD5, SHA1, Time zone, workload queue timeout, approximate count
distinct, distributed tables, concurrency increased to 50 from 15, user defined functions
• Ingestion – S3 Manifest, LZOP/LZO, JSON built-ins, UTF-8 4byte, invalid character substitution,
CSV, auto datetime format detection, epoch, load from EMR/HDFS/SSH…
Full list: http://docs.aws.amazon.com/redshift/latest/dg/doc-history.html
Amazon Redshift architecture
• Leader Node
– SQL endpoint
– Stores metadata
– Coordinates query execution
• Compute Nodes
– Local, columnar storage
– Isolated from end-user
– Execute queries in parallel
– Load, backup, restore via Amazon S3
– Parallel load from Amazon
DynamoDB
– Load from EMR/HDFS/SSH *New*
• Single node version available
10 GigE
(HPC)
Ingestion
Backup
Restore
JDBC/ODBC
Amazon Redshift lets you start small and grow big
Single Node (2 TB)
Cluster 2-32 Nodes
(4 TB – 64 TB)
Cluster 2-100 Nodes (32 TB – 1.6 PB)
Dense Storage Node (dw1.xlarge)
2 TB, 16 GB RAM, 2 cores
Dense Compute Node (dw2.large)
0.16 TB, 16 GB RAM, 2 cores
8XL Dense Storage Node (dw1.8xlarge)
16 TB, 128 GB RAM, 16 cores, 10 GigE
8XL Dense Compute Node (dw2.8xlarge)
2.56 TB, 128 GB RAM, 16 cores, 10 GigE
Amazon Redshift integrates with multiple data sources
Dashboarding
Reporting & BI
Ad Hoc Analysis
Amazon Redshift
Amazon EMRAmazon DynamoDBAmazon RDS Amazon S3
On-Premise
database
Flat
files
On-Premise
Warehouse
Amazon Redshift works with your existing analysis tools
JDBC/ODBC
Amazon Redshift
Common customer use cases
• Reduce costs by
extending DW rather than
adding HW
• Respond faster to
business drivers by
avoiding procurement
loop.
• Improve performance by
an order of magnitude
• Make more data
available for analysis
• Access business data via
standard reporting tools
• Add analytic functionality
to applications
• Scale DW capacity as
demand grows
• Reduce HW and SW costs
by an order of magnitude
Extend On-Premise DW Companies with Big Data SaaS Companies
Redshift powers Clickstream Analytics for Amazon.com
• Performance
– Scan 2.25 trillion rows of data: 14 minutes
– Load 5 billion rows data: 10 minutes
– Backfill 150 billion rows of data: 9.75 hours
– Pig  Amazon Redshift: 2 days to 1 hr
• 10B row join with 700 M rows
– Oracle  Amazon Redshift: 90 hours to 8 hrs
• Reduced number of SQLs by a factor of 3
• Cost
– 1.6 PB cluster
– 100 node dw1.8xl (3-yr RI)
– $180/hr
• Complexity
– 20% time of one DBA
• Backup
• Restore
• Resizing
Resources
• Detail Pages
– http://aws.amazon.com/redshift
– https://aws.amazon.com/marketplace/redshift/
• Presentations & Webinars:
– http://www.youtube.com/watch?v=JxLpj_TnisM (SF Summit Presentation)
– http://www.youtube.com/watch?v=R1m-fwzXMow (Best Practices 1 of 2)
– http://www.youtube.com/watch?v=7ySzRTOyK6o (Best Practices 2 of 2)
Agenda
• Overview of Attunity CloudBeam by
Rodan Zadeh, Director of Product
Management, Attunity
Right Data. Right Place. Right Time.
DW
HD
To Analytic Platform To Many Locations
To One Location
Enable Analytics Share & Distribute
Scale Reporting Consolidate
Away from
OLTP
Attunity CloudBeam – Highlights
• Simplicity – Automated, “Click-to-Load” Solution
• Performance – Optimized data transfer
• Real-time Data – Incremental/Continuous data loads
• Free Trial – “Try before you Buy”
• One-Stop Shop – Integrated into the AWS Redshift
16
Attunity CloudBeam – Solutions for AWS
• Amazon Redshift:
– Load data for BI / Analytics
– One time and incrementally
• Amazon RDS / DBs on EC2:
– Migrate data from on-premise data centers
– Enable disaster recovery (on-premise to cloud)
• Amazon S3:
– Transfer data for archiving, storage, content availability
– Enable disaster recovery
Attunity CloudBeam for Amazon Redshift & S3
Simplicity: Automated,
Click-2-Load Solution
Performance: Optimized
data transfer
Real-time: Continuous
data loads
Free Trial: Try before you
Buy
1-Stop-Shop: Integrated
with AWS Marketplace
Pricing Options: Express (hourly), Premium (hourly) and BYOL
How Attunity CloudBeam Works
1. Extract data from source database
2. Apply filters and transformations
3. Generate data files for transfer
4. Make optimized transfer to Amazon S3
5. Orchestrate COPY and MERGE for Amazon
Redshift
6. Provide CDC (change data capture) for
incremental loading AWS Region
EC2 Machine
M3.Large
Attunity AMI
RedshiftS3
On-Premises
Source DB(s)
Agenda
• Glidewell Customer Story by Mike Selberis,
CIO, Glidewell Dental Laboratories
Glidewell Laboratories
• Founded in 1970 and based in California
• Largest dental lab in the United States with a market presence in Europe and
Latin America
• Provides high-quality dental lab products, services, and materials to dental
professionals
• Over 4K employees worldwide
• Over 3M custom units processed and manufactured annually
Technology
• Hybrid cloud environment with multiple on-premise datacenters
and AWS as public cloud provider
• Use Attunity CloudBeam to synchronize data between on-
premises databases and Amazon Redshift
• Provide end users with timely access to data on Amazon Redshift
to support business analytics and business intelligence
• Tableau Software and Dundas Dashboards are used for analytics
and data visualization
CAD/CAM Digital Manufacturing
MillScan Design
Glidewell Data Explosion
• Lab management system (ERP/CRM)
• Case tracking, logistics, technician performance
• Manufacturing data (CAD/CAM)
• Manufacturing Automation
• Robotics
• Cloud-based manufacturing system (CloudPoint)
• IoT (Future)
Introduction of Digital Manufacturing
Our Challenge
• Managing an explosion of data and disparate data
sources
• Bridging on-premise data with cloud-generated data
• Providing a robust data analytics platform for our
employees, customers, and engineers
• Development resource constraints
• Global expansion
• Speed of action
Our Solution
Dundas
Dashboards
Tableau
Analytics
Ad Hoc reporting
and analysis
Amazon Redshift
Transactional Data
(ERP/CRM)
CloudPoint
Customer DataScan, Design,
& Manufacturing Data
Scan, Design,
& Manufacturing Data
Legacy Manufacturing Data
Glidewell Laboratories Moves Data to Amazon
Redshift with Attunity CloudBeam
Click here to watch the
recorded webinar

How Glidewell Moves Data to Amazon Redshift

  • 1.
    Glidewell Laboratories MovesData to Amazon Redshift with Attunity CloudBeam October 20, 2015
  • 2.
    Agenda • Overview ofAmazon Redshift by Mike Ruiz, Solution Architect, AWS Emerging Partner Team • Overview of Attunity CloudBeam by Rodan Zadeh, Director of Product Management, Attunity • Glidewell Customer Story by Mike Selberis, CIO, Glidewell Dental Laboratories
  • 3.
    Agenda • Overview ofAmazon Redshift by Mike Ruiz, Solution Architect, AWS Emerging Partner Team
  • 4.
    Amazon Redshift Fast, simple,petabyte-scale data warehousing for less than $1,000/TB/Year
  • 5.
    Data warehousing donethe AWS way • Easy to provision • Pay as you go, no up front costs • Fast, cheap, easy to use • SQL
  • 6.
    Constant Innovation: 95+new features since launch… • Regions – N. Virginia, Oregon, Dublin, Tokyo, Singapore, Sydney, Frankfurt • Certifications – PCI, SOC 1/2/3, HIPAA and FedRamp • Security – Load/unload encrypted files, Resource-level IAM, Temporary credentials, HSM/CloudHSM, Audit Logging • Manageability – Snapshot sharing, backup/restore progress indicators, SNS Alerts, faster cluster creation, cross-region backups, faster resize, WLM resource management • Query – Regex, Cursors, MD5, SHA1, Time zone, workload queue timeout, approximate count distinct, distributed tables, concurrency increased to 50 from 15, user defined functions • Ingestion – S3 Manifest, LZOP/LZO, JSON built-ins, UTF-8 4byte, invalid character substitution, CSV, auto datetime format detection, epoch, load from EMR/HDFS/SSH… Full list: http://docs.aws.amazon.com/redshift/latest/dg/doc-history.html
  • 7.
    Amazon Redshift architecture •Leader Node – SQL endpoint – Stores metadata – Coordinates query execution • Compute Nodes – Local, columnar storage – Isolated from end-user – Execute queries in parallel – Load, backup, restore via Amazon S3 – Parallel load from Amazon DynamoDB – Load from EMR/HDFS/SSH *New* • Single node version available 10 GigE (HPC) Ingestion Backup Restore JDBC/ODBC
  • 8.
    Amazon Redshift letsyou start small and grow big Single Node (2 TB) Cluster 2-32 Nodes (4 TB – 64 TB) Cluster 2-100 Nodes (32 TB – 1.6 PB) Dense Storage Node (dw1.xlarge) 2 TB, 16 GB RAM, 2 cores Dense Compute Node (dw2.large) 0.16 TB, 16 GB RAM, 2 cores 8XL Dense Storage Node (dw1.8xlarge) 16 TB, 128 GB RAM, 16 cores, 10 GigE 8XL Dense Compute Node (dw2.8xlarge) 2.56 TB, 128 GB RAM, 16 cores, 10 GigE
  • 9.
    Amazon Redshift integrateswith multiple data sources Dashboarding Reporting & BI Ad Hoc Analysis Amazon Redshift Amazon EMRAmazon DynamoDBAmazon RDS Amazon S3 On-Premise database Flat files On-Premise Warehouse
  • 10.
    Amazon Redshift workswith your existing analysis tools JDBC/ODBC Amazon Redshift
  • 11.
    Common customer usecases • Reduce costs by extending DW rather than adding HW • Respond faster to business drivers by avoiding procurement loop. • Improve performance by an order of magnitude • Make more data available for analysis • Access business data via standard reporting tools • Add analytic functionality to applications • Scale DW capacity as demand grows • Reduce HW and SW costs by an order of magnitude Extend On-Premise DW Companies with Big Data SaaS Companies
  • 12.
    Redshift powers ClickstreamAnalytics for Amazon.com • Performance – Scan 2.25 trillion rows of data: 14 minutes – Load 5 billion rows data: 10 minutes – Backfill 150 billion rows of data: 9.75 hours – Pig  Amazon Redshift: 2 days to 1 hr • 10B row join with 700 M rows – Oracle  Amazon Redshift: 90 hours to 8 hrs • Reduced number of SQLs by a factor of 3 • Cost – 1.6 PB cluster – 100 node dw1.8xl (3-yr RI) – $180/hr • Complexity – 20% time of one DBA • Backup • Restore • Resizing
  • 13.
    Resources • Detail Pages –http://aws.amazon.com/redshift – https://aws.amazon.com/marketplace/redshift/ • Presentations & Webinars: – http://www.youtube.com/watch?v=JxLpj_TnisM (SF Summit Presentation) – http://www.youtube.com/watch?v=R1m-fwzXMow (Best Practices 1 of 2) – http://www.youtube.com/watch?v=7ySzRTOyK6o (Best Practices 2 of 2)
  • 14.
    Agenda • Overview ofAttunity CloudBeam by Rodan Zadeh, Director of Product Management, Attunity
  • 15.
    Right Data. RightPlace. Right Time. DW HD To Analytic Platform To Many Locations To One Location Enable Analytics Share & Distribute Scale Reporting Consolidate Away from OLTP
  • 16.
    Attunity CloudBeam –Highlights • Simplicity – Automated, “Click-to-Load” Solution • Performance – Optimized data transfer • Real-time Data – Incremental/Continuous data loads • Free Trial – “Try before you Buy” • One-Stop Shop – Integrated into the AWS Redshift 16
  • 17.
    Attunity CloudBeam –Solutions for AWS • Amazon Redshift: – Load data for BI / Analytics – One time and incrementally • Amazon RDS / DBs on EC2: – Migrate data from on-premise data centers – Enable disaster recovery (on-premise to cloud) • Amazon S3: – Transfer data for archiving, storage, content availability – Enable disaster recovery
  • 18.
    Attunity CloudBeam forAmazon Redshift & S3 Simplicity: Automated, Click-2-Load Solution Performance: Optimized data transfer Real-time: Continuous data loads Free Trial: Try before you Buy 1-Stop-Shop: Integrated with AWS Marketplace Pricing Options: Express (hourly), Premium (hourly) and BYOL
  • 19.
    How Attunity CloudBeamWorks 1. Extract data from source database 2. Apply filters and transformations 3. Generate data files for transfer 4. Make optimized transfer to Amazon S3 5. Orchestrate COPY and MERGE for Amazon Redshift 6. Provide CDC (change data capture) for incremental loading AWS Region EC2 Machine M3.Large Attunity AMI RedshiftS3 On-Premises Source DB(s)
  • 20.
    Agenda • Glidewell CustomerStory by Mike Selberis, CIO, Glidewell Dental Laboratories
  • 21.
    Glidewell Laboratories • Foundedin 1970 and based in California • Largest dental lab in the United States with a market presence in Europe and Latin America • Provides high-quality dental lab products, services, and materials to dental professionals • Over 4K employees worldwide • Over 3M custom units processed and manufactured annually
  • 22.
    Technology • Hybrid cloudenvironment with multiple on-premise datacenters and AWS as public cloud provider • Use Attunity CloudBeam to synchronize data between on- premises databases and Amazon Redshift • Provide end users with timely access to data on Amazon Redshift to support business analytics and business intelligence • Tableau Software and Dundas Dashboards are used for analytics and data visualization
  • 23.
  • 24.
    Glidewell Data Explosion •Lab management system (ERP/CRM) • Case tracking, logistics, technician performance • Manufacturing data (CAD/CAM) • Manufacturing Automation • Robotics • Cloud-based manufacturing system (CloudPoint) • IoT (Future) Introduction of Digital Manufacturing
  • 25.
    Our Challenge • Managingan explosion of data and disparate data sources • Bridging on-premise data with cloud-generated data • Providing a robust data analytics platform for our employees, customers, and engineers • Development resource constraints • Global expansion • Speed of action
  • 26.
    Our Solution Dundas Dashboards Tableau Analytics Ad Hocreporting and analysis Amazon Redshift Transactional Data (ERP/CRM) CloudPoint Customer DataScan, Design, & Manufacturing Data Scan, Design, & Manufacturing Data Legacy Manufacturing Data
  • 27.
    Glidewell Laboratories MovesData to Amazon Redshift with Attunity CloudBeam Click here to watch the recorded webinar