Session slide at db tech showcase 2012
How Rakuten Reduced Database Management Spending by 90% through Clustrix implementation
- About Rakuten
- Rakuten database environment and operational issues
- What is Clustrix?
- Clustrix verification results and implementation effectiveness
- Summary
Industry Digitalisation with 5G : Smart ManufacturingMahmoud Dasser
20-21 April 2019: 5G Malaysia Showcase
Accelerate Malaysia's transformation into a. Smart and Modern Manufacturing System
As the Fourth Industrial Revolution sweeps over the global front, the Malaysian Communications and Multimedia Commission (MCMC) is entasked with the challenges to bring the nation one step ahead in the digital era. Understanding that this new industrial revolution brings on bigger and more intricate challenges, MCMC has positioned itself as the key driver to prepare Malaysia with the platform for borderless information, brought forward by the next big trends namely 5G, Internet of Things (IoT), Smart Cities, Big Data, Artificial Intelligence (AI), Blockchain, etc.
https://www.mcmc.gov.my/digital-outlook-series-2019/digital-outlook-series-2019-shaping-the-digital
Industry Digitalisation with 5G : Smart ManufacturingMahmoud Dasser
20-21 April 2019: 5G Malaysia Showcase
Accelerate Malaysia's transformation into a. Smart and Modern Manufacturing System
As the Fourth Industrial Revolution sweeps over the global front, the Malaysian Communications and Multimedia Commission (MCMC) is entasked with the challenges to bring the nation one step ahead in the digital era. Understanding that this new industrial revolution brings on bigger and more intricate challenges, MCMC has positioned itself as the key driver to prepare Malaysia with the platform for borderless information, brought forward by the next big trends namely 5G, Internet of Things (IoT), Smart Cities, Big Data, Artificial Intelligence (AI), Blockchain, etc.
https://www.mcmc.gov.my/digital-outlook-series-2019/digital-outlook-series-2019-shaping-the-digital
Bonduelle - Rapport d'activité et de Développement Durable 2008/2009Bonduelle
Vous trouverez dans ce rapport toutes les informations sur l'activité et l'engagement pour le développement durable du Groupe Bonduelle.
Pour connaître tous les chiffres clés et rapport du groupe : http://www.bonduelle.com/fr/finance/rapports-annuels.html
The project contains the analysis of Nokia's Supply Chain Management published by Kellogg Institute of Management. The project includes solutions as well as recommendations for Philips, Ericsson and Nokia/
Venture capital firm LETA Capital issues a report called “State of Phygital 2022”. It is our annual report highlighting the concept of “Phygital” and introduce a fresh look at the very fundamentals of “Phygital”, which is a set of technologies that combine the physical (offline) world with digital (online).
We share our vision and insights on the rising merge of a few technology directions, such as XR, IoT, Edge Computing etc., into a new paradigm called Phygital. In contrast to the digital-only Metaverse future, Phygital is trying to enhance objects from the real world with digital content and capabilities. It is a trend that has been gaining momentum in recent years and will continue to grow in the coming years.
We hope you will pay particular attention to page #11 of our report because we tried to prepare an ultimate market map of startups and vendors, involved in the creation of the Phygital world. Please, feel free to reach out if you know more solution providers, we will add them to the list next year.
In the previous report, we decided to focus on macroeconomic factors of the Phygital economy's emergence, challenges, and projections. Most of them are still relevant today. We believe that many upcoming events, such as Meta's Next VR Headset release this October (Meta Cambria, aka Quest Pro), will accelerate the penetration into everyday life.
Sergey Toporov, Partner at LETA Capital: "This year, in State of the Phygital 2022, we decided to elaborate more on existing use-cases of the Phygital approach. We want to help businesses and people from various industries, from retail to construction and logistics, see what has been done already and let them generate more new cases and brilliant ideas."
The long and difficult 13- year journey to the marketplace for Pfizer’s viagr...P&CO
Explain why it was so necessary to ensure marketing was involved in the early stages of this new product development project
Explain how, despite the enormous resources of Pfizer, a lack of available information made the evaluation of the new product proposal so very difficult.
Explain how the Viagra case needs to be viewed as a successful example of excellent applied science but also an excellent example of good marketing.
How can Pfizer manage the threat posed to Viagra by new entrants to the market?
How has Pfizer helped create a market for Viagra and thereby contributed to disease mongering?
Industry 4.0 (IIoT) is changing the face of manufacturing like never before. Global situation of tariff war and re-shoring are catalysts driving this change in addition to technology adaption.
Check out this Comparative Analysis between Industry 4.0 (IIoT) and SCADA systems to understand the basic differences between the two.
Bonduelle - Rapport d'activité et de Développement Durable 2008/2009Bonduelle
Vous trouverez dans ce rapport toutes les informations sur l'activité et l'engagement pour le développement durable du Groupe Bonduelle.
Pour connaître tous les chiffres clés et rapport du groupe : http://www.bonduelle.com/fr/finance/rapports-annuels.html
The project contains the analysis of Nokia's Supply Chain Management published by Kellogg Institute of Management. The project includes solutions as well as recommendations for Philips, Ericsson and Nokia/
Venture capital firm LETA Capital issues a report called “State of Phygital 2022”. It is our annual report highlighting the concept of “Phygital” and introduce a fresh look at the very fundamentals of “Phygital”, which is a set of technologies that combine the physical (offline) world with digital (online).
We share our vision and insights on the rising merge of a few technology directions, such as XR, IoT, Edge Computing etc., into a new paradigm called Phygital. In contrast to the digital-only Metaverse future, Phygital is trying to enhance objects from the real world with digital content and capabilities. It is a trend that has been gaining momentum in recent years and will continue to grow in the coming years.
We hope you will pay particular attention to page #11 of our report because we tried to prepare an ultimate market map of startups and vendors, involved in the creation of the Phygital world. Please, feel free to reach out if you know more solution providers, we will add them to the list next year.
In the previous report, we decided to focus on macroeconomic factors of the Phygital economy's emergence, challenges, and projections. Most of them are still relevant today. We believe that many upcoming events, such as Meta's Next VR Headset release this October (Meta Cambria, aka Quest Pro), will accelerate the penetration into everyday life.
Sergey Toporov, Partner at LETA Capital: "This year, in State of the Phygital 2022, we decided to elaborate more on existing use-cases of the Phygital approach. We want to help businesses and people from various industries, from retail to construction and logistics, see what has been done already and let them generate more new cases and brilliant ideas."
The long and difficult 13- year journey to the marketplace for Pfizer’s viagr...P&CO
Explain why it was so necessary to ensure marketing was involved in the early stages of this new product development project
Explain how, despite the enormous resources of Pfizer, a lack of available information made the evaluation of the new product proposal so very difficult.
Explain how the Viagra case needs to be viewed as a successful example of excellent applied science but also an excellent example of good marketing.
How can Pfizer manage the threat posed to Viagra by new entrants to the market?
How has Pfizer helped create a market for Viagra and thereby contributed to disease mongering?
Industry 4.0 (IIoT) is changing the face of manufacturing like never before. Global situation of tariff war and re-shoring are catalysts driving this change in addition to technology adaption.
Check out this Comparative Analysis between Industry 4.0 (IIoT) and SCADA systems to understand the basic differences between the two.
Rakuten’s Journey with Splunk - Evolution of Splunk as a ServiceRakuten Group, Inc.
This is a presentation material of splunklive 2016 Tokyo.
(Japanese) Splunk Live 2016 での発表資料です。
楽天社内で展開しているSplunkの共通基盤である、Splunk as a Serviceのご紹介をします。
ユーザーの活用事例とともに、これまでのSplunkサービスの歩みを振り返り、今後の展望についてもお話します。
また、OSS化され、さらにパワーアップしたウェブツールのご紹介をしつつ、サービスのユーザー拡大と運用のコツをお伝えします。
Jubatus: Realtime deep analytics for BIgData@Rakuten Technology Conference 2012Preferred Networks
Currently, we face new challenges in realtime analytics of BigData, such as social monitoring, M2M sensor, online advertising optimization, smart energy management and security monitoring. To analyze these data, scalable machine learning technologies are essential. Jubatus is the open source platform for online distributed machine learning on the data streams of BigData. we explain the inside technologies of Jubatus and show how jubatus can achieve realtime analytics in various problems.
NewSQL overview:
- History of RDBMs
- The reasons why NoSQL concept appeared
- Why NoSQL was not enough, the necessity of NewSQL
- Characteristics of NewSQL
- 7 DBs that belongs to NewSQL
- Overview Table with main properties
Our (Olle from King and myself) session at OOW2014 (MySQL Central).
You will learn about the setup at King and also have a brief introduction to scaling MySQL.
Introducing the ultimate MariaDB cloud, SkySQLMariaDB plc
SkySQL is the first and only database-as-a-service (DBaaS) engineered for MariaDB by MariaDB, to use a state-of-the-art multi-cloud architecture built on Kubernetes and ServiceNow, and to deploy databases and data warehouses for transactional, analytical and hybrid transactional/analytical workloads.
In this session, we’ll lay out the vision for SkySQL, provide an overview of its capabilities, take a tour of its architecture, and discuss the long-term roadmap. We’ll wrap things up with a live demo of SkySQL, including a preview of its deep learning–based workload analysis and visualization interface.
In 2018's user conference keynote MariaDB CEO, Michael Howard, announced an initiative to build a MariaDB DBaaS platform. In this session, the DBaaS team shares how MariaDB is approaching DBaaS, then discusses the role of containers and Kubernetes, the need for infrastructure-agnostic provisioning, support for day-two operations and enterprise requirements for large-scale DBaaS deployments.
The bulk of the NoSQL Technologies focus on achieving scale-out ability by building their architecture around a simple, distributed hash, key-value store. This works well for partitioning simple data, but in reality, your information models are not simple. As a result, you may have to build enormous layers of code to manage an explicit structure baked into the persistence tier. In this session, take a look at a NoSQL solution which allows you to store naturally clustered, richly linked object networks beneath your key partitioned roots. The result is that you do not have to write extensive code to deal with the physical structure in the persistence tier even when dealing with complex information models like predictive models, timeseries, recursive relations, compositions, etc. We will explore how such an implementation works in practice by looking at a case study of an advanced model analytics and visualization solution built on the clustered NoSQL database solution Versant Database Engine.
DataStax C*ollege Credit: What and Why NoSQL?DataStax
In the first of our bi-weekly C*ollege Credit series Aaron Morton, DataStax MVP for Apache Cassandra and Apache Cassandra committer and Robin Schumacher, VP of product management at DataStax, will take a look back at the history of NoSQL databases and provide a foundation of knowledge for people looking to get started with NoSQL, or just wanting to learn more about this growing trend. You will learn how to know that NoSQL is right for your application, and how to pick a NoSQL database. This webinar is C* 101 level.
Vote NO for MySQL - Election 2012: NoSQL. Researchers predict a dark future for MySQL. Significant market loss to come. Are things that bad, is MySQL falling behind? A look at NoSQL, an attempt to identify different kinds of NoSQL stores, their goals and how they compare to MySQL 5.6. Focus: Key Value Stores and Document Stores. MySQL versus NoSQL means looking behind the scenes, taking a step back and looking at the building blocks.
(ARC309) Getting to Microservices: Cloud Architecture PatternsAmazon Web Services
Gilt, a billion dollar e-commerce company, implemented a sophisticated microservices architecture on AWS to handle millions of customers visiting their site at noon every day. The microservices architecture pattern enables independent service scaling, faster deployments, better fault isolation, and graceful degradation. In this session, Derek Chiles, AWS solutions architect, will review best practices and recommended architectures for deploying microservices on AWS. Adrian Trenaman, SVP of engineering at Gilt, will share Gilt's experiences and lessons learned during their evolution from a single monolithic Rails application in a traditional data center to more than 300 Scala/Java microservices deployed in the cloud.
SkySQL implements a groundbreaking, state-of-the-art architecture based on Kubernetes and ServiceNow, and with a strong emphasis on cloud security – using compartmentalization and indirect access to secure and protect customer databases.
In this session, we’ll walk through the architecture of SkySQL and discuss how MariaDB leverages an advanced Kubernetes operator and powerful ServiceNow configuration/workflow management to deploy and manage databases on cloud infrastructure.
AWS Certified Cloud Practitioner Course S11-S17Neal Davis
This deck contains the slides from our AWS Certified Cloud Practitioner video course. It covers:
Section 11 Databases and Analytics
Section 12 Management and Governance
Section 13 AWS Cloud Security and Identity
Section 14 Architecting for the Cloud
Section 15 Accounts, Billing and Support
Section 16 Migration, Machine Learning and More
Section 17 Exam Preparation and Tips
Full course can be found here: https://digitalcloud.training/courses/aws-certified-cloud-practitioner-video-course/
Commit Conf 2018 - Hotelbeds' journey to a microservice cloud-based architectureJordi Puigsegur Figueras
Hotelbeds is one of the leading bedbanks in the world, a global distributor of accommodation based in Palma de Mallorca, with an offering of more than 170.000 hotels and up to 2.500 Million request per day.
In the last two years Hotelbeds Group has undergone a profound technical transformation from an old monolythic on premise architecture to a microservices cloud-native architecture. Nowadays a big chunk of our monolyth has been transformed into java microservices deployed on cloud and we are already processing all our bookings on cloud. We have also moved to the cloud our availability engine, distributed to several locations accross the globe to be closer to the clients and reduce latencies.
This talk is about the path we followed on this fascinating journey, the decisions taken and lessons learned, as well as some insight on our future challenges.
Video available at https://www.youtube.com/watch?v=5H0X65QVF34
* All images used in this presentation are for illustration purposes only. They have been created specifically or obtained from publicly accessible sources. If any image is incorrectly used please contact me and it will be immediately removed and not used in the future.
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #5: Oracle’s InnoDB ClusterContinuent
Oracle’s InnoDB Cluster vs. Continuent Tungsten Clusters for MySQL
Building a Geo-Distributed, Multi-Region and Highly Available MySQL Cloud Back-End
This is the fifth of our High Noon series covering MySQL clustering solutions for high availability (HA), disaster recovery (DR), and geographic distribution.
InnoDB Cluster uses MySQL’s group replication to handle the replication. It’s also known as semi-synchronous replication. Learn about this and more in this webinar!
You may use Tungsten Clustering with native MySQL, MariaDB or Percona Server for MySQL in GCP, AWS, Azure, and/or on-premises data centers for better technological capabilities, control, and flexibility. But learn about the pros and cons!
AGENDA
- Goals for the High Noon Webinar Series
- High Noon Series: Tungsten Clustering vs Others
- Oracle InnoDB Cluster
- Key Characteristics
- Certification-based Replication
- InnoDB Cluster Multi-Site Requirements
- Limitations Using InnoDB Cluster
- How to do better MySQL HA / DR / Geo-Distribution?
- InnoDB Cluster vs Tungsten Clustering
- About Continuent & Its Solutions
PRESENTER
Matthew Lang - Customer Success Director – Americas, Continuent - has over 25 years of experience in database administration, database programming, and system architecture, including the creation of a database replication product that is still in use today. He has designed highly available, scaleable systems that have allowed startups to quickly become enterprise organizations, utilizing a variety of technologies including open source projects, virtualization and cloud.
Benchmark Showdown: Which Relational Database is the Fastest on AWS?Clustrix
Do you have a high-value, high throughput application running on AWS? Are you moving part or all of your infrastructure to AWS? Do you have a high-transaction workload that is only expected to grow as your company grows? Choosing the right database for your move to AWS can make you a hero or a goat. Be a hero!
Databases are the mission-critical lifeline of most businesses. For years MySQL has been the easy choice -- but the popularity of the cloud and new products like Aurora, RDS MySQL and ClustrixDB have given customers choices and options that can help them work smarter and more efficiently.
Enterprise Strategy Group (ESG) presents their findings from a recent performance benchmark test configured for high-transaction, low-latency workloads running on AWS.
In this webinar, you will learn:
How high-transaction, high-value database workloads perform when run on three popular databases solutions running on AWS.
How key metrics like transactions per second (tps) and database response time (latency) can affect performance and customer satisfaction.
How the ability to scale both database reads and writes is the key to unlocking performance on AWS
Similar to How Rakuten Reduced Database Management Spending by 90% through Clustrix implementation (20)
In my presentation, I will summarize the applied and practical aspects of creating sustainable software products. What does it mean - "green" software for users and developers? I want to explain how creating “green” software can be driven by multiple organizational layers. And how building “green” software products can help the organization increase overall software product efficiency.
This presentation introduces the OWASP Top 10:2021.
It explains how to look at the data related to OWASP Top 10:2021, and provides detailed explanations of items with distinctive data. It also introduces the OWASP Project related to each item.
How Rakuten Reduced Database Management Spending by 90% through Clustrix implementation
1. 楽天事例紹介: Clustrix導入による
DB管理コストの削減
How Rakuten Reduced Database
Management Spending by 90% through
Clustrix implementation
October 17th, 2012
Ryutaro Yada (矢田 龍太郎)
Database Platform Group
Global Infrastructure Development Dept.
Rakuten, Inc.
2. Introduction
Ryutaro Yada
First employed by Rakuten in 2008
Present job
Development of a platform database to support Rakuten
Testing and discussion of new techniques and new architecture in view of having it
adopted for use.
Previous functions
Promotion of Oracle business with specified customer
Establish collaborative network with Oracle, develop and verify new solutions, etc..
LinkedIn profile: http://www.linkedin.com/pub/ryutaro-yada/32/368/4b0
1
3. Agenda
About Rakuten
Rakuten database environment and
operational issues
What is Clustrix?
Clustrix verification results and
implementation effectiveness
Summary
2
4. Introduction to Rakuten
About 3000 employees: (Group approx. 7000)
Market / more than 40 services provided including travel
More than 120,000 contracted firms; more than 80,000,000
registered products
Group distribution total: 3.2 trillion yen (2011)
Rakuten market
5. Rakuten Global Expansion
Our Goal is to become the No. 1 Internet Service in the World
LS(UK)
★★
★
★ ★
★★
★
★
★★
★
★ ★ ★
★
★ ★
★ ★ ★
★ ★ ★ ★
★
★
★
★★
★
★★
★ ★
★ ★★
★
★
★
★★
★ ★★
Taiwan ★
★
★
★
*To be open soon ★
★
★
★
★ Ichiba (EC)
★ ★ Travel ★ Performance marketing
★ ★
6. Rakuten’s Global Position
Rakuten is aiming to be the world’s largest internet firm.
Firm and highly flexible infrastructure is required to achieve this
goal
Retail / auction site global ranking 2011 based on unique (no. of) visitors
300000
250000
200000
150000
100000
50000
0
Amazon e-Bay Alibaba Apple Rakuten Walmart
Source: comScore Media Metrics
7. Rakuten Database
Breakdown according to the number of databases:
approx. 80% MySQL (more than 1100)
More than 350 MySQL database servers
MySQL has the largest share
Oracle PostgreSQL Teraddata
No. of databases according
Informix
to actual environment
RDBMS
Same number of databases
for each STG and DEV
MySQL
6
8. MySQL Database Issue (1)
Data Sharding Operations
Required for functionality scaling
Instance/database/table splitting, data redistribution
Correction of application code, control of database access
Data Protection, HA Securing
Replication cannot realize zero data loss at failure
Switch back/switch over management takes a lot of effort
7
9. MySQL Database Issue (2)
Online Maintainability
Schema modification and index addition, rebuild
Lock, access concentration
Number of Units Tends to Increase
Load distribution slave, redundant configuration of slave
Tendency for preparations on an individual service basis (service level
differences, maintenance adjustment diversion)
CPU efficiency decreases; increases in data center costs
8
10. Clustrix Characteristics
What is Clustrix?
Appliance-style database server
Cluster database
NewSQL = LegacySQL + NoSQL
LegacySQL: SQL access, transaction consistency
NoSQL: Scalability, high performance
Fault-tolerance function
MySQL compatibility
Usually access is through MySQL protocol
9
23. Complex and Heavy SQL Comparison
Clustrix IA with SSD SPARC with SAN
J) Count+GroupBy+OrderBy+Limit 1.9s (3.4s) 2.1s (8.5s) 3.4s (409.32s)
K) Count+GroupBy+OrderBy+Limit 0.7s (1.13s) 5.9s (7.49s) 13.0s (39.41s)
L) 2000 of IN+GroupBy 3.8s (8.97s) 106.5s (103.77s) 193.0s (321.68s)
M) Case+OrderBy 31.0s (45.66s) 47.3s (60.9s) 22 90.5s (112.24s)
24. Example of Performance Improvements
Example improvements regarding a particular service
Before: 116.8ms
After: 21.4ms
23
25. Fault-Tolerance Inspection
Failure Test Items Downtime
1 Front network (port1) No
2 Front network (port2) No
3 Internal network (primary) < 12s
4 Internal network (standby) No
Front SW1 Front SW2
5 MySQL instance < 4s
6 Node OS < 4s 1
Online data disk 11
7 < 5s 2
(SSD) failure
Log/work data disk
8 No DB DB DB DB
(SATA) failure 5,6 7,8
9 4 12
Infiniband switch (primary) < 12s
3
10 Infiniband switch (standby) No
11 Front network (port1&2) < 18s 10
9
Internal network
12 < 12s Infiniband SW1 Infiniband SW2
(primary & standby)
24
26. Time Required for Online Maintenance
Table Rows and Size
Small Medium Large
Row 50,000 500,000 5,000,000
Size (byte) 113,639,424 1,063,190,528 10,696,130,560
Implementation Time
Small Medium Large
Create Column 1.6s 13.5 149.8
Create Index 1.6s 13.0s 172.7s
Drop Column 1.5s 13.8s 125.5s
Drop Index 0.5s 0.5s 0.5s
25
27. Impacts During Online Scheme Modification
No impact on access performance in areas other than those subject to work operations
Some impact on performance of access to table being subject to work operations (taking
periods with little impacts, such as night service, into consideration)
Online execution –
5 million
cases, total tables
10G
26
29. Clustrix Implementation Impacts Release from Sharding (2)
No need for correction of application
No need for DB distribution
Sharding production costs reduction (over 90%) for
both application engineer and DBA
In case of large-scale
as-i
s sharding
project, actual
production costs
DBA
compared to
to-be
APP
original
0 2 4 6 8 10 12 14
m an-m onth
28
30. Clustrix Implementation Impacts Cost Reductions due to Consolidation (1)
Sufficient performance scalability
Fault-tolerance ready for mission critical
No data loss
High online maintainability that doesn’t affect other services
Possibility of consolidation to Clustrix of existing MySQL
database
29
31. Clustrix Implementation Impacts Cost Reductions due to Consolidation (2)
Consolidation of all existing MySQL within Clustrix
Number of servers will be reduced to 10%
Monthly system costs will be reduced to 40%
30
32. Back-up Structure
Clustrix
DB DB DB
…
Node 1 Node 2 Node 3
Replication
Slave as first
backup
Backup by mysqldump
MySQL
DB
NFS
NAS
31
33. Data Migration Procedure
Replication to DEV for verification
Replication to PRO for migration
Conversion of application access point to PRO
MySQL
DB
Replication
Replication
Clustrix DEV
Clustrix PRO DB DB DB
DB DB DB
32
34. Other Advantages of Clustrix
Auto-Defrag
Cordial Support Service
Advice regarding structure
Troubleshooting
Tuning advice
Etc.
33
35. Operational Issues Resolved with Clustrix
Data sharding operations Unnecessary, operational
cost reduction
Data protection, HA securing Possible
Online maintenance Possible
Tendency for large number of units Consolidation possible
Cost reduction
34
36. Clustrix at Rakuten
An important database platform
Provided as Database-as-a-Service
No lead-time
Usage volume rate structure
35