NTT Data is an IT service company. Kirin is one of the largest beverages companies in Japan. In this presentation, we will present the user story of migrating all applications from creaky infrastructure to OpenStack private cloud including actual challenges, know-hows and future prospects. The key concept of this project is: * Mission Critical: Migrate all Kirin enterprise applications to OpenStack private cloud. * Think Big, Start Small: Start from small number of apps, and expand rapidly. * Agility and elasticity: Adopt a PaaS-like automation approach, targeting 50% less development cost and 40% less operational cost. In order to achieve all items above, we have decided to use OpenStack IaaS, ICO, which is an automation product by IBM, serverspec for testing, and Hinemos for monitoring management. Starting from Aug 2014, the project expects 100 VM / 100 TB storage as the first-stage migration by end of 2015. We're planning to migrate 500 VM / 300 TB by end of 2016 and 2000 VM / 1 PB finally.

1. Copyright © 2015 NTT DATA Corporation
Wednesday, 28 October 2015
Atsushi Koga, Motoki Kakinuma
User Story:
Migrate all Kirin enterprise applications to
OpenStack private cloud

2. Copyright © 2015 NTT DATA Corporation 2

3. 3Copyright © 2015 NTT DATA Corporation
Troubleshooting

4. 4Copyright © 2015 NTT DATA Corporation
Corporate Data
■ Name:
NTT DATA Corporation
■ Head Office:
Toyosu Center Building,
3-3, Toyosu 3-chome,
Koto-ku, Tokyo 135-6033, Japan
■ Established:
May 23, 1988
■ Common Stock:
142,520 million (as of March 31, 2014)
■ Fiscal Year:
April 1 to March 31 of the following year
■ URL:
http://www.nttdata.com
■ Business Areas:
- System integration
- Networking system services
- Other business activities related
to the above
1. About NTT DATA
1988 NTT DATA Communications Systems Corporation spun
off into a separate company from NTT
1995 Listed on the Second Section of the Tokyo Stock Exchange
1996 Changed English name to NTT DATA Corporation
Listed on the First Section of the Tokyo Stock Exchange
2007 Achieved consolidated net sales of 1 trillion
2008 20th anniversary of establishment
2012 Start of four-year Medium-Term Management Policy
(covering the period to March 31, 2016)
2013 25th anniversary of establishment
Overview of NTT DATA History

5. 5Copyright © 2015 NTT DATA Corporation
Provides high-value-added global IT services
that support manufacturing, distribution,
services, media, communications
and other business activities.
Offers platforms and solutions
that support cutting-edge IT services.
Provides high-value-added IT services
that support public administration,
healthcare, finance, settlement and other
systems that form the social foundation.
Segments of NTT DATA
Enterprise IT Services
(E-IT)
Public & Financial
IT Services
(P&F)
Provides high-value-added IT
services in overseas markets.
Solutions & Technologies
(S&T)
Global Business
(GB)
Note: Per-segment data includes internal transactions.
1. About NTT DATA

6. 6Copyright © 2015 NTT DATA Corporation
Bases
Status of Overseas Bases
1. About NTT DATA

7. 7Copyright © 2015 NTT DATA Corporation
Kirin Group Overview （December, 2014）
Trade Name
Date of Incorporation
Paid-in Capital
Head Office
Number of Employees
Business
Kirin Holdings Company, Limited
February 23, 1907（Kirin Brewery Co., Ltd.）
102 billion yen
Nakano, Tokyo
39,922
Alcohol Beverages, Soft Drinks, Pharmaceuticals,
Biochemical, etc…
Corporate Philosophy

8. 8Copyright © 2015 NTT DATA Corporation
8
(Unit: 100 million yen)
Business Segment （December, 2014）

9. Copyright © 2015 NTT DATA Corporation 9

10. 10Copyright © 2015 NTT DATA Corporation

11. 11Copyright © 2015 NTT DATA Corporation
nfrastructure
 Solving creaky infrastructure becomes an big issue in Kirin.
• Kirin has managed various types of infrastructure as below.
BCPBCP

12. 12Copyright © 2015 NTT DATA Corporation
Issues to be solved in Kirin (1)
0
100
200
300
400
500
600
700
FAT シンクラ2015 2016 2017 20182014……

13. 13Copyright © 2015 NTT DATA Corporation
Drastic reduction in
Infrastructure cost
2015 2018
Issues to be solved in Kirin (2)

14. 14Copyright © 2015 NTT DATA Corporation
Why we adopt OpenStack?
- Simple Architecture.
- Extensive enterprise level experience over the world.
- Quick bug report and fix by community members.
- Not just an IaaS, there is a possibility to be a PaaS
with customize.
- Start small, go big.
- “Selecting optimal hardware for each customer” is
NTTDATA’s way.
Sophisticated
Flexibility
Vendor
Neutral

15. 15Copyright © 2015 NTT DATA Corporation
Our Challenges
Full
Automation
• Deployment
automation
• Testing automation
Mass
migration
• Mission Critical
• Minimal downtime
• Quick migration
Drastic cost
reduction
• Operational
efficiency
• Small start to avoid
buying extra server
scale
migration
main
topic
One-Stop deployment

16. 16Copyright © 2015 NTT DATA Corporation
Estimated introduction effects
Apply
for
deploy
confirm
by
admin
Internal
review
Requirement
definition
Design
server
Build
&
Test
Release
Full automated deployment methods
Usual deployment methods
Apply
for
deploy
confirm
by
admin
Internal
review
Build & Test Release
FULL AUTOMATION
Cost reduction effect estimate: 75% OFF per server

17. Copyright © 2015 NTT DATA Corporation 17

18. 18Copyright © 2015 NTT DATA Corporation
Kirin OpenStack Overview
 Kirin OpenStack is fully automated and optimized private
cloud system.
Kirin
OpenStack
3. Consistent inventory management
1. Deployment automation 2. Testing automation

19. 19Copyright © 2015 NTT DATA Corporation
 Kirin EA user sends a deployment request to the Kirin EA(web
application), then build process is started automatically.
1. Deployment automation
User Kirin EA
(Web application)
OpenStack
Request Build
Kirin OpenStack

20. 20Copyright © 2015 NTT DATA Corporation
1. Deployment automation
 Kirin EA enables to us,
 3 Operating systems deployment
• Windows Server 2012 R2/2008 R2
• Red Hat Enterprise Linux 6.6
 11 middleware deployment
• Java, Apache, Tomcat, Hinemos, JP1, Security Software,
SQL Server, etc..
 800+ parameters setting
• Kernel parameter, create OS user/group, create DB user,
• Active Directory, Windows-Feature, etc...

21. 21Copyright © 2015 NTT DATA Corporation
Kirin EA screen sample
password
disk
hostname
network
middleware
select
version
check if
install
1. Deployment automation

22. 22Copyright © 2015 NTT DATA Corporation
Kirin EA screen sample (e.g. Apace HTTP Server)
change paramater of
kernel settings (/etc/sysctl.conf)
change parameter of
/etc/sysconfig/httpd
change parameter of
/etc/httpd/httpd.conf
1. Deployment automation

23. 23Copyright © 2015 NTT DATA Corporation
 Kirin EA registers instances with Big-IP/Hinemos automatically.
Operator
1. Deployment
Request
2. Deployment automation
1. Deployment automation
Kirin EA
3. register
Big-IP

24. 24Copyright © 2015 NTT DATA Corporation
 Kirin EA suspends balancing/monitoring setting from Big-
IP/Hinemos automatically.
 Resume also works.
Operator
1. Instance
stop
3.Stop instances
1. Deployment automation
Kirin EA
2. suspend
Big-IP

25. 25Copyright © 2015 NTT DATA Corporation
2. Testing automation
 Mission: “Getting automatically tested server.”
 Testing automation enables to reduce the costs almost exactly
zero compared to usual manually methods.
 Kirin EA testing automation consists of,
• Parameter test
• Function test (e.g. process-start/stop, log error check)
• Apache-Tomcat(modjk) connectivity test
• DB access test
• Load-balancing(BIG-IP) test
• Monitoring(Hinemos) test ..etc

26. 26Copyright © 2015 NTT DATA Corporation
2. Testing automation
• Testing tools
1. Serverspec
• Parameter check, state check, command result check..etc
2. Original scripts
 BIG-IP test
• Health check works correctly?
• Failover works correctly in the case of server down?
 Hinemos test
• Registered correctly?
• Monitoring is started correctly?
 DB (Oracle) test
• Databases access works correctly with new user?
describe file('/etc/sysconfig/i18n') do
its(:content) { should match("^LANG=ja_JP.UTF-8") }
end
e.g.

27. 27Copyright © 2015 NTT DATA Corporation
• Test result is reported automatically.
• In the case of Linux deployment with full middleware, +140
test cases will be executed.
2. Testing automation
Test results summary
Automated test report
Test result summary
OK or NG
Server information
Test score

28. 28Copyright © 2015 NTT DATA Corporation
• Before Kirin OpenStack implementation, IT assets were managed
inconsistent way.
3. Consistent inventory management
…by the way, how many servers we have?
does anyone know it?

29. 29Copyright © 2015 NTT DATA Corporation
3. Consistent inventory management
• Kirin EA uses “metadata service” in order to tag servers with
metadata.
• Kirin EA user can get all server list anytime.
“application number": "12345",
“sector name”: “sales group",
“ipaddress-01": “10.1.10.23",
"os": "Windows2012R2“,
“cpu”: “2”,
“memory”:”8GB”
middlewares:[
“Java JRE 8u60”
“Apache Tomcat”
・・・
metadata sample
get sever list!

30. 30Copyright © 2015 NTT DATA Corporation
3. Consistent inventory management
• Kirin EA also manages servers which are moved from other
infrastructure by P2V or V2V.
• Operator inputs metadata manually on each server after
migration.
P2V
V2V
EOL
EOL
metadata
metadata

31. 31Copyright © 2015 NTT DATA Corporation
How we implemented OpenStack?
• Install OpenStack was completely easy.
 IBM OpenStack provides each OpenStack Component as a
VMWare virtual server.
• Implementation of availability.
 Use the VMware HA technology.
Compute Cluster
OpenStack
components
VMware HA
Storage
Controller Cluster
VMware #1 VMware #2
nova
cinder
neutron
keystone
VMware HA

32. 32Copyright © 2015 NTT DATA Corporation
How we use the OpenStack?
• OpenStack version: Icehouse
• Virtual technology: VMware
• Used components:
• Nova, Cinder, Neutron, Keystone.
 Implementing OpenStack is NOT our primary purpose.
 Stable, cost effective, easy to use and easy to operate
platform is customer requirement.
We implemented OpenStack as simple as possible.

33. Copyright © 2015 NTT DATA Corporation 33
４．Summary and suggestions

34. 34Copyright © 2015 NTT DATA Corporation
Summary and suggestion
Summary
Suggestion
 It is sufficient to implement IaaS with OpenStack as an
enterprise level.
 Not only automated building, but also automated
testing is effective.
 Retaining aging IT infrastructure with OpenStack will
spreads more and more in Japan.
1. Using OpenStack as PaaS
 Fast and easy implementation of PaaS function on
OpenStack is helpful.
2. Hybrid Cloud
 API compatibility for AWS
3. Software-defined infrastructure with converged
infrastructure
 Free from management of Storage on OpenStack.

35. Copyright © 2011 NTT DATA Corporation
Copyright © 2015 NTT DATA Corporation
All product names mentioned are trademarks or registered trademarks of the respective companies.
記載されている商品名はみな各社の商標または登録商標です