Datacenter and cloud developments and challenges in China
1. Datacenter and Cloud developments and
challenges in China
Terence Graham
Associate Researcher, BroadGroup
and Hong Kong University
2. AGENDA
BroadGroup Study
China’s Mushrooming Cloud
Scale of Investment and Optimism
Inner Mongolia
Market Drivers
Key Challenges – Power and Cooling
Key Challenges – Level of Market Maturity
Key Challenges – Licensing
Business Models
Pricing
Modular
2 Case Studies for Market Entry
Cloud Ecosystem in China
Datacenter Capacity and Projections
Conclusions
3. BROADGROUP – CHINA STUDY
New study of China
Assessed Datacenter players, markets, strategies and Cloud in China
Conducted over past five months
Will be largest study of its kind
Publication November 2012
Some of the key findings …..
6. SCALE OF INVESTMENT AND OPTIMISM
• More than 17 provinces and municipalities constructing 30 large-scale
datacenters
• Total investment 270 billion RMB (US$43.26 billion)
• Total investment in cloud computing projects in China expected to reach
US$154 billion in coming few years (Asia Cloud Computing Association)
• Rackspace to accommodate more than 10 million server units, 5x current
total national datacenter capacity (人民邮电报 )
• Heavy construction, light applications
• Hype?
• From 5 to glut to cloud as colo?
7. INNER MONGOLIA
• Power supply and lower costs
• New eco system - companies announcing set up business processing
centres
• Investment occurring in datacenter sector
• Access huge amounts of space (unlike Tier 1 cities)
• Not for all enterprises – latency issue
• Young educated workforce?
8. MARKET DRIVERS
Growth in China being driven by increasing consumer and business online
Social Networking (Behaviour)
120
70
Gaming
60
100
50
80
40
60
30
40
20
20
10
0
0
2007
2009
2012E
Social Networking Site Users (millions)
1400
Transactions/Data Storage
1200
1000
800
600
2007
2009
2013E
Online Gaming Market (RMB Billions)
“More than one-third of today's Internet
traffic is devoted to video consumption, and
we expect that to grow to 80 per cent by the
end of 2015“
400
Scott Guthrie, corporate vice president, Windows Azure
application platform, Microsoft
200
0
2007
2009
2012E
Online Sales Volume (RMB Billions)
Source: National Bureau of Statistics of China
+ MOBILE GROWTH
Text, Video, Photos, Mobile Internet ….
9. MARKET DRIVERS
12-5 Targets
1.2 billion mobile users
More than 800 million Internet users, Internet penetration rate of over 57%
Fixed broadband Internet access to more than 250 million, more than 40 million
FTTH users
Over 450 million 3G users, accounting for over 36% of total number of mobile
phone users
International bandwidth of ８Ｔｂｐｓ
Achieve LTE commercialization
IPv6 broadband access users more than 25 million
5 Year Plan
2012 (Year 1) Server virtualization and desktop virtualization (VDI)
2013 (Year 2) Try to upgrade to cloud
2014 (Year 3) Private cloud plan
2015 (Year 4) Public cloud plan
2016 (Year 5) Private/public/hybrid cloud
10. MARKET DRIVERS
SEI Strategic Emerging Industries
Cloud computing industry chain expected to reach 750 billion to 1 trillion RMB
by 2015
“There is no doubt datacenters are an indispensable and integral part of the core
resources that make up a country’s economic, political, social, and cultural activities,
and will become one of the country‘s most important strategic assets.” (周宏仁)
National broadband plan
Broadband Development
1.6 trillion RMB, of which 570 billion RMB in broadband fixed access networks
Encouragement of private capital into telecommunications sector (June 2012)
11. KEY CHALLENGES – POWER AND COOLING
Price of Power
Price 80RMB per kVA (Beijing) – Can negotiate to 38RMB in Shanghai
High Power Consumption
2011 -- total datacenter power consumption in China 70 billion kWh - 1.5% of the
country’s total (Other estimates put at 5% of total output)
Medium Sized Datacenter
To use 200w powered servers in a medium-sized computing center of 10,000
servers, annual electricity bill > 30 million yuan
New energy efficiency focus
Majority of current datacenters – coal powered
MIIT 工业节能“十二五”规划 calls for the establishment of China IDC green standards and
an 8% reduction in datacenter PUE by 2015
PUE values
Need to move from current high 2.5 to below 1.5
12. KEY CHALLENGES – LEVEL OF MARKET MATURITY
CLOUD
Complexity of Networks and connectivity procurement
Still not realized saves cost on OPEX not CAPEX
Still in early stage
ENTERPRISE
IT Dept gap with
CLOUD
Long refreshment cycles
IT departments - Fear of centralization,
Internal gap with business depts
Customization at odds with monthly fee model
+ strong influence of 3rd party ISVs and SIs
14. KEY CHALLENGES - LICENSING
Perhaps the most significant current challenge
Policy uncertainty: Very black and very grey
“In building our first datacenter in China, we have learned a lot, how to
communicate with authorities, what applications are needed, what
government approvals are required, how to deal with delays and setbacks.”
Current – one license to build datacenter; another to arrange connectivity
VIE scheme
15. PRICING
End user (list) price is 4000-6000 RMB per month (per rack), with some
areas cheaper
In Tianjin there is one site, a reseller, offering 2400 RMB per month
16. MODULAR
Fast deployment – 3 months - but not cheap
Sites like Ordoss (Inner Mongolia) might be more suitable
Not an easy sell in China/not perceived as flexible
21Vianet own patents on modular datacenters
17. MARKET BARRIERS
Access to capital / funding – scale issue for China key market coverage –
Very high capex requirements
There are 3 kinds of IDC license – (1) like carrier (China Telecom, Unicom) they
don’t need a license can do IDC using telecoms license, (2) (domestic)
companies like SDS or GDS, can build DC and provide service, (3) just buy
capacity, rent DC from China Telecom or other IDC, and become a reseller.
Access to Power – limitations of grid – customized solution for higher power
Very expensive
Access to bandwidth – especially cross city/province
Fibre link and last mile
18. DATACENTER COMPETITION - LANDSCAPE
“Big 3”
China Telecom
Unicom
China Mobile
300
Carrier Neutral
21 Vianet
GDS
Dr Peng
SDS
Different cloud strategies
Planning to build
Alibaba
Baidu
Tencent
Massive Projects
Range International
19. 2 CASE STUDIES FOR CHINA MARKET ENTRY
Case Study 1 - SST - Shanghai Symphony Telecom – 信天通信
Importance of a clear cloud story
Case Study 2 – Equinix
Following the US$230.5 million acquisition of Asia Tone in July 2012, Equinix
gained a total of 6 datacenters and one disaster recovery center located across
Hong Kong, Shanghai and Singapore, including SH5. The first phase of SH5 will
provide 200 cabinet equivalents, expanding to 900 with a gross area of 80,000
square feet when all three phases of SH5 are completed
20. CONCLUSIONS
Who will be the winners?
Experience elsewhere shows winning datacentre companies:
-
Have size and scale
Achieved superior energy efficiency
Secured renewable energy supply
Have established clients and service offerings
Vertical market expertise
Regional expertise (multi-country)
Cost/Technical Differentiation
28. MARKET PLAYER LANDSCAPE
3 state-owned operators
From Red Ocean (红海) to Blue Ocean (蓝海 ) back to Red Ocean
“In China the 3 operators [start] with 3 different cloud strategies but in the
future will do similar things. We always compete, red ocean, price is important.”
TAB: Tencent Alibaba Baidu
3rd party or carrier-neutral datacenters
ISVs
SIs
29. CHINA TELECOM
•
•
•
•
•
•
•
•
•
Backbone network bandwidth (骨干网带宽) of 22 Tbps
International bandwidth (国际出口带宽) 925 Gbps
Number of broadband users 80 million
Datacenter space over 1 million square meters
Datacenters 300+
3 star Tier ? and above datacenters 265
5 overseas datacenters
Racks over 100,000, with another 80,000 on the way
Currently operating 20,000 VMs (高性能虚拟主机) in 3
datacenters in Shanghai, Guangdong, and Sichuan
• Storage capacity of 20,000 TB IaaS services
• Installing 300,000 WiFi hotspots to reach a total of 900,000
around country
30. CHINA TELECOM – NEW BUSINESS UNIT
Quest for the IT holy grail:
March 2012 China Telecom creates new business unit bringing
IDC and cloud computing business together, 中国电信天翼云
公司 or 中国电信云计算公司, led by Dr. Jerry Xie (谢朝阳)
Key implications:
31. BAIDU / TENCENT / ALIBABA
Aim to build nationwide network of datacenters
Still relatively young companies so will take some time
Fear of becoming obsolete like Yahoo (company DNA)
Use cloud-like technologies to build platform that delivers better end user
experience for content (main competitive advantage)
Specs: commodity-PC server & rack power & bandwidth-hungry & own
management software (“special sauce”)
Like-Google approach to datacenters?
Change in relationship with China Telecom & China Unicom
Openness & Zynga problem 70,000 web-based applications run on Baidu
platform or run on their own servers with interface defined by Baidu
32. GOVERNMENT AS ENABLER
• SEI Strategic Emerging Industries
– Cloud computing industry chain expected to reach 750 billion to 1
trillion RMB by 2015
– “There is no doubt datacenters are an indispensable and integral part
of the core resources that make up a country’s
economic, political, social, and cultural activities, and will become one
of the country‘s most important strategic assets.” (周宏仁)
• National broadband plan
– The campaign against fake broadband
– 1.6 trillion RMB, of which 570 billion RMB in broadband fixed access
networks
• Encouragement of private capital into
telecommunications sector (June 2012)
33. TARGETS
• 1.2 billion mobile users
• More than 800 million Internet users, Internet
penetration rate of over 57%
• Fixed broadband Internet access to more than 250
million, more than 40 million FTTH users
• Over 450 million 3G users, accounting for over 36% of
total number of mobile phone users
• International bandwidth of ８Ｔｂｐｓ
• Achieve LTE commercialization
• IPv6 broadband access users more than 25 million
36. CLOUD ECOSYSTEM (2)
Architecting Next Wave of Innovation: Building Blocs over the coming 2-3 years – Cloud Ecosystem
Server
Low-cost, lowpower chips
x86
From generalpurpose
computers to
applicationspecific
servers?
IBM
HP
Dell
Atom (see
SeaMicro)
Virtualization +
automation +
consolidation +
deduplication
Telcos 1
Cloud Stack
[Infrastructure and
Platform]
[Platform]
standardization?
[Data center
optimization]
Security,
availability,
scalability,
reliability, latency,
redundancy,
resiliency, end-toend SLAs
Public/private/hybrid
(federated) <>
Workloads
IPVPN
Devices
BT
Verizon Business
Maturity
Vendor lock-in?
Amazon Web Services
(AWS)
Google App Engine
(GAE)
Juniper
EMC + VMware +
Cisco
MS’s Hyper-V
Citrix Systems
XenServer
Configurations +
Pricing options:
nebulous
Web-centric or Cloud
Applications/Solutions2
Abstract
Bare iron
[SaaS]
Security +
Privacy +
Censorship
Back-end and front-end
components
Commonly available
technologies
Early adopters,
mainstream
companies,
conservative
users
3rd party services
Legacy systems
Inertia
Web 2.0
LAMP: Linux + Apache
+ MySQL + PHP/Perl
Memcache
Consumer:
gaming, social
networking
Java [Oracle acquires Sun
and MySQL]
Ruby on Rails (RoR)
SMBs
Salesforce.com
VMForce
SKT + IBM
KT
(LG) + MS
Nebula (interoperable)
NASA + Rackspace
Python (server language)
Groovy/Scala
Cassandra (Twitter)
MongoDB/CouchDB
NGB (China)
IT services
System integrators
Location of
user’s data
(jurisdiction)
Mission-critical?
MS’s Azure
Singtel (Alatum)
NTT + DiData +
NEC
Chunghwa
Equinix
Cloudera/Eucalyptus/
Cloudcat
Apple (?)
ISVs: raw access to
APIs?
1
Governance
[OSS development tools]
ARM (Marvell)
Commodity
hardware
Verticals
Hadoop (Hive) (big data)
Tornado (Facebook)
Scribe/Thrift
Healthcare
Financial services
Public sector
Education
Smarter Planet
Transportation
Logistics
(IOT)
BI/Analytics
(real-time)
RIM
Governments
to ease
regulations
that hinder
external
storage of
data and
outsourcing
of services +
establish
data
utilization
and
distribution
rules that
account for
privacy +
anonymity
The new
Asian
innovators?
What new
services and
products will
they be
providing?
[“Vibrant,
young Internet
companies”?] 3
.Net
If they can pool all the expertise and make the platform really work, they’re on to something.
“We have crossed a point of scale where we have to spread things across lots of small machines to be cost-effective.’’ (Engineering director, Facebook)
3
What defines the next generation of creators – “They grew up assuming that the computer's GUI was its primary interface. They assume you can learn by copying and modifying code, because
that's what the browser affords with 'view source'. Most of them don't actually want to be programmers, they just want to use programming and circuits to get things done.” IS this true in Asia?
2
37. THE GROWTH OF CLOUD
In China - Cloud = IDC + CDN
China Mobile rolling out IaaS in Guangdong province by end of 2012
VDI licensing “3 for 1”
China Unicom in Shanghai
38. In a Nutshell
“They have no idea how difficult it will be to replace MS and
VMware and are scrambling to develop a ‘Chinese solution.’ It is
a really hard problem. They will need 10s of 1000s of
engineers, over multiple years, to engineer an alternative. Then
find a way to lock it down and create a trade barrier with a
‘national ecosystem.’ Ha! The Chinese have been good at import
substitution with a lot of things but software is not a strong
point, especially large-scale systems. As for CT, like bellheads
everywhere, they are going to have a really difficult time
[figuring out] life-beyond-transport… With shared services they
will start with storage and disaster recovery, maybe managed
security. But that is a long way from server virtualization and
cloud, or application services. You are right, though, they have to
get started somewhere.”
- IT expert
39. Aliyun
“We are catching up quickly, very focused, do not lack
[for] connectivity of customer. We know what is
missing, what needs must be fulfilled. What is important
is that we would rather have 3 of the best cloud services
instead of 5 mediocre ones.”
“We are aiming for 1 billion RMB in revenue from Aliyun.
We currently have less than 50,000 machines, we are
shooting for 300,000 in the next 16-18 months. We have
more than 1000 developers.”
[July 2012]
40. 3rd-party IDCs (第三方IDC运营商)
“To create a cross-cloud, cross-region, real-time application
support network.”
“Learn from the semiconductor industry how to *foster+ value
creation for the whole ecosystem. Greater China has become
the foundry, the middle layer between commercial production
and value-added design. We will play a role in the cloud akin
to that played by a foundry and provide a basic foundation for
cloud services, drop down the cost of innovation.”
Conflict of interest? Financial resources?
41. ISVs: Who will be the IBM of China?
SIs: Who will be the Huawei of cloud?
42.
43. Not another 5-Year Plan
• 2012 (Year 1) Server virtualization and desktop
virtualization (VDI)
• 2013 (Year 2) Try to upgrade to cloud
• 2014 (Year 3) Private cloud plan
• 2015 (Year 4) Public cloud plan
• 2016 (Year 5) Private/public/hybrid cloud
Data integration on backend is difficultHpibmaccenture working with municipal champions on smart city smart gird smart meter projectsSerious projects takes time before one can see resultsAlso many tier 2 cities vertically integrated city and suburbs textiles hub, pharmaceutical hub, chemicals hubOracle working with these cities on vertical clouds tailored to the demands of those specific industries
Bt healthcare cloud tier 2 city in gansu provinceWhy?Someone in the office has personal connection with someone powerful enough to cut through the red tape and get the project off the groundBasically every tier 1 and tier 2 city has a cloud planPowerful local city officials competitive
From 5 to glut to coloMention tier 2 spending on cloud3-4 million on cloud infra, 500,000-2 million on storageLess infighting easier to get projects off the groundNumber of experts and research institutes have made projections and estimates for the capacity coming onlineHype the numbers are so bigone natural reaction is cynicism Build streets first, worry about cars later
Starter flatBargain limited resource give them coal mining rightsHalf price of bjNorway/finland/iceland location agnostic people fixed price 15 yearsLatency online gaming/fast tradersDc within half an hour carCITIC telecom ifcOne of the most expensive per square meters dcs in the worldCheap power = Iceland/Finland/Norway fixed prices 15 years for those that are location agnostic Unicom Langfang/Beijing/Inner Mongolia
Building blocs being put in place now When see public cloud
Do not like paying royalties to foreign ip holdersOne track build national ecosystem for cloudDisappointed if govt didn’t come out and say it would try to form a china cloud standardIndigenous innovationBb ubiquitous HK example21th century economy280 billion yuanct to increase access speed by factor 10 in 3 years
Baidupue 1.321vianet free air coolingpoor design 200 watt powered annualto achieve green targetrecognize international enterprises to outsource to them their datacenter footprint in China CSR statements at shareholder meetings – why important internal targets enterprises want assurances about power increasingly a major issue Baidu did not care much about PUE concept but when it is now processing 100 PB a day through its datacenter and Tencent announced number of servers and traffic growing at 50-60% annually take a much harder look at PUE and cloud-like technologies like virtualizationBaidu now 1.3 PUE
North south divide31 provinces 31 telcosPoor interconnectionOnly 3 internet peeringFirewallMaintain each provincial monopoly position – ct and unicom could resolve the problem overnight but don’t No peering – piece network togetherfirewall in/out ChinaKorean gaming rude awakeningVirtualization can’t do everything 1000 pcs 100 broken if datacenter breaks downVmware selling lot of virtualization software for testing and developmentOracle erp us 80% packaged software 15% youngyou in china 50% own erp – private companies only 10 years oldNo standardized interfaces or messaging buses// Complexity of cloud no definition Academic conference no one could really define cloud in one sentence. 20 vendors 20 different definitions someone joked Chinese govt should step in and set a definition. 8-9bn RMB in turnover only spend 15 million on IT – limits the market – SMBs not spending in US and Europe SaaS sold directly to SMEs. Build as powerful as Google. Amazon 3-5 years ahead. If Amazon and Google were to enter the market no holds barred Baidu would disappear overnight but Tencent would be a formidable competitor. SLA – data security multitenancy
Cloud complexity build as good as google/amazonShandaPopular gaming platformUsed cloud like technologies DDoS – competitors would tryno idea of how many servers it hasReceived Govt handout to show govt officials to demonstrate what an amazon type platform looked likeVery poor monitoring systemsYou can really expect a great platformPersonalities in internet flamboyant used to work for amazon s3Cloud incubator no IaaSBig cloud centers not interested in attracting startupsNot spending 8-9bn rmb 15rmb on it/ sla/multitenancy/BT puts together a great product Haggle on price end up with a bare bone version of cloud service, these companies ROI does not include other elements such as business agility – cost savings and make up their mind whether they are going to use you or hire 2-3 more people
Private capita lFujitsu 80-20licensesSpace/power foreign company like kddi or fujitsu8020Identified a gap in market they could exploitVisit 4 star dc left a lot to be desiredFujitsu boss visit many timesCustomer has to deal with ct/cu/cmWinwinHogan want to win cisco communications hub greater asia, several tens of thousands for rackspace, couple million to china telecom for bwDid this because it had worked with unicom in shanghai and upset local incumbent a bridge u don’t want to burnRumors licensing Examples of JVs – Akamai, Symantec/Huawei, sds and datapipe, joyent, emc and Lenovo, citrix has done well Private capital into DC conference release guidelines postponed indefinitely wait till after nov leadership changeVIE Sina badge of honor for Internet companiesVIE scheme 3 years billionaires strong personalities Shandacloud person amazon S3 /CaymanAlipay taken out of equation no legal recourse to pursue the Chinese holder of the licenseVIE Energy and mineral companies seeking to use VIE vehicle > govt forced to actTRUSTFinding entrepreneurs 3 years they want to become all billionaires social gaming 21vianet iporaised a lot of money for expansion and 4g licensebad cloud/cdn no biz bleeding you have to be very careful about bottom line
People expensive bandwidth cheapBeijing and shanghai competitive even for title of most expensiveLicensed 21vianet bgp you get as # from cnnicChina Internet Network Information Center Configure routerDepending on where enduser is who is accessing your website or product use automatically routes to use network offering least congestion2000 rmb a meg startup 100 megGo to hangzhouBeijing and ShanghaiEven same quality BGP there is a price differential, BJ is 50% cheaper than Shanghai – maybe it’s the business culture in Shanghai. Shanghainese are more wise to doing business (and not getting fooled). Compromise, if CT/Unicom is selling at 100, the other telco (CT/Uni) will do 85/90, there is a little bit of difference, but they (Unicom in Shanghai) won’t sell at 60/70 to compete with CT.2000 RMB a meg bgp got to tier 2 city if you are a cloud city.foreigners always get charged premium chinacache quotes 10x
ordoss west solar power21 has patents on modular
200mrmbNot counting land“To create a cross-cloud, cross-region, real-time application support network.”Beijing/Shanghai/Guangzhou/Shenzhen/ - Chengdu/Xian/DalianTalk to shareholders200m rmb a datacenter not counting on landfacility over which you have complete controlnot 21vianet franchise model with local telco > reduce capx exposure but once biz growsnational presence a stronger business – not like UK London or France with ParisChina is like several countries stuffed within the bordersGain scale you become like Telecity with a presence in 14-15 countriesScale of operations – multiplicity – achieve economies – unified design and server configuration on the backend – improved ROI Foreign companies are not getting licenses full stopRunning back and forth 6.8 kva per rack customized but will pay a lot 1.5 EDC you could get cheaper bw in another province but the provincial pta will step in and prevent you – protect each provincial ct – difficult to create own network/NO SPOT market21 vianet certain amount of fiberShanghai sdszhangjiangDr. peng great wall broadbandAlibaba
Range crossborderCarrier conflict of interestUnicom tail end of a drawn out merger between it and netcomExcitedFirst time building real datacentersStarting to seize control of its destinyWhy carrier neutral good cannot commoditize quality of service db story / 24 hours china mobile
TrustGood internet citizen awardPull the plugDc in guangdong province 70gigBlog/pornography/internet scamChild of a loveless marriage
Focus on verticals/market experienceSST not selling to engineer bandwidthNeed to understand industry Hiring new peopleThink what the enduser is going to do with the bandwidth Finance companies superfast hilatencyGovt datacenters UK special audit comply security requirementsNot general approach dig deep into that verticalAmerican healthcare datacenters manage compliance rules these companies exist under multiprovinceCost and technical differentiatorOnce build dc constantly have to invest in the facility to keep it upgraded and up to new standards.
