Cloud fusion concept fujitsu scientific tech journal april 2012


Published on

Cloud Fusion has three defining attributes:
• It delivers advanced integration of multiple clouds – interconnecting the clouds of today to deliver the “cloud of clouds” for business
• It enables actionable insights from big data, analyzed and aggregated to support real-time business navigation
• It delivers extended services from the cloud for distributed applications which handle mobile devices/sensors and provides all necessary resources/services in an on-demand self-service model.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cloud fusion concept fujitsu scientific tech journal april 2012

  1. 1. Cloud Fusion Concept  Yoshitaka Sakashita  Kuniharu Takayama  Akihiko Matsuo  Hidetoshi Kurihara Cloud is expected to develop from a single-purpose cloud to a hybrid cloud that links clouds or existing systems, or to a fusion of two or more clouds in the future. Fujitsu Laboratories named this advanced form of coordination “Cloud Fusion” at the start of 2010. This paper explains the aim of this coordination and the direction in which research should head. It goes on to describe the relationship between Cloud Fusion and Fujitsu’s and Fujitsu Laboratories’ vision—enabling Human- Centric Intelligent Society. It describes the five pillars of research on Cloud Fusion and its outline. In particular, it introduces details about the development and execution environment that is one of the pillars.1. Introduction 2010.note) i We have conducted research while Fujitsu Laboratories started working on closely working with the product developmentcloud computing (hereafter “cloud”) together division and FUJITSU Family Association/LSwith Fujitsu in the spring of 2007 and started up Research Committee, a customers’ researchthe present Cloud Computing Research Center committee, from the beginning. In the processin April 2009. This organization is an aggregate of studying what to realize with clouds andof groups engaged in cloud-related research such communicating with customers, we becameas infrastructure control, system management aware that technology capable of linking existingand development and execution environment. It systems and clouds was necessary. Hence, wehas been launched as a center with the nature started to work on achieving a hybrid cloudof a cross-laboratory organization. Fujitsu and around the summer of 2009. With this idea stillFujitsu Laboratories had been researching promoted as Fujitsu’s cloud strategy, we havethemes such as grid computing and virtual proposed the optimum system construction bysystem control since before the launch of the linking public and private clouds and existingcenter. As an initial achievement, we developed systems and offered products. Subsequently, wea virtual system control engine, which provides have proposed various types of clouds to give risethe core of a cloud system.1) The engine was to healthcare, local government and other clouds.adopted as an in-house practice system of Under the circumstances, Fujitsu Laboratoriesthe On-demand Virtual System Service. It is felt it was necessary to develop a system linkedcapable of providing an infrastructure for the with various clouds and existing systems, anddevelopment of necessary systems such as user named this form of coordination “Cloud Fusion”management and billing as well as operation and note) i In June 2011, the On-demand Virtualservice design. Moreover, it contributed to the System Service was named as Fujitsusubsequent start of commercial service in October Global Cloud Platform “FGCP/S5” service in the Japanese market and Global Cloud Platform (GCP) service in overseas markets.FUJITSU Sci. Tech. J., Vol. 48, No. 2, pp. 143–150 (April 2012) 143
  2. 2. Y. Sakashita et al.: Cloud Fusion Conceptat the start of 2010. described above, we estimated that a system for This paper describes the aims and overall linking and mutually using private and publicconcept of this Cloud Fusion. clouds would be important in terms of business as well. We aimed to link various clouds together2. Aims of Cloud Fusion by Cloud Fusion to link and share information, In the cloud market, the public cloud and thereby expand the fields of information andbusiness and software as a service (SaaS) communications technology (ICT) applicationbusiness based on it were the mainstream in and create a new market. A similar conceptthe initial phase, and we now expect customer- is referred to as “inter-cloud” by the Smartspecific private cloud services will develop. IDC Cloud Study Group of the Ministry of InternalJapan released a forecast on April 7, 2011 saying Affairs and Communications (MIC).2) Whilethat the scale of the software market for public similar concepts have been recently announcedclouds and that for private clouds will be almost by other companies as well, we believe thatequal at 145 billion yen in 2012, and the latter Fujitsu Laboratories has taken the lead in thiswill overtake the former after that. Meanwhile, research. Furthermore, Fujitsu Laboratoriesa form of sharing a cloud by an industry or proposes a way to eliminate companies’ anxietycommunity has appeared. At present, we are of being locked in to a specific cloud by usingconducting research on media and engineering Cloud Fusion to link different types of clouds.clouds as a service in addition to healthcare and As technologies required for realizing these aimslocal government clouds. In addition, we are and ideas, we are intensively engaged in R&Dcurrently linking clouds by using sensors and under the following five phone handsets as input devices, which • Secure data connectionmay be referred to as the Internet of Things. • Big data parallel distributed processing Fujitsu and Fujitsu Laboratories set forth • Cloud development and executiona vision of Human-Centric Intelligent Society in environmentthe spring of 2010 and have since been working • Cloud mangementfor the construction of a cloud environment • Managed networkthat supports this vision. Based on the trends Figure 1 shows the background, concepts From hybrid cloud to Cloud Fusion, multiple clouds freely combined for use Highly-integrated cloud services Community Hybrid cloud Industry Medical care Enterprise Cloud Media Engineering Single cloud Corporate information center Figure 1 Fujitsu Laboratories’ approach to cloud. Figure 1144 FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012) Fujitsu Laboratories’ approach to cloud.
  3. 3. Y. Sakashita et al.: Cloud Fusion Conceptand major research themes. The following entire system. For that purpose, aside from thesections describe the R&D for realizing Cloud research group for Cloud Fusion, we have otherFusion with the focus on the development and groups such as one engaged in research on Greenexecution environment. For other themes, see Internet Data Center (IDC), for example, whichthe respective papers contained in this special puts together servers, storage, networks andissue. facilities. Above Cloud Fusion, there is also a group working on the technologies for enabling3. Technologies for realizing Human-Centric Intelligent Society, which is Cloud Fusion Fujitsu’s and Fujitsu Laboratories’ vision. In An overview of the technologies for realizing this field of research, Cloud Fusion takes chargeCloud Fusion is shown in Figure 2. of the development and execution environment Cloud is a synthesis of technologies for high-speed accumulation and analysis ofwidely ranging from infrastructures such as various pieces of event data and other big data.hardware and networks to technologies that Human-Centric Intelligent Society supportsuse them to make contributions as business data collection in the actual field and analysis,and social infrastructure. Accordingly, Fujitsu prediction and proposal for use in the field.Laboratories does not only conduct world-class The Cloud Fusion group has defined fiveR&D in the respective fields of these technologies pillars of R&D as mentioned in the previousbut also links other companies’ technologies and section. The following outlines the respectiveopen source materials. It does so by vertical pillars of research.integration in an attempt to sophisticate the 1) Secure data connection Human-Centric Intelligent Society Fusion Cloud Fusion technologies Management Cross-industry cloud Secure data connection Other company’s cloud Big data PaaS management parallel distributed Information hiding processing Information traceability Infrastructure Linking Large-scale parallel integrated visualization/ Development and distributed Industry cloud Logic movement diagnosis processing execution environment between clouds Group company cloud Development simplification Volume event Predictive fault processing detection Data analysis application development and Managed Single cloud integration environment Distributed KVS network Other company’s SaaS Application execution Volume batch environment management processing Increase of throughput platform of communication Public cloud Enterprise mashup between DCs, etc. technology Private cloud Test automation/ efficiency improvement Virtual platform control/construction Existing system Green IDC Cloud management standardization Figure 2 Fujitsu Laboratories’ overall technology for Cloud Fusion. Figure 2 Fujitsu Laboratories’ overall technology for Cloud Fusion.FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012) 145
  4. 4. Y. Sakashita et al.: Cloud Fusion Concept Security, which is said to be a major of IT management, aiming for its standardization.hindering factor to the application of cloud, may In addition, we have commercialized part of thehave a significant impact if information is leaked technology to comprehensively visualize andand a system to allow use with more security diagnose the performance of network and serverthan before is required. We want customers to infrastructure at the end of 2010. Since theentrust Fujitsu’s cloud with their information beginning of 2011, we have been aiming to applywithout anxiety. Hence, we aim to be able to predictive fault detection technology to the realhide information between the existing systems environment. To this end, we have been movingand clouds and to identify how information has ahead with new research on PaaS managementbeen used (this function will be commercialized in consideration of application life the middle of FY 2012). In addition, we need 5) Managed networkto be able to process data without handing them In this age of cloud, we believe networks willto a cloud. Therefore, we have researched and experience a dramatic increase in data volume,developed a technology to have SaaS applications and this has given rise to demand for researchdownloaded to the own center and impose a to address various challenges. For example,restriction to prevent the applications from research on the performance and managementsending information outside. technology of a network between clouds and2) Big data parallel distributed processing between clouds and terminals is necessary and a The essential aim of Cloud Fusion is to new perspective is and share information to expand the fieldsof ICT application and create a new market. To 4. Overview of development anddo this, big data processing is a key technology. execution environmentWe are basing our work on technologies such as The essence of cloud is aggregation andthe key value store (KVS) technology, suitable distribution. That is, cloud is intended tofor storing big data, and open source Hadoop, aggregate on a large scale customers’ systemscapable of quickly processing big data. And we and data to realize economies of scale andare investigating even more advanced parallel their distributed processing by commoditydistributed processing for close linkage with infrastructure (such as servers, storage andHuman-Centric Intelligent Society. networks) to realize cost efficiency.3) Cloud development and execution In terms of execution environment, environment aggregating systems on a cloud provides cost This involves research on technology to and quality benefits. For example, it eliminatesallow customers to easily connect clouds with the need for infrastructure on the customer’sexisting systems and big data analysis platforms. site, makes systems available for use only whenThe following section gives the details. and as much as required, and acts as measures4) Cloud management for disaster recovery by using a backup cloud. Up to now, we have conducted research Meanwhile, aggregating data on a cloud alsoon the efficient control and management of provides cost, quality and efficiency benefits invirtualization systems. We have also proposed terms of development environment. For instance,the management interface for On-demand it centralizes the development environmentVirtual System Service. It became commercially and makes it easy to ensure governance onavailable in October 2010, and we proposed it to the development processes and results. Withthe Distributed Management Task Force (DMTF), this situation in the background, softwarean industry group promoting the standardization development and execution environment on146 FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012)
  5. 5. Y. Sakashita et al.: Cloud Fusion Concepta cloud, or the so-called platform as a service fusion of various application systems will be(PaaS), has recently been evolving and becoming important including the existing and newwidespread. areas. With general Web applications, linking is It must be noted that implementing existing roughly classified into linking in the front end inbusiness applications on a cloud has a scale-out charge of interaction with the user and that inproblem. With the conventional infrastructure, the back end, which takes charge of processingbusiness applications with large amounts of and interchange with databases according toaccesses and processing are often built by using the access from the front end. One well-knownhigh-performance servers and databases, which technology of the former is mashup. For theis called scale-up. On the other hand, clouds are latter, service-oriented architecture (SOA) is acomposed of large numbers of similar commodity popular technology. One important challenge inservers and databases and cannot be scaled up. such linking is how to effectively use the existingAccordingly, scale-out is necessary, in which systems. Accordingly, we have developedprocessing is distributed among many servers WebAPI creation assistance technology,and databases. Relational databases (RDBs), which semi-automatically extracts applicationwhich are typical conventional databases, programming interfaces (APIs) from the existingare good at complicated data retrieval and systems to allow them to be mashed up withaggregation. However, they take their processing other applications.steps one at a time so as to maintain consistency, Of these, big data analysis PaaS andwhich makes them unsuitable for distributing WebAPI creation assistance technology will bemultiple accesses for parallel processing. In explained in more detail in the following sections.clouds, on the other hand, new types of datastorage are used such as KVS, which is capable 5. Big data analysis PaaSof high-speed parallel distributed processing of From now on, it will be important to makea simple data structure with sets of keys and use of data gathered in large volumes fromvalues. For this reason, we have developed sensors and terminals to propose new servicesoptimum deployment technology for data storage. and solutions that could not be readily realizedIt automatically selects between multiple types with the conventional ICT. In the analysisof data storage such as RDB and distributed KVS and development environment based on Cloudin a cloud system according to the characteristics Fusion, we will need to create new servicesof the data or accesses handled. In this way, it together with customers by combining dataappropriately switches data storage accesses across the boundaries of business categories andfrom Web applications.3),4) industries starting from the big data owned by This paper has so far covered the existing customers.areas of business. New business areas that take However, analyzing big data and developingadvantage of big data such as sensor data and event-driven services requires skills to widelybusiness logs are being launched. To analyze master and implement various processinglarge amounts of data and develop event-driven technologies including data parallel distributedservices, we need a system capable of easily and processing (such as Hadoop), event processingcomprehensively analyzing and developing batch (such as Esper) and analysis processing (suchanalysis processing of big data and real-time as Mahout). An analysis and developmentprocessing of event data. We have developed a environment that comprehensively supportsbig data analysis PaaS for that purpose. them has not been realized. In the future, we expect that linking and To address this need, we have test-built theFUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012) 147
  6. 6. Y. Sakashita et al.: Cloud Fusion Conceptenvironment shown in Figure 3 that supports We have taken a real case of business as athe cycle of service planning by data analysis reference to develop cluster analysis processing[2) in Figure 3], simplified development of and coupon issuing application and evaluatedservice applications [3) in Figure 3] and service this environment. We have confirmed that withevaluation [4) in Figure 3]. In this environment, this environment it is possible to reduce thebatch analysis processing of big data and real- period of time taken to provide the service, whichtime processing of event data can be handled in an was conventionally two months, to two weeks.integrated manner. As an interface, a data flowdiagram (DFD) editor environment (HTML5) is 6. WebAPI and mashupprovided [1) in Figure 3]. The architecture allows technologythe defined DFD to be configured as a program More than one business system is operatedaccording to Service Component Architecture in an enterprise and a user often needs to(SCA), a standard component specification, for use multiple systems to perform a series ofdeployment and execution. operations. Along with the expansion of cloud, The data analysis personnel can easily companies now need to link with applicationsexecute analysis and event processing without provided by external clouds in addition to thoseneeding knowledge about the individual systems. For this reason, we have developedimplementation technologies, simply by WebAPI creation assistance technology to calldeploying input/output (I/O) data and processing existing systems as Web services without makingcomponents as a DFD and providing execution changes to allow linking by mashup.parameters. To realize this system, we have Mashup is a technique of combiningdeveloped as a differentiating technology a new multiple Web services published on the Internetfunction to convert components into the optimum to build Web applications with new value,implementation component according to the as exemplified by Google Maps. Applicationtypes of processing and data. developers can easily use functions such as data Feedback of points of improvement Evaluation Planning 4) Service effectiveness analysis 1) Analysis process description by DFD 2) Static analysis of large-scale data Multidimensional Cluster Effect Service analysis analysis measurement use Accumulated data Hive Mahout event data RDB Hadoop HDFS/MapReduce HTML5 Service customization Simplified Service development development reflecting business rules 3) Real-time processing by CEP Service operation Service Event Filter Matching notification Queue CEP engine Queue CEP: Complex event processing Figure 3 Outline of PaaS for large-scale data analysis. Figure 3148 FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012) Outline of PaaS for large-scale data analysis.
  7. 7. Y. Sakashita et al.: Cloud Fusion Conceptacquisition and screen display/operation simply them to be used from mashup applications. Withby executing calls according to the APIs offered this technology, the user’s screen operations onby Web services. It may be possible to apply this the system are reproduced to obtain the resultsmashup technique to enterprise systems to use of processing from the screen, thereby addingthe functions of existing systems and applications WebAPIs without making changes to existingthat meet the needs of user departments. If such systems. Figure 4 shows an overall view of thisan environment can be built easily then higher technique.operations efficiency can be expected.5) To create an HTTP request queue as To use system functions by mashup, a reproduction of screen operations, thishowever, the functions must be provided as APIs technology has the user execute the operationsand callable from a different system. Existing of the applications concerned and records datasystems in enterprises are often not provided sent and received between the browser andwith such APIs and are unavailable from outside Web applications. This eliminates the needas components. Making changes to systems to develop a program for generating a requestto add APIs or data-linking function requires queue and operations that need many screensmany person-hours for development and testing. before producing results can be easily turnedThis has made it difficult up to now to use the into WebAPIs. In addition, it has a function tofunctions of existing systems by mashup. edit the recorded request queue for extraction of To deal with this challenge, we have parameters to be provided from outside such asdeveloped WebAPI creation assistance the user ID, password and search key. Hence,technology. It adds APIs without making authentication information essential to businesschanges to existing Web applications to allow systems can be provided by mashup applications. Developer Enterprise mashup proxy 1) Execute operations HTML Business application JSON Communication/response recorder 3) Edit and provide components 2) Record communication log Communication/response records Process SaaS application executed GET / HTTP/1.0 according to GET /login.cgi the records HTTP/1.0 POST /search User HTTP/1.0 Key=XXX&id=YYY Generalized business component Present development 4) Mashup by using components Figure 4 Overall support technology for making WebAPI. Figure 4FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012) Overall support technology for making WebAPI. 149
  8. 8. Y. Sakashita et al.: Cloud Fusion Concept In a trial using this technology, we have Fujitsu Laboratories intends to continue to leadconfirmed that WebAPI can be created with one- the development of cloud technology so as to helptenth the person-hours as compared with adding create and pioneer new business.WebAPI to an existing system by manuallycreating a request queue. References 1) M. Kishimoto: Fujitsu’s Trusted-Service Platform. (in Japanese), Cloud Computing7. Conclusion Technology to Grasp the Other Side of the World Fujitsu Laboratories’ approach to the area of Clouds. ASCII, 2009, pp. 58–65. 2) MIC: Smart Cloud Study Group Report—Smartof cloud presented here is based on the vision Cloud Strategy—(May 2010).called Cloud Fusion. At the beginning of the joho_tsusin/eng/councilreport/pdf/development of On-demand Virtual System 100517_1.pdfService in October 2010, it was referred to as the 3) M. Adachi et al.: Fujitsu’s Approach to the Age of Cloud—Development of Construction andService-Oriented Platform. We thought of a “site” Operation Technology for a Cloud System. (inwhere information is gathered to conduct new Japanese), ASCII technologies, June Issue, pp. 62–69 (2010).communication and business as a “platform.” We 4) Y. Mizobuchi et al.: Automatic Optimized Use ofhave conducted research based on this idea since Multiple Datastores in Cloud Computing Era. (in Japanese), IPSJ Special Interest Group onbefore the name “cloud” became popular and we Software Engineering Report, Vol. 2010-SE-170,think that the essence of Cloud Fusion is the No. 14, pp. 1–8 (2010). 5) A. Matsuo et al.: LivePoplet: Technology Thatsame. In the future, cloud is expected to change Enables Mashup of Existing Applications.the business, social infrastructure and lifestyles Fujitsu Sci. Tech. J., Vol. 45, No. 3, pp. 304–312 (2009).of people in the world more than we can imagine. Yoshitaka Sakashita Akihiko Matsuo Fujitsu Laboratories Ltd. Fujitsu Laboratories Ltd. Mr. Sakashita is currently engaged in Mr. Matsuo is currently engaged research strategy and promotion for in research and development of entire cloud computing. inter-cloud linking technology and technology to improve efficiency of software maintenance. Kuniharu Takayama Hidetoshi Kurihara Fujitsu Laboratories Ltd. Fujitsu Laboratories Ltd. Mr. Takayama is currently engaged Mr. Kurihara is currently engaged in in research and development of research and development of software application development and execution development environment and envi-ronment in cloud. up-stream process technology.150 FUJITSU Sci. Tech. J., Vol. 48, No. 2 (April 2012)