Congresso Sociedade Brasileira de Computação CSBC2016 Porto Alegre (Brazil) Workshop on Cloud Networks & Cloudscape Brazil Philippe O.A. Navaux, Professor at the Federal University of Rio Grande do Sul and Computer Science Area Representative at CAPES, Brazil Collaborative Research in Cloud Computing: future and challenges

1. 1º WCN - Workshop on Cloud Networks
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPCCloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. Navaux
GPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento
Paralelo e Distribuído

2. 1º WCN - Workshop on Cloud Networks July 06, 2016 2
High performance applicationsHigh performance applications

Applications that require a lot of computing
 Fluid dynamics
 Computational biology
 Seismic models
 Climatological models

BRAMS and OLAM

HPC is traditionally obtained with the use of supercomputers, clusters and grids

3. 3
Fluids DinamicFluids Dinamic
Sterling e Brandt 2012

4. 1º WCN - Workshop on Cloud Networks July 06, 2016 4
1997 2007DoE
Combustion EngineCombustion Engine

5. 5
Molecular DinamicMolecular Dinamic

Sterling e Brandt 2012

6. 6
HuricanesHuricanes
Disasters PreventionDisasters Prevention

7. 
Click to edit the outline text
format
 Second Outline Level

Third Outline Level
Fourth Outline Level

Fifth Outline Level

Sixth Outline Level

Click to edit the outline text
format
 Second Outline Level

Third Outline Level
Fourth Outline Level

Fifth Outline Level
“Numerous reports have documented the technical challenges and nonviability of simply scalingNumerous reports have documented the technical challenges and nonviability of simply scaling
existing computer designs to reach exascale,” said Dongarra. “Drawing from these reports andexisting computer designs to reach exascale,” said Dongarra. “Drawing from these reports and
experience, our subcommittee has identified the top 10 computing technology advancements thatexperience, our subcommittee has identified the top 10 computing technology advancements that
are critical to making a productive, economically viable exascale system.” (Dongarra 2014)are critical to making a productive, economically viable exascale system.” (Dongarra 2014)
Top 10 avancemments:Top 10 avancemments:
• Energy efficiency,Energy efficiency,
• Interconnect technology,Interconnect technology,
• Memory Technology,Memory Technology,
• Scalable System Software,Scalable System Software,
• Programming systemsProgramming systems
Data managementData management
Exascale Algorithms,Exascale Algorithms,
Algorithms for discovery, design,Algorithms for discovery, design,
and decision,and decision,
Resilience and correctness,Resilience and correctness,
Scientic productivity.Scientic productivity.

8. 1º WCN - Workshop on Cloud Networks July 06, 2016 8
Supercomputers TitanSupercomputers Titan

9. 1º WCN - Workshop on Cloud Networks July 06, 2016 9
Questions ?Questions ?

Can we process applications that need high performance processing on cloud?

E Science

Did the performance suitable?

10. 1º WCN - Workshop on Cloud Networks July 06, 2016 10
Cluster vs CloudCluster vs Cloud

Traditional hardware Cluster
 Hardware cost
 Hardware maintenance
 People for maintenance

Cloud Computing
 Without acquisition and maintenance cost
 Minimum staff
 Pay per use

11. 1º WCN - Workshop on Cloud Networks July 06, 2016 11
Challenges in applications migration to cloudChallenges in applications migration to cloud

Portability:

Lack of application and code documentation, slow down migration.

Scientific applications have a very specific problem domains.

Migration requires a high level knowlegde of application .

Applications developed without version control or comments creates a bottleneck in migration (legacy code)

Costs

Porting application code to PaaS.

Number of processing instances

Acceptable time to solution

12. 1º WCN - Workshop on Cloud Networks July 06, 2016 12
Challenges in applications migration to cloudChallenges in applications migration to cloud

Resources
 Network

Scaling, instance migration, resilience.
 Storage

Local and distributed file systems.
 Processing

Number of instances vs number of processors.
 Applications

CPU-bound, IO-bound, Network-bound.

13. 1º WCN - Workshop on Cloud Networks July 06, 2016 13
Case: Weather ForecastingCase: Weather Forecasting
Numerical Weather Forecast
Used to accurately predict atmospheric
behavior for a future time period.
Represents the state of the atmosphere
at a xed time (temperature,wind
components, etc) over a discretized
domain.
The results are denominated weather
or climate forecasts, a few days for
weather, months for climate.

14. 1º WCN - Workshop on Cloud Networks July 06, 2016 14

15. 1º WCN - Workshop on Cloud Networks July 06, 2016 15
BRAMS Climate ModelBRAMS Climate Model

16. 1º WCN - Workshop on Cloud Networks July 06, 2016 16
BRAMS ModelBRAMS Model
Brazilian Regional Atmospheric Modeling System
Mesoscale model based on
RAMS, developed by the Atmospheric
Science Department at the CSU,
adapted to Brasil Climate.
The main objective of BRAMS
is to provide a single model
to Brazilian Regional Weather
Centers. Each center execute
mesoscale forecasts in their geographical area.

17. 1º WCN - Workshop on Cloud Networks July 06, 2016 17
BRAMS is a freeBRAMS is a free
softwaresoftware

18. 1º WCN - Workshop on Cloud Networks July 06, 2016 18
BRAMS Ensemble ForecastBRAMS Ensemble Forecast
Ensemble Forecast
The impact of initial condition error
propagation in the forecast integration
time can be alleviated by
the ensemble forecast technique.
A set of independent integrations
are done, with small variations
in initial conditions among each
other, resulting in distinct forecasts.

19. 1º WCN - Workshop on Cloud Networks July 06, 2016 19
BRAMS - ECMWF MethodBRAMS - ECMWF Method

12 executions per year (for 10 years)

20. 1º WCN - Workshop on Cloud Networks July 06, 2016 20
BRAMS Result example: Rain PrecipitationBRAMS Result example: Rain Precipitation

21. 1º WCN - Workshop on Cloud Networks July 06, 2016 21
BRAMS in Azure
Affinity groups.
Wrapped in a Cloud Service
Processing VMs Created on demand
Frontend is the only fixed instance
Billed by forecast run
Service is fully automated
Data stored in Azure file service

22. 1º WCN - Workshop on Cloud Networks July 06, 2016 22
Approach:
Using Azure File Srvice:
Pros:
•Legacy Compatibility
•Using SMB
•Highly scalable, replication
•Easy to setup
Cons:
•No file permissions
•Throughput limited
•(Preview)
•Metadata is lost

23. 1º WCN - Workshop on Cloud Networks July 06, 2016 23
Shared Usage

24. Multiples instances of BRAMS in Cloud
72 hours
4 regions of South America
Sharing input data

25. 1º WCN - Workshop on Cloud Networks July 06, 2016 25
Conclusions of HPC in Cloud: Climate caseConclusions of HPC in Cloud: Climate case
The main objective of the work was to migrate the Brazilian Regional
Atmospheric Modeling System (BRAMS) to a new platform that
uses the advantages provided by a cloud computing infrastructure
and evaluate the challenges of this procedure.
The goals of the work was:

Migrate Brams to Azure.

Measure the performance of the migrated version.

Analyse the cost of execution to achieve useful results.

Study the viability of usage of the implemented model by other weather centers.

26. 1º WCN - Workshop on Cloud Networks July 06, 2016 26
Cloud and SupercomputingCloud and Supercomputing
Cloud Companies are installing Supercomputing
facilities to improve their processing power in
executing in cloud.

27. 1º WCN - Workshop on Cloud Networks July 06, 2016 27
Conclusion of HPC in Cloud and with SupercomputingConclusion of HPC in Cloud and with Supercomputing
facilitiesfacilities
- we can use HPC applications in cloud
-.the performance is not the same
- but cloud are changing including supercomputers features
- cloud hadd VM facilities
BUT
- interconnection is one of the challenges
- I/O is another challenge

28. 1º WCN - Workshop on Cloud Networks
Thanks!Thanks!
Collaborative Research in Cloud Computing: future and challenges
Cloud Computing for HPCCloud Computing for HPC
Philippe O. A. NavauxPhilippe O. A. Navaux
GPPD - Informatics Institute – UFRGSGPPD - Informatics Institute – UFRGS
Grupo de Processamento
Paralelo e Distribuído