Solvcon PyCon APAC 2014


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Solvcon PyCon APAC 2014

  1. 1. SOLVCON: Software- Engineering Simulations of Conservation Laws Yung-Yu Chen! 
 SOLVCON Project! @ PyConAPAC 2014 May 17
  2. 2. Introduction ❖ I am: an engineer using computers to solve problems in mechanics.! ❖ I want to solve conservation laws. I want to solve for various physical processes. I should build a framework.! ❖ Examples of the simulations: Aerodynamics Benchmarks Stress Waves in! Anisotropic Solids Supersonic Jet in Cross Flow
  3. 3. Goals ❖ We want multiple solvers for physical processes governed by the equations:! ! ❖ We use the space-time Conservation Element and Solution Element (CESE) method.! ❖ We want an extensible system, modularized for various physical processes.! ❖ Then plan for interoperation.! ❖ We want parallel code (scale from 1,000 cores to more). We want hybrid parallelism for heterogeneous architecture.! ❖ We want the code to be organized. We don’t want spaghetti code patched from paper to paper. We want to code systematically.
  4. 4. It’s an Engineering Problem ❖ “Develop a software framework for the CESE method.”! ❖ Balance the following points:! ❖ Extensibility and modularity.! ❖ Parallel computing.! ❖ Maintainability.! ❖ All goals are achieved, with rough ends.
  5. 5. Python ❖ Python is slow, but provides a good infrastructure for working with low-level number-crunching code.! ❖ ndarray (N-dimensional array) of NumPy (http://! ❖ Cython ( for interfacing between Python and C.! ❖ The versatile scientific Python ecosystem simplifies coding supportive functionalities.! ❖ Practices emerged from other fields flow into sciences:! ❖ Version control, unit tests, documenting tools, etc.
  6. 6. Scripting Is a Must ❖ There are too many parameters and analyses required.! ❖ No interface is sufficient for the simulations: SOLVCON needs to be a library.! ❖ Controlling a library by using low-level languages like C and C++ is too painful to be realistic.
  7. 7. Design for Productivity ❖ The outstanding capability to interface to C makes Python an ideal manager for low-level, high-performance C code.! ❖ C++ works as well but SOLVCON uses only C to avoid the complexity.! ❖ The hybrid approach allows us to architect the system with the high-level Python and still have the high performance of C.! ❖ Ideas can be quickly prototyped with Python. After results get validated, the fast C version can then developed iteratively.
  8. 8. Discretize Space ❖ The conservation laws we are solving assume continuum model.! ❖ In computer memory nothing is continuous.! ❖ There must be a way to discretize the space for the equations.! ❖ “Meshing”.
  9. 9. Unstructured Meshes ❖ We chose to employ the unstructured meshes of mixed elements to model complex geometry.! ❖ In computer memory they consist of several look-up tables. quadrilateral triangle hexahedron tetrahedron prism pyramid
  10. 10. Look-Up Tables for Meshes ❖ SOLVCON categorize the look-up tables into three: (i) coordinate, (ii) meta-data, and (iii) connectivity.! ❖ They are either 1-D or 2-D arrays.! ❖ These arrays contain “ghost cells” for boundary- condition treatments.! ❖ The ghost cells are outside the computing space.! ❖ The CESE method needs only one layer of ghost cells.
  11. 11. Domain Decomposition ❖ Millions or billions of unstructured mesh elements can be automatically decomposed into multiple blocks for parallel computing.! ❖ A graph is built for representing the connectivity among all the mesh elements.! ❖ The partitioned graphs are used to decompose the original mesh into thousands of blocks.
  12. 12. Two-Level Nested Loops ❖ The time-marching nature of the CESE method dictates a structure of two-level nested loops.! ❖ The outer loop is for time- marching.! ❖ Multiple inner loops “sweep” over the discretized space.! ❖ All facilities can built around this execution flow.
  13. 13. Module Structure
  14. 14. Case-Solver Relation ❖ the outer temporal loop.! ❖ solvcon.solver.MeshSolver: house of the inner spatial loops.
  15. 15. In-Situ Visualization/Analysis ❖ What’s in-situ visualization/analysis (ISV)?! ❖ Perform result visualization/analysis while time-marching.! ❖ Why in-situ?! ❖ [50 million-element mesh] * [10,000 time steps] * [6 single-precision solution fields (CFD)] mean 12 TB of data. It is one simulation case.! ❖ Segregating result analysis from simulation code is difficult.! ❖ Architecture of most research codes isn’t good enough for the segregation.! ❖ Most codes simply produce the data and use another software to post- process.! ❖ A common practice is to reduce the resolution.! ❖ Outputting every 100 time steps reduces the output dat to 120 GB.! ❖ But it defeats the purpose of high-resolution simulations.
  16. 16. SOLVCON’s Segregation ❖ SOLVCON provides companions of MeshCase and MeshSolver: MeshHook and MeshAnchor, respectively.! ❖ MeshCase and MeshSolver are for equation solutions.! ❖ MeshHook and MeshAnchor are for solution analysis.! ❖ And there’re still a lot to do.
  17. 17. Documenting ❖ SOLVCON documents should contain what’s in conventional research papers.! ❖ What we want is more than conventional papers.! ❖ Bi-directional linking between the code and the documents.! ❖ Some part of the documents should be runnable. It’s related to validation.
  18. 18. Sphinx ❖ Sphinx ( is the default tool for Python documentation.! ❖ ReadTheDocs ( is a service to host Sphinx documents.! ❖ SOLVCON authors and organizes its documents with Sphinx.! ❖ The documents are published at http:// (ReadTheDocs is behind the scenes).
  19. 19. Authoring, Editing, and Reviewing ❖ We use Mercurial for version control and BitBucket ( to host the project and its source code.! ❖ Reviewing the documents goes through the pull-request flow of the code review.! ❖ We treat the reStructuredText source files of the documents like the program source code.
  20. 20. Integrated Testing ❖ Code without verification can’t be taken as working code, no matter how carefully we wrote it.! ❖ With the testing system, refactoring and quick prototyping become possible.! ❖ Many research codes don’t have a regression testing system, so while coding the developers can’t be sure how the changes impact the system.! ❖ Two kinds of tests: unit tests and answer tests. Both need automation.
  21. 21. Unit Tests ❖ Unit tests are short and quick, and check for simple output of elementary API behaviors.! ❖ SOLVCON uses standard Python unittest module (! ❖ nose ( is used as the automatic test runner.
  22. 22. Answer Tests ! ❖ Answer tests check for the correctness of simulations. Usually an answer test is a meaningful simulation, and can serve as an example to how to use SOLVCON to simulate other problems. ! ❖ Jenkins ( is installed at http:// to automatically run the answer tests for every commit of SOLVCON in a testing farm.
  23. 23. Challenges (Conclusions) ❖ The simulations were done with older version of SOLVCON; it takes a lot of efforts to upgrade (ongoing).! ❖ Unfamiliarity of the collaborators to the authoring system (Sphinx):! ❖ Unification of code and document needs extra efforts.! ❖ Feeding back to the main framework.! ❖ Server farm for continuous integration.
  24. 24. Current/Future Works ❖ New physical processes:! ❖ solvcon.parcel.bulk: acoustic wave solver based on bulk modulus.! ❖ solvcon.parcel.vewave: visco-elastic wave solver.! ❖ Document the CESE method.! ❖ Documenting with iPython in addition to Sphinx?! ❖ Get more people to play together!