Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Solvcon PyCon APAC 2014


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Solvcon PyCon APAC 2014

  1. 1. SOLVCON: Software- Engineering Simulations of Conservation Laws Yung-Yu Chen! 
 SOLVCON Project! @ PyConAPAC 2014 May 17
  2. 2. Introduction ❖ I am: an engineer using computers to solve problems in mechanics.! ❖ I want to solve conservation laws. I want to solve for various physical processes. I should build a framework.! ❖ Examples of the simulations: Aerodynamics Benchmarks Stress Waves in! Anisotropic Solids Supersonic Jet in Cross Flow
  3. 3. Goals ❖ We want multiple solvers for physical processes governed by the equations:! ! ❖ We use the space-time Conservation Element and Solution Element (CESE) method.! ❖ We want an extensible system, modularized for various physical processes.! ❖ Then plan for interoperation.! ❖ We want parallel code (scale from 1,000 cores to more). We want hybrid parallelism for heterogeneous architecture.! ❖ We want the code to be organized. We don’t want spaghetti code patched from paper to paper. We want to code systematically.
  4. 4. It’s an Engineering Problem ❖ “Develop a software framework for the CESE method.”! ❖ Balance the following points:! ❖ Extensibility and modularity.! ❖ Parallel computing.! ❖ Maintainability.! ❖ All goals are achieved, with rough ends.
  5. 5. Python ❖ Python is slow, but provides a good infrastructure for working with low-level number-crunching code.! ❖ ndarray (N-dimensional array) of NumPy (http://! ❖ Cython ( for interfacing between Python and C.! ❖ The versatile scientific Python ecosystem simplifies coding supportive functionalities.! ❖ Practices emerged from other fields flow into sciences:! ❖ Version control, unit tests, documenting tools, etc.
  6. 6. Scripting Is a Must ❖ There are too many parameters and analyses required.! ❖ No interface is sufficient for the simulations: SOLVCON needs to be a library.! ❖ Controlling a library by using low-level languages like C and C++ is too painful to be realistic.
  7. 7. Design for Productivity ❖ The outstanding capability to interface to C makes Python an ideal manager for low-level, high-performance C code.! ❖ C++ works as well but SOLVCON uses only C to avoid the complexity.! ❖ The hybrid approach allows us to architect the system with the high-level Python and still have the high performance of C.! ❖ Ideas can be quickly prototyped with Python. After results get validated, the fast C version can then developed iteratively.
  8. 8. Discretize Space ❖ The conservation laws we are solving assume continuum model.! ❖ In computer memory nothing is continuous.! ❖ There must be a way to discretize the space for the equations.! ❖ “Meshing”.
  9. 9. Unstructured Meshes ❖ We chose to employ the unstructured meshes of mixed elements to model complex geometry.! ❖ In computer memory they consist of several look-up tables. quadrilateral triangle hexahedron tetrahedron prism pyramid
  10. 10. Look-Up Tables for Meshes ❖ SOLVCON categorize the look-up tables into three: (i) coordinate, (ii) meta-data, and (iii) connectivity.! ❖ They are either 1-D or 2-D arrays.! ❖ These arrays contain “ghost cells” for boundary- condition treatments.! ❖ The ghost cells are outside the computing space.! ❖ The CESE method needs only one layer of ghost cells.
  11. 11. Domain Decomposition ❖ Millions or billions of unstructured mesh elements can be automatically decomposed into multiple blocks for parallel computing.! ❖ A graph is built for representing the connectivity among all the mesh elements.! ❖ The partitioned graphs are used to decompose the original mesh into thousands of blocks.
  12. 12. Two-Level Nested Loops ❖ The time-marching nature of the CESE method dictates a structure of two-level nested loops.! ❖ The outer loop is for time- marching.! ❖ Multiple inner loops “sweep” over the discretized space.! ❖ All facilities can built around this execution flow.
  13. 13. Module Structure
  14. 14. Case-Solver Relation ❖ the outer temporal loop.! ❖ solvcon.solver.MeshSolver: house of the inner spatial loops.
  15. 15. In-Situ Visualization/Analysis ❖ What’s in-situ visualization/analysis (ISV)?! ❖ Perform result visualization/analysis while time-marching.! ❖ Why in-situ?! ❖ [50 million-element mesh] * [10,000 time steps] * [6 single-precision solution fields (CFD)] mean 12 TB of data. It is one simulation case.! ❖ Segregating result analysis from simulation code is difficult.! ❖ Architecture of most research codes isn’t good enough for the segregation.! ❖ Most codes simply produce the data and use another software to post- process.! ❖ A common practice is to reduce the resolution.! ❖ Outputting every 100 time steps reduces the output dat to 120 GB.! ❖ But it defeats the purpose of high-resolution simulations.
  16. 16. SOLVCON’s Segregation ❖ SOLVCON provides companions of MeshCase and MeshSolver: MeshHook and MeshAnchor, respectively.! ❖ MeshCase and MeshSolver are for equation solutions.! ❖ MeshHook and MeshAnchor are for solution analysis.! ❖ And there’re still a lot to do.
  17. 17. Documenting ❖ SOLVCON documents should contain what’s in conventional research papers.! ❖ What we want is more than conventional papers.! ❖ Bi-directional linking between the code and the documents.! ❖ Some part of the documents should be runnable. It’s related to validation.
  18. 18. Sphinx ❖ Sphinx ( is the default tool for Python documentation.! ❖ ReadTheDocs ( is a service to host Sphinx documents.! ❖ SOLVCON authors and organizes its documents with Sphinx.! ❖ The documents are published at http:// (ReadTheDocs is behind the scenes).
  19. 19. Authoring, Editing, and Reviewing ❖ We use Mercurial for version control and BitBucket ( to host the project and its source code.! ❖ Reviewing the documents goes through the pull-request flow of the code review.! ❖ We treat the reStructuredText source files of the documents like the program source code.
  20. 20. Integrated Testing ❖ Code without verification can’t be taken as working code, no matter how carefully we wrote it.! ❖ With the testing system, refactoring and quick prototyping become possible.! ❖ Many research codes don’t have a regression testing system, so while coding the developers can’t be sure how the changes impact the system.! ❖ Two kinds of tests: unit tests and answer tests. Both need automation.
  21. 21. Unit Tests ❖ Unit tests are short and quick, and check for simple output of elementary API behaviors.! ❖ SOLVCON uses standard Python unittest module (! ❖ nose ( is used as the automatic test runner.
  22. 22. Answer Tests ! ❖ Answer tests check for the correctness of simulations. Usually an answer test is a meaningful simulation, and can serve as an example to how to use SOLVCON to simulate other problems. ! ❖ Jenkins ( is installed at http:// to automatically run the answer tests for every commit of SOLVCON in a testing farm.
  23. 23. Challenges (Conclusions) ❖ The simulations were done with older version of SOLVCON; it takes a lot of efforts to upgrade (ongoing).! ❖ Unfamiliarity of the collaborators to the authoring system (Sphinx):! ❖ Unification of code and document needs extra efforts.! ❖ Feeding back to the main framework.! ❖ Server farm for continuous integration.
  24. 24. Current/Future Works ❖ New physical processes:! ❖ solvcon.parcel.bulk: acoustic wave solver based on bulk modulus.! ❖ solvcon.parcel.vewave: visco-elastic wave solver.! ❖ Document the CESE method.! ❖ Documenting with iPython in addition to Sphinx?! ❖ Get more people to play together!