Docker for software development stack optimization
•
3 likes•1,331 views
The document describes using Docker containers to build and test different software stack configurations for OpenFOAM. Key points: 1. Dockerfiles are used to create base images with compilers then images adding MPI and OpenFOAM for iterative development and testing of configurations. 2. This allows isolating combinations of applications, libraries, compilers and MPI implementations to explore horizontal and vertical scaling across platforms. 3. A group development approach is suggested where different experts focus on specific components and an application developer combines them for testing.
Recommended
More Related Content
What's hot
What's hot (13)
Similar to Docker for software development stack optimization
Similar to Docker for software development stack optimization (20)
More from inside-BigData.com
More from inside-BigData.com (20)
Recently uploaded
Recently uploaded (20)
Docker for software development stack optimization
- 1. 109 207 246 Dark 1 255 255 255 Light 1 131 56 155 Dark 2 0 99 190 Light 2 85 165 28 Accent 1 214 73 42 Accent 2 185 175 164 Accent 3 151 75 7 Accent 4 193 187 0 Accent 5 255 221 62 Accent 6 255 255 255 Hyperlink 236 137 29 Followed Hyperlink 127 175 221 Tata Blue 50% 203 215 238 Tata Blue 25% 179 149 197 Purple 50 % 212 195 223 Purple 25 % 255 242 171 Yellow 50 % 255 249 213 Yellow 25 % 229 205 186 Brown 50 % 248 241 235 Brown 25 % 180 213 154 Green 50 % 214 231 200 Green 25 % 241 240 202 Light Green 50% 251 251 241 Light Green 25% Title and Content Docker for Software Development Nishant Agrawal & Ravi Ojha TCS Innovation Labs – Performance Engineering http://www.tcs.com/research Mumbai, India
- 2. 2 Challenges in S/W Development OpenFOAM 2.1.x OpenFOAM 2.2.x OpenFOAM 2.3.2 ICC GCC MVAPICH OpenMP Intel MPI OpenMP CILK TBB OpenFOAM 2.2.x ICC Intel MPI TBBRun each combination? 3 6 18 50
- 3. 3 Container Development Iteration towards End Result Rinse & Repeat Base Image Docker File Docker pull Run Docker Image Try 1 2 Update Dockerfile & build new Image SUCESS ERROR
- 4. 4 OpenFOAM Docker File Installed compilers on base image Build new image named “tcs/gcc” Build another image named “tcs/openmpi” 1. Installed openMPI incrementally2. Installed OpenFOAM3. FROM centos:centos6 Maintainer “Nishant Agrawal <nishant.agrawal@tcs.com>” Run echo “proxy=http://192.168.140.15:3128” >> /etc/yum.conf Run echo “proxy_username=username” >> /etc/yum.conf Run echo “proxy_password=password” >> /etc/yum.conf Run yum install –y gcc Run yum install –y g++ FROM tcs/gcc Maintainer “Nishant Agrawal <nishant.agrawal@tcs.com>” Run yum groupinstall –y ‘Development Tools’ Run yum install –y openmpi openmpi-devel qt zlib-devel cmake FROM tcs/openmpi Maintainer “Nishant Agrawal <nishant.agrawal@tcs.com>” RUN yum install -y yum-utils RUN yum-config-manager --nogpgcheck --add-repo http://dl.atrpms.net/el6-x86_64/atrpms/stable RUN yum install -y --nogpgcheck qtwebkit qtwebkit-devel RUN yum install -y --nogpgcheck CGAL CGAL-devel RUN yum install -y http://www.openfoam.org/download/rhel/6.5/x86_64/OpenFOAM-2.3.0- 1.x86_64.rpm
- 5. 5 Docker – Our OpenFOAM stack ICC 13.0 Intel MPI 5.0 ICC 13.0 Intel MPI 5.1 ICC 13.0 GCC 4.9.X OpenMPI 1.8.6 GCC 4.9.X OpenMPI 1.7.5 GCC 4.9.X ICC 13.0 Intel MPI 5.1 OpenMP 3 ICC 14.0 Intel MPI 5.1 OpenMP 4 ICC 14.0 Intel MPI 5.1 OpenMP 3.0 OpenFOAM 2.3.x ICC 14.0 Intel MPI 5.1 OpenMP 3.0 OpenFOAM 2.2.x Base Image . . . . . . . . . . . . . . . . . . . .
- 6. 6 Docker Software - stack Incremental Image
- 7. 7 General Benefits Creating isolated combinations of applications and third party libraries to be run as container. Such that application specific dependencies and exports limited within a container. Explore horizontal and vertical scaling by running isolated bundles of software setups across different platforms. Leverage the ideal configuration achieved to other projects.
- 8. 8 Group Development Standing on the shoulders of giants – Compiler NERD is in charge of compiler image – MPI expert creates MPI image – Threading experts build CLIK/OPENMP/TBB – Developer puts Application Layer Introduce an automated CI (Jenkins) Be productive after a short Docker introduction
- 9. 9 Docker – Portable solution Determine best software stack for a given hardware. – GPU/MIC – InfiniBand / OmniPath Images can be shipped to repository or saved if security concerns are there and loaded later. docker save <image>:<tagname> > image.tar.gz docker load –i image.tar.gz
- 10. 10 Docker – Scalable development solution Host/Server Run as many Docker containers on Host 1……N Containers running in parallel Docker1 Docker NDocker 4Docker 3Docker 2
- 11. 11 Thank You!