SlideShare a Scribd company logo
1 of 32
Download to read offline
Reduce, Reuse, Recycle:
Repeatable library images for Docker.
Steven Lembark
Workhorse Computing
lembark@wrkhors.com
Distributing Docker Images
Stock doc's use Ubuntu distro image.
Includes systemd, among other things.
Bloated
Distributing Docker Images
Why distro?
/bin?
/usr/bin?
/sbin?
/var/spool?
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Your executable + Ubuntu == runnable.
Distributing Docker Images
Why distro?
Shared Object Lib's (.so).
Your executable + Ubuntu != runnable.
Except when it isn't.
Docker is not a VM
Execuatables use versions of lib's.
Not all versions are [bug-] compatible.
RH/SuSE/Arch/Gentoo + Ubuntu == ???
libc differences are the worst.
What you need are your libs.
Q: How to distribute?
One approach: Dup /lib*, /usr/lib* .
4 /lib32
38 /lib64
372 /usr/lib32
2526 /usr/lib64
About 3GB on my notebook.
Development server? (ouch)
What you need is all you want
Q: What do you need?
What you need is all you want
Q: What do you need?
A: Ask.
ldd lists SO paths.
What you need: ask ldd
# Path to .so and virtual offset of entry point.
$ ldd /opt/bin/perl
linux-vdso.so.1 (0x00007ffee215e000)
libperl.so => /opt/perl/5.22/lib/5.22.0/x86_64-
linux/CORE/libperl.so (0x00007f6b40026000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6b3fe0a000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6b3fbf2000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f6b3f9ee000)
libm.so.6 => /lib64/libm.so.6 (0x00007f6b3f6f2000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f6b3f4bb000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f6b3f2b8000)
libc.so.6 => /lib64/libc.so.6 (0x00007f6b3ef1d000)
/lib64/ld-linux-x86-64.so.2 (0x00007f6b40417000)
What don't you need?
Then again, not everything needs shared lib's:
$ ldd /opt/perl/lib/5.22.0/NEXT.pm
ldd: warning: you do not have execution permission for
`/opt/perl/lib/5.22.0/NEXT.pm'
not a dynamic executable
Q: How do you ask for what you need?
More than one way...
1. Scan it all.
Scan all of perl5's lib.
ldd everything.
filter out rejects.
uniq the rest.
Good: One image for everything.
Bad: Big (?)
2. Scan current
Construct docker's source directory.
Scan that with ldd.
App From "-libs" image.
Good: Small.
Bad: More work.
2. Scan current
Construct docker's source directory.
Scan that with ldd.
App From "-libs" image.
Good: Small.
Bad: More work.
Good: Easy to automate.
Q: Which PM's do I need?
A: Ask perl.
%INC has all of your module paths.
x %INC
0 HASH(0x24d8070)
'/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-
linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' =>
'/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-
linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix'
'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm'
'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
Q: Which PM's do I need?
A: Ask perl.
%INC has all of your module paths.
Hard to export for arbitrary program.
x %INC
0 HASH(0x24d8070)
'/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-
linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' =>
'/opt/perl/5.22/lib/site_perl/5.22.0/x86_64-
linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix'
'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm'
'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
So: Bulk is it.
cd /opt/perl;
find -H lib bin -type f |
xargs ldd 2>/dev null |
grep '.so' | grep -v ':$';
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff244e37000)
/lib64/ld-linux-x86-64.so.2 (0x00005622e44ad000)
linux-vdso.so.1 (0x00007fffe4075000)
libc.so.6 => /lib64/libc.so.6 (0x00007f92f978f000)
/lib64/ld-linux-x86-64.so.2 (0x00005622e63a1000)
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)
Q: Why no path?
A: This is a kernel binding.
No file on disk.
Menagrie of sharing
linux-vdso.so.1 (0x00007ffe8d3f6000)
Q: Why no path?
A: This is a kernel binding.
No file on disk.
Be careful what you search for!
ldd uses fat commas
find -H /opt/perl/lib |
xargs ldd 2>/dev/null |
grep '=>' | cut -d ' ' -f3|
sort -d | uniq ;
ldd uses fat commas
Except when it doesn't:
/lib64/libresolv.so.2
/lib64/librt.so.1
/lib64/libtinfo.so.6
/lib64/libutil.so.1
/lib64/libz.so.1
not
/usr/lib32/libatk-1.0.so.0
/usr/lib32/libbz2.so.1
/usr/lib32/libcairo.so.2
cpio copies paths
Copy to build dir.
cpio ignores "not", keeps going.
#!/bin/bash
cd $(dirname $0);
dirs='/opt/perl/{lib,bin}';
find $dirs | xargs ldd | cut | sort | uniq |
cpio -pdv . ;
Result:
$ ls -a -1
.dockerignore
build
Dockerfile
lib32
lib64
usr
$ du -msx; # compare to 3000MB!
8
Dockerfile
FROM empty
MAINTAINER lembark@wrkhors.com
Alternative:
FROM busybox
.dockerignore
One line: build
Command line
$ docker build --rm . 
http://host:5000/lembark/perl-so;
Not so bad.
build script
#!/bin/bash
cd $(dirname $0);
name=$(basename $(dirname $PWD));
find -H /opt/perl/{bin,lib} -type f |
xargs ldd 2>/dev/null |
grep '=>' | cut -d' ' -f3 |
uniq | sort -d | uniq |
cpio -pd . ;
docker build --rm . 
http://localhost:5000/$(whoami)/$name;
perl-exec from perl-so
FROM .../lembark/perl-so
All the libs you need.
In one place.
Summary
Code requires SO's.
If SO's match things work.
Get what you need using ldd.
Dockerize perl from SO image.
Result: Truly Lazy Docker.

More Related Content

What's hot

2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku
ronnywang_tw
 

What's hot (20)

DevOps(2) : Vagrant - (MOSG)
DevOps(2) : Vagrant  -  (MOSG)DevOps(2) : Vagrant  -  (MOSG)
DevOps(2) : Vagrant - (MOSG)
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
 
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
 
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
 
DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)DevOps(4) : Ansible(2) - (MOSG)
DevOps(4) : Ansible(2) - (MOSG)
 
Test-Driven Puppet Development - PuppetConf 2014
Test-Driven Puppet Development - PuppetConf 2014Test-Driven Puppet Development - PuppetConf 2014
Test-Driven Puppet Development - PuppetConf 2014
 
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
WordPress Plugin Unit Tests (FR - WordCamp Paris 2015)
 
Designing net-aws-glacier
Designing net-aws-glacierDesigning net-aws-glacier
Designing net-aws-glacier
 
Puppet loves RSpec, why you should, too
Puppet loves RSpec, why you should, tooPuppet loves RSpec, why you should, too
Puppet loves RSpec, why you should, too
 
Asynchronous Systems with Fn Flow
Asynchronous Systems with Fn FlowAsynchronous Systems with Fn Flow
Asynchronous Systems with Fn Flow
 
Lessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containersLessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containers
 
Py conkr 20150829_docker-python
Py conkr 20150829_docker-pythonPy conkr 20150829_docker-python
Py conkr 20150829_docker-python
 
DevOps with Fabric
DevOps with FabricDevOps with Fabric
DevOps with Fabric
 
Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014
 
2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku
 
Nginx Workshop Aftermath
Nginx Workshop AftermathNginx Workshop Aftermath
Nginx Workshop Aftermath
 
Puppet @ Seat
Puppet @ SeatPuppet @ Seat
Puppet @ Seat
 
Concurrency in Python
Concurrency in PythonConcurrency in Python
Concurrency in Python
 
Docker, c'est bonheur !
Docker, c'est bonheur !Docker, c'est bonheur !
Docker, c'est bonheur !
 
Ex407
Ex407Ex407
Ex407
 

Viewers also liked

ACG_09_Advertorial
ACG_09_AdvertorialACG_09_Advertorial
ACG_09_Advertorial
Tony Wayne
 
Get a good education
Get a good educationGet a good education
Get a good education
blockdavid44
 
Habilidade h13 definitivo
Habilidade h13 definitivoHabilidade h13 definitivo
Habilidade h13 definitivo
Ludz_Tamboro
 

Viewers also liked (14)

ACG_09_Advertorial
ACG_09_AdvertorialACG_09_Advertorial
ACG_09_Advertorial
 
Get a good education
Get a good educationGet a good education
Get a good education
 
Pagina 1
Pagina   1Pagina   1
Pagina 1
 
Ingenieria economica
Ingenieria economicaIngenieria economica
Ingenieria economica
 
Seres vivos
Seres vivos Seres vivos
Seres vivos
 
91ROCKweb formatos
91ROCKweb formatos91ROCKweb formatos
91ROCKweb formatos
 
Imperi napoleònic i restauració
Imperi napoleònic i restauracióImperi napoleònic i restauració
Imperi napoleònic i restauració
 
Final Copy French Culture Project and copyright use
Final Copy French Culture Project and copyright useFinal Copy French Culture Project and copyright use
Final Copy French Culture Project and copyright use
 
PJM 3102 Pergerakan Asas (SEMESTER 2)
PJM 3102 Pergerakan Asas (SEMESTER 2)PJM 3102 Pergerakan Asas (SEMESTER 2)
PJM 3102 Pergerakan Asas (SEMESTER 2)
 
project report file on telecommunication(report file on vodafone)
project report file on telecommunication(report file on vodafone)project report file on telecommunication(report file on vodafone)
project report file on telecommunication(report file on vodafone)
 
Habilidade h13 definitivo
Habilidade h13 definitivoHabilidade h13 definitivo
Habilidade h13 definitivo
 
Branding - marka wnetrz
Branding - marka wnetrzBranding - marka wnetrz
Branding - marka wnetrz
 
Bolero musica guitarra1233289 hot
Bolero musica guitarra1233289 hotBolero musica guitarra1233289 hot
Bolero musica guitarra1233289 hot
 
Uso de las tics
Uso de las ticsUso de las tics
Uso de las tics
 

Similar to Shared Object images in Docker: What you need is what you want.

Developing and Deploying PHP with Docker
Developing and Deploying PHP with DockerDeveloping and Deploying PHP with Docker
Developing and Deploying PHP with Docker
Patrick Mizer
 
Os Grossupdated
Os GrossupdatedOs Grossupdated
Os Grossupdated
oscon2007
 

Similar to Shared Object images in Docker: What you need is what you want. (20)

6202942
62029426202942
6202942
 
Puppet control-repo 
to the next level
Puppet control-repo 
to the next levelPuppet control-repo 
to the next level
Puppet control-repo 
to the next level
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Zenoh Tutorial
Zenoh TutorialZenoh Tutorial
Zenoh Tutorial
 
Puppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction KitPuppet Systems Infrastructure Construction Kit
Puppet Systems Infrastructure Construction Kit
 
Introduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud RunIntroduction to JIB and Google Cloud Run
Introduction to JIB and Google Cloud Run
 
Packaging perl (LPW2010)
Packaging perl (LPW2010)Packaging perl (LPW2010)
Packaging perl (LPW2010)
 
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
 
Developing IT infrastructures with Puppet
Developing IT infrastructures with PuppetDeveloping IT infrastructures with Puppet
Developing IT infrastructures with Puppet
 
Take care of hundred containers and not go crazy
Take care of hundred containers and not go crazyTake care of hundred containers and not go crazy
Take care of hundred containers and not go crazy
 
App container rkt
App container rktApp container rkt
App container rkt
 
PHP selber bauen
PHP selber bauenPHP selber bauen
PHP selber bauen
 
Developing and Deploying PHP with Docker
Developing and Deploying PHP with DockerDeveloping and Deploying PHP with Docker
Developing and Deploying PHP with Docker
 
Docker 활용법: dumpdocker
Docker 활용법: dumpdockerDocker 활용법: dumpdocker
Docker 활용법: dumpdocker
 
Composer - The missing package manager for PHP
Composer - The missing package manager for PHPComposer - The missing package manager for PHP
Composer - The missing package manager for PHP
 
maXbox Starter87
maXbox Starter87maXbox Starter87
maXbox Starter87
 
Os Grossupdated
Os GrossupdatedOs Grossupdated
Os Grossupdated
 
Docker module 1
Docker module 1Docker module 1
Docker module 1
 
codemotion-docker-2014
codemotion-docker-2014codemotion-docker-2014
codemotion-docker-2014
 

More from Workhorse Computing

More from Workhorse Computing (20)

Wheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility ModulesWheels we didn't re-invent: Perl's Utility Modules
Wheels we didn't re-invent: Perl's Utility Modules
 
mro-every.pdf
mro-every.pdfmro-every.pdf
mro-every.pdf
 
Paranormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add UpParanormal statistics: Counting What Doesn't Add Up
Paranormal statistics: Counting What Doesn't Add Up
 
The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.The $path to knowledge: What little it take to unit-test Perl.
The $path to knowledge: What little it take to unit-test Perl.
 
Generating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in PosgresqlGenerating & Querying Calendar Tables in Posgresql
Generating & Querying Calendar Tables in Posgresql
 
Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!Hypers and Gathers and Takes! Oh my!
Hypers and Gathers and Takes! Oh my!
 
BSDM with BASH: Command Interpolation
BSDM with BASH: Command InterpolationBSDM with BASH: Command Interpolation
BSDM with BASH: Command Interpolation
 
Findbin libs
Findbin libsFindbin libs
Findbin libs
 
Memory Manglement in Raku
Memory Manglement in RakuMemory Manglement in Raku
Memory Manglement in Raku
 
BASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic InterpolationBASH Variables Part 1: Basic Interpolation
BASH Variables Part 1: Basic Interpolation
 
Metadata-driven Testing
Metadata-driven TestingMetadata-driven Testing
Metadata-driven Testing
 
The W-curve and its application.
The W-curve and its application.The W-curve and its application.
The W-curve and its application.
 
Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.Keeping objects healthy with Object::Exercise.
Keeping objects healthy with Object::Exercise.
 
Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.Perl6 Regexen: Reduce the line noise in your code.
Perl6 Regexen: Reduce the line noise in your code.
 
Smoking docker
Smoking dockerSmoking docker
Smoking docker
 
Getting Testy With Perl6
Getting Testy With Perl6Getting Testy With Perl6
Getting Testy With Perl6
 
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
Neatly Hashing a Tree: FP tree-fold in Perl5 & Perl6
 
Neatly folding-a-tree
Neatly folding-a-treeNeatly folding-a-tree
Neatly folding-a-tree
 
Paranormal stats
Paranormal statsParanormal stats
Paranormal stats
 
Selenium sandwich-3: Being where you aren't.
Selenium sandwich-3: Being where you aren't.Selenium sandwich-3: Being where you aren't.
Selenium sandwich-3: Being where you aren't.
 

Recently uploaded

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

Shared Object images in Docker: What you need is what you want.

  • 1. Reduce, Reuse, Recycle: Repeatable library images for Docker. Steven Lembark Workhorse Computing lembark@wrkhors.com
  • 2. Distributing Docker Images Stock doc's use Ubuntu distro image. Includes systemd, among other things. Bloated
  • 3. Distributing Docker Images Why distro? /bin? /usr/bin? /sbin? /var/spool?
  • 4. Distributing Docker Images Why distro? Shared Object Lib's (.so).
  • 5. Distributing Docker Images Why distro? Shared Object Lib's (.so). Your executable + Ubuntu == runnable.
  • 6. Distributing Docker Images Why distro? Shared Object Lib's (.so). Your executable + Ubuntu != runnable. Except when it isn't.
  • 7. Docker is not a VM Execuatables use versions of lib's. Not all versions are [bug-] compatible. RH/SuSE/Arch/Gentoo + Ubuntu == ??? libc differences are the worst. What you need are your libs.
  • 8. Q: How to distribute? One approach: Dup /lib*, /usr/lib* . 4 /lib32 38 /lib64 372 /usr/lib32 2526 /usr/lib64 About 3GB on my notebook. Development server? (ouch)
  • 9. What you need is all you want Q: What do you need?
  • 10. What you need is all you want Q: What do you need? A: Ask. ldd lists SO paths.
  • 11. What you need: ask ldd # Path to .so and virtual offset of entry point. $ ldd /opt/bin/perl linux-vdso.so.1 (0x00007ffee215e000) libperl.so => /opt/perl/5.22/lib/5.22.0/x86_64- linux/CORE/libperl.so (0x00007f6b40026000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6b3fe0a000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6b3fbf2000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f6b3f9ee000) libm.so.6 => /lib64/libm.so.6 (0x00007f6b3f6f2000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f6b3f4bb000) libutil.so.1 => /lib64/libutil.so.1 (0x00007f6b3f2b8000) libc.so.6 => /lib64/libc.so.6 (0x00007f6b3ef1d000) /lib64/ld-linux-x86-64.so.2 (0x00007f6b40417000)
  • 12. What don't you need? Then again, not everything needs shared lib's: $ ldd /opt/perl/lib/5.22.0/NEXT.pm ldd: warning: you do not have execution permission for `/opt/perl/lib/5.22.0/NEXT.pm' not a dynamic executable
  • 13. Q: How do you ask for what you need? More than one way...
  • 14. 1. Scan it all. Scan all of perl5's lib. ldd everything. filter out rejects. uniq the rest. Good: One image for everything. Bad: Big (?)
  • 15. 2. Scan current Construct docker's source directory. Scan that with ldd. App From "-libs" image. Good: Small. Bad: More work.
  • 16. 2. Scan current Construct docker's source directory. Scan that with ldd. App From "-libs" image. Good: Small. Bad: More work. Good: Easy to automate.
  • 17. Q: Which PM's do I need? A: Ask perl. %INC has all of your module paths. x %INC 0 HASH(0x24d8070) '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64- linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' => '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64- linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' 'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm' 'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
  • 18. Q: Which PM's do I need? A: Ask perl. %INC has all of your module paths. Hard to export for arbitrary program. x %INC 0 HASH(0x24d8070) '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64- linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' => '/opt/perl/5.22/lib/site_perl/5.22.0/x86_64- linux/auto/Term/ReadLine/Gnu/XS/autosplit.ix' 'AutoLoader.pm' => '/opt/perl/5.22/lib/5.22.0/AutoLoader.pm' 'B.pm' => '/opt/perl/5.22/lib/5.22.0/x86_64-linux/B.pm'
  • 19. So: Bulk is it. cd /opt/perl; find -H lib bin -type f | xargs ldd 2>/dev null | grep '.so' | grep -v ':$';
  • 20. Menagrie of sharing linux-vdso.so.1 (0x00007ffe8d3f6000) libc.so.6 => /lib64/libc.so.6 (0x00007ff244e37000) /lib64/ld-linux-x86-64.so.2 (0x00005622e44ad000) linux-vdso.so.1 (0x00007fffe4075000) libc.so.6 => /lib64/libc.so.6 (0x00007f92f978f000) /lib64/ld-linux-x86-64.so.2 (0x00005622e63a1000)
  • 21. Menagrie of sharing linux-vdso.so.1 (0x00007ffe8d3f6000) Q: Why no path? A: This is a kernel binding. No file on disk.
  • 22. Menagrie of sharing linux-vdso.so.1 (0x00007ffe8d3f6000) Q: Why no path? A: This is a kernel binding. No file on disk. Be careful what you search for!
  • 23. ldd uses fat commas find -H /opt/perl/lib | xargs ldd 2>/dev/null | grep '=>' | cut -d ' ' -f3| sort -d | uniq ;
  • 24. ldd uses fat commas Except when it doesn't: /lib64/libresolv.so.2 /lib64/librt.so.1 /lib64/libtinfo.so.6 /lib64/libutil.so.1 /lib64/libz.so.1 not /usr/lib32/libatk-1.0.so.0 /usr/lib32/libbz2.so.1 /usr/lib32/libcairo.so.2
  • 25. cpio copies paths Copy to build dir. cpio ignores "not", keeps going. #!/bin/bash cd $(dirname $0); dirs='/opt/perl/{lib,bin}'; find $dirs | xargs ldd | cut | sort | uniq | cpio -pdv . ;
  • 26. Result: $ ls -a -1 .dockerignore build Dockerfile lib32 lib64 usr $ du -msx; # compare to 3000MB! 8
  • 29. Command line $ docker build --rm . http://host:5000/lembark/perl-so; Not so bad.
  • 30. build script #!/bin/bash cd $(dirname $0); name=$(basename $(dirname $PWD)); find -H /opt/perl/{bin,lib} -type f | xargs ldd 2>/dev/null | grep '=>' | cut -d' ' -f3 | uniq | sort -d | uniq | cpio -pd . ; docker build --rm . http://localhost:5000/$(whoami)/$name;
  • 31. perl-exec from perl-so FROM .../lembark/perl-so All the libs you need. In one place.
  • 32. Summary Code requires SO's. If SO's match things work. Get what you need using ldd. Dockerize perl from SO image. Result: Truly Lazy Docker.