SlideShare a Scribd company logo
1 of 54
Download to read offline
“warpdrive”, making Python
web application
deployment magically easy.
Graham Dumpleton
@GrahamDumpleton
PyCon New Zealand - September 2016
Is deploying Python web
applications too hard?
$ virtualenv venv
New python executable in /usr/local/www/mysite/venv/bin/python
Installing setuptools, pip, wheel...done.
$ source venv/bin/activate
(venv) $ pip install Django
Collecting Django
Using cached Django-1.9.7-py2.py3-none-any.whl
Installing collected packages: Django
Successfully installed Django-1.9.7
(venv) $ python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
You have unapplied migrations; your app may not work
properly until they are applied.
Run 'python manage.py migrate' to apply them.
June 17, 2016 - 01:02:22
Django version 1.9.7, using settings 'hello_world.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
(venv) $ python manage.py migrate
Operations to perform:
Apply all migrations: admin, contenttypes, auth, sessions
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying sessions.0001_initial... OK
(venv) $ python manage.py createsuperuser
Username (leave blank to use 'graham'): grumpy
Email address: grumpy@example.com
Password:
Password (again):
Superuser created successfully.
(venv) $ python manage.py runserver 0.0.0.0:8080
Performing system checks...
System check identified no issues (0 silenced).
June 17, 2016 - 01:06:17
Django version 1.9.7, using settings 'hello_world.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.
<VirtualHost *:80>
ServerName www.example.com
Alias /static/ /usr/local/www/mysite/static/
<Directory /usr/local/www/mysite/static>
Require all granted
</Directory>



WSGIDaemonProcess mysite threads=5 
request-timeout=30 queue-timeout=45 
python-home=/usr/local/www/mysite/venv 

python-path=/usr/local/www/mysite
WSGIScriptAlias / /usr/local/www/mysite/mysite/wsgi.py 

process-group=mysite application-group=%{GLOBAL}
<Directory /usr/local/www/mysite/mysite>
Require all granted
</Directory>
</VirtualHost>
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
(venv) $ python manage.py collectstatic --noinput
Copying ‘/…/static/admin/css/base.css’
…
56 static files copied to ‘/…/static’.
• Initialisation steps
• Create Python virtual environment
• Activate Python virtual environment
• Build steps
• Install required Python packages
• Collect together static file assets
• Deployment steps
• Initialise or migrate database data
• Configure and start the WSGI server
Why should you need to
care about the details?
What if all the build and
deployment steps were
managed for you?
“warpdrive” is the

magic glue that can
make that possible!
For local development,
as well as production
deployments.
Initialisation steps
$ warpdrive project django
Build steps
(warpdrive+django) $ warpdrive build
-----> Installing dependencies with pip (requirements.txt)
Collecting Django (from -r requirements.txt (line 1))
Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB)
100% |████████████████████████████████| 6.6MB 365kB/s
Collecting mod_wsgi (from -r requirements.txt (line 2))
Downloading mod_wsgi-4.5.2.tar.gz (1.8MB)
100% |████████████████████████████████| 1.8MB 837kB/s
Installing collected packages: Django, mod-wsgi
Running setup.py install for mod-wsgi ... done
Successfully installed Django-1.9.7 mod-wsgi-4.5.2
-----> Collecting static files for Django
+ python manage.py collectstatic --noinput
Copying ‘/…/base.css’
56 static files copied to ‘/…/static’.
Configure and start the
WSGI server
(warpdrive+django) $ warpdrive start
-----> Configuring for deployment type of 'auto'
-----> Default WSGI server type is 'mod_wsgi'
-----> Running server script start-mod_wsgi
-----> Executing server command 'mod_wsgi-express start-server
--log-to-terminal --startup-log --port 8080 --application-type
module --entry-point hello_world.wsgi --callable-object
application --url-alias /static/ /usr/local/www/mysite/static'
[Sun Jun 19 22:00:59.819762 2016] [mpm_prefork:notice] [pid
67483] AH00163: Apache/2.4.18 (Unix) mod_wsgi/4.5.2 Python/2.7.10
configured -- resuming normal operations
Automatic hosting detection
• shell -> app.sh
• python -> app.py
• wsgi -> wsgi.py
• django -> manage.py
Choice of WSGI servers
• mod_wsgi
• gunicorn
• uwsgi
• waitress
Initialise the database
(warpdrive+django) $ warpdrive setup
-----> Running .warpdrive/action_hooks/setup
-----> Checking database is running
-----> Initialising database.
Operations to perform:
Apply all migrations: admin, contenttypes, auth, sessions
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
…
Applying sessions.0001_initial... OK
-----> Running Django super user creation
Username (leave blank to use 'graham'): grumpy
Email address: grumpy@example.com
Password:
Password (again):
Superuser created successfully.
Migrate the database
(warpdrive+django) $ warpdrive migrate
-----> Running .warpdrive/action_hooks/migrate
-----> Checking database is running
-----> Running Django database migration
Operations to perform:
Apply all migrations: admin, contenttypes, auth, sessions
Running migrations:
No migrations to apply.
Database migration hook.warpdrive/action_hooks/migrate
#!/bin/bash
CHECK_DATABASE="$WARPDRIVE_SRC_ROOT/scripts/check-database.py"
case "$DATABASE_URL" in
sqlite:*)
;;
*)
(cat - | python manage.py shell) << !
import runpy
_ = runpy.run_path('$CHECK_DATABASE')
!
;;
esac
echo " -----> Running Django database migration"
python manage.py migrate
Action hooks
• pre-build
• build-env
• build
• deploy-env
• deploy-cfg
• deploy
• setup
• migrate
• verify
• ready
• alive
Interactive shell
(warpdrive+django) $ warpdrive shell
bash-3.2$ python manage.py migrate
Command execution
(warpdrive+django) $ warpdrive exec env | grep WARPDRIVE
WARPDRIVE_SRC_ROOT=/Users/graham/Projects/warpdrive-django-
auto
WARPDRIVE_APP_ROOT=/Users/graham/.warpdrive/warpdrive
+django
WARPDRIVE_ACTION=exec
WARPDRIVE_VERSION=0.20.1
WARPDRIVE_HTTP_PORT=8080
WARPDRIVE_ENV_NAME=django
Create Docker image
(warpdrive+django) $ warpdrive image django
I0619 22:14:22.783544 67609 install.go:251] Using "assemble" installed from
"image:///opt/app-root/s2i/bin/assemble"
I0619 22:14:22.783688 67609 install.go:251] Using "run" installed from
"image:///opt/app-root/s2i/bin/run"
I0619 22:14:22.783712 67609 install.go:251] Using "save-artifacts" installed
from "image:///opt/app-root/s2i/bin/save-artifacts"
---> Installing application source
---> Building application from source
-----> Installing dependencies with pip (requirements.txt)
Collecting Django (from -r requirements.txt (line 1))
Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB)
Installing collected packages: Django
Successfully installed Django-1.9.7
-----> Collecting static files for Django
Copying ‘…/urlify.js’
…
56 static files copied to '/opt/app-root/src/static'.
---> Fix permissions on application source
Run Docker image
$ docker run --rm -p 8080:8080 django
---> Executing the start up script
-----> Configuring for deployment type of 'auto'
-----> Default WSGI server type is 'mod_wsgi'
-----> Running server script start-mod_wsgi
-----> Executing server command 'mod_wsgi-express start-server --log-to-terminal --
startup-log --port 8080 --application-type module --entry-point hello_world.wsgi --
callable-object application --url-alias /static/ /opt/app-root/src/static/'
[Sun Jun 19 07:44:57.955455 2016] [mpm_event:notice] [pid 48:tid 139683988789312]
AH00489: Apache/2.4.6 (CentOS) mod_wsgi/4.5.2 Python/3.4.2 configured -- resuming
normal operations
Manually build image
FROM grahamdumpleton/warp0-centos7-python34
COPY . ${WARPDRIVE_SRC_ROOT}
RUN warpdrive build
CMD [ "warpdrive", "start" ]
Source to Image (S2I)
https://github.com/openshift/source-to-image
$ s2i build https://github.com/GrahamDumpleton/warpdrive-django-auto.git
grahamdumpleton/warp0-centos7-python34 django
I0619 17:34:38.784337 66356 docker.go:352] Image "grahamdumpleton/warp0-centos7-
python34:latest" not available locally, pulling ...
I0619 17:40:23.793610 66356 clone.go:32] Downloading "https://github.com/
GrahamDumpleton/warpdrive-django-modwsgi.git" ...
I0619 17:40:25.979028 66356 install.go:251] Using "assemble" installed from
"image:///opt/app-root/s2i/bin/assemble"
I0619 17:40:25.979075 66356 install.go:251] Using "run" installed from "image:///
opt/app-root/s2i/bin/run"
I0619 17:40:25.979099 66356 install.go:251] Using "save-artifacts" installed from
"image:///opt/app-root/s2i/bin/save-artifacts"
---> Installing application source
---> Building application from source
-----> Installing dependencies with pip (requirements.txt)
Collecting Django (from -r requirements.txt (line 1))
Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB)
…
E0619 17:40:51.152317 66356 util.go:91] + python manage.py collectstatic --noinput
-----> Collecting static files for Django
Copying ‘/…/urlify.js’
…
56 static files copied to '/opt/app-root/src/static'.
---> Fix permissions on application source
$ oc new-app warpdrive-python34 --param APPLICATION_NAME=django --param
REPOSITORY_URL=https://github.com/GrahamDumpleton/warpdrive-django-auto.git
--> Deploying template warpdrive-python34 for "warpdrive-python34"
With parameters:
Name=django
Git Repository URL=https://github.com/GrahamDumpleton/warpdrive-django-
auto.git
Hostname=
Deployment Mode=auto
Server Type=mod_wsgi
Shell Script=app.sh
Python Script=app.py
Application Module=wsgi
Application Callable=application
Static URL=
Static Root=
Builder Version=
Script Debugging=
--> Creating resources with label app=django ...
imagestream "django" created
buildconfig "django" created
deploymentconfig "django" created
service "django" created
route "django" created
--> Success
Build scheduled, use 'oc logs -f bc/django' to track its progress.
Run 'oc status' to view your app.
Integration possibilities
• Direct to a host
• Using Docker or Rocket
• Kubernetes/OpenShift cluster
• Other container platforms
• Legacy PaaS environments
The goals!
Best of breed Docker
image for Python web
application deployment
Integrated builder scripts which
handle application image creation
and running of the application
Local development

using the same scripts
and workflow
Works with major
hosting services
What I am looking for!
Feedback.
Reviewers.
Graham.Dumpleton@gmail.com
@GrahamDumpleton
www.getwarped.org
https://github.com/GrahamDumpleton/warpdrive
blog.dscpl.com.au
https://www.openshift.com/promotions/for-developers.html

More Related Content

What's hot

Django deployment best practices
Django deployment best practicesDjango deployment best practices
Django deployment best practices
Erik LaBianca
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
Puppet
 

What's hot (20)

Django deployment best practices
Django deployment best practicesDjango deployment best practices
Django deployment best practices
 
Augeas, swiss knife resources for your puppet tree
Augeas, swiss knife resources for your puppet treeAugeas, swiss knife resources for your puppet tree
Augeas, swiss knife resources for your puppet tree
 
Preparation study of_docker - (MOSG)
Preparation study of_docker  - (MOSG)Preparation study of_docker  - (MOSG)
Preparation study of_docker - (MOSG)
 
Zero Downtime Deployment with Ansible
Zero Downtime Deployment with AnsibleZero Downtime Deployment with Ansible
Zero Downtime Deployment with Ansible
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 Edition
 
Getting started with Ansible
Getting started with AnsibleGetting started with Ansible
Getting started with Ansible
 
Towards Continuous Deployment with Django
Towards Continuous Deployment with DjangoTowards Continuous Deployment with Django
Towards Continuous Deployment with Django
 
Dev ninja -> vagrant + virtualbox + chef-solo + git + ec2
Dev ninja  -> vagrant + virtualbox + chef-solo + git + ec2Dev ninja  -> vagrant + virtualbox + chef-solo + git + ec2
Dev ninja -> vagrant + virtualbox + chef-solo + git + ec2
 
Python virtualenv & pip in 90 minutes
Python virtualenv & pip in 90 minutesPython virtualenv & pip in 90 minutes
Python virtualenv & pip in 90 minutes
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
 
10 Million hits a day with WordPress using a $15 VPS
10 Million hits a day  with WordPress using a $15 VPS10 Million hits a day  with WordPress using a $15 VPS
10 Million hits a day with WordPress using a $15 VPS
 
DevOps(2) : Vagrant - (MOSG)
DevOps(2) : Vagrant  -  (MOSG)DevOps(2) : Vagrant  -  (MOSG)
DevOps(2) : Vagrant - (MOSG)
 
Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.
 
Automate with Ansible basic (2/e, English)
Automate with Ansible basic (2/e, English)Automate with Ansible basic (2/e, English)
Automate with Ansible basic (2/e, English)
 
Django Deployment-in-AWS
Django Deployment-in-AWSDjango Deployment-in-AWS
Django Deployment-in-AWS
 
Ansible Meetup Hamburg / Quickstart
Ansible Meetup Hamburg / QuickstartAnsible Meetup Hamburg / Quickstart
Ansible Meetup Hamburg / Quickstart
 
Ansible new paradigms for orchestration
Ansible new paradigms for orchestrationAnsible new paradigms for orchestration
Ansible new paradigms for orchestration
 
Instruction: dev environment
Instruction: dev environmentInstruction: dev environment
Instruction: dev environment
 
Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)
 
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
 

Similar to “warpdrive”, making Python web application deployment magically easy.

Deploying
DeployingDeploying
Deploying
soon
 
Deployment with Fabric
Deployment with FabricDeployment with Fabric
Deployment with Fabric
andymccurdy
 
Creating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on HerokuCreating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on Heroku
Joe Kutner
 
2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku
ronnywang_tw
 

Similar to “warpdrive”, making Python web application deployment magically easy. (20)

Heroku pycon
Heroku pyconHeroku pycon
Heroku pycon
 
Deploying
DeployingDeploying
Deploying
 
Deploying Symfony | symfony.cat
Deploying Symfony | symfony.catDeploying Symfony | symfony.cat
Deploying Symfony | symfony.cat
 
Django Architecture Introduction
Django Architecture IntroductionDjango Architecture Introduction
Django Architecture Introduction
 
Building a Django App on Heroku
Building a Django App on HerokuBuilding a Django App on Heroku
Building a Django App on Heroku
 
Continuous Delivery w projekcie Open Source - Marcin Stachniuk - DevCrowd 2017
Continuous Delivery w projekcie Open Source - Marcin Stachniuk - DevCrowd 2017Continuous Delivery w projekcie Open Source - Marcin Stachniuk - DevCrowd 2017
Continuous Delivery w projekcie Open Source - Marcin Stachniuk - DevCrowd 2017
 
Continuous delivery w projekcie open source - Marcin Stachniuk
Continuous delivery w projekcie open source - Marcin StachniukContinuous delivery w projekcie open source - Marcin Stachniuk
Continuous delivery w projekcie open source - Marcin Stachniuk
 
Deploying Django with Ansible
Deploying Django with AnsibleDeploying Django with Ansible
Deploying Django with Ansible
 
Django deployment with PaaS
Django deployment with PaaSDjango deployment with PaaS
Django deployment with PaaS
 
Django
DjangoDjango
Django
 
Nginx3
Nginx3Nginx3
Nginx3
 
Deployment with Fabric
Deployment with FabricDeployment with Fabric
Deployment with Fabric
 
Intro django
Intro djangoIntro django
Intro django
 
Creating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on HerokuCreating Scalable JVM/Java Apps on Heroku
Creating Scalable JVM/Java Apps on Heroku
 
Instrumentación de entrega continua con Gitlab
Instrumentación de entrega continua con GitlabInstrumentación de entrega continua con Gitlab
Instrumentación de entrega continua con Gitlab
 
Continuous Delivery: The Next Frontier
Continuous Delivery: The Next FrontierContinuous Delivery: The Next Frontier
Continuous Delivery: The Next Frontier
 
J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"
J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"
J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"
 
Continous Delivering a PHP application
Continous Delivering a PHP applicationContinous Delivering a PHP application
Continous Delivering a PHP application
 
2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku2012 coscup - Build your PHP application on Heroku
2012 coscup - Build your PHP application on Heroku
 
Arbeiten mit distribute, pip und virtualenv
Arbeiten mit distribute, pip und virtualenvArbeiten mit distribute, pip und virtualenv
Arbeiten mit distribute, pip und virtualenv
 

More from Graham Dumpleton

Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.
Graham Dumpleton
 
PyCon US 2012 - Web Server Bottlenecks and Performance Tuning
PyCon US 2012 - Web Server Bottlenecks and Performance TuningPyCon US 2012 - Web Server Bottlenecks and Performance Tuning
PyCon US 2012 - Web Server Bottlenecks and Performance Tuning
Graham Dumpleton
 
DjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New RelicDjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New Relic
Graham Dumpleton
 

More from Graham Dumpleton (13)

Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.
 
Not Tom Eastman
Not Tom EastmanNot Tom Eastman
Not Tom Eastman
 
Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.
 
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
 
OpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaSOpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaS
 
Automated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and KubernetesAutomated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and Kubernetes
 
PyCon HK 2015 - Monitoring the performance of python web applications
PyCon HK 2015 -  Monitoring the performance of python web applicationsPyCon HK 2015 -  Monitoring the performance of python web applications
PyCon HK 2015 - Monitoring the performance of python web applications
 
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
PyCon AU 2015  - Using benchmarks to understand how wsgi servers workPyCon AU 2015  - Using benchmarks to understand how wsgi servers work
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
 
PyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating DecoratorsPyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating Decorators
 
PyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applicationsPyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applications
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web Applications
 
PyCon US 2012 - Web Server Bottlenecks and Performance Tuning
PyCon US 2012 - Web Server Bottlenecks and Performance TuningPyCon US 2012 - Web Server Bottlenecks and Performance Tuning
PyCon US 2012 - Web Server Bottlenecks and Performance Tuning
 
DjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New RelicDjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New Relic
 

Recently uploaded

Recently uploaded (20)

Test Automation Design Patterns_ A Comprehensive Guide.pdf
Test Automation Design Patterns_ A Comprehensive Guide.pdfTest Automation Design Patterns_ A Comprehensive Guide.pdf
Test Automation Design Patterns_ A Comprehensive Guide.pdf
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdf
 
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
Workshop -  Architecting Innovative Graph Applications- GraphSummit MilanWorkshop -  Architecting Innovative Graph Applications- GraphSummit Milan
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
Encryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key ConceptsEncryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key Concepts
 
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
 
GraphSummit Milan - Visione e roadmap del prodotto Neo4j
GraphSummit Milan - Visione e roadmap del prodotto Neo4jGraphSummit Milan - Visione e roadmap del prodotto Neo4j
GraphSummit Milan - Visione e roadmap del prodotto Neo4j
 
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
Auto Affiliate  AI Earns First Commission in 3 Hours..pdfAuto Affiliate  AI Earns First Commission in 3 Hours..pdf
Auto Affiliate AI Earns First Commission in 3 Hours..pdf
 
Weeding your micro service landscape.pdf
Weeding your micro service landscape.pdfWeeding your micro service landscape.pdf
Weeding your micro service landscape.pdf
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeCon
 
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
OpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCAOpenChain Webinar: AboutCode and Beyond - End-to-End SCA
OpenChain Webinar: AboutCode and Beyond - End-to-End SCA
 
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
COMPUTER AND ITS COMPONENTS PPT.by naitik sharma Class 9th A mittal internati...
 
From Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptxFrom Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptx
 
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
 
GraphSummit Milan & Stockholm - Neo4j: The Art of the Possible with Graph
GraphSummit Milan & Stockholm - Neo4j: The Art of the Possible with GraphGraphSummit Milan & Stockholm - Neo4j: The Art of the Possible with Graph
GraphSummit Milan & Stockholm - Neo4j: The Art of the Possible with Graph
 
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 

“warpdrive”, making Python web application deployment magically easy.

  • 1. “warpdrive”, making Python web application deployment magically easy. Graham Dumpleton @GrahamDumpleton PyCon New Zealand - September 2016
  • 2. Is deploying Python web applications too hard?
  • 3.
  • 4. $ virtualenv venv New python executable in /usr/local/www/mysite/venv/bin/python Installing setuptools, pip, wheel...done.
  • 6. (venv) $ pip install Django Collecting Django Using cached Django-1.9.7-py2.py3-none-any.whl Installing collected packages: Django Successfully installed Django-1.9.7
  • 7. (venv) $ python manage.py runserver Performing system checks... System check identified no issues (0 silenced). You have unapplied migrations; your app may not work properly until they are applied. Run 'python manage.py migrate' to apply them. June 17, 2016 - 01:02:22 Django version 1.9.7, using settings 'hello_world.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C.
  • 8. (venv) $ python manage.py migrate Operations to perform: Apply all migrations: admin, contenttypes, auth, sessions Running migrations: Rendering model states... DONE Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying admin.0002_logentry_remove_auto_add... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying sessions.0001_initial... OK
  • 9. (venv) $ python manage.py createsuperuser Username (leave blank to use 'graham'): grumpy Email address: grumpy@example.com Password: Password (again): Superuser created successfully.
  • 10. (venv) $ python manage.py runserver 0.0.0.0:8080 Performing system checks... System check identified no issues (0 silenced). June 17, 2016 - 01:06:17 Django version 1.9.7, using settings 'hello_world.settings' Starting development server at http://0.0.0.0:8080/ Quit the server with CONTROL-C.
  • 11.
  • 12.
  • 13. <VirtualHost *:80> ServerName www.example.com Alias /static/ /usr/local/www/mysite/static/ <Directory /usr/local/www/mysite/static> Require all granted </Directory>
 
 WSGIDaemonProcess mysite threads=5 request-timeout=30 queue-timeout=45 python-home=/usr/local/www/mysite/venv 
 python-path=/usr/local/www/mysite WSGIScriptAlias / /usr/local/www/mysite/mysite/wsgi.py 
 process-group=mysite application-group=%{GLOBAL} <Directory /usr/local/www/mysite/mysite> Require all granted </Directory> </VirtualHost>
  • 15. (venv) $ python manage.py collectstatic --noinput Copying ‘/…/static/admin/css/base.css’ … 56 static files copied to ‘/…/static’.
  • 16. • Initialisation steps • Create Python virtual environment • Activate Python virtual environment
  • 17. • Build steps • Install required Python packages • Collect together static file assets
  • 18. • Deployment steps • Initialise or migrate database data • Configure and start the WSGI server
  • 19. Why should you need to care about the details?
  • 20. What if all the build and deployment steps were managed for you?
  • 21. “warpdrive” is the
 magic glue that can make that possible!
  • 22. For local development, as well as production deployments.
  • 24. Build steps (warpdrive+django) $ warpdrive build -----> Installing dependencies with pip (requirements.txt) Collecting Django (from -r requirements.txt (line 1)) Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB) 100% |████████████████████████████████| 6.6MB 365kB/s Collecting mod_wsgi (from -r requirements.txt (line 2)) Downloading mod_wsgi-4.5.2.tar.gz (1.8MB) 100% |████████████████████████████████| 1.8MB 837kB/s Installing collected packages: Django, mod-wsgi Running setup.py install for mod-wsgi ... done Successfully installed Django-1.9.7 mod-wsgi-4.5.2 -----> Collecting static files for Django + python manage.py collectstatic --noinput Copying ‘/…/base.css’ 56 static files copied to ‘/…/static’.
  • 25. Configure and start the WSGI server (warpdrive+django) $ warpdrive start -----> Configuring for deployment type of 'auto' -----> Default WSGI server type is 'mod_wsgi' -----> Running server script start-mod_wsgi -----> Executing server command 'mod_wsgi-express start-server --log-to-terminal --startup-log --port 8080 --application-type module --entry-point hello_world.wsgi --callable-object application --url-alias /static/ /usr/local/www/mysite/static' [Sun Jun 19 22:00:59.819762 2016] [mpm_prefork:notice] [pid 67483] AH00163: Apache/2.4.18 (Unix) mod_wsgi/4.5.2 Python/2.7.10 configured -- resuming normal operations
  • 26. Automatic hosting detection • shell -> app.sh • python -> app.py • wsgi -> wsgi.py • django -> manage.py
  • 27. Choice of WSGI servers • mod_wsgi • gunicorn • uwsgi • waitress
  • 28. Initialise the database (warpdrive+django) $ warpdrive setup -----> Running .warpdrive/action_hooks/setup -----> Checking database is running -----> Initialising database. Operations to perform: Apply all migrations: admin, contenttypes, auth, sessions Running migrations: Rendering model states... DONE Applying contenttypes.0001_initial... OK … Applying sessions.0001_initial... OK -----> Running Django super user creation Username (leave blank to use 'graham'): grumpy Email address: grumpy@example.com Password: Password (again): Superuser created successfully.
  • 29. Migrate the database (warpdrive+django) $ warpdrive migrate -----> Running .warpdrive/action_hooks/migrate -----> Checking database is running -----> Running Django database migration Operations to perform: Apply all migrations: admin, contenttypes, auth, sessions Running migrations: No migrations to apply.
  • 30. Database migration hook.warpdrive/action_hooks/migrate #!/bin/bash CHECK_DATABASE="$WARPDRIVE_SRC_ROOT/scripts/check-database.py" case "$DATABASE_URL" in sqlite:*) ;; *) (cat - | python manage.py shell) << ! import runpy _ = runpy.run_path('$CHECK_DATABASE') ! ;; esac echo " -----> Running Django database migration" python manage.py migrate
  • 31. Action hooks • pre-build • build-env • build • deploy-env • deploy-cfg • deploy • setup • migrate • verify • ready • alive
  • 32. Interactive shell (warpdrive+django) $ warpdrive shell bash-3.2$ python manage.py migrate
  • 33. Command execution (warpdrive+django) $ warpdrive exec env | grep WARPDRIVE WARPDRIVE_SRC_ROOT=/Users/graham/Projects/warpdrive-django- auto WARPDRIVE_APP_ROOT=/Users/graham/.warpdrive/warpdrive +django WARPDRIVE_ACTION=exec WARPDRIVE_VERSION=0.20.1 WARPDRIVE_HTTP_PORT=8080 WARPDRIVE_ENV_NAME=django
  • 34.
  • 35. Create Docker image (warpdrive+django) $ warpdrive image django I0619 22:14:22.783544 67609 install.go:251] Using "assemble" installed from "image:///opt/app-root/s2i/bin/assemble" I0619 22:14:22.783688 67609 install.go:251] Using "run" installed from "image:///opt/app-root/s2i/bin/run" I0619 22:14:22.783712 67609 install.go:251] Using "save-artifacts" installed from "image:///opt/app-root/s2i/bin/save-artifacts" ---> Installing application source ---> Building application from source -----> Installing dependencies with pip (requirements.txt) Collecting Django (from -r requirements.txt (line 1)) Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB) Installing collected packages: Django Successfully installed Django-1.9.7 -----> Collecting static files for Django Copying ‘…/urlify.js’ … 56 static files copied to '/opt/app-root/src/static'. ---> Fix permissions on application source
  • 36. Run Docker image $ docker run --rm -p 8080:8080 django ---> Executing the start up script -----> Configuring for deployment type of 'auto' -----> Default WSGI server type is 'mod_wsgi' -----> Running server script start-mod_wsgi -----> Executing server command 'mod_wsgi-express start-server --log-to-terminal -- startup-log --port 8080 --application-type module --entry-point hello_world.wsgi -- callable-object application --url-alias /static/ /opt/app-root/src/static/' [Sun Jun 19 07:44:57.955455 2016] [mpm_event:notice] [pid 48:tid 139683988789312] AH00489: Apache/2.4.6 (CentOS) mod_wsgi/4.5.2 Python/3.4.2 configured -- resuming normal operations
  • 37.
  • 38. Manually build image FROM grahamdumpleton/warp0-centos7-python34 COPY . ${WARPDRIVE_SRC_ROOT} RUN warpdrive build CMD [ "warpdrive", "start" ]
  • 39. Source to Image (S2I) https://github.com/openshift/source-to-image $ s2i build https://github.com/GrahamDumpleton/warpdrive-django-auto.git grahamdumpleton/warp0-centos7-python34 django I0619 17:34:38.784337 66356 docker.go:352] Image "grahamdumpleton/warp0-centos7- python34:latest" not available locally, pulling ... I0619 17:40:23.793610 66356 clone.go:32] Downloading "https://github.com/ GrahamDumpleton/warpdrive-django-modwsgi.git" ... I0619 17:40:25.979028 66356 install.go:251] Using "assemble" installed from "image:///opt/app-root/s2i/bin/assemble" I0619 17:40:25.979075 66356 install.go:251] Using "run" installed from "image:/// opt/app-root/s2i/bin/run" I0619 17:40:25.979099 66356 install.go:251] Using "save-artifacts" installed from "image:///opt/app-root/s2i/bin/save-artifacts" ---> Installing application source ---> Building application from source -----> Installing dependencies with pip (requirements.txt) Collecting Django (from -r requirements.txt (line 1)) Downloading Django-1.9.7-py2.py3-none-any.whl (6.6MB) … E0619 17:40:51.152317 66356 util.go:91] + python manage.py collectstatic --noinput -----> Collecting static files for Django Copying ‘/…/urlify.js’ … 56 static files copied to '/opt/app-root/src/static'. ---> Fix permissions on application source
  • 40.
  • 41.
  • 42.
  • 43.
  • 44. $ oc new-app warpdrive-python34 --param APPLICATION_NAME=django --param REPOSITORY_URL=https://github.com/GrahamDumpleton/warpdrive-django-auto.git --> Deploying template warpdrive-python34 for "warpdrive-python34" With parameters: Name=django Git Repository URL=https://github.com/GrahamDumpleton/warpdrive-django- auto.git Hostname= Deployment Mode=auto Server Type=mod_wsgi Shell Script=app.sh Python Script=app.py Application Module=wsgi Application Callable=application Static URL= Static Root= Builder Version= Script Debugging= --> Creating resources with label app=django ... imagestream "django" created buildconfig "django" created deploymentconfig "django" created service "django" created route "django" created --> Success Build scheduled, use 'oc logs -f bc/django' to track its progress. Run 'oc status' to view your app.
  • 45. Integration possibilities • Direct to a host • Using Docker or Rocket • Kubernetes/OpenShift cluster • Other container platforms • Legacy PaaS environments
  • 47. Best of breed Docker image for Python web application deployment
  • 48. Integrated builder scripts which handle application image creation and running of the application
  • 49. Local development
 using the same scripts and workflow
  • 51. What I am looking for!