SlideShare a Scribd company logo
Unix Philosophy & Django projects
Applying the Unix Philosophy to
Django projects
a report from the real world
Who am I?
Federico Capoano (A.K.A. Nemesis)
OpenWISP core developer
Working on public wif at Cineca since 2012
What is OpenWISP?
OpenWISP is a set of software modules that can be
used to deploy and manage wireless networks
(public wif, backbone, mesh networks)
OpenWISP 1
OpenWISP 1: started in 2008-2016
Focused on public wif for Italian cities
Fast growth of features to accommodate requirements
Problems of OpenWISP 1
● monolithic ruby on rails apps
Problems of OpenWISP 1
● monolithic ruby on rails apps
● tight coupling
Problems of OpenWISP 1
● monolithic ruby on rails apps
● tight coupling
● duplicated logic
Problems of OpenWISP 1
● monolithic ruby on rails apps
● tight coupling
● duplicated logic
● policy not separated from mechanisms
Problems of OpenWISP 1
● monolithic ruby on rails apps
● tight coupling
● duplicated logic
● policy not separated from mechanisms
● complexity
Negative consequences (OpenWISP 1)
These problems caused several
very negative consequences
1. "Giant ball of mud" problems
Many features in a single codebase
Hard to add new features in a clean way
Hard to maintain over time (eg: upgrade
dependencies, fx bugs)
2. Inflexibility
Hard to extend without changing the core
Hard to adapt it and reuse it in different contexts
Specialized features available in the
codebase are not easily reusable
3. Little or no contributions
Complexity scared away
potential external contributors
We only got low quality patches
which were never merged
4. Costs and risk of death
Increased costs to keep the project alive
Many years of work risked to fade into oblivion
5. Duplication of efforts
Little reusability means:
reinventing the wheel
no ecosystem of interoperable tools
Nikola is not happy
This can't be...
There must be a better way!
The Unix Philosophy
What is the unix philosophy?
1. Approach to software development
● Bottom-up
1. Approach to software development
● Bottom-up
● Pragmatic
1. Approach to software development
● Bottom-up
● Pragmatic
● Grounded in experience
1. Approach to software development
● Bottom-up
● Pragmatic
● Grounded in experience
● Founded in empirical skepticism
2. Mature
Evolved in the Unix community since 1970
3. Change tolerant
Born during the digital revolution 1970-2010
It kept pace with the dramatic
evolution of technology
The Unix Philosophy in short
What are the main concepts
of the Unix philosophy?
"Do one thing and do it well"
● Break down big problems in smaller problems
"Do one thing and do it well"
● Break down big problems in smaller problems
● Write simple programs that solve one problem well
"Do one thing and do it well"
● Break down big problems in smaller problems
● Write simple programs that solve one problem well
● Write programs to work together
"Do one thing and do it well"
● Break down big problems in smaller problems
● Write simple programs that solve one problem well
● Write programs to work together
● Value simplicity and maintainability over
performance and complexity
What advantages does it bring?
What are the main advantages
of applying the Unix Philosophy?
1. Saner development
Solving one problem at time is simpler
2. Maintainability
Maintaining a simple program is less expensive
3. Usability
A simple program is easier to use
4. Readability
A simple program is easier to read
and understand
5. Agility
A simple program will be more easily
readapted to new requirements
6. Reuse
Being easier to use, understand, read and maintain
it will be readapted to work in different contexts
and will attract more contributors
7. Ecosystem
Many such programs form an ecosystem of
interoperable tools
(like basic lego building blocks)
and
Allow to solve complex problems faster
ME GUSTA
The Unix Philosophy in detail
The 17 rules of the Unix Philosophy
From "The art of unix programming"
by Eric Raymond
1. Rule of Modularity
Write simple parts connected
by clean interfaces
source
2. Rule of Clarity
Clarity is better than cleverness
source
3. Rule of Composition
Design programs to be connected
to other programs
source
4. Rule of Separation
Separate policy from mechanism;
separate interfaces from engines
source
5. Rule of Simplicity
Design for simplicity;
add complexity only where you must
source
6. Rule of Parsimony
Write a big program only when it is clear by
demonstration that nothing else will do
source
7. Rule of Transparency
Design for visibility to make inspection
and debugging easier
source
8. Rule of Robustness
Software is said to be robust when it performs well
under unexpected conditions which stress the
designer's assumptions, as well as under normal
conditions
source
9. Rule of Representation
Fold knowledge into data so program logic
can be stupid and robust
source
10. Rule of Least Surprise
In interface design, always do
the least surprising thing
source
11. Rule of Silence
When a program has nothing surprising
to say, it should say nothing
source
12. Rule of Repair
When you must fail, fail noisily
and as soon as possible
source
13. Rule of Economy
Programmer time is expensive; conserve it
in preference to machine time
source
14. Rule of Generation
Avoid hand-hacking; write programs to
write programs when you can
source
15. Rule of Optimization
Prototype before polishing;
get it working before you optimize it
source
16. Rule of Diversity
Distrust all claims for “one true way”
source
17. Rule of Extensibility
Design for the future, because it will
be here sooner than you think
source
How to apply it to Django?
How to apply the Unix philosophy
to django projects?
Let's see some real world
examples from OpenWISP 2
1. Rule of Modularity
● Develop main features as reusable django apps
1. Rule of Modularity
● Develop main features as reusable django apps
● One django app for each group of related features
1. Rule of Modularity
● Develop main features as reusable django apps
● One django app for each group of related features
● Document public API
1. Rule of Modularity
● Develop main features as reusable django apps
● One django app for each group of related features
● Document its public API
● Include a license and a changelog
1. Rule of Modularity
● Develop main features as reusable django apps
● One django app for each group of related features
● Document its public API
● Include a license and a changelog
● Publish your app on pypi and djangopackages.org
1. Real world examples
● Confgurations of routers and VPNs:
django-netjsonconfg
1. Real world examples
● Confgurations of routers and VPNs:
django-netjsonconfg
● PKI management (x509 certifcates): django-x509
1. Real world examples
● Confgurations of routers and VPNs:
django-netjsonconfg
● PKI management (x509 certifcates): django-x509
● Multi-tenancy: contributed to django-organizations
reuse existing projects when possible!
2. Rule of Clarity
Explicit is better than implicit
This is already a widely accepted concept in
the python world
3. Rule of Composition
Combine, extend and customize django
reusable apps in your fnal django project
3. Real world examples
● openwisp-users: extends django-organizations
3. Real world examples
● openwisp-users: extends django-organizations
● openwisp-controller: depends on openwisp-users
and extends django-netjsonconfg and django-x509
3. Real world examples
● openwisp-users: extends django-organizations
● openwisp-controller: depends on openwisp-users
and extends django-netjsonconfg and django-x509
● Final result handled by ansible-openwisp2 in an
(almost) transparent manner
4. Rule of Separation
● implement mechanisms as libraries
4. Rule of Separation
● implement mechanisms as libraries
● implement policy as a confguration
4. Rule of Separation
● implement mechanisms as libraries
● implement policy as a confguration
● make these libraries highly confgurable
4. Rule of Separation
● implement mechanisms as libraries
● implement policy as a confguration
● make these libraries highly confgurable
● make these libraries work with data
4. Rule of Separation
● implement mechanisms as libraries
● implement policy as a confguration
● make these libraries highly confgurable
● make these libraries work with data
● clearly defne input and output
4. Real world examples
● netjsonconfg: python library for generating
router & VPN confgurations from NetJSON
objects
4. Real world examples
● netjsonconfg: python library for generating
router & VPN confgurations from NetJSON
objects
● django-netjsonconfg: web interface that uses
netjsonconfg under the hood
>>> from netjsonconfig import OpenWrt
>>> router = OpenWrt({
... "general": {"hostname": "HomeRouter"}
... })
>>> print(router.render())
package system
config system 'system'
option hostname 'HomeRouter'
option timezone 'UTC'
option zonename 'UTC'
5. Rule of Simplicity
● start with very basic features
5. Rule of Simplicity
● start with very basic features
● release your project early
(even if you feel it's incomplete)
5. Rule of Simplicity
● start with very basic features
● release your project early
(even if you feel it's incomplete)
● add one feature at time as your understanding grows
5. Rule of Simplicity
● start with very basic features
● release your project early
(even if you feel it's incomplete)
● add one feature at time as your understanding grows
● add complexity only when necessary
5. Real world examples
as of April 2017:
5. Real world examples
as of April 2017:
● 24 releases of netjsonconfg
5. Real world examples
as of April 2017:
● 24 releases of netjsonconfg
● 25 releases of django-netjsonconfg
5. Real world examples
as of April 2017:
● 24 releases of netjsonconfg
● 25 releases of django-netjsonconfg
● 5 releases of django-x509
5. Real world examples
as of April 2017:
● 24 releases of netjsonconfg
● 25 releases of django-netjsonconfg
● 5 releases of django-x509
OpenWISP 2 still lacks some features of OpenWISP 1
6. Rule of Parsimony
Prefer creating new reusable apps
when adding big features
unless doing this complicates things a lot
with no real advantages
6. Real world examples
PKI management in new app: django-x509
VPN confgurations added to existing app:
django-netjsonconfg
7. Rule of Transparency
Log unexpected events using
the python logging facility
Logging of bad requests in django-netjsonconfg
import logging
logger = logging.getLogger(__name__)
def invalid_response(request, error, status):
logger.warning(error, extra={'request':
request, 'stack': True})
return ControllerResponse(error,
status=status)
7. Real world examples
Provide good default logging
Provide support for sentry
Take a look at a real example
8. Rule of Robustness
Add constraints to your reusable django apps only
when necessary:
● avoid very strict validation rules
● provide confgurable settings
8. Real world examples
● settings in django-netjsonconfg
● settings in django-x509
9. Rule of Representation
Fold complex information in data structures
Process these data structures with algorithms
Try to make algorithms framework-agnostic
(if possible)
9. Real world examples
in OpenWISP 2, confguration of routers is
implemented as a single text feld formatted as
NetJSON
(in OpenWISP 1 each available confguration had its
own database table and model,
very hard to maintain and evolve)
# NetJSON DeviceConfiguration example
{
"general": {"hostname": "HomeRouter"},
"interfaces": [
{
"name": "eth0",
"type": "ethernet",
"addresses": [
{
"address": "192.168.1.1",
"mask": 24,
"proto": "static",
"family": "ipv4"
}
]
}
]
}
10. Rule of Least Surprise
This has become a widely accepted
concept in the IT industry
11. Rule of Silence
Just don't make your apps annoying
It's not that hard :-P
12. Rule of Repair
Errors should never pass silently
Unless explicitly silenced
(from the zen of python)
Let's add: fail fast, noisily and early
from django.core.exceptions import ImproperlyConfigured
from .settings import REGISTRATION_ENABLED, SHARED_SECRET
if REGISTRATION_ENABLED and not SHARED_SECRET:
msg = 'NETJSONCONFIG_SHARED_SECRET not set!'
raise ImproperlyConfigured(msg)
Full real world example available in
django-netjsonconfg
13. Rule of Economy
When Unix was born this was a radical idea:
assembler was the norm, C was considered a higher
level language
Python is a consequence of the success of that
radical idea
Embrace this concept
14. Rule of Generation
When you fnd yourself writing lots of boilerplate
code, try to use
meta-programming or code generators
14. meta-programmed urlpatterns
urls.py in openwisp-controller
14. Auto-generated UI from json-schema
JSON-Schema of this UI available on github
15. Rule of Optimization
● write a frst early version of your django app
● do not forget to write automated tests
● refactor and polish
16. Rule of Diversity
Maintain a skeptic attitude towards
dogmas and fads
Measure the effectiveness of an
approach before embracing it
17. Rule of Extensibility
Provide ways to extend and customize the
behaviour of modules without the
need of changing the core code
17. Extensibility: abstract models
● Provide abstract models in your
most important modules
17. Extensibility: abstract models
● Provide abstract models in your
most important modules
● Store these in a python fle which does not import
concrete models
(otherwise other django apps won't be able to import them)
17. Extensibility: abstract models
abstract models in django-netjsonconfg
example usage
17. Extensibility: abstract models
abstract models in django-x509
example usage
17. Extensibility: base admin
● Provide base admin classes in your main modules
(avoid duplication)
17. Extensibility: base admin
● Provide base admin classes in your main modules
(avoid duplication)
● Store them in a python fle which does not import
concrete models
(otherwise other django-apps won't be able to import them)
17. Extensibility: base admin
base admin classes django-netjsonconfg
example usage
17. Extensibility: base views
● If your reusable django app has views, provide
generic views that can be extended
● Store them in a python fle which does not import
concrete models
(otherwise other django-apps won't be able to import them)
17. Extensibility: base views
base views in django-netjsonconfg
example usage
17. Extensibility: reusable urls
If your app provides views that can be extended,
third party apps will have to redefne their URLs
You may want to avoid this required duplication by
providing a mechanism to import urls
17. Extensibility: reusable urls
reusable urls in django-netjsonconfg
example usage:
from django_netjsonconfig.utils import get_controller_urls
from . import views # customized views
# creates new url patterns hooked to customized views
urlpatterns = get_controller_urls(views)
17. Extensibility: AppConfig
If your reusable django app relies on signal
connection for some features,
provide a base AppConfig class
17. Extensibility: AppConfig
base AppConfig class in django-netjsonconfg
from django_netjsonconfig.apps import OpenWispAppConfig
class MyOwnApp(OpenWispAppConfig):
name = 'yourapp.config'
label = 'config'
def __setmodels__(self):
# these are your custom models
from .models import Config, VpnClient
self.config_model = Config
self.vpnclient_model = VpnClient
Positive results
Positive results achieved in OpenWISP
1. Faster release cycle & evolution
We release new features more often
The project is evolving rapidly
2. Easier maintainance
Once a bug in a specifc module is replicated
fxing it is easier compared to the work needed to
fx bugs in OpenWISP 1
3. More derivative work
OpenWISP 2 was released officially
less than 1 year ago (as of April 2017)
notwithstanding that, there are already
a couple of derivative works
4. Growth
The user base has been growing rapidly
Users send feedback and patches
4. Growth: mailing list
4. Growth: GSOC 2017
OpenWISP has also been accepted
as a mentoring organization
for the Google Summer of Code 2017
Disadvantages
Is not a bed of roses
1. Integration issues
Combining more django apps may
result in integration issues
Changes to project confguration can result
in bugs that are not caught by unit tests
1. Solution
Integration tests are important in this case
Shortcut: I was able to import tests of a base app
and repeat them in the extension app
This easy fx was good enough for my case
2. Big features
When I needed to introduce a new major feature I
needed to change at least a couple of modules
2. Solution
many times this will be necessary
If you have to change many modules every time,
refactor your code to be more loosely coupled
3. Repository management overhead
● working with many repositories can be overwhelming
3. Repository management overhead
● working with many repositories can be overwhelming
● more git tags, pypi releases, versioning, changelogs
3. Repository management overhead
● working with many repositories can be overwhelming
● more git tags, pypi releases, versioning, changelogs
● more announcements for new releases
3. Solutions
● import several projects in a single window of your editor
3. Solutions
● import several projects in a single window of your editor
● automate versioning and changelogs
(I haven't done this yet)
3. Solutions
● import several projects in a single window of your editor
● automate versioning and changelogs
(I haven't done this yet)
● do not send announcements for minor releases which
contain no real advantage for end users
4. Documentation fragmentation
We have many repositories with
their own READMEs and docs
We don't have a single documentation website yet
4. Solutions
I added links to specifc module docs from the website
I often have to send those links on the mailing list
But a central comprehensive documentation website
would be better (we don't have this yet)
5. Bug reports
Users don't know where to send bug reports
and they write to the mailing list
5. Solution
● For the moment I reply by pointing to them where
to open issues
5. Solution
● For the moment I reply by pointing to them where
to open issues
● But a single issue tracker for all the repositories
would be better
(even though it may not really solve the "which module?" problem)
6. Dispersion of popularity
We no longer have a single popular
repository with many github stars
6. Solution
Stop caring about this
Github stars are not an accurate indicator
of the success of a project
Conclusions
The creators of Unix are not newcomers
Following their heritage is a good idea
Delve deep
Further resources on the Unix Philosophy
and related concepts:
● the art of unix programming
● the pragmatic programmer
Practice
Start to gradually apply these concepts in your projects
EG: extract big features to separate projects
NO to dogmas
● Practice empirical (scientifc) skepticism
NO to dogmas
● Practice empirical (scientifc) skepticism
● Test your ideas
NO to dogmas
● Practice empirical (scientifc) skepticism
● Test your ideas
● Measure your results
Read existing code
Read the code of OpenWISP
@openwisp organization on github
(or other projects like @openstack)
Unix Philosophy for the win!
Thank you
Ideas, critical feedback, suggestions?
Talk to me!
Find me also on:
● twitter (@nemesisdesign)
● github (@nemesisdesign)
● linkedin (Federico Capoano)
You can fnd the slides on slideshare

More Related Content

What's hot

Bao cao tom tat de tai tot nghiep
Bao cao tom tat de tai tot nghiepBao cao tom tat de tai tot nghiep
Bao cao tom tat de tai tot nghiep
Nguyễn Công Huy
 
lý thuyết cơ sở dữ liệu phân tán
lý thuyết cơ sở dữ liệu phân tánlý thuyết cơ sở dữ liệu phân tán
lý thuyết cơ sở dữ liệu phân tánNgo Trung
 
Hệ điều hành (chương 5)
Hệ điều hành (chương 5)Hệ điều hành (chương 5)
Hệ điều hành (chương 5)realpotter
 
Quản trị dự án công nghệ thông tin
Quản trị dự án công nghệ thông tinQuản trị dự án công nghệ thông tin
Quản trị dự án công nghệ thông tinAnh Dam
 
Ntfs and computer forensics
Ntfs and computer forensicsNtfs and computer forensics
Ntfs and computer forensics
Gaurav Ragtah
 
He dieu hanh
He dieu hanhHe dieu hanh
He dieu hanh
VNG
 
5 bo nho may tinh
5 bo nho may tinh5 bo nho may tinh
5 bo nho may tinh
Ly hai
 
Mise en conformité des bases de données MySQL avec le RGPD
Mise en conformité des bases de données MySQL avec le RGPDMise en conformité des bases de données MySQL avec le RGPD
Mise en conformité des bases de données MySQL avec le RGPD
Valentin Traën
 
Giáo trình kiến trúc máy tính ICTU
Giáo trình kiến trúc máy tính ICTUGiáo trình kiến trúc máy tính ICTU
Giáo trình kiến trúc máy tính ICTU
Ngô Doãn Tình
 
Cơ sở dữ liệu phân tán
Cơ sở dữ liệu phân tánCơ sở dữ liệu phân tán
Cơ sở dữ liệu phân tánQuy Nguyen
 
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQLXây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
AskSock Ngô Quang Đạo
 
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise ViettinsoftTài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
Phần mềm tính lương | phan mem tinh luong
 
Computer forensics
Computer forensicsComputer forensics
Computer forensics
Hiren Selani
 
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAYLuận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
Dịch vụ viết bài trọn gói ZALO 0917193864
 

What's hot (14)

Bao cao tom tat de tai tot nghiep
Bao cao tom tat de tai tot nghiepBao cao tom tat de tai tot nghiep
Bao cao tom tat de tai tot nghiep
 
lý thuyết cơ sở dữ liệu phân tán
lý thuyết cơ sở dữ liệu phân tánlý thuyết cơ sở dữ liệu phân tán
lý thuyết cơ sở dữ liệu phân tán
 
Hệ điều hành (chương 5)
Hệ điều hành (chương 5)Hệ điều hành (chương 5)
Hệ điều hành (chương 5)
 
Quản trị dự án công nghệ thông tin
Quản trị dự án công nghệ thông tinQuản trị dự án công nghệ thông tin
Quản trị dự án công nghệ thông tin
 
Ntfs and computer forensics
Ntfs and computer forensicsNtfs and computer forensics
Ntfs and computer forensics
 
He dieu hanh
He dieu hanhHe dieu hanh
He dieu hanh
 
5 bo nho may tinh
5 bo nho may tinh5 bo nho may tinh
5 bo nho may tinh
 
Mise en conformité des bases de données MySQL avec le RGPD
Mise en conformité des bases de données MySQL avec le RGPDMise en conformité des bases de données MySQL avec le RGPD
Mise en conformité des bases de données MySQL avec le RGPD
 
Giáo trình kiến trúc máy tính ICTU
Giáo trình kiến trúc máy tính ICTUGiáo trình kiến trúc máy tính ICTU
Giáo trình kiến trúc máy tính ICTU
 
Cơ sở dữ liệu phân tán
Cơ sở dữ liệu phân tánCơ sở dữ liệu phân tán
Cơ sở dữ liệu phân tán
 
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQLXây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
Xây dựng hệ thống thông tin quản lý khách hàng bằng PHP và MySQL
 
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise ViettinsoftTài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
Tài liệu hướng dẫn phần mềm quản lý chấm công Att_Paradise Viettinsoft
 
Computer forensics
Computer forensicsComputer forensics
Computer forensics
 
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAYLuận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
Luận văn: Xây dựng ứng dụng Android xem video trực tuyến, HAY
 

Similar to Applying the Unix Philosophy to Django projects: a report from the real world

DockerDay2015: Keynote
DockerDay2015: KeynoteDockerDay2015: Keynote
DockerDay2015: Keynote
Docker-Hanoi
 
DockerCon SF 2015: Keynote Day 1
DockerCon SF 2015: Keynote Day 1DockerCon SF 2015: Keynote Day 1
DockerCon SF 2015: Keynote Day 1
Docker, Inc.
 
Use open source software to develop ideas at work
Use open source software to develop ideas at workUse open source software to develop ideas at work
Use open source software to develop ideas at work
Sammy Fung
 
Teaching Open Source In The University
Teaching Open Source In The UniversityTeaching Open Source In The University
Teaching Open Source In The University
Dominique Cimafranca
 
All You need to Know about Secure Coding with Open Source Software
All You need to Know about Secure Coding with Open Source SoftwareAll You need to Know about Secure Coding with Open Source Software
All You need to Know about Secure Coding with Open Source Software
Javier Perez
 
Leveraging Android's Linux Heritage
Leveraging Android's Linux HeritageLeveraging Android's Linux Heritage
Leveraging Android's Linux Heritage
Opersys inc.
 
Android Application Development Training by NITIN GUPTA
Android Application Development Training by NITIN GUPTA Android Application Development Training by NITIN GUPTA
Android Application Development Training by NITIN GUPTA
NITIN GUPTA
 
Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019
Olivier Loverde
 
Leveraging Android's Linux Heritage at ELC-E 2011
Leveraging Android's Linux Heritage at ELC-E 2011Leveraging Android's Linux Heritage at ELC-E 2011
Leveraging Android's Linux Heritage at ELC-E 2011
Opersys inc.
 
stackconf 2022: It’s Time to Debloat the Cloud with Unikraft
stackconf 2022: It’s Time to Debloat the Cloud with Unikraftstackconf 2022: It’s Time to Debloat the Cloud with Unikraft
stackconf 2022: It’s Time to Debloat the Cloud with Unikraft
NETWAYS
 
Microservice pitfalls
Microservice pitfalls Microservice pitfalls
Microservice pitfalls
Mite Mitreski
 
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
OW2
 
Dublin Unity User Group Meetup Sept 2015
Dublin Unity User Group Meetup Sept 2015Dublin Unity User Group Meetup Sept 2015
Dublin Unity User Group Meetup Sept 2015
Dominique Boutin
 
Making Strongly-typed NETCONF Usable
Making Strongly-typed NETCONF UsableMaking Strongly-typed NETCONF Usable
Making Strongly-typed NETCONF Usable
Open Networking Summit
 
Iot development from prototype to production
Iot development from prototype to productionIot development from prototype to production
Iot development from prototype to production
Mender.io
 
Build and deploy scientific Python Applications
Build and deploy scientific Python Applications  Build and deploy scientific Python Applications
Build and deploy scientific Python Applications
Ramakrishna Reddy
 
Effectively using Open Source with conda
Effectively using Open Source with condaEffectively using Open Source with conda
Effectively using Open Source with conda
Travis Oliphant
 
A Summary about Hykes' Keynote on Dockercon 2015
A Summary about Hykes' Keynote on Dockercon 2015A Summary about Hykes' Keynote on Dockercon 2015
A Summary about Hykes' Keynote on Dockercon 2015
Henry Huang
 
Top 10 Best DevOps tools in 2020
Top 10 Best DevOps tools in 2020Top 10 Best DevOps tools in 2020
Top 10 Best DevOps tools in 2020
prafulIQBusiness
 
NLLUG 2012 - XPages Extensibility API - going deep!
NLLUG 2012 - XPages Extensibility API - going deep!NLLUG 2012 - XPages Extensibility API - going deep!
NLLUG 2012 - XPages Extensibility API - going deep!
René Winkelmeyer
 

Similar to Applying the Unix Philosophy to Django projects: a report from the real world (20)

DockerDay2015: Keynote
DockerDay2015: KeynoteDockerDay2015: Keynote
DockerDay2015: Keynote
 
DockerCon SF 2015: Keynote Day 1
DockerCon SF 2015: Keynote Day 1DockerCon SF 2015: Keynote Day 1
DockerCon SF 2015: Keynote Day 1
 
Use open source software to develop ideas at work
Use open source software to develop ideas at workUse open source software to develop ideas at work
Use open source software to develop ideas at work
 
Teaching Open Source In The University
Teaching Open Source In The UniversityTeaching Open Source In The University
Teaching Open Source In The University
 
All You need to Know about Secure Coding with Open Source Software
All You need to Know about Secure Coding with Open Source SoftwareAll You need to Know about Secure Coding with Open Source Software
All You need to Know about Secure Coding with Open Source Software
 
Leveraging Android's Linux Heritage
Leveraging Android's Linux HeritageLeveraging Android's Linux Heritage
Leveraging Android's Linux Heritage
 
Android Application Development Training by NITIN GUPTA
Android Application Development Training by NITIN GUPTA Android Application Development Training by NITIN GUPTA
Android Application Development Training by NITIN GUPTA
 
Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019Node.js Service - Best practices in 2019
Node.js Service - Best practices in 2019
 
Leveraging Android's Linux Heritage at ELC-E 2011
Leveraging Android's Linux Heritage at ELC-E 2011Leveraging Android's Linux Heritage at ELC-E 2011
Leveraging Android's Linux Heritage at ELC-E 2011
 
stackconf 2022: It’s Time to Debloat the Cloud with Unikraft
stackconf 2022: It’s Time to Debloat the Cloud with Unikraftstackconf 2022: It’s Time to Debloat the Cloud with Unikraft
stackconf 2022: It’s Time to Debloat the Cloud with Unikraft
 
Microservice pitfalls
Microservice pitfalls Microservice pitfalls
Microservice pitfalls
 
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
Enabling DevOps for IoT software development, powered by Open Source, OW2onli...
 
Dublin Unity User Group Meetup Sept 2015
Dublin Unity User Group Meetup Sept 2015Dublin Unity User Group Meetup Sept 2015
Dublin Unity User Group Meetup Sept 2015
 
Making Strongly-typed NETCONF Usable
Making Strongly-typed NETCONF UsableMaking Strongly-typed NETCONF Usable
Making Strongly-typed NETCONF Usable
 
Iot development from prototype to production
Iot development from prototype to productionIot development from prototype to production
Iot development from prototype to production
 
Build and deploy scientific Python Applications
Build and deploy scientific Python Applications  Build and deploy scientific Python Applications
Build and deploy scientific Python Applications
 
Effectively using Open Source with conda
Effectively using Open Source with condaEffectively using Open Source with conda
Effectively using Open Source with conda
 
A Summary about Hykes' Keynote on Dockercon 2015
A Summary about Hykes' Keynote on Dockercon 2015A Summary about Hykes' Keynote on Dockercon 2015
A Summary about Hykes' Keynote on Dockercon 2015
 
Top 10 Best DevOps tools in 2020
Top 10 Best DevOps tools in 2020Top 10 Best DevOps tools in 2020
Top 10 Best DevOps tools in 2020
 
NLLUG 2012 - XPages Extensibility API - going deep!
NLLUG 2012 - XPages Extensibility API - going deep!NLLUG 2012 - XPages Extensibility API - going deep!
NLLUG 2012 - XPages Extensibility API - going deep!
 

Recently uploaded

Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Pitangent Analytics & Technology Solutions Pvt. Ltd
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 

Recently uploaded (20)

Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 

Applying the Unix Philosophy to Django projects: a report from the real world

  • 1. Unix Philosophy & Django projects Applying the Unix Philosophy to Django projects a report from the real world
  • 2. Who am I? Federico Capoano (A.K.A. Nemesis) OpenWISP core developer Working on public wif at Cineca since 2012
  • 3.
  • 4. What is OpenWISP? OpenWISP is a set of software modules that can be used to deploy and manage wireless networks (public wif, backbone, mesh networks)
  • 5. OpenWISP 1 OpenWISP 1: started in 2008-2016 Focused on public wif for Italian cities Fast growth of features to accommodate requirements
  • 6. Problems of OpenWISP 1 ● monolithic ruby on rails apps
  • 7. Problems of OpenWISP 1 ● monolithic ruby on rails apps ● tight coupling
  • 8. Problems of OpenWISP 1 ● monolithic ruby on rails apps ● tight coupling ● duplicated logic
  • 9. Problems of OpenWISP 1 ● monolithic ruby on rails apps ● tight coupling ● duplicated logic ● policy not separated from mechanisms
  • 10. Problems of OpenWISP 1 ● monolithic ruby on rails apps ● tight coupling ● duplicated logic ● policy not separated from mechanisms ● complexity
  • 11. Negative consequences (OpenWISP 1) These problems caused several very negative consequences
  • 12. 1. "Giant ball of mud" problems Many features in a single codebase Hard to add new features in a clean way Hard to maintain over time (eg: upgrade dependencies, fx bugs)
  • 13. 2. Inflexibility Hard to extend without changing the core Hard to adapt it and reuse it in different contexts Specialized features available in the codebase are not easily reusable
  • 14. 3. Little or no contributions Complexity scared away potential external contributors We only got low quality patches which were never merged
  • 15. 4. Costs and risk of death Increased costs to keep the project alive Many years of work risked to fade into oblivion
  • 16. 5. Duplication of efforts Little reusability means: reinventing the wheel no ecosystem of interoperable tools
  • 17. Nikola is not happy
  • 18. This can't be... There must be a better way!
  • 19.
  • 20. The Unix Philosophy What is the unix philosophy?
  • 21. 1. Approach to software development ● Bottom-up
  • 22. 1. Approach to software development ● Bottom-up ● Pragmatic
  • 23. 1. Approach to software development ● Bottom-up ● Pragmatic ● Grounded in experience
  • 24. 1. Approach to software development ● Bottom-up ● Pragmatic ● Grounded in experience ● Founded in empirical skepticism
  • 25. 2. Mature Evolved in the Unix community since 1970
  • 26. 3. Change tolerant Born during the digital revolution 1970-2010 It kept pace with the dramatic evolution of technology
  • 27. The Unix Philosophy in short What are the main concepts of the Unix philosophy?
  • 28. "Do one thing and do it well" ● Break down big problems in smaller problems
  • 29. "Do one thing and do it well" ● Break down big problems in smaller problems ● Write simple programs that solve one problem well
  • 30. "Do one thing and do it well" ● Break down big problems in smaller problems ● Write simple programs that solve one problem well ● Write programs to work together
  • 31. "Do one thing and do it well" ● Break down big problems in smaller problems ● Write simple programs that solve one problem well ● Write programs to work together ● Value simplicity and maintainability over performance and complexity
  • 32. What advantages does it bring? What are the main advantages of applying the Unix Philosophy?
  • 33. 1. Saner development Solving one problem at time is simpler
  • 34. 2. Maintainability Maintaining a simple program is less expensive
  • 35. 3. Usability A simple program is easier to use
  • 36. 4. Readability A simple program is easier to read and understand
  • 37. 5. Agility A simple program will be more easily readapted to new requirements
  • 38. 6. Reuse Being easier to use, understand, read and maintain it will be readapted to work in different contexts and will attract more contributors
  • 39. 7. Ecosystem Many such programs form an ecosystem of interoperable tools (like basic lego building blocks) and Allow to solve complex problems faster
  • 41. The Unix Philosophy in detail The 17 rules of the Unix Philosophy From "The art of unix programming" by Eric Raymond
  • 42. 1. Rule of Modularity Write simple parts connected by clean interfaces source
  • 43. 2. Rule of Clarity Clarity is better than cleverness source
  • 44. 3. Rule of Composition Design programs to be connected to other programs source
  • 45. 4. Rule of Separation Separate policy from mechanism; separate interfaces from engines source
  • 46. 5. Rule of Simplicity Design for simplicity; add complexity only where you must source
  • 47. 6. Rule of Parsimony Write a big program only when it is clear by demonstration that nothing else will do source
  • 48. 7. Rule of Transparency Design for visibility to make inspection and debugging easier source
  • 49. 8. Rule of Robustness Software is said to be robust when it performs well under unexpected conditions which stress the designer's assumptions, as well as under normal conditions source
  • 50. 9. Rule of Representation Fold knowledge into data so program logic can be stupid and robust source
  • 51. 10. Rule of Least Surprise In interface design, always do the least surprising thing source
  • 52. 11. Rule of Silence When a program has nothing surprising to say, it should say nothing source
  • 53. 12. Rule of Repair When you must fail, fail noisily and as soon as possible source
  • 54. 13. Rule of Economy Programmer time is expensive; conserve it in preference to machine time source
  • 55. 14. Rule of Generation Avoid hand-hacking; write programs to write programs when you can source
  • 56. 15. Rule of Optimization Prototype before polishing; get it working before you optimize it source
  • 57. 16. Rule of Diversity Distrust all claims for “one true way” source
  • 58. 17. Rule of Extensibility Design for the future, because it will be here sooner than you think source
  • 59.
  • 60. How to apply it to Django? How to apply the Unix philosophy to django projects? Let's see some real world examples from OpenWISP 2
  • 61. 1. Rule of Modularity ● Develop main features as reusable django apps
  • 62. 1. Rule of Modularity ● Develop main features as reusable django apps ● One django app for each group of related features
  • 63. 1. Rule of Modularity ● Develop main features as reusable django apps ● One django app for each group of related features ● Document public API
  • 64. 1. Rule of Modularity ● Develop main features as reusable django apps ● One django app for each group of related features ● Document its public API ● Include a license and a changelog
  • 65. 1. Rule of Modularity ● Develop main features as reusable django apps ● One django app for each group of related features ● Document its public API ● Include a license and a changelog ● Publish your app on pypi and djangopackages.org
  • 66. 1. Real world examples ● Confgurations of routers and VPNs: django-netjsonconfg
  • 67. 1. Real world examples ● Confgurations of routers and VPNs: django-netjsonconfg ● PKI management (x509 certifcates): django-x509
  • 68. 1. Real world examples ● Confgurations of routers and VPNs: django-netjsonconfg ● PKI management (x509 certifcates): django-x509 ● Multi-tenancy: contributed to django-organizations reuse existing projects when possible!
  • 69. 2. Rule of Clarity Explicit is better than implicit This is already a widely accepted concept in the python world
  • 70. 3. Rule of Composition Combine, extend and customize django reusable apps in your fnal django project
  • 71. 3. Real world examples ● openwisp-users: extends django-organizations
  • 72. 3. Real world examples ● openwisp-users: extends django-organizations ● openwisp-controller: depends on openwisp-users and extends django-netjsonconfg and django-x509
  • 73. 3. Real world examples ● openwisp-users: extends django-organizations ● openwisp-controller: depends on openwisp-users and extends django-netjsonconfg and django-x509 ● Final result handled by ansible-openwisp2 in an (almost) transparent manner
  • 74.
  • 75. 4. Rule of Separation ● implement mechanisms as libraries
  • 76. 4. Rule of Separation ● implement mechanisms as libraries ● implement policy as a confguration
  • 77. 4. Rule of Separation ● implement mechanisms as libraries ● implement policy as a confguration ● make these libraries highly confgurable
  • 78. 4. Rule of Separation ● implement mechanisms as libraries ● implement policy as a confguration ● make these libraries highly confgurable ● make these libraries work with data
  • 79. 4. Rule of Separation ● implement mechanisms as libraries ● implement policy as a confguration ● make these libraries highly confgurable ● make these libraries work with data ● clearly defne input and output
  • 80. 4. Real world examples ● netjsonconfg: python library for generating router & VPN confgurations from NetJSON objects
  • 81. 4. Real world examples ● netjsonconfg: python library for generating router & VPN confgurations from NetJSON objects ● django-netjsonconfg: web interface that uses netjsonconfg under the hood
  • 82.
  • 83. >>> from netjsonconfig import OpenWrt >>> router = OpenWrt({ ... "general": {"hostname": "HomeRouter"} ... }) >>> print(router.render()) package system config system 'system' option hostname 'HomeRouter' option timezone 'UTC' option zonename 'UTC'
  • 84. 5. Rule of Simplicity ● start with very basic features
  • 85. 5. Rule of Simplicity ● start with very basic features ● release your project early (even if you feel it's incomplete)
  • 86. 5. Rule of Simplicity ● start with very basic features ● release your project early (even if you feel it's incomplete) ● add one feature at time as your understanding grows
  • 87. 5. Rule of Simplicity ● start with very basic features ● release your project early (even if you feel it's incomplete) ● add one feature at time as your understanding grows ● add complexity only when necessary
  • 88. 5. Real world examples as of April 2017:
  • 89. 5. Real world examples as of April 2017: ● 24 releases of netjsonconfg
  • 90. 5. Real world examples as of April 2017: ● 24 releases of netjsonconfg ● 25 releases of django-netjsonconfg
  • 91. 5. Real world examples as of April 2017: ● 24 releases of netjsonconfg ● 25 releases of django-netjsonconfg ● 5 releases of django-x509
  • 92. 5. Real world examples as of April 2017: ● 24 releases of netjsonconfg ● 25 releases of django-netjsonconfg ● 5 releases of django-x509 OpenWISP 2 still lacks some features of OpenWISP 1
  • 93. 6. Rule of Parsimony Prefer creating new reusable apps when adding big features unless doing this complicates things a lot with no real advantages
  • 94. 6. Real world examples PKI management in new app: django-x509 VPN confgurations added to existing app: django-netjsonconfg
  • 95. 7. Rule of Transparency Log unexpected events using the python logging facility
  • 96. Logging of bad requests in django-netjsonconfg import logging logger = logging.getLogger(__name__) def invalid_response(request, error, status): logger.warning(error, extra={'request': request, 'stack': True}) return ControllerResponse(error, status=status)
  • 97. 7. Real world examples Provide good default logging Provide support for sentry Take a look at a real example
  • 98. 8. Rule of Robustness Add constraints to your reusable django apps only when necessary: ● avoid very strict validation rules ● provide confgurable settings
  • 99. 8. Real world examples ● settings in django-netjsonconfg ● settings in django-x509
  • 100. 9. Rule of Representation Fold complex information in data structures Process these data structures with algorithms Try to make algorithms framework-agnostic (if possible)
  • 101. 9. Real world examples in OpenWISP 2, confguration of routers is implemented as a single text feld formatted as NetJSON (in OpenWISP 1 each available confguration had its own database table and model, very hard to maintain and evolve)
  • 102. # NetJSON DeviceConfiguration example { "general": {"hostname": "HomeRouter"}, "interfaces": [ { "name": "eth0", "type": "ethernet", "addresses": [ { "address": "192.168.1.1", "mask": 24, "proto": "static", "family": "ipv4" } ] } ] }
  • 103.
  • 104.
  • 105. 10. Rule of Least Surprise This has become a widely accepted concept in the IT industry
  • 106. 11. Rule of Silence Just don't make your apps annoying It's not that hard :-P
  • 107. 12. Rule of Repair Errors should never pass silently Unless explicitly silenced (from the zen of python) Let's add: fail fast, noisily and early
  • 108. from django.core.exceptions import ImproperlyConfigured from .settings import REGISTRATION_ENABLED, SHARED_SECRET if REGISTRATION_ENABLED and not SHARED_SECRET: msg = 'NETJSONCONFIG_SHARED_SECRET not set!' raise ImproperlyConfigured(msg) Full real world example available in django-netjsonconfg
  • 109. 13. Rule of Economy When Unix was born this was a radical idea: assembler was the norm, C was considered a higher level language Python is a consequence of the success of that radical idea Embrace this concept
  • 110. 14. Rule of Generation When you fnd yourself writing lots of boilerplate code, try to use meta-programming or code generators
  • 111. 14. meta-programmed urlpatterns urls.py in openwisp-controller
  • 112. 14. Auto-generated UI from json-schema JSON-Schema of this UI available on github
  • 113. 15. Rule of Optimization ● write a frst early version of your django app ● do not forget to write automated tests ● refactor and polish
  • 114. 16. Rule of Diversity Maintain a skeptic attitude towards dogmas and fads Measure the effectiveness of an approach before embracing it
  • 115. 17. Rule of Extensibility Provide ways to extend and customize the behaviour of modules without the need of changing the core code
  • 116. 17. Extensibility: abstract models ● Provide abstract models in your most important modules
  • 117. 17. Extensibility: abstract models ● Provide abstract models in your most important modules ● Store these in a python fle which does not import concrete models (otherwise other django apps won't be able to import them)
  • 118. 17. Extensibility: abstract models abstract models in django-netjsonconfg example usage
  • 119. 17. Extensibility: abstract models abstract models in django-x509 example usage
  • 120. 17. Extensibility: base admin ● Provide base admin classes in your main modules (avoid duplication)
  • 121. 17. Extensibility: base admin ● Provide base admin classes in your main modules (avoid duplication) ● Store them in a python fle which does not import concrete models (otherwise other django-apps won't be able to import them)
  • 122. 17. Extensibility: base admin base admin classes django-netjsonconfg example usage
  • 123. 17. Extensibility: base views ● If your reusable django app has views, provide generic views that can be extended ● Store them in a python fle which does not import concrete models (otherwise other django-apps won't be able to import them)
  • 124. 17. Extensibility: base views base views in django-netjsonconfg example usage
  • 125. 17. Extensibility: reusable urls If your app provides views that can be extended, third party apps will have to redefne their URLs You may want to avoid this required duplication by providing a mechanism to import urls
  • 126. 17. Extensibility: reusable urls reusable urls in django-netjsonconfg example usage: from django_netjsonconfig.utils import get_controller_urls from . import views # customized views # creates new url patterns hooked to customized views urlpatterns = get_controller_urls(views)
  • 127. 17. Extensibility: AppConfig If your reusable django app relies on signal connection for some features, provide a base AppConfig class
  • 128. 17. Extensibility: AppConfig base AppConfig class in django-netjsonconfg from django_netjsonconfig.apps import OpenWispAppConfig class MyOwnApp(OpenWispAppConfig): name = 'yourapp.config' label = 'config' def __setmodels__(self): # these are your custom models from .models import Config, VpnClient self.config_model = Config self.vpnclient_model = VpnClient
  • 129. Positive results Positive results achieved in OpenWISP
  • 130. 1. Faster release cycle & evolution We release new features more often The project is evolving rapidly
  • 131. 2. Easier maintainance Once a bug in a specifc module is replicated fxing it is easier compared to the work needed to fx bugs in OpenWISP 1
  • 132. 3. More derivative work OpenWISP 2 was released officially less than 1 year ago (as of April 2017) notwithstanding that, there are already a couple of derivative works
  • 133. 4. Growth The user base has been growing rapidly Users send feedback and patches
  • 135. 4. Growth: GSOC 2017 OpenWISP has also been accepted as a mentoring organization for the Google Summer of Code 2017
  • 136.
  • 137. Disadvantages Is not a bed of roses
  • 138.
  • 139. 1. Integration issues Combining more django apps may result in integration issues Changes to project confguration can result in bugs that are not caught by unit tests
  • 140. 1. Solution Integration tests are important in this case Shortcut: I was able to import tests of a base app and repeat them in the extension app This easy fx was good enough for my case
  • 141. 2. Big features When I needed to introduce a new major feature I needed to change at least a couple of modules
  • 142. 2. Solution many times this will be necessary If you have to change many modules every time, refactor your code to be more loosely coupled
  • 143. 3. Repository management overhead ● working with many repositories can be overwhelming
  • 144. 3. Repository management overhead ● working with many repositories can be overwhelming ● more git tags, pypi releases, versioning, changelogs
  • 145. 3. Repository management overhead ● working with many repositories can be overwhelming ● more git tags, pypi releases, versioning, changelogs ● more announcements for new releases
  • 146. 3. Solutions ● import several projects in a single window of your editor
  • 147. 3. Solutions ● import several projects in a single window of your editor ● automate versioning and changelogs (I haven't done this yet)
  • 148. 3. Solutions ● import several projects in a single window of your editor ● automate versioning and changelogs (I haven't done this yet) ● do not send announcements for minor releases which contain no real advantage for end users
  • 149. 4. Documentation fragmentation We have many repositories with their own READMEs and docs We don't have a single documentation website yet
  • 150. 4. Solutions I added links to specifc module docs from the website I often have to send those links on the mailing list But a central comprehensive documentation website would be better (we don't have this yet)
  • 151. 5. Bug reports Users don't know where to send bug reports and they write to the mailing list
  • 152. 5. Solution ● For the moment I reply by pointing to them where to open issues
  • 153. 5. Solution ● For the moment I reply by pointing to them where to open issues ● But a single issue tracker for all the repositories would be better (even though it may not really solve the "which module?" problem)
  • 154. 6. Dispersion of popularity We no longer have a single popular repository with many github stars
  • 155. 6. Solution Stop caring about this Github stars are not an accurate indicator of the success of a project
  • 156. Conclusions The creators of Unix are not newcomers Following their heritage is a good idea
  • 157. Delve deep Further resources on the Unix Philosophy and related concepts: ● the art of unix programming ● the pragmatic programmer
  • 158. Practice Start to gradually apply these concepts in your projects EG: extract big features to separate projects
  • 159. NO to dogmas ● Practice empirical (scientifc) skepticism
  • 160. NO to dogmas ● Practice empirical (scientifc) skepticism ● Test your ideas
  • 161. NO to dogmas ● Practice empirical (scientifc) skepticism ● Test your ideas ● Measure your results
  • 162. Read existing code Read the code of OpenWISP @openwisp organization on github (or other projects like @openstack)
  • 163. Unix Philosophy for the win!
  • 164. Thank you Ideas, critical feedback, suggestions? Talk to me! Find me also on: ● twitter (@nemesisdesign) ● github (@nemesisdesign) ● linkedin (Federico Capoano) You can fnd the slides on slideshare