Managing Creativity Pycon UK, Sept 2007 Michael Sparks Kamaelia Project BBC Research

Conflicting needs Research Don't know what the problem is Don't know the solution Don't know suitable approach Unknown requirements Production Know the problem inside out Have a solution which must be reliable Approach is known and engineered Requirements clear, but with scope for growth

Research

Don't know what the problem is

Don't know the solution

Don't know suitable approach

Unknown requirements

Production

Know the problem inside out

Have a solution which must be reliable

Approach is known and engineered

Requirements clear, but with scope for growth

Research Don't know what the problem is Don't know the solution Don't know suitable approach Unknown requirements Production Know the problem inside out Have a solution which must be reliable Approach is known and engineered Requirements clear, but with scope for growth Conflicting needs How to go from here to here?

Research

Don't know what the problem is

Don't know the solution

Don't know suitable approach

Unknown requirements

Production

Know the problem inside out

Have a solution which must be reliable

Approach is known and engineered

Requirements clear, but with scope for growth

Research Don't know what the problem is Don't know the solution Don't know suitable approach Unknown requirements Production Know the problem inside out Have a solution which must be reliable Approach is known and engineered Requirements clear, but with scope for growth Conflicting needs More like this!

Research

Don't know what the problem is

Don't know the solution

Don't know suitable approach

Unknown requirements

Production

Know the problem inside out

Have a solution which must be reliable

Approach is known and engineered

Requirements clear, but with scope for growth

Process Have an idea Build something to see if idea is good Review & build systems using it it is a good idea Make possible to use in production system Make production quality Exploration is very different from polishing

Have an idea

Build something to see if idea is good

Review & build systems using it it is a good idea

Make possible to use in production system

Make production quality

Exploration is very different from polishing

hackers&painters vs CS Hackers & Painters essentially postulates that coding is like painting Computer Science essentially postulates that coding is like engineering with solid foundations we can follow sound rules & constraints Both are right even though in conflict Huh?

Hackers & Painters essentially postulates that coding is like painting

Computer Science essentially postulates that coding is like engineering with solid foundations we can follow sound rules & constraints

Both are right even though in conflict

Huh?

Building a Building Sketch a design Collect system requirements Build models Refine Architect building based on physical stresses etc Refine Build Both engineered & crafted

Sketch a design

Collect system requirements

Build models

Refine

Architect building based on physical stresses etc

Refine

Build

Both engineered & crafted

Key Trick – Code as Wiki In python terms: Namespaces are one honking great idea -- let's do more of those! -- import this Or wiki terms: Enlarge Space - In order to preserve GlobalResources, create more public space. This reduces limited resource tension. Unlike the RealWorld, land is cheap online.

In python terms:

Namespaces are one honking great idea

-- let's do more of those! -- import this

Or wiki terms:

Enlarge Space - In order to preserve GlobalResources, create more public space. This reduces limited resource tension. Unlike the RealWorld, land is cheap online.

Core Approach Encourage play - capture results (sketchbooks) Encourage system building, component extraction Reuse & refine components Make testable Reimplement as TDD – potentially formalise tests as formal spec Version control as core enabler

Encourage play - capture results (sketchbooks)

Encourage system building, component extraction

Reuse & refine components

Make testable

Reimplement as TDD – potentially formalise tests as formal spec

Version control as core enabler

Core Feature Flow Idea & Initial System /trunk/Sketches/<initials> Componentisation & Systems /branches/private_<initials>_Scratch (sometimes) Reuse & Candidate for main tree /branches/private_<initials>_anything Merge into Main Tree /trunk/Code

Idea & Initial System

/trunk/Sketches/<initials>

Componentisation & Systems

/branches/private_<initials>_Scratch (sometimes)

Reuse & Candidate for main tree

/branches/private_<initials>_anything

Merge into Main Tree

/trunk/Code

Feature Consolidation Testable /trunk/Code/<path>/Example/<example> /trunk/Code/<path>/Tests/test_<component>.py TDD /branches/private_<initials>_FeatureRewrite Merge /branches/private_<initials>_FeatureRewrite

Testable

/trunk/Code/<path>/Example/<example>

/trunk/Code/<path>/Tests/test_<component>.py

TDD

/branches/private_<initials>_FeatureRewrite

Merge

/branches/private_<initials>_FeatureRewrite

Feature Future Proofing Tests as formal spec /trunk/Tests/Python/<project>/test_.... eg /trunk/Tests/Python/Axon/test_*

Tests as formal spec

/trunk/Tests/Python/<project>/test_....

eg /trunk/Tests/Python/Axon/test_*

Feature Extension Idea /trunk/Sketches/<initials> Componentisation & Systems /branches/private_<initials>_Scratch (sometimes) Reuse & Candidate for main tree /branches/private_<initials>_anything Merge into Main Tree /trunk/Code

Idea

/trunk/Sketches/<initials>

Componentisation & Systems

/branches/private_<initials>_Scratch (sometimes)

Reuse & Candidate for main tree

/branches/private_<initials>_anything

Merge into Main Tree

/trunk/Code

Alternative View Idea Shared Sketchbooks Componentisation & Systems Visible integration & demo of use Reuse & Candidate for main tree I'm handing this off for review Merge into Main Tree Release to general user base

Idea

Shared Sketchbooks

Componentisation & Systems

Visible integration & demo of use

Reuse & Candidate for main tree

I'm handing this off for review

Merge into Main Tree

Release to general user base

Alternative View Idea Shared Sketchbooks Componentisation & Systems Visible integration & demo of use Reuse & Candidate for main tree I'm handing this off for review Merge into Main Tree Release to general user base Can use code for idea at all stages of development

Idea

Shared Sketchbooks

Componentisation & Systems

Visible integration & demo of use

Reuse & Candidate for main tree

I'm handing this off for review

Merge into Main Tree

Release to general user base

Shared Sketchbooks /Sketches

Shared Sketchbooks: /Sketches /Sketches/MPS – this is mine, I can do what I like there. Does not have to work Can be imperfect Everyone else can see it eg /Sketches/MPS/Systems/Paint

/Sketches/MPS – this is mine, I can do what I like there.

Does not have to work

Can be imperfect

Everyone else can see it

eg

/Sketches/MPS/Systems/Paint

Shared Sketchbooks: /Sketches /Sketches/MH – this is Matt's, he can do what he likes there. Does not have to work, can be imperfect, etc Can copy other people's sketches & extend eg /Sketches/MH/Sketcher

/Sketches/MH – this is Matt's, he can do what he likes there.

Does not have to work, can be imperfect, etc

Can copy other people's sketches & extend

eg

/Sketches/MH/Sketcher

Shared Sketchbooks: /Sketches Can also play with new ideas Handwriting recognition 3D Page turning a “record for me” PVR GestureRecognition/StrokeRecogniser.py , OpenGL/3fFolding.py, DVB_PSI/PVR.py

Can also play with new ideas

Handwriting recognition

3D Page turning

a “record for me” PVR

GestureRecognition/StrokeRecogniser.py , OpenGL/3fFolding.py, DVB_PSI/PVR.py

Shared Sketchbooks: /Sketches Same goes for anyone else Trainee engineers Summer of code students Anyone else (you?) Can get feedback much sooner, and do fast diffs

Same goes for anyone else

Trainee engineers

Summer of code students

Anyone else (you?)

Can get feedback much sooner, and do fast diffs

What's there? Video annotating whiteboard, P2P swarming radio, IRC clients, pygame experiments, topology visualisation, paint programs, a multicast toolset, shedskin (python to C++) compatible micro-axon, simple reliable multicast experiments, sub-component level experiments, new tools for integration outside kamaelia, networked audio mixer matrix, record everything tv experiments, seaside-like HTTP server, random experiments, tools for multicast of DVB & playback, networking of pipes distribution, gesture (and basic handwriting) recognition, custom PVRs, mobile reframers, 3D video playback, PVR content viewer for mobiles, trusted communications experiments, speex tools, Icecast clients, subtitling tickers, libao tests, early 3D work, bittorrent integration work, DVB tools, weather scraping etc

Video annotating whiteboard, P2P swarming radio, IRC clients, pygame experiments, topology visualisation, paint programs, a multicast toolset, shedskin (python to C++) compatible micro-axon, simple reliable multicast experiments, sub-component level experiments, new tools for integration outside kamaelia, networked audio mixer matrix, record everything tv experiments, seaside-like HTTP server, random experiments, tools for multicast of DVB & playback, networking of pipes distribution, gesture (and basic handwriting) recognition, custom PVRs, mobile reframers, 3D video playback, PVR content viewer for mobiles, trusted communications experiments, speex tools, Icecast clients, subtitling tickers, libao tests, early 3D work, bittorrent integration work, DVB tools, weather scraping etc

Visible Integration

Visible Integration /branches/private_<initials>_Scratch Personal version of trunk – may or may not actually be shared. Benefits of sharing Not at the whims of a merger to use new namespaces Don't corrupt others Provide mechanism to demo is “OK”

/branches/private_<initials>_Scratch

Personal version of trunk – may or may not actually be shared.

Benefits of sharing

Not at the whims of a merger to use new namespaces

Don't corrupt others

Provide mechanism to demo is “OK”

Visible Integration As you go through this process – more detail may emerge, and become clearer making more generally useful – better case for mainline merge

As you go through this process – more detail may emerge, and become clearer making more generally useful – better case for mainline merge

Hand off branches

Hand off branches Always named /branches/private_<initials>_feature Never merged to /trunk by their creator, unless agreed by all admins – encourage review & feedback.

Always named

/branches/private_<initials>_feature

Never merged to /trunk by their creator, unless agreed by all admins – encourage review & feedback.

Main Code Tree: /Code Code only reaches main code tree if: Has proved itself useful in /Sketches Merge onto pseudo trunk works well Passes code review

Code only reaches main code tree if:

Has proved itself useful in /Sketches

Merge onto pseudo trunk works well

Passes code review

Key points Always have permission to create new branches named appropriately Always have permission to check in new code “ Force of gravity” approach to consolidation Clear & simple code migration rules Version control as a tool to support process management tool

Always have permission to create new branches named appropriately

Always have permission to check in new code

“ Force of gravity” approach to consolidation

Clear & simple code migration rules

Version control as a tool to support process management tool

Key points Encourages play, encourages innovation, but in a manageable way, that also encourages development of an engineered usable system Really about “who owns a namespace”

Encourages play, encourages innovation, but in a manageable way, that also encourages development of an engineered usable system

Really about “who owns a namespace”