Moodle Development Best Practices

           Presented by:
            Justin Filip
Presentation outline


●
    Intro / development environment
●
    Best practices
●
    Peer reviews
●
    Working with the Moodle tracker
●
    Maintaining a plugin with Github
●
    Submitting a core patch to Moodle HQ
Development environment

●
    Web server:
     –   Apache
     –   Lighttpd
     –   Nginx
●
    Database server:
     –   MySQL
     –   PostgreSQL
●
    PHP (5.3.2 or later for Moodle 2.2)
●
    Code editor:
     –   Console (vim, Emacs)
     –   IDE (Eclipse, Netbeans, Sublime Text)
Console editor - Emacs
IDE - Netbeans
General development settings


●
    Enable developer debugging from Debugging section of the
    Development section of the Site administration menu
●
    If working with real user data, prevent emails from being sent
    out by adding $CFG->noemailever = true; to your
    config.php file
●
    If working on theme / CSS changes, enable Theme designer
    mode from the Theme settings section of the Site
    administration menu
     –   Important! Do not leave this setting enabled on
         production sites as it has a large performance hit
Best practices


●
    Correct usage of plugins and APIs
●
    Security
●
    Performance
●
    Testing
●
    Coding style
Key concepts: Correct usage of
plugins and APIs
Correct usage of plugins and APIs


●
    Key concepts:
    –   Plugin types
    –   Core APIs
    –   Database system
Key concepts: Moodle plugins


●
    http://docs.moodle.org/dev/Plugins
     –   Activity modules
     –   Authentication plugins
     –   Blocks
     –   Course formats
     –   Enrolment plugins
     –   Filters
     –   Local plugins
     –   Reports
     –   Repository plugins
     –   Themes
Extra: Moodle 2.3 plugin changes


●
    Assignment module
     –   /mod/assign/submission/
     –   /mod/assign/feedback/
●
    Repository plugins
     –   Support for alias / shortcut to external content
     –   supported_returntypes() should add
         FILE_REFERENCE to the return values to indicate this
Key concepts: database system


●
    http://docs.moodle.org/dev/Data_definition_API
●
    Plugin specific documentation – http://tinyurl.com/a3gvpjj
●
    Always use a primary key column called id
●
    Add indexes and foreign keys to your database tables
●
    If writing full SQL queries make sure to put table names within
          braces without the prefix
     –   {assignment_submissions} instead of
            mdl_assignment_submissions
●
    Use the built-in XMLDB editor in Moodle for managing your
        XML install files
Key concepts: Moodle core APIs

●
    Access
●
    Data manipulation (DML)
●
    File
●
    Form
●
    Logging
●
    Navigation
●
    Page
●
    Strings
●
    Upgrade
●
    Extra: data definition (DDL)
Correct usage of plugins and APIs:
Summary


    ●
        Key concepts:
        –   Plugin types
        –   Core APIs
        –   Database system
Security
Security


●
    Key concepts:
    –   Authentication
    –   Roles and capabilities
    –   User input
Key concepts: authentication


●
    How users get into the system
●
    Can connect to external systems:
     –   Problems if that system is unavailable
     –   SSO & SSI
●
    Confirming users and deletion
Key concepts: roles and capabilities


●
    Context levels:
     –   CONTEXT_SYSTEM, CONTEXT_COURSE, CONTEXT_USER,
         etc.
●
    Roles:
     –   Definitions
     –   Applicable contexts
●
    Capabilities
     –   Use the most specific capability possible
     –   Check in the most specific context level
Extra: context hierarchy
Key concepts: user input

●
    Never ever trust user input!!!
     –   Common web application problem
     –   Always sanitise input data
     –   required_param() & optional_param()
     –   required_param_array() &
         optional_param_array()
●
    Input parameter types:
     –   PARAM_ALPHA, PARAM_ALPHANUM, PARAM_NOTAGS,
         PARAM_CLEANHTML
     –   PARAM_EMAIL, PARAM_URL, PARAM_SAFEDIR
●
    Moodle form API (formslib) and SESSKEY validation
Extra: protecting access to your code


●
    Command-line only scripts:
    –   define('CLI_SCRIPT', true);
●
    Preventing directly loading a PHP file:
    –   defined('MDL_INTERNAL') ||
        die();
Security: Summary


●
    Authentication
●
    Roles and capabilities
●
    User input
Performance
Performance


●
    Key concepts:
    –   Database IO
    –   Profiling
Key concepts: database IO


●
    Limit returned dataset using $fields parameter
●
    Use result set paging with $limitfrom and $limitnum
●
    Use record sets to not load all returned data into memory
●
    Writing custom SQL queries
     –   Database agnostic
     –   Caution when using JOINs and subqueries
Key concepts: profiling


●
    Use good testing data
     –   Try to use a large dataset
     –   If at all available, use a copy of data from a production
         site
          ●
              Set $CFG->noemailever = true; in your Moodle
              config.php file
     –   Enable performance output on each Moodle page
          ●
              Displays execution time, memory usage, DB read /
              write, etc.
          ●
              http://tinyurl.com/axkuqjz
Key concepts: profiling with XHProf


●
    Facebook-developed live profiling of PHP code
●
    Built into Moodle
●
    Quantitative analysis of results from a page execution
●
    Pin-pointing performance problems
●
    Critical execution path
●
    http://techportal.inviqa.com/2009/12/01/profiling-with-xhprof/
     –   Ignore everything before the How to use XHProf section
XHProf: summary data
XHProf: execution call graph
Performance: Summary


●
    Database IO
●
    Profiling
Testing
Testing


●
    Key concepts:
    –   Unit testing
    –   Functional testing
    –   Performance testing
Key concepts: unit testing


●
    PHPUnit
     –   https://github.com/sebastianbergmann/phpunit/
●
    Testing single functions and methods in isolation
●
    Added to Moodle core in 2.3.0
●
    PHPUnit tests menu in the Development section of the Site
    administration menu contains more information how to setup
    and use them
●
    See examples from other core code and plugins
Key concepts: functional testing


●
    Testing interaction with your code via web browser or a
    simulated web browser
●
    Can be used to find UI display problems across multiple
    browsers / OSs
●
    Selenium – http://seleniumhq.org/
●
    Moodle HQ Behat testing
     –   https://moodle.org/mod/forum/discuss.php?d=221638
     –   To be available for plugin authors as well as used in core
         Moodle
Key concepts: performance testing


●
    Jmeter
    –   http://jmeter.apache.org/
●
    Stress testing
●
    Simulates load from many concurrent users
Extra: Moodle Universal Cache (MUC)


●
    http://tinyurl.com/by7gs3p
●
    A generic caching system that any code can use
●
    Can plug into different back-end caching systems
●
    Introduced in Moodle 2.4.0
●
    Available to add-ons now, core components to use it in
    Moodle 2.0
Testing: Summary


●
    Unit testing
●
    Functional testing
●
    Performance testing
Coding style
Coding style


●
    More about how you write your code, not necessarily what you
    write
●
    Consistency allows for familiarity when looking at new areas
●
    Can prevent “bad” or sub-optimal code from being released
●
    CodeChecker plugin:
     –   http://tinyurl.com/a9z9d8o
Best practices: Summary


●
    Correct usage of plugins and APIs
●
    Security
●
    Performance
●
    Testing
●
    Coding style
Peer reviews


●
    Attempt to find problems before testing
●
    Ensure consistency in new code
●
    Make sure required information is present and correct
●
    Teaching tool for new developers
●
    Enforces style and correctness of solution
●
    Verify that new code is not using deprecated functionality
Peer review checklist

●
    Syntax
●
    Whitespace
●
    Output
●
    Language
●
    Databases
●
    Testing
●
    Security
●
    Documentation
●
    Git
●
    Sanity check
Working with the Moodle tracker


●
    Key concepts:
    –   Creating new issues
    –   Working on an existing issue
Key concepts: creating new issues


●
    Always make sure to see if the issue you want to create
    already exists
●
    Make sure to report as much information as possible
     –   Affects Version/s
     –   Database
     –   Testing instructions
     –   Workaround
●
    Important! Always include debugging messages and
    screenshots if reporting a bug
Key concepts: working on an existing issue


●
    Make sure nobody is working on it already
●
    Put in a comment to ask for the issue to be assigned to you
●
    Put your work in a publicly available Git repository and provide
    this information in the issue
●
    Make sure that you provide testing instructions for your work
●
    Request peer review when your work is finished
Maintaing a plugin with Github


●
    Use correct repository name (see Moodle Frankenstyle)
     –   http://docs.moodle.org/dev/Frankenstyle
●
    Create branches for supported Moodle versions (e.g.
    MOODLE_23_STABLE, MOODLE_24_STABLE)
●
    Provide documentation in the MoodleDocs wiki
●
    Submit the plugin to the Moodle.org plugins database
●
    Keep track of bugs and new features in the official tracker
     –   Request a component be created for your plugin in the
         Non-core contributed modules project
     –   http://tinyurl.com/b7xc7nv
Submitting core patches to HQ


●
    Fork the Moodle Github repository
    –   https://github.com/moodle/moodle
●
    Create branches for affected versions
●
    If this is for an existing issue, post the
    information for your repository into the issue
●
    If this is for something new, create a new
    issue and start a discussion in the Moodle.org
    forums
Thank you


●
    This presentation is available on SlideShare
    –   http://tinyurl.com/a3w7m2f


●
    Find me on Twitter
    –   @jfilip

Moodle Development Best Pracitces

  • 1.
    Moodle Development BestPractices Presented by: Justin Filip
  • 2.
    Presentation outline ● Intro / development environment ● Best practices ● Peer reviews ● Working with the Moodle tracker ● Maintaining a plugin with Github ● Submitting a core patch to Moodle HQ
  • 3.
    Development environment ● Web server: – Apache – Lighttpd – Nginx ● Database server: – MySQL – PostgreSQL ● PHP (5.3.2 or later for Moodle 2.2) ● Code editor: – Console (vim, Emacs) – IDE (Eclipse, Netbeans, Sublime Text)
  • 4.
  • 5.
  • 6.
    General development settings ● Enable developer debugging from Debugging section of the Development section of the Site administration menu ● If working with real user data, prevent emails from being sent out by adding $CFG->noemailever = true; to your config.php file ● If working on theme / CSS changes, enable Theme designer mode from the Theme settings section of the Site administration menu – Important! Do not leave this setting enabled on production sites as it has a large performance hit
  • 7.
    Best practices ● Correct usage of plugins and APIs ● Security ● Performance ● Testing ● Coding style
  • 8.
    Key concepts: Correctusage of plugins and APIs
  • 9.
    Correct usage ofplugins and APIs ● Key concepts: – Plugin types – Core APIs – Database system
  • 10.
    Key concepts: Moodleplugins ● http://docs.moodle.org/dev/Plugins – Activity modules – Authentication plugins – Blocks – Course formats – Enrolment plugins – Filters – Local plugins – Reports – Repository plugins – Themes
  • 11.
    Extra: Moodle 2.3plugin changes ● Assignment module – /mod/assign/submission/ – /mod/assign/feedback/ ● Repository plugins – Support for alias / shortcut to external content – supported_returntypes() should add FILE_REFERENCE to the return values to indicate this
  • 12.
    Key concepts: databasesystem ● http://docs.moodle.org/dev/Data_definition_API ● Plugin specific documentation – http://tinyurl.com/a3gvpjj ● Always use a primary key column called id ● Add indexes and foreign keys to your database tables ● If writing full SQL queries make sure to put table names within braces without the prefix – {assignment_submissions} instead of mdl_assignment_submissions ● Use the built-in XMLDB editor in Moodle for managing your XML install files
  • 13.
    Key concepts: Moodlecore APIs ● Access ● Data manipulation (DML) ● File ● Form ● Logging ● Navigation ● Page ● Strings ● Upgrade ● Extra: data definition (DDL)
  • 14.
    Correct usage ofplugins and APIs: Summary ● Key concepts: – Plugin types – Core APIs – Database system
  • 15.
  • 16.
    Security ● Key concepts: – Authentication – Roles and capabilities – User input
  • 17.
    Key concepts: authentication ● How users get into the system ● Can connect to external systems: – Problems if that system is unavailable – SSO & SSI ● Confirming users and deletion
  • 18.
    Key concepts: rolesand capabilities ● Context levels: – CONTEXT_SYSTEM, CONTEXT_COURSE, CONTEXT_USER, etc. ● Roles: – Definitions – Applicable contexts ● Capabilities – Use the most specific capability possible – Check in the most specific context level
  • 19.
  • 20.
    Key concepts: userinput ● Never ever trust user input!!! – Common web application problem – Always sanitise input data – required_param() & optional_param() – required_param_array() & optional_param_array() ● Input parameter types: – PARAM_ALPHA, PARAM_ALPHANUM, PARAM_NOTAGS, PARAM_CLEANHTML – PARAM_EMAIL, PARAM_URL, PARAM_SAFEDIR ● Moodle form API (formslib) and SESSKEY validation
  • 21.
    Extra: protecting accessto your code ● Command-line only scripts: – define('CLI_SCRIPT', true); ● Preventing directly loading a PHP file: – defined('MDL_INTERNAL') || die();
  • 22.
    Security: Summary ● Authentication ● Roles and capabilities ● User input
  • 23.
  • 24.
    Performance ● Key concepts: – Database IO – Profiling
  • 25.
    Key concepts: databaseIO ● Limit returned dataset using $fields parameter ● Use result set paging with $limitfrom and $limitnum ● Use record sets to not load all returned data into memory ● Writing custom SQL queries – Database agnostic – Caution when using JOINs and subqueries
  • 26.
    Key concepts: profiling ● Use good testing data – Try to use a large dataset – If at all available, use a copy of data from a production site ● Set $CFG->noemailever = true; in your Moodle config.php file – Enable performance output on each Moodle page ● Displays execution time, memory usage, DB read / write, etc. ● http://tinyurl.com/axkuqjz
  • 27.
    Key concepts: profilingwith XHProf ● Facebook-developed live profiling of PHP code ● Built into Moodle ● Quantitative analysis of results from a page execution ● Pin-pointing performance problems ● Critical execution path ● http://techportal.inviqa.com/2009/12/01/profiling-with-xhprof/ – Ignore everything before the How to use XHProf section
  • 28.
  • 29.
  • 30.
    Performance: Summary ● Database IO ● Profiling
  • 31.
  • 32.
    Testing ● Key concepts: – Unit testing – Functional testing – Performance testing
  • 33.
    Key concepts: unittesting ● PHPUnit – https://github.com/sebastianbergmann/phpunit/ ● Testing single functions and methods in isolation ● Added to Moodle core in 2.3.0 ● PHPUnit tests menu in the Development section of the Site administration menu contains more information how to setup and use them ● See examples from other core code and plugins
  • 34.
    Key concepts: functionaltesting ● Testing interaction with your code via web browser or a simulated web browser ● Can be used to find UI display problems across multiple browsers / OSs ● Selenium – http://seleniumhq.org/ ● Moodle HQ Behat testing – https://moodle.org/mod/forum/discuss.php?d=221638 – To be available for plugin authors as well as used in core Moodle
  • 35.
    Key concepts: performancetesting ● Jmeter – http://jmeter.apache.org/ ● Stress testing ● Simulates load from many concurrent users
  • 36.
    Extra: Moodle UniversalCache (MUC) ● http://tinyurl.com/by7gs3p ● A generic caching system that any code can use ● Can plug into different back-end caching systems ● Introduced in Moodle 2.4.0 ● Available to add-ons now, core components to use it in Moodle 2.0
  • 37.
    Testing: Summary ● Unit testing ● Functional testing ● Performance testing
  • 38.
  • 39.
    Coding style ● More about how you write your code, not necessarily what you write ● Consistency allows for familiarity when looking at new areas ● Can prevent “bad” or sub-optimal code from being released ● CodeChecker plugin: – http://tinyurl.com/a9z9d8o
  • 40.
    Best practices: Summary ● Correct usage of plugins and APIs ● Security ● Performance ● Testing ● Coding style
  • 41.
    Peer reviews ● Attempt to find problems before testing ● Ensure consistency in new code ● Make sure required information is present and correct ● Teaching tool for new developers ● Enforces style and correctness of solution ● Verify that new code is not using deprecated functionality
  • 42.
    Peer review checklist ● Syntax ● Whitespace ● Output ● Language ● Databases ● Testing ● Security ● Documentation ● Git ● Sanity check
  • 43.
    Working with theMoodle tracker ● Key concepts: – Creating new issues – Working on an existing issue
  • 44.
    Key concepts: creatingnew issues ● Always make sure to see if the issue you want to create already exists ● Make sure to report as much information as possible – Affects Version/s – Database – Testing instructions – Workaround ● Important! Always include debugging messages and screenshots if reporting a bug
  • 45.
    Key concepts: workingon an existing issue ● Make sure nobody is working on it already ● Put in a comment to ask for the issue to be assigned to you ● Put your work in a publicly available Git repository and provide this information in the issue ● Make sure that you provide testing instructions for your work ● Request peer review when your work is finished
  • 46.
    Maintaing a pluginwith Github ● Use correct repository name (see Moodle Frankenstyle) – http://docs.moodle.org/dev/Frankenstyle ● Create branches for supported Moodle versions (e.g. MOODLE_23_STABLE, MOODLE_24_STABLE) ● Provide documentation in the MoodleDocs wiki ● Submit the plugin to the Moodle.org plugins database ● Keep track of bugs and new features in the official tracker – Request a component be created for your plugin in the Non-core contributed modules project – http://tinyurl.com/b7xc7nv
  • 47.
    Submitting core patchesto HQ ● Fork the Moodle Github repository – https://github.com/moodle/moodle ● Create branches for affected versions ● If this is for an existing issue, post the information for your repository into the issue ● If this is for something new, create a new issue and start a discussion in the Moodle.org forums
  • 48.
    Thank you ● This presentation is available on SlideShare – http://tinyurl.com/a3w7m2f ● Find me on Twitter – @jfilip