Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenERP 6.1 Framework Changes

16,413 views

Published on

http://openerp.com

Technical Presentation of the main changes in OpenERP 6.1 at the API and framework level: architecture changes, API evolution, new UI features, etc.

See also the related presentation about the new 6.1 Web layer: http://www.slideshare.net/openobject/openerp-61-web-framework-tutorial

Published in: Technology, Education

OpenERP 6.1 Framework Changes

  1. 1. OpenERP 6.1 Technical Changes at API/Framework level @odony – 6.1 Framework Changes – 05.2012
  2. 2. OpenERP 6.1 ● Released February 2011 ● Normal release – non LTS = limited support ● Release Notes ● http://bit.ly/openerp61RN ● Focus ● Usability ● Social features ● New web client ● Cleanup @odony – 6.1 Framework Changes – 05.2012
  3. 3. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  4. 4. Evolution: Architecture changes ● Unified server-side deployment ● Scalability through multi-processing ● Use all those cores on modern hardware ● Bypass Python GIL issues ● Reliability (cpu/memory/crashes) ● WSGI ● Standard Python solution – OpenERP 6.1 is compliant ● Gunicorn - a WSGI compliant HTTP server ● Statelessness ● Distributed scheduled tasks (cron) @odony – 6.1 Framework Changes – 05.2012
  5. 5. Unified server-side deployment ● No more need to deploy web client ● Web client is now an OpenERP module ● Only one package to install ● Pre-loaded server-wide via –-load=web ● All modules in same place (addons_path) ● openerp-web still available stand-alone ● For testing/debugging purposes ● Needs to have copy of normal modules too! @odony – 6.1 Framework Changes – 05.2012
  6. 6. Scalability through multi-processing ● Use all those cores on modern hardware ● Bypass Python GIL limitations ● Improve reliability (cpu/memory/crashes) → What does this all mean?? @odony – 6.1 Framework Changes – 05.2012
  7. 7. Multi-processing and GIL (1) ● CPython has a Global Interpreter Lock/GIL ● Simplifies low-level memory management, C calls, etc. ● 1 Python thread = 1 OS thread ● Parallel execution of threads forbidden ● Cooperative multi-tasking on 1 CPU ● Worse performance on multi-CPU/core! ● Battle for GIL on the different virtual CPUs! @odony – 6.1 Framework Changes – 05.2012
  8. 8. Multi-processing and GIL (2) ● Cooperative multi-tasking ● Release on I/O or every 100 ticks ● 1 tick ≈ 1 interpreter instruction @odony – 6.1 Framework Changes – 05.2012
  9. 9. Scalability through multi-processing ● multiprocessing module was an option ● Using Unix tools and real processes was more flexible and easier to scale up ● Managing the processes was not necessary, many tools exist to do that (e.g. Gunicorn) ● Just had to make OpenERP WSGI-compliant @odony – 6.1 Framework Changes – 05.2012
  10. 10. WSGI (1) ● Web Server Gateway Interface for Python ● WSGI applications can be embedded in WSGI-compliant web servers ● Many web frameworks support WSGI ● Django, CherryPy, … ● Many HTTP servers support WSGI ● Apache(+mod), nginx(+mod), tornado, gunicorn, … @odony – 6.1 Framework Changes – 05.2012
  11. 11. WSGI (2) ● Very simple API: one method ● environ = HTTP environment map ● HTTP_HOST, REQUEST_METHOD, etc. ● start_response = callback(status,headers) ● Returns response as unicode string ● OpenERPs entry point: wsgi.core.application @odony – 6.1 Framework Changes – 05.2012
  12. 12. Gunicorn (1) ● WGSI compliant HTTP Server ● Python port of Ruby Unicorn ● Pre-fork multi-process model ● Flexible and easily configurable ● UNIX only (fork support needed) ● See also gunicorn.org @odony – 6.1 Framework Changes – 05.2012
  13. 13. Gunicorn (2) ● Master “Arbiter” process ● Starts and monitors workers (crash/timeout/...) ● Restarts killed workers → fork ● Increment/Decrement worker count (TTIN/TTOU signals) ● Doesnt speak HTTP → naive dispatch ● Forked worker processes ● Speak HTTP → Perform WSGI translation ● Dispatch to application entry point ● Can restart after given number of requests @odony – 6.1 Framework Changes – 05.2012
  14. 14. Gunicorn (3) ● OpenERP cant use normal config file or command-line options ● Config has to be provided via Gunicorn ● gunicorn.conf, written in Python! ● Contains Gunicorn configuration (workers, limits, etc.) ● Contains OpenERP configuration ● Sets up requests/application hooks (pre/post request...) ● Arbiter controls max-requests limit ● Workers control memory/cpu limits @odony – 6.1 Framework Changes – 05.2012
  15. 15. Gunicorn (4)CONFRUN $ gunicorn openerp:wsgi.core.application -c gunicorn.conf.py @odony – 6.1 Framework Changes – 05.2012
  16. 16. Good to know (1) ● Gunicorn → must be used explicitly ● Linear speed-up for CPU-bound tasks, less for I/O or DB ● Werkzeug → built-in WSGI-compliant server ● openerp-server script ● multi-threaded mono-process ● Proxy mode specific handling (X-Forwarded..) ● When deploying behind reverse proxy (e.g. with ProxyPass) ● Gunicorn: wsgi.proxied.application entry point instead ● Werkzeug: --proxy_mode @odony – 6.1 Framework Changes – 05.2012
  17. 17. Good to know (2) ● No cron job with Gunicorn ● Could be killed at any point! ● Start separate openerp-cron-worker job ● Anywhere! More about cron jobs in a minute... @odony – 6.1 Framework Changes – 05.2012
  18. 18. OpenERP 6.0 deployment model Requests OpenERP OpenERP PostgreSQL Web Client Server @odony – 6.1 Framework Changes – 05.2012
  19. 19. OpenERP 6.1 deployment model (1) OpenERP Worker Requests OpenERP Worker PostgreSQL OpenERP Worker OpenERP Worker @odony – 6.1 Framework Changes – 05.2012
  20. 20. OpenERP 6.1 deployment model (2) OpenERP Worker OpenERP Worker OpenERP Worker OpenERP WorkerRequests Load Balancer PostgreSQL OpenERP Worker OpenERP Worker OpenERP Worker OpenERP Worker @odony – 6.1 Framework Changes – 05.2012
  21. 21. Statelessness (1) ● OpenERP 6.0 was still partially stateful: 1) Model registry (pool) 2) Wizards (osv_memory) kept in-memory 3) Application caches – menus, perms, … ! 4) Background task processing: cron ● Not suitable for multi-processing @odony – 6.1 Framework Changes – 05.2012
  22. 22. Caches and Model Registries ● Caches and Registries have state ● Must be cleared on all workers when invalidated – across multiple machines! ● Signaling works with 2 DB sequences ● base_registry_signaling ● base_cache_signaling ● Before each request, both are checked ● After each request, increment if changed @odony – 6.1 Framework Changes – 05.2012
  23. 23. TransientModel ● osv_memory → TransientModel ● Real database-backed model ● Automatically vacuum-cleaned ● Based on size limit or time limit ● Default: 2 hours ● Has special and implicit access rights ● Everyone can perform CRUD ● But only access on your own records ● Can be used indifferently by multiple workers @odony – 6.1 Framework Changes – 05.2012
  24. 24. Background Tasks (cron) (1) ● Problem: ● Each worker possibly runs a cron processor ● Must prevent duplicate run of same task ● Must benefit from multi-processing when multiple cron processors are running (each runs different tasks) ● Solution: ● Database row-level mutex using LOCK ● Cron processor can be run on any number of servers ● Disabled on Gunicorn workers by default ● Can be run separately with openerp-cron-worker script added in 6.1 ● Cron tasks cant be changed while running (locked) @odony – 6.1 Framework Changes – 05.2012
  25. 25. Background Tasks (cron) (2) ● And by the way, cron tasks are executed in parallel in OpenERP 6.1 ● Also if running mono-process, up to number or thread specified with –-max-cron-threads (default: 4) ● Allows multiple tasks to be executed at the same time @odony – 6.1 Framework Changes – 05.2012
  26. 26. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  27. 27. Date Storage and Computation (1) ● 6.0 stored timestamps in “server timezone” ● Worked well for small teams with only one TZ ● Database not portable, Reports incorrect for cross-TZ teams, DST issues! ● 6.1 stores timestamps as UTC always! ● Big change, but Right Thing to Do™ ● Database becomes portable ● Dates monotonically incremented in DB ● Simplifies conversion for display UTC ↔ Client TZ ● Timestamps computation works as before on server-side ● Watch out! ● “Dates” have no TZ ● Server-side dates must be initialized in client TZ @odony – 6.1 Framework Changes – 05.2012
  28. 28. Date Storage and Computation (2) ● Server-side ● All computation in UTC ● PDF reports use client TZ from context/user ● Database ● All timestamps stored as TIMESTAMP WITHOUT TZ ● SELECT now() at time zone ‘UTC’ (should not need it) ● Client ● Converted back and forth between UTC and client TZ ● Web uses browser TZ for display (JS limitation) @odony – 6.1 Framework Changes – 05.2012
  29. 29. Date Storage and Computation (3) @odony – 6.1 Framework Changes – 05.2012
  30. 30. Date Storage and Computation (4) ● Be very careful with date (that)! ● Pure dates should be taken in user TZ ● Its very tempting to use datetime.date.today() ● But may be wrong for half of the planet! ● Use fields.date.context_today() instead @odony – 6.1 Framework Changes – 05.2012
  31. 31. Unified Mail Subsystem ● 6.0 had 3+ independent mail stacks ● 6.1 has one unified mail stack (mail) ● Per-database SMTP settings (SSL, Priority, Debug) ● Single point of extension ir.mail_server ● Single mail queue/archive ● Common features mail.thread mail.message ● Scheduler scheduler composition ● Template (email_template) ● Composition wizard email_template @odony – 6.1 Framework Changes – 05.2012
  32. 32. Many2Many auto-naming and inheritance ● What happens when you _inherit a model containing m2m field, w/ different _name? ● Problem more apparent with abstract osv_memory models ● In 6.1 many2many have auto names ● rel_table, rel_from, rel_to are automatically computed ● Simpler/Partial declaration, clean inherit ● rel_table override needed for multiple relationships @odony – 6.1 Framework Changes – 05.2012
  33. 33. YAML improvements (1) ● 6.1 YAML tests: ● 65% coverage, 150 scenarios, 1800 test steps ● Simulating workflows works well ● But what about UI: views, on_change, … ? ● Done! YAML will now automatically use form views and on_change methods ● Better coverage ● Less code: on_change can fill in implicit values ● Can be overridden with view attribute @odony – 6.1 Framework Changes – 05.2012
  34. 34. YAML improvements (2) @odony – 6.1 Framework Changes – 05.2012
  35. 35. Simplified Configuration (1) ● Setup process streamlined – 1-click install ● Kanban module view – ordered by sequence ● Configuration Overview Dashboard ● Based on ir.action.todo and ir.actions.todo.category ● New ir.actions.todo ● Reminder: group_ids, action_id, and note ● States: Todo or Done ● Types: Manual, Manual Once, Automatic ● Most are Manual or Manual Once now ● New category_id for Configuration Overview @odony – 6.1 Framework Changes – 05.2012
  36. 36. Simplified Configuration (2) @odony – 6.1 Framework Changes – 05.2012
  37. 37. Logging ● Python has a decent logging system ● netsvc.notifyChannel() deprecated in 6.0 ● Convenience hierarchical loggers ● One _logger per file recommended _logger = logging.getLogger(__name__) ● New –-log-handler config parameter ● --log-handler=PREFIX:LEVEL ● --log-handler=openerp.netsvc.rpc.request:DEBUG ● --log-handler=werkzeug:CRITICAL @odony – 6.1 Framework Changes – 05.2012
  38. 38. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  39. 39. Kanban (1) ● Kanban ( カンバン ): Japanese billboard ● Toyotas Lean/JIT production: “pull” or “demand-driven” SCM ● Kanban cards signal need to produce @odony – 6.1 Framework Changes – 05.2012
  40. 40. Kanban (2) ● Kanban method, applied to Management ● Signboard shows whole workflow at a glance ● Limit WIP (work-in-progress) for each state (Pull!) ● Monitor flow @odony – 6.1 Framework Changes – 05.2012
  41. 41. Kanban (3) ● kanban views are written in Qweb ● Automatic column layout with Group By ● Drag & Drop reassigns grouped column ● One mandatory template “kanban-box” ● Can use normal OpenERP fields ● Any CSS/HTML you like @odony – 6.1 Framework Changes – 05.2012
  42. 42. Kanban (4) ● Typical structure @odony – 6.1 Framework Changes – 05.2012
  43. 43. Kanban (5) ● Drag&Drop is nice, but group_by only shows existing data ● How about missing columns? ● Solution: add a _group_by_full map ● Return type must be like read(), depending on the column type @odony – 6.1 Framework Changes – 05.2012
  44. 44. Kanban (6) ● group_by_full, complex example @odony – 6.1 Framework Changes – 05.2012
  45. 45. Kanban (7) Exercise: add test kanban view for res.partner @odony – 6.1 Framework Changes – 05.2012
  46. 46. Kanban (8) ● More details about Kanban API: http://pad.openerp.com/kanban-api (temporary location) @odony – 6.1 Framework Changes – 05.2012
  47. 47. Customizable dashboards ● Dashboards now feature multiple layouts ● Saved in ir.ui.view_custom, per user ● Filters menu now has “Add to dashboard” ● Works with kanban views too @odony – 6.1 Framework Changes – 05.2012
  48. 48. New many2one widget ● Many2One fields = eternal UI problem ● 6.1 widget: ● Auto-complete selection as you type ● Intuitive quick creation ● New ORM method: name_create(cr, uid, name, context) ● If fails, default to normal form creation ● Disabled with options={"quick_create": false} @odony – 6.1 Framework Changes – 05.2012
  49. 49. New statusbar widget ● Old state selection was not user-friendly ● statusbar widget shows workflow steps ● Only for selection fields ● Options: list of visible values, colors @odony – 6.1 Framework Changes – 05.2012
  50. 50. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  51. 51. Module Manifest changes ● Web-related keys: js, css, qweb ● active → auto_install (backwards-compat): depends triggers! ● complexity (easy|normal|expert) ● application (bool): Application or Extra ● sequence (integer): Application install screen order ● New category: Hidden – Technical, hidden by default ● Module icon for install screen /static/src/img/icon.png ● OpenERP Apps ● Description in RST, one main section ● images: [/path/to/screenshot.png, /path/to/screenshot2.png] @odony – 6.1 Framework Changes – 05.2012
  52. 52. EDI Subsystem (1) ● Goal: bring companies up to par with people and social network tools ● EDI module installed with Sales, Account ● EDI Setup ● Company Addres & Logo ● Outgoing Email Server (SMTP) ● User emails ● Customers/Contacts emails @odony – 6.1 Framework Changes – 05.2012
  53. 53. EDI Subsystem (2) @odony – 6.1 Framework Changes – 05.2012
  54. 54. EDI Subsystem (3) ● Fine-tuning ● Payment options ● Paypal account on company ● Bank accounts on company ● Email templates ● EDI Preview templates: Qweb ● Opt-out ● Per-customer ● Global – workflow level ● Global – template @odony – 6.1 Framework Changes – 05.2012
  55. 55. EDI Subsystem (4) ● EDI Document is simple JSON ● EDI engine is generic @odony – 6.1 Framework Changes – 05.2012
  56. 56. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  57. 57. Exercises ● Gunicorn: ● Install Gunicorn and start OpenERP w/ it ● Kanban ● Add a kanban view to the model of your choice ● Set a default group_by on the menu action ● Implement a working _group_by_full ● Add the Kanban view to a dashboard ● EDI ● Modify an EDI Preview template to show Quotations @odony – 6.1 Framework Changes – 05.2012

×