OpenERP 6.1 Framework Changes

15,241 views
15,115 views

Published on

http://openerp.com

Technical Presentation of the main changes in OpenERP 6.1 at the API and framework level: architecture changes, API evolution, new UI features, etc.

See also the related presentation about the new 6.1 Web layer: http://www.slideshare.net/openobject/openerp-61-web-framework-tutorial

Published in: Technology, Education
1 Comment
12 Likes
Statistics
Notes
No Downloads
Views
Total views
15,241
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
722
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

OpenERP 6.1 Framework Changes

  1. 1. OpenERP 6.1 Technical Changes at API/Framework level @odony – 6.1 Framework Changes – 05.2012
  2. 2. OpenERP 6.1 ● Released February 2011 ● Normal release – non LTS = limited support ● Release Notes ● http://bit.ly/openerp61RN ● Focus ● Usability ● Social features ● New web client ● Cleanup @odony – 6.1 Framework Changes – 05.2012
  3. 3. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  4. 4. Evolution: Architecture changes ● Unified server-side deployment ● Scalability through multi-processing ● Use all those cores on modern hardware ● Bypass Python GIL issues ● Reliability (cpu/memory/crashes) ● WSGI ● Standard Python solution – OpenERP 6.1 is compliant ● Gunicorn - a WSGI compliant HTTP server ● Statelessness ● Distributed scheduled tasks (cron) @odony – 6.1 Framework Changes – 05.2012
  5. 5. Unified server-side deployment ● No more need to deploy web client ● Web client is now an OpenERP module ● Only one package to install ● Pre-loaded server-wide via –-load=web ● All modules in same place (addons_path) ● openerp-web still available stand-alone ● For testing/debugging purposes ● Needs to have copy of normal modules too! @odony – 6.1 Framework Changes – 05.2012
  6. 6. Scalability through multi-processing ● Use all those cores on modern hardware ● Bypass Python GIL limitations ● Improve reliability (cpu/memory/crashes) → What does this all mean?? @odony – 6.1 Framework Changes – 05.2012
  7. 7. Multi-processing and GIL (1) ● CPython has a Global Interpreter Lock/GIL ● Simplifies low-level memory management, C calls, etc. ● 1 Python thread = 1 OS thread ● Parallel execution of threads forbidden ● Cooperative multi-tasking on 1 CPU ● Worse performance on multi-CPU/core! ● Battle for GIL on the different virtual CPUs! @odony – 6.1 Framework Changes – 05.2012
  8. 8. Multi-processing and GIL (2) ● Cooperative multi-tasking ● Release on I/O or every 100 ticks ● 1 tick ≈ 1 interpreter instruction @odony – 6.1 Framework Changes – 05.2012
  9. 9. Scalability through multi-processing ● multiprocessing module was an option ● Using Unix tools and real processes was more flexible and easier to scale up ● Managing the processes was not necessary, many tools exist to do that (e.g. Gunicorn) ● Just had to make OpenERP WSGI-compliant @odony – 6.1 Framework Changes – 05.2012
  10. 10. WSGI (1) ● Web Server Gateway Interface for Python ● WSGI applications can be embedded in WSGI-compliant web servers ● Many web frameworks support WSGI ● Django, CherryPy, … ● Many HTTP servers support WSGI ● Apache(+mod), nginx(+mod), tornado, gunicorn, … @odony – 6.1 Framework Changes – 05.2012
  11. 11. WSGI (2) ● Very simple API: one method ● environ = HTTP environment map ● HTTP_HOST, REQUEST_METHOD, etc. ● start_response = callback(status,headers) ● Returns response as unicode string ● OpenERPs entry point: wsgi.core.application @odony – 6.1 Framework Changes – 05.2012
  12. 12. Gunicorn (1) ● WGSI compliant HTTP Server ● Python port of Ruby Unicorn ● Pre-fork multi-process model ● Flexible and easily configurable ● UNIX only (fork support needed) ● See also gunicorn.org @odony – 6.1 Framework Changes – 05.2012
  13. 13. Gunicorn (2) ● Master “Arbiter” process ● Starts and monitors workers (crash/timeout/...) ● Restarts killed workers → fork ● Increment/Decrement worker count (TTIN/TTOU signals) ● Doesnt speak HTTP → naive dispatch ● Forked worker processes ● Speak HTTP → Perform WSGI translation ● Dispatch to application entry point ● Can restart after given number of requests @odony – 6.1 Framework Changes – 05.2012
  14. 14. Gunicorn (3) ● OpenERP cant use normal config file or command-line options ● Config has to be provided via Gunicorn ● gunicorn.conf, written in Python! ● Contains Gunicorn configuration (workers, limits, etc.) ● Contains OpenERP configuration ● Sets up requests/application hooks (pre/post request...) ● Arbiter controls max-requests limit ● Workers control memory/cpu limits @odony – 6.1 Framework Changes – 05.2012
  15. 15. Gunicorn (4)CONFRUN $ gunicorn openerp:wsgi.core.application -c gunicorn.conf.py @odony – 6.1 Framework Changes – 05.2012
  16. 16. Good to know (1) ● Gunicorn → must be used explicitly ● Linear speed-up for CPU-bound tasks, less for I/O or DB ● Werkzeug → built-in WSGI-compliant server ● openerp-server script ● multi-threaded mono-process ● Proxy mode specific handling (X-Forwarded..) ● When deploying behind reverse proxy (e.g. with ProxyPass) ● Gunicorn: wsgi.proxied.application entry point instead ● Werkzeug: --proxy_mode @odony – 6.1 Framework Changes – 05.2012
  17. 17. Good to know (2) ● No cron job with Gunicorn ● Could be killed at any point! ● Start separate openerp-cron-worker job ● Anywhere! More about cron jobs in a minute... @odony – 6.1 Framework Changes – 05.2012
  18. 18. OpenERP 6.0 deployment model Requests OpenERP OpenERP PostgreSQL Web Client Server @odony – 6.1 Framework Changes – 05.2012
  19. 19. OpenERP 6.1 deployment model (1) OpenERP Worker Requests OpenERP Worker PostgreSQL OpenERP Worker OpenERP Worker @odony – 6.1 Framework Changes – 05.2012
  20. 20. OpenERP 6.1 deployment model (2) OpenERP Worker OpenERP Worker OpenERP Worker OpenERP WorkerRequests Load Balancer PostgreSQL OpenERP Worker OpenERP Worker OpenERP Worker OpenERP Worker @odony – 6.1 Framework Changes – 05.2012
  21. 21. Statelessness (1) ● OpenERP 6.0 was still partially stateful: 1) Model registry (pool) 2) Wizards (osv_memory) kept in-memory 3) Application caches – menus, perms, … ! 4) Background task processing: cron ● Not suitable for multi-processing @odony – 6.1 Framework Changes – 05.2012
  22. 22. Caches and Model Registries ● Caches and Registries have state ● Must be cleared on all workers when invalidated – across multiple machines! ● Signaling works with 2 DB sequences ● base_registry_signaling ● base_cache_signaling ● Before each request, both are checked ● After each request, increment if changed @odony – 6.1 Framework Changes – 05.2012
  23. 23. TransientModel ● osv_memory → TransientModel ● Real database-backed model ● Automatically vacuum-cleaned ● Based on size limit or time limit ● Default: 2 hours ● Has special and implicit access rights ● Everyone can perform CRUD ● But only access on your own records ● Can be used indifferently by multiple workers @odony – 6.1 Framework Changes – 05.2012
  24. 24. Background Tasks (cron) (1) ● Problem: ● Each worker possibly runs a cron processor ● Must prevent duplicate run of same task ● Must benefit from multi-processing when multiple cron processors are running (each runs different tasks) ● Solution: ● Database row-level mutex using LOCK ● Cron processor can be run on any number of servers ● Disabled on Gunicorn workers by default ● Can be run separately with openerp-cron-worker script added in 6.1 ● Cron tasks cant be changed while running (locked) @odony – 6.1 Framework Changes – 05.2012
  25. 25. Background Tasks (cron) (2) ● And by the way, cron tasks are executed in parallel in OpenERP 6.1 ● Also if running mono-process, up to number or thread specified with –-max-cron-threads (default: 4) ● Allows multiple tasks to be executed at the same time @odony – 6.1 Framework Changes – 05.2012
  26. 26. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  27. 27. Date Storage and Computation (1) ● 6.0 stored timestamps in “server timezone” ● Worked well for small teams with only one TZ ● Database not portable, Reports incorrect for cross-TZ teams, DST issues! ● 6.1 stores timestamps as UTC always! ● Big change, but Right Thing to Do™ ● Database becomes portable ● Dates monotonically incremented in DB ● Simplifies conversion for display UTC ↔ Client TZ ● Timestamps computation works as before on server-side ● Watch out! ● “Dates” have no TZ ● Server-side dates must be initialized in client TZ @odony – 6.1 Framework Changes – 05.2012
  28. 28. Date Storage and Computation (2) ● Server-side ● All computation in UTC ● PDF reports use client TZ from context/user ● Database ● All timestamps stored as TIMESTAMP WITHOUT TZ ● SELECT now() at time zone ‘UTC’ (should not need it) ● Client ● Converted back and forth between UTC and client TZ ● Web uses browser TZ for display (JS limitation) @odony – 6.1 Framework Changes – 05.2012
  29. 29. Date Storage and Computation (3) @odony – 6.1 Framework Changes – 05.2012
  30. 30. Date Storage and Computation (4) ● Be very careful with date (that)! ● Pure dates should be taken in user TZ ● Its very tempting to use datetime.date.today() ● But may be wrong for half of the planet! ● Use fields.date.context_today() instead @odony – 6.1 Framework Changes – 05.2012
  31. 31. Unified Mail Subsystem ● 6.0 had 3+ independent mail stacks ● 6.1 has one unified mail stack (mail) ● Per-database SMTP settings (SSL, Priority, Debug) ● Single point of extension ir.mail_server ● Single mail queue/archive ● Common features mail.thread mail.message ● Scheduler scheduler composition ● Template (email_template) ● Composition wizard email_template @odony – 6.1 Framework Changes – 05.2012
  32. 32. Many2Many auto-naming and inheritance ● What happens when you _inherit a model containing m2m field, w/ different _name? ● Problem more apparent with abstract osv_memory models ● In 6.1 many2many have auto names ● rel_table, rel_from, rel_to are automatically computed ● Simpler/Partial declaration, clean inherit ● rel_table override needed for multiple relationships @odony – 6.1 Framework Changes – 05.2012
  33. 33. YAML improvements (1) ● 6.1 YAML tests: ● 65% coverage, 150 scenarios, 1800 test steps ● Simulating workflows works well ● But what about UI: views, on_change, … ? ● Done! YAML will now automatically use form views and on_change methods ● Better coverage ● Less code: on_change can fill in implicit values ● Can be overridden with view attribute @odony – 6.1 Framework Changes – 05.2012
  34. 34. YAML improvements (2) @odony – 6.1 Framework Changes – 05.2012
  35. 35. Simplified Configuration (1) ● Setup process streamlined – 1-click install ● Kanban module view – ordered by sequence ● Configuration Overview Dashboard ● Based on ir.action.todo and ir.actions.todo.category ● New ir.actions.todo ● Reminder: group_ids, action_id, and note ● States: Todo or Done ● Types: Manual, Manual Once, Automatic ● Most are Manual or Manual Once now ● New category_id for Configuration Overview @odony – 6.1 Framework Changes – 05.2012
  36. 36. Simplified Configuration (2) @odony – 6.1 Framework Changes – 05.2012
  37. 37. Logging ● Python has a decent logging system ● netsvc.notifyChannel() deprecated in 6.0 ● Convenience hierarchical loggers ● One _logger per file recommended _logger = logging.getLogger(__name__) ● New –-log-handler config parameter ● --log-handler=PREFIX:LEVEL ● --log-handler=openerp.netsvc.rpc.request:DEBUG ● --log-handler=werkzeug:CRITICAL @odony – 6.1 Framework Changes – 05.2012
  38. 38. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  39. 39. Kanban (1) ● Kanban ( カンバン ): Japanese billboard ● Toyotas Lean/JIT production: “pull” or “demand-driven” SCM ● Kanban cards signal need to produce @odony – 6.1 Framework Changes – 05.2012
  40. 40. Kanban (2) ● Kanban method, applied to Management ● Signboard shows whole workflow at a glance ● Limit WIP (work-in-progress) for each state (Pull!) ● Monitor flow @odony – 6.1 Framework Changes – 05.2012
  41. 41. Kanban (3) ● kanban views are written in Qweb ● Automatic column layout with Group By ● Drag & Drop reassigns grouped column ● One mandatory template “kanban-box” ● Can use normal OpenERP fields ● Any CSS/HTML you like @odony – 6.1 Framework Changes – 05.2012
  42. 42. Kanban (4) ● Typical structure @odony – 6.1 Framework Changes – 05.2012
  43. 43. Kanban (5) ● Drag&Drop is nice, but group_by only shows existing data ● How about missing columns? ● Solution: add a _group_by_full map ● Return type must be like read(), depending on the column type @odony – 6.1 Framework Changes – 05.2012
  44. 44. Kanban (6) ● group_by_full, complex example @odony – 6.1 Framework Changes – 05.2012
  45. 45. Kanban (7) Exercise: add test kanban view for res.partner @odony – 6.1 Framework Changes – 05.2012
  46. 46. Kanban (8) ● More details about Kanban API: http://pad.openerp.com/kanban-api (temporary location) @odony – 6.1 Framework Changes – 05.2012
  47. 47. Customizable dashboards ● Dashboards now feature multiple layouts ● Saved in ir.ui.view_custom, per user ● Filters menu now has “Add to dashboard” ● Works with kanban views too @odony – 6.1 Framework Changes – 05.2012
  48. 48. New many2one widget ● Many2One fields = eternal UI problem ● 6.1 widget: ● Auto-complete selection as you type ● Intuitive quick creation ● New ORM method: name_create(cr, uid, name, context) ● If fails, default to normal form creation ● Disabled with options={"quick_create": false} @odony – 6.1 Framework Changes – 05.2012
  49. 49. New statusbar widget ● Old state selection was not user-friendly ● statusbar widget shows workflow steps ● Only for selection fields ● Options: list of visible values, colors @odony – 6.1 Framework Changes – 05.2012
  50. 50. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  51. 51. Module Manifest changes ● Web-related keys: js, css, qweb ● active → auto_install (backwards-compat): depends triggers! ● complexity (easy|normal|expert) ● application (bool): Application or Extra ● sequence (integer): Application install screen order ● New category: Hidden – Technical, hidden by default ● Module icon for install screen /static/src/img/icon.png ● OpenERP Apps ● Description in RST, one main section ● images: [/path/to/screenshot.png, /path/to/screenshot2.png] @odony – 6.1 Framework Changes – 05.2012
  52. 52. EDI Subsystem (1) ● Goal: bring companies up to par with people and social network tools ● EDI module installed with Sales, Account ● EDI Setup ● Company Addres & Logo ● Outgoing Email Server (SMTP) ● User emails ● Customers/Contacts emails @odony – 6.1 Framework Changes – 05.2012
  53. 53. EDI Subsystem (2) @odony – 6.1 Framework Changes – 05.2012
  54. 54. EDI Subsystem (3) ● Fine-tuning ● Payment options ● Paypal account on company ● Bank accounts on company ● Email templates ● EDI Preview templates: Qweb ● Opt-out ● Per-customer ● Global – workflow level ● Global – template @odony – 6.1 Framework Changes – 05.2012
  55. 55. EDI Subsystem (4) ● EDI Document is simple JSON ● EDI engine is generic @odony – 6.1 Framework Changes – 05.2012
  56. 56. Agenda  ● Framework & API ● Architecture changes ● API evolution ● New views and UI features ● Other important bits ● Exercises, Q&A @odony – 6.1 Framework Changes – 05.2012
  57. 57. Exercises ● Gunicorn: ● Install Gunicorn and start OpenERP w/ it ● Kanban ● Add a kanban view to the model of your choice ● Set a default group_by on the menu action ● Implement a working _group_by_full ● Add the Kanban view to a dashboard ● EDI ● Modify an EDI Preview template to show Quotations @odony – 6.1 Framework Changes – 05.2012

×