http://openerp.com
Technical Presentation of the main changes in OpenERP 6.1 at the API and framework level: architecture changes, API evolution, new UI features, etc.
See also the related presentation about the new 6.1 Web layer: http://www.slideshare.net/openobject/openerp-61-web-framework-tutorial
2. OpenERP 6.1
● Released February 2011
● Normal release – non LTS = limited support
● Release Notes
● http://bit.ly/openerp61RN
● Focus
● Usability
● Social features
● New web client
● Cleanup
@odony – 6.1 Framework Changes – 05.2012
3. Agenda
●
Framework & API
●
Architecture changes
●
API evolution
●
New views and UI features
●
Other important bits
●
Exercises, Q&A
@odony – 6.1 Framework Changes – 05.2012
4. Evolution: Architecture changes
● Unified server-side deployment
● Scalability through multi-processing
● Use all those cores on modern hardware
● Bypass Python GIL issues
● Reliability (cpu/memory/crashes)
● WSGI
● Standard Python solution – OpenERP 6.1 is compliant
● Gunicorn - a WSGI compliant HTTP server
● Statelessness
● Distributed scheduled tasks (cron)
@odony – 6.1 Framework Changes – 05.2012
5. Unified server-side deployment
●
No more need to deploy web client
●
Web client is now an OpenERP module
●
Only one package to install
●
Pre-loaded server-wide via –-load=web
●
All modules in same place (addons_path)
●
openerp-web still available stand-alone
●
For testing/debugging purposes
●
Needs to have copy of normal modules too!
@odony – 6.1 Framework Changes – 05.2012
6. Scalability through multi-processing
●
Use all those cores on modern hardware
●
Bypass Python GIL limitations
●
Improve reliability (cpu/memory/crashes)
→ What does this all mean??
@odony – 6.1 Framework Changes – 05.2012
7. Multi-processing and GIL (1)
●
CPython has a Global Interpreter Lock/GIL
●
Simplifies low-level memory management,
C calls, etc.
●
1 Python thread = 1 OS thread
●
Parallel execution of threads forbidden
●
Cooperative multi-tasking on 1 CPU
●
Worse performance on multi-CPU/core!
●
Battle for GIL on the different virtual CPUs!
@odony – 6.1 Framework Changes – 05.2012
8. Multi-processing and GIL (2)
●
Cooperative multi-tasking
●
Release on I/O or every 100 ticks
●
1 tick ≈ 1 interpreter instruction
@odony – 6.1 Framework Changes – 05.2012
9. Scalability through multi-processing
●
multiprocessing module was an option
●
Using Unix tools and real processes was
more flexible and easier to scale up
●
Managing the processes was not
necessary, many tools exist to do that
(e.g. Gunicorn)
●
Just had to make OpenERP WSGI-compliant
@odony – 6.1 Framework Changes – 05.2012
10. WSGI (1)
●
Web Server Gateway Interface for Python
●
WSGI applications can be embedded in
WSGI-compliant web servers
●
Many web frameworks support WSGI
●
Django, CherryPy, …
●
Many HTTP servers support WSGI
●
Apache(+mod), nginx(+mod), tornado, gunicorn, …
@odony – 6.1 Framework Changes – 05.2012
11. WSGI (2)
● Very simple API: one method
● environ = HTTP environment map
● HTTP_HOST, REQUEST_METHOD, etc.
● start_response = callback(status,headers)
● Returns response as unicode string
● OpenERP's entry point: wsgi.core.application
@odony – 6.1 Framework Changes – 05.2012
12. Gunicorn (1)
●
WGSI compliant HTTP Server
●
Python port of Ruby Unicorn
●
Pre-fork multi-process model
●
Flexible and easily configurable
●
UNIX only (fork support needed)
●
See also gunicorn.org
@odony – 6.1 Framework Changes – 05.2012
13. Gunicorn (2)
●
Master “Arbiter” process
●
Starts and monitors workers (crash/timeout/...)
●
Restarts killed workers → fork
●
Increment/Decrement worker count (TTIN/TTOU signals)
●
Doesn't speak HTTP → naive dispatch
●
Forked worker processes
●
Speak HTTP → Perform WSGI translation
●
Dispatch to application entry point
●
Can restart after given number of requests
@odony – 6.1 Framework Changes – 05.2012
14. Gunicorn (3)
●
OpenERP can't use normal config file or
command-line options
●
Config has to be provided via Gunicorn
●
gunicorn.conf, written in Python!
●
Contains Gunicorn configuration (workers, limits, etc.)
●
Contains OpenERP configuration
●
Sets up requests/application hooks (pre/post request...)
●
Arbiter controls max-requests limit
●
Workers control memory/cpu limits
@odony – 6.1 Framework Changes – 05.2012
16. Good to know (1)
● Gunicorn → must be used explicitly
● Linear speed-up for CPU-bound tasks, less for I/O or DB
● Werkzeug → built-in WSGI-compliant server
● openerp-server script
● multi-threaded mono-process
● Proxy mode specific handling (X-Forwarded..)
● When deploying behind reverse proxy (e.g. with ProxyPass)
● Gunicorn: wsgi.proxied.application entry point instead
● Werkzeug: --proxy_mode
@odony – 6.1 Framework Changes – 05.2012
17. Good to know (2)
●
No cron job with Gunicorn
●
Could be killed at any point!
●
Start separate openerp-cron-worker job
●
Anywhere!
More about cron jobs in a minute...
@odony – 6.1 Framework Changes – 05.2012
18. OpenERP 6.0 deployment model
Requests OpenERP OpenERP
PostgreSQL
Web Client Server
@odony – 6.1 Framework Changes – 05.2012
21. Statelessness (1)
●
OpenERP 6.0 was still partially stateful:
1) Model registry (pool)
2) Wizards (osv_memory) kept in-memory
3) Application caches – menus, perms, … !
4) Background task processing: cron
●
Not suitable for multi-processing
@odony – 6.1 Framework Changes – 05.2012
22. Caches and Model Registries
●
Caches and Registries have state
●
Must be cleared on all workers when
invalidated – across multiple machines!
●
Signaling works with 2 DB sequences
● base_registry_signaling
● base_cache_signaling
●
Before each request, both are checked
●
After each request, increment if changed
@odony – 6.1 Framework Changes – 05.2012
23. TransientModel
●
osv_memory → TransientModel
●
Real database-backed model
●
Automatically vacuum-cleaned
●
Based on size limit or time limit
●
Default: 2 hours
●
Has special and implicit access rights
●
Everyone can perform CRUD
●
But only access on your own records
●
Can be used indifferently by multiple workers
@odony – 6.1 Framework Changes – 05.2012
24. Background Tasks (cron) (1)
● Problem:
● Each worker possibly runs a cron processor
● Must prevent duplicate run of same task
● Must benefit from multi-processing when multiple cron processors
are running (each runs different tasks)
● Solution:
● Database row-level mutex using LOCK
● Cron processor can be run on any number of servers
● Disabled on Gunicorn workers by default
● Can be run separately with openerp-cron-worker script added in 6.1
● Cron tasks can't be changed while running (locked)
@odony – 6.1 Framework Changes – 05.2012
25. Background Tasks (cron) (2)
●
And by the way, cron tasks are executed
in parallel in OpenERP 6.1
●
Also if running mono-process, up to
number or thread specified with
–-max-cron-threads (default: 4)
●
Allows multiple tasks to be executed at
the same time
@odony – 6.1 Framework Changes – 05.2012
26. Agenda
●
Framework & API
●
Architecture changes
●
API evolution
●
New views and UI features
●
Other important bits
●
Exercises, Q&A
@odony – 6.1 Framework Changes – 05.2012
27. Date Storage and Computation (1)
● 6.0 stored timestamps in “server timezone”
● Worked well for small teams with only one TZ
● Database not portable, Reports incorrect for cross-TZ teams, DST issues!
● 6.1 stores timestamps as UTC always!
● Big change, but Right Thing to Do™
● Database becomes portable
● Dates monotonically incremented in DB
● Simplifies conversion for display UTC ↔ Client TZ
● Timestamps computation works as before on server-side
● Watch out!
● “Dates” have no TZ
● Server-side dates must be initialized in client TZ
@odony – 6.1 Framework Changes – 05.2012
28. Date Storage and Computation (2)
●
Server-side
●
All computation in UTC
●
PDF reports use client TZ from context/user
●
Database
●
All timestamps stored as TIMESTAMP WITHOUT TZ
●
SELECT now() at time zone ‘UTC’ (should not need it)
●
Client
●
Converted back and forth between UTC and client TZ
●
Web uses browser TZ for display (JS limitation)
@odony – 6.1 Framework Changes – 05.2012
29. Date Storage and Computation (3)
@odony – 6.1 Framework Changes – 05.2012
30. Date Storage and Computation (4)
●
Be very careful with date (that)!
●
Pure dates should be taken in user TZ
●
It's very tempting to use
datetime.date.today()
●
But may be wrong for half of the planet!
●
Use fields.date.context_today() instead
@odony – 6.1 Framework Changes – 05.2012
31. Unified Mail Subsystem
●
6.0 had 3+ independent mail stacks
●
6.1 has one unified mail stack (mail)
●
Per-database SMTP settings (SSL, Priority, Debug)
●
Single point of extension ir.mail_server
●
Single mail queue/archive
●
Common features mail.thread mail.message
●
Scheduler
scheduler composition
●
Template (email_template)
●
Composition wizard
email_template
@odony – 6.1 Framework Changes – 05.2012
32. Many2Many auto-naming and inheritance
●
What happens when you _inherit a model
containing m2m field, w/ different _name?
●
Problem more apparent with abstract
osv_memory models
●
In 6.1 many2many have auto names
●
rel_table, rel_from, rel_to are automatically computed
●
Simpler/Partial declaration, clean inherit
●
rel_table override needed for multiple relationships
@odony – 6.1 Framework Changes – 05.2012
33. YAML improvements (1)
● 6.1 YAML tests:
● 65% coverage, 150 scenarios, 1800 test steps
● Simulating workflows works well
● But what about UI: views, on_change, … ?
● Done! YAML will now automatically use form
views and on_change methods
● Better coverage
● Less code: on_change can fill in implicit values
● Can be overridden with view attribute
@odony – 6.1 Framework Changes – 05.2012
35. Simplified Configuration (1)
●
Setup process streamlined – 1-click install
●
Kanban module view – ordered by sequence
●
Configuration Overview Dashboard
●
Based on ir.action.todo and ir.actions.todo.category
●
New ir.actions.todo
●
Reminder: group_ids, action_id, and note
●
States: Todo or Done
●
Types: Manual, Manual Once, Automatic
●
Most are Manual or Manual Once now
●
New category_id for Configuration Overview
@odony – 6.1 Framework Changes – 05.2012
37. Logging
●
Python has a decent logging system
●
netsvc.notifyChannel() deprecated in 6.0
●
Convenience hierarchical loggers
●
One _logger per file recommended
_logger = logging.getLogger(__name__)
●
New –-log-handler config parameter
● --log-handler=PREFIX:LEVEL
● --log-handler=openerp.netsvc.rpc.request:DEBUG
● --log-handler=werkzeug:CRITICAL
@odony – 6.1 Framework Changes – 05.2012
38. Agenda
●
Framework & API
●
Architecture changes
●
API evolution
●
New views and UI features
●
Other important bits
●
Exercises, Q&A
@odony – 6.1 Framework Changes – 05.2012
39. Kanban (1)
●
Kanban ( カンバン ): Japanese billboard
●
Toyota's Lean/JIT production: “pull” or
“demand-driven” SCM
●
Kanban cards signal need to produce
@odony – 6.1 Framework Changes – 05.2012
40. Kanban (2)
●
Kanban method, applied to Management
●
Signboard shows whole workflow at a glance
●
Limit WIP (work-in-progress) for each state (Pull!)
●
Monitor flow
@odony – 6.1 Framework Changes – 05.2012
41. Kanban (3)
●
kanban views are written in Qweb
●
Automatic column layout with Group By
●
Drag & Drop reassigns grouped column
●
One mandatory template “kanban-box”
●
Can use normal OpenERP fields
●
Any CSS/HTML you like
@odony – 6.1 Framework Changes – 05.2012
43. Kanban (5)
●
Drag&Drop is nice, but group_by only
shows existing data
●
How about missing columns?
●
Solution: add a _group_by_full map
●
Return type must be like read(),
depending on the column type
@odony – 6.1 Framework Changes – 05.2012
45. Kanban (7)
Exercise: add test kanban view for res.partner
@odony – 6.1 Framework Changes – 05.2012
46. Kanban (8)
●
More details about Kanban API:
http://pad.openerp.com/kanban-api
(temporary location)
@odony – 6.1 Framework Changes – 05.2012
47. Customizable dashboards
●
Dashboards now feature multiple layouts
●
Saved in ir.ui.view_custom, per user
●
Filters menu now has “Add to dashboard”
●
Works with kanban views too
@odony – 6.1 Framework Changes – 05.2012
48. New many2one widget
●
Many2One fields = eternal UI problem
●
6.1 widget:
●
Auto-complete selection as you type
●
Intuitive quick creation
●
New ORM method: name_create(cr, uid, name, context)
●
If fails, default to normal form creation
●
Disabled with options='{"quick_create": false}'
@odony – 6.1 Framework Changes – 05.2012
49. New statusbar widget
●
Old state selection was not user-friendly
●
statusbar widget shows workflow steps
●
Only for selection fields
●
Options: list of visible values, colors
@odony – 6.1 Framework Changes – 05.2012
50. Agenda
●
Framework & API
●
Architecture changes
●
API evolution
●
New views and UI features
●
Other important bits
●
Exercises, Q&A
@odony – 6.1 Framework Changes – 05.2012
51. Module Manifest changes
● Web-related keys: js, css, qweb
● active → auto_install (backwards-compat): 'depends' triggers!
● complexity (easy|normal|expert)
● application (bool): Application or Extra
● sequence (integer): Application install screen order
● New category: 'Hidden' – Technical, hidden by default
● Module icon for install screen
/static/src/img/icon.png
● OpenERP Apps
● Description in RST, one main section
● 'images': ['/path/to/screenshot.png', '/path/to/screenshot2.png']
@odony – 6.1 Framework Changes – 05.2012
52. EDI Subsystem (1)
●
Goal: bring companies up to par with
people and social network tools
●
EDI module installed with Sales, Account
●
EDI Setup
●
Company Addres & Logo
●
Outgoing Email Server (SMTP)
●
User emails
●
Customers/Contacts emails
@odony – 6.1 Framework Changes – 05.2012
54. EDI Subsystem (3)
●
Fine-tuning
●
Payment options
●
Paypal account on company
●
Bank accounts on company
●
Email templates
●
EDI Preview templates: Qweb
●
Opt-out
●
Per-customer
●
Global – workflow level
●
Global – template
@odony – 6.1 Framework Changes – 05.2012
55. EDI Subsystem (4)
● EDI Document is simple JSON
● EDI engine is generic
@odony – 6.1 Framework Changes – 05.2012
56. Agenda
●
Framework & API
●
Architecture changes
●
API evolution
●
New views and UI features
●
Other important bits
●
Exercises, Q&A
@odony – 6.1 Framework Changes – 05.2012
57. Exercises
●
Gunicorn:
●
Install Gunicorn and start OpenERP w/ it
●
Kanban
●
Add a kanban view to the model of your choice
●
Set a default group_by on the menu action
●
Implement a working _group_by_full
●
Add the Kanban view to a dashboard
●
EDI
●
Modify an EDI Preview template to show Quotations
@odony – 6.1 Framework Changes – 05.2012