- Oslo.versionedobjects is an object model used in Nova to version and serialize objects for remote procedure calls (RPCs). Each change is versioned to allow backwards-compatible evolution. - Objects define fields and methods, with fields serialized independently of code. Versioning allows controlled schema changes without database changes. - Objects can be remotely called via an indirection service, with copies sent and returned. This decouples services from databases, enabling independent upgrades.

1. oslo.versionedobjects:
Deep Dive
Dan Smith

2. What is it?
● Object model to solve problems we had in
nova
● Each RPC-visible change is versioned
● Remotable, serializable
● Backportable

3. What is it? (versioning)
● Attempt to get away from dicts (of madness)
● Fields are defined and typed, with
serialization routines
● Defined at the edges, no embedding code or
class names in the serialized form
● Controlled evolution of object/RPC schema
independent of DB schema over time

4. What is it? (remotable)
● You implement Indirection API (nova-
conductor implements for nova)
● @remotable methods get called via
indirection
● Self-serializing

5. obj.do_thing()
def do_thing()
def do_thing()
DB
Indirection
Service
@remotable
When indirection
is active
● Remotable methods wrapped with
decorator
● Calls may avoid the local
implementation and make the call run
through the indirection service
● Same implementation both places!
● Copy of the object (including version)
is sent
● Object (may be) modified on remote,
changes copied back to the caller
Remotable Methods

6. What is it not?
● Amazingly novel
● ...going to solve world hunger
● ...going to solve your problems (alone)
● A lemming

7. from oslo.versionedobjects import base
from oslo.versionedobjects import base
class MyObject(base.VersionedObject):
# Version 1.0: Initial
# Version 1.1: Added ‘name’ field
VERSION = ‘1.1’
fields = {
‘id’: fields.IntegerField(nullable=False),
‘name’: fields.StringField(nullable=True),
‘metadata’: fields.DictOfStringsField(nullable=True),
}
@base.remotable_classmethod
def get_by_id(cls, context, id):
db_thingy = db.get_thingy(context, id)
obj = cls(context=context,
id=db_thingy[‘id’],
name=db_thingy.get(‘thingy_name’),
metadata=db.get_metadata_for(context, id))
obj.obj_reset_changes()
return obj
Careful, humans will put non-
versioned stuff in here and cause
problems (as humans do)!

8. class MyObject(base.VersionedObject):
# … continued
@base.remotable
def save(self):
changes = self.obj_what_changed()
db_updates = {}
if ‘id’ in changes:
raise Exception(‘Can’t change the id!’)
if ‘name’ in changes:
db_updates[‘thingy_name’] = self.name
with db.transaction():
if ‘metadata’ in changes:
db.update_metadata_for(self.context, self.id,
self.metadata)
if db_updates:
db.update_thingy(self.context, self.id, db_updates)
self.obj_reset_changes()

9. Lazy-Loading
● Attributes can be set or unset, independent
of nullability (i.e. “can be None”)
● Loadable attributes handled by a method
● In practice, only some (heavy) attributes are
loadable

10. class MyObject(base.VersionedObject):
# … continued
# Not remotable!
def obj_load_attr(self, attrname):
if attrname == ‘metadata’:
obj = self.get_by_id(self.context, self.id)
self.metadata = obj.metadata
self.obj_reset_changes()
else:
# Fail as not-lazy-loadable
super(MyObject, self).obj_load_attr(attrname)

11. Serialization
● Convert object to/from serialized form
● Field implementations are responsible for
serializing their contents (not your object)
● Serializer object actually produces the result
● Nova uses JSON, but you can do whatever
● Object mentions no classnames or code,
relies on the model to deserialize

12. Serialization (continued)
● Most services will need to have a request
context for all operations
● Object stores this for ease and security
● Each RPC trip re-attaches the context of the
RPC call, so context is not serialized!
● Can be anything or ignored if you don’t need
it

13. obj = MyObj(id=123, name=’foo’, metadata={‘speed’: ‘88mph’})
prim = obj.obj_to_primitive()
string = json.dumps(prim)
reprim = json.loads(string)
reobj = VersionedObject.obj_from_primitive(reprim, context)

14. class MyRPCClientAPI(object):
def __init__(self):
# ...
serializer = base.VersionedObjectSerializer()
self.client = self.get_client(target, version_cap, serializer)
class MyRPCServerAPI(service.Service):
def start():
# ...
serializer = base.VersionedObjectSerializer()
self.rpcserver = rpc.get_server(target, endpoints, serializer)
self.rpcserver.start()

15. Versioning
● Requires diligence (with some tools to help
catch offenders)
● Backwards-compatible changes
● Recipients should honor old versions
● Senders can pre-backport to older versions
● Nova does this automatically, but simply
pinning versions is an easy way

16. class MyObject(base.VersionedObject):
# … continued
def obj_make_compatible(self, primitive, version):
if version == ‘1.0’:
del primitive[‘name’]
obj = MyObject(id=123, name=’foo’, metadata={‘speed’: ‘88mph’})
prim = obj.obj_to_primitive(target_version=’1.0’)
# Now prim won’t have the ‘name’ field
oldobj = VersionedObject.obj_from_primitive(prim, context)
self.assertFalse(oldobj.obj_attr_is_set(‘name’))

17. Why do this?
● Get a better handle on what you’re sending
over RPC and what happens if you change
● Decouples the database from services,
allowing DB schema upgrades independent
of object schemas
● Easy serialization without sacrificing rich
typing and bundled methods

18. Why not do this?
● Because it will magically solve upgrade
problems (it won’t)
● Because you have a better way
● Because it doesn’t fit your model or process
● Because the name is too boring