PofEAA and SQLAlchemy

PofEAA and
SQLAlchemy
INADA Naoki
@methane

Public Activity
MessagePack-Python
JSON like binary protocol
wsaccel
WebSocket accelerator for Tornado, ws4py
MyKaze
PyMySQL in Tornado (WIP)
Python for PHPer
Quick guide of Python for PHPer (Japanese)

Online Game Developer
R&D and Architect
Tuning server side middleware and application
KLab Inc.

Patterns of
Enterprise Application
Architecture

Design patterns
good to know
for Web Programmers

SQLAlchemy
Quickly
(from tutorial)

● Dialects -- Various DB-API wrapper
● Engine -- Connection management
● Schema and Types
● SQL expression
● and O/R Mapper
SQLAlchemy features

Create Engine
from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:',
echo=True)
# Using engine without O/R mapper
con = engine.connect()
with con.begin() as trx:
con.execute('SELECT 1+1')
con.close()

Define Object and Schema
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
email = Column(String, nullable=False, unique=True)
password = Column(String, nullable=False)

Create table on DB
Base.metadata.create_all(engine)
Output:
CREATE TABLE users (
id INTEGER NOT NULL,
name VARCHAR,
email VARCHAR NOT NULL,
password VARCHAR NOT NULL,
PRIMARY KEY (id),
UNIQUE (email)
)

Create an instance
>>> ed_user = User(name='ed',
... email='ed@example.com',
... password='edspassword')
...
>>> ed_user.password
'edspassword'
>>> str(ed_user.id)
'None'

Save it
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session()
session.add(ed_user)
session.commit()

Why not just
engine.save(user)
What's session?

Unit of Work is
a Pattern
in PofEAA

Part 1. The Narratives
3. Mapping to RDB
Part 2. The Patterns
10. Data Source Architectural Patterns
11. O/R Behavioral Patterns
12. O/R Structural Patterns
13. O/R Metadata Mapping Patterns
O/R chapters in the P of EAA

11. O/R Behavioral Patterns
● Unit of Work
● Identity map
● Lazy load

Unit of Work
Target: Maintains set of objects to save
in transaction.
How:
1. Wrap transaction.
2. Keep track of new, modified and deleted
objects.
3. Save objects when commit

Why Unit of Work
Remove boilerplate saving code from domain
logic.
Avoid objects saved too often.

Unit of Work in SQLAlchemy
# Create an Unit of Work
session = Session()
# Get connection and start transaction
ed_user = session.query(User).
filter(User.name=='ed').one()
ed_user.name='E.D.'
session.commit() # save ed_user and commit

Object states
user = User(...)
# Transient (not saved)
session.add(user)
# pending (saved)
session.flush()
# persistent (saved if modified)
session.expunge(user)
# Detached (not saved)

Flushing
Execute insert / update /delete query.
Session flushes automatically when querying
to avoid inconsistent result. (autoflush)
You can stop autoflush.
You can manually flush.

expire -- cause reload
session.expire(user)
print(user.name) # Cause reloading
commit() # expires all objects for consistency.
You can avoid this by
expire_on_commit=False.

Contextual Session
from sqlalchemy.orm import (sessionmaker,
scoped_session)
Session = scoped_session(
sessionmaker(bind=engine))
Session.add(ed_user)
Session.commit()
...
Session.remove()

Target:
Avoid two objects for one record.
How:
Keep mapping of (table, pk) -> object.
Check the map when querying.
Bonus: It acts like a easy cache
Identity map

Identity map in SQLAlchemy
Session has identity map.
# Looks up id map after querying
session.query(User).filter(User.id==3).one()
# Looks up id map before and after querying.
session.query(User).get(3)
NOTE: Id map is weak reference by default.

Lazy Load
Load relation or heavy column on demand.

Lazy loading relationship
http://docs.sqlalchemy.
org/en/rel_0_8/orm/loading.html
SQLAlchemy uses lazy loading by default for
relationship.
You can choose eager loading strategy.

Lazy loading big column
org/en/rel_0_8/orm/mapper_config.
html#deferred-column-loading
SQLAlchemy supports deferred column to
lazy loading big BLOB/TEXT.

12. O/R Structural Pattern
● Identity Field
● Foreign Key Mapping
● Association Table Mapping
● Dependent Mapping
● Embedded Value
● Serialized LOB
● Single Table Inheritance
● Class Table Inheritance
● Concrete Table Inheritance
● Inheritance Mappers

Object have PK in database as a field.
class Player(Base):
id = Column(INTEGER, primary=True)
Identity Field

Map one to one or one to many relation to
Object reference.
Not:
session.query(Address).get(user.address_id)
Yes:
user.address
Foreign Key Mapping

With property
from werkzeug.utils.cached_property
class User(Base):
...
@cached_property
def address(self):
session.query(Address).
get(self.address_id)
This is enough to achieve ``user.address``.
But no SQLA support.

relationship
class User(Base):
...
addresses = relationship('Address',
backref='user')
class Address(Base):
...
user_id = Column(Integer,
ForeignKey('user.id'))

Association Table Mapping
Mapping many-to-many association table to
object reference.
In SQLAlchemy:
Pass “secondary” argument to relationship()
http://docs.sqlalchemy.org/en/rel_0_8/orm/tutorial.html#building-a-many-
to-many-relationship

Dependent Mapping
O/R map ownership without identity.
PofEAA says:
I don’t recommend Dependent Mapping
if you’re using Unit of Work.

Embedded Value
Map an object to some columns in a row.
class Duration:
start_date
end_date
class Account:
duration
CREATE TABLE account (
…
start_date DATE,
end_date DATE,
…
);

Embedded Value by property
@property
def duration(self):
return Duration(
self.start_date, self.end_date)
@duration.setter
def duration(self, duration):
self.start_date = duration.start
self.end_date = duration.end

Embedded Value by Composite
http://docs.sqlalchemy.org/en/rel_0_8/orm/mapper_config.html#mapper-
composite
class Account(Base):
...
start_date = Column(DATE)
end_date = Column(DATE)
duration = composite(Duration, start_date, end_date)
class Duration(namedtuple(‘Duration’, ‘start end’)):
def __composite_value__(self):
return self

Serialized LOB
Serialize objects and save to XLOB column.

Using property
_info = Column('info', BLOB)
@property
def info(self):
return json.loads(self._info)
@info.setter
def _set_info(self, info):
self._info = json.dumps(self)

Custom Type
SQLAlchemy provides PickleType for serialized
LOB.
You can define custom type via TypeDecorator:
http://docs.sqlalchemy.org/en/rel_0_8/core/types.
html#marshal-json-strings

Single Table Inheritance
Player
SoccerPlayer BaseballPlayer
CREATE TABLE player(
id INTEGER PRIMARY KEY,
type INTEGER NOT NULL,
position INTEGER,
)
All classes saved
into one table.

Single Table Inheritance in SQLA
class Player(Base):
__tablename__ = ‘player’
id = Column(INTEGER, primary_key=True)
position = Column(INTEGER)
type = Column(INTEGER, nullable=False)
__mapper_args__ = {‘polymorphic_on’: type}
SOCCER_PLAYER = 1
BASEBALL_PLAYER = 2
class SoccerPlayer(Player):
__mapper_args__ = {
‘polymorhic_identity’: Player.SOCCER_PLAYER}

Defining columns in subclass
Defining columns in subclass may cause
conflict.
SQLA provides way to avoid it. See below.
org/en/latest/orm/extensions/declarative.html#resolving-
column-conflicts

Class Table Inheritance
Player
CREATE TABLE player(
type INTEGER NOT NULL
)
CREATE TABLE soccer_player(
position INTEGER NOT NULL
)
CREATE TABLE baseball_player(
)
Tables for each
classes.

Class table inheritance in SQLA
SQLA call it “Joined table inheritance”
http://docs.sqlalchemy.org/en/latest/orm/extensions/declarative.html#joined-
table-inheritance

class Player(Base):
__tablename__ = 'player'
type = Column(INTEGER, nullable=False)
SOCCER_PLAYER = 1
BASEBALL_PLAYER = 2
__mapper_args__ = {'polymorphic_on': type}
class SoccerPlayer(Player):
__tablename__ = 'soccer_player'
id = Column(ForeignKey('player.id'), primary_key=True)
position = Column(INTEGER, nullable=False)
__mapper_args__ = {
'polymorhic_identity': Player.SOCCER_PLAYER}

Concrete Table Inheritance
Player
CREATE TABLE soccer_player(
name VARCHAR(32) NOT NULL,
)
CREATE TABLE baseball_player(
name VARCHAR(32) NOT NULL,
)
Tables for each
concrete classes

Mix-in
class BasePlayer(object):
name = Column(VARCHAR(32), nullable=False)
class SoccerPlayer(BasePlayer, Base):
__tablename__ = ‘soccer_player’
position = Column(INTEGER, nullable=False)
Pros) Simple.
Cons) You can’t get support from SQLAlchemy

Concrete Table Inheritance in SQLA
http://docs.sqlalchemy.org/en/latest/orm/inheritance.
html#concrete-table-inheritance

10. Data Source Architectural Patterns
● Table Data Gateway
● Row Data Gateway
● Active Record
● Data Mapper
● Architectural Pattern and SQLAlchemy

Table Data Gateway
Target: Split querying from your domain logic
How: Create gateway for each table.
class PlayerGateway:
def find_by_id(self, id)
def find_by_name(self, name)
def update(self, id, name=None, age=None)
def insert(self, id, name, age)

Row Data Gateway
Target: Split querying from your domain logic
How: Create gateway for each record.
class PlayerGateway:
@classmethod
def find_by_id(cls, id):
…
return cls(id, name, age)
…
def insert(self)
def update(self)

Active Record
Row Data Gateway with Domain Logic
class Player:
@classmethod
def find_by_id(cls, id)
...
def birthday(self):
self.age += 1

Data Mapper
Target: Split out column/attribute mapping code
from your domain logic.
class PersonMapper(BaseMapper):
def map(self, row):
return Person(id=row.id,
name=row.name,
age=row.age)

Architectural Patterns
and SQLAlchemy
My personal consideration

Querying in SQLAlchemy
Unit of Work handles most of update and insert
queries.
Easy query can be written very easy.
(Building complex query is hard)

Table Gateway?
Most tables doesn’t require Table Gateway.
When there are some complex queries,
Table Gateway is good to have.

Row Data Gateway?
Active Record?
Separating model object and row data gateway
cause additional complexity.
For example, does your model need identity
map?
Person
Person
PersonRow

Active Record drawbacks
Problem: Good table design is not good class
design always.
My answer: Use structural patterns like
Embedded Value.
P: Mixing database access and domain logic is
bad idea.
A: Use Table Gateway to separate them.

Fat Model (™)
Person.register(...) # class method
person.unregister()
person.request_friend(other_person_id)
...

Simple ActiveRecord & Service
class AccountService:
def register(self, name, email, age,...)
def unregister(self, person_id)
class FriendService:
def request(self,
from_person_id, to_person_id)

Finder in Service
When you use service classes, you may be able to
put finders there and don’t use Table Gateway.
class PersonService:
...
def find_by_name(self, name):
return Session.query(Person).
filter_by(name=name).all()

Anemic Domain Model
http://www.martinfowler.
com/bliki/AnemicDomainModel.html
P: Implementing all domain logic in service is
not a OOP.
A: Simple Active Record + Medium sized
classes + Service

When you design large
application or framework,
PofEAA helps you lot.

PofEAA and SQLAlchemy

More Related Content

What's hot

Similar to PofEAA and SQLAlchemy

Recently uploaded

PofEAA and SQLAlchemy