Install● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5       1. easy_install, if needed          ● wget http://p...
Michael BayerMichael Bayer is a software architect in New York City and is the creator of SQLAlchemy.http://techspot.zzzee...
ORM 101 – the bad old daysc = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c....
ORM 101 – and there was lightsession = Session()user = session.query(User).filter(name=name).one()user.last_login_at = tim...
The devil is in the details● Compound WHERE clauses, subqueries,  outer joins, sql functions, ...● Eager/lazy loading● Sup...
What SQLAlchemy is not
[Demolition photo]
Tough love?“Disproving the myth of the best database layer isthe one that makes the database invisible is aprimary philoso...
Technical excellence●   PK: multi-column is fine; mutable is fine; any    data type is fine; doesnt have to be named    “i...
Supported databasesPostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess
Database dependence: a feature●   Performance     –   Functions, partial indexes, bitmap indexes,         partitioning, re...
CachingBeaker integration is an example w/ the 0.6distribution (currently in beta)
Questions
Todays agenda: fundamentalsData Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRela...
Agenda 2: More ORM detailsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensio...
Agenda 3: Extensions and        related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
Two ORM patternsActive RecordData Mapper
Active Record
Data Mapper
SQLAlchemy supports both●   Declarative plugin for common simple    situations●   Full data mapper power when you need it
Tables for this tutorialusersaddressesordersorderitemskeywordsitemkeywords
Tablesusers = Table(users, metadata,    Column(user_id, Integer, primary_key = True),    Column(name, String(40)))users = ...
Legacy columnsack = Table(ACK110030, metadata,            Column(ITMNUMBER, Integer,                   primary_key=True, k...
Setup from scratchengine = create_engine(sqlite:///:memory:, echo=True)metadata = MetaData()metadata.bind = engineSession ...
Quick setuphttp://old.utahpython.org/sqla2010/from tutorial_tables import *create()data()
Table + mapped class togetherclass Order(Base):    __tablename__ = orders    order_id = Column(Integer, primary_key=True) ...
Full data mapper patternorders = Table(orders, metadata,    Column(order_id, Integer, ...),    Column(user_id, Integer, Fo...
The way we will do mappingclass Order(Base):    __table__ = orders
Queryingsession = Session()q = session.query(Order)print q.all.get.first, .one
Query modificationq = session.query(Order).filter.filter_by.order_by [desc, asc].limit, .offset
Operators== >= <= > <~ | &not_ or_ and_in_between like startswith endswith
Some examplesq = session.query(Order)q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like(...
Slicing: limit/offset sugarq = session.query(Order)q.limit(1).offset(2)q[2:3]
Questions
Exercise● Map the orderitems table to an OrderItem  class● Get a list of all OrderItems    –   Where they belong to order ...
clear_mappers()
Creating, updatingo = Order()o.user_id = 7o.description = order 6session.add(o)o.order_id is Nonesession.commit()o.order_i...
Scoped (“smart”) sessionsSession = scoped_session(    sessionmaker(autoflush=True, autocommit=False))assert Session() == S...
Scoped sessions 2Base = declarative_base(metadata=Session.metadata)class Order(Base): ...o = Order()o.user_id = 7o.descrip...
Direct updates, deletesSQL layer alert!orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_i...
One-to-many relationsclass User(Base):    orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first...
Editing collectionso = Order(description=An order)u.orders.append(o)session.commit()
Why sessions are your friendsSome ORMs rely on explicit saveu = User.get(1)u.orders[0].description = An orderu.save() # no...
Identity mapRows with same PK get mapped to sameobject (per-session)Limited caching for get()Only for get()
Managing the identity mapsesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)e...
Questions
ExerciseLoad the user named jack (lowercase)Remove his first orderSave changes to the db
Fun with collectionsu = session.query(User).filter_by(name=jack).one()u.orders = u.orders[1:]session.commit()>>> session.q...
Two solutionso = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=uorder 1,isopen=0)session.delete(o)session.commit(...
#2: delete-orphanclass User(Base):    __table__ = users    orders = relation(Order,                      cascade="all, del...
Questions
Exercisedef user_for_order(order):    session = Session.object_session(order)    return ?
Backrefsclass User(Base):    __table__ = users    orders = relation(Order,                      backref=user,             ...
Thats it for fundamentalsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtension...
ExerciseList all users who have open orders (isopen isnonzero)
A taste of advanced queryingAll orders with an open order:q = session.query(User).join(User.orders)q.filter(Order.isopen==...
Selecting multiple classesWhat if we want the user and the order?(efficiently)q = session.query(User, Order) # cartesian j...
Dropping down to SQLsql = """select u.*from users uwhere u.user_id in (    select user_id from orders where isopen = 1)"""...
Dropping down a little lessq = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()
ExerciseIn a single query, select the users and orderswhere the order description is like order%
One to oneclass Address(Base):    __table__ = addressesclass User(Base):    orders = relation(Order,                      ...
Many to manyclass Keyword(Base):    __table__ = keywordsclass Item(Base):    __table__ = orderitems    keywords = relation...
Relation queriesuser = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order....
Relation queries 2q = session.query(Order)q.filter(Order.user.has(name=jack)).all()q.filter(Order.user.has((User.name==jac...
Relation queries 3q = session.query(User)q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any(       ...
Relation queries 4Just need a raw EXISTS clause? Use .any() or.has() without extra parameters.
Relation queries 5Keyword = session.query(Keyword).filter_by(name=red).one()q = session.query(Item)q.filter(Item.keywords....
ExerciseRetrieve all users that do not have any orders.
Breathe EasyThat was our last exercise!
Eager loadingclass User(Base):    orders = relation(Order,                      order_by=[orders.c.order_id],             ...
Transactions are simple●   session: autocommit=False     –   this is the default     –   commit() / rollback() manually●  ...
Other transaction features●   begin_nested●   manual transaction management at the    Connection level
__init____init__ is for object creation, not loadinguse @reconstructorclass Order(Base):    __table__ = orders    def __in...
Questions
Related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
Migrate# one-time setupmigrate create path/to/upgradescripts "comment"migrate manage dbmanage.py--repository=path/to/upgra...
FormAlchemyorder1 = session.query(Order).first()from formalchemy import FieldSetfs = FieldSet(order1)print fs.render()
FormAlchemy, cont.from formalchemy import Gridorders = session.query(Order).all()g = Grid(Order, orders)print g.render()
SqlSoup>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)>>> db.users.filter(db.users.user_id < 10)....
SqlSoup 2>>> db.users.relate(orders, db.orders)>>> db.users.first().orders[MappedOrders(...)]>>> db.users.filter(db.users....
SqlSoup 3s = select([func.count(*)],           users.c.user_id==orders.c.user_id,           from_obj=[orders],           s...
Elixir   class Person(Entity):       has_field(name, Unicode)       acts_as_taggable()   ...   some_person_instance.add_ta...
Resourcesirc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir
Final Questions
PyCon 2010 SQLAlchemy tutorial
PyCon 2010 SQLAlchemy tutorial
PyCon 2010 SQLAlchemy tutorial
Upcoming SlideShare
Loading in …5
×

PyCon 2010 SQLAlchemy tutorial

9,167 views

Published on

Links to files are broken, sorry.

Published in: Technology
1 Comment
11 Likes
Statistics
Notes
  • nice tutorial / overview
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
9,167
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
117
Comments
1
Likes
11
Embeds 0
No embeds

No notes for slide

PyCon 2010 SQLAlchemy tutorial

  1. 1. Install● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5 1. easy_install, if needed ● wget http://peak.telecommunity.com/dist/ez_setup.py ● python ez_setup.py 2. easy_install sqlalchemy==0.5.8● http://old.utahpython.org/sqla2010/
  2. 2. Michael BayerMichael Bayer is a software architect in New York City and is the creator of SQLAlchemy.http://techspot.zzzeek.org/@zzzeek
  3. 3. ORM 101 – the bad old daysc = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c.fetchone())user.last_login_at = time.time()sql = "UPDATE users SET last_login_at = %s WHERE name = %s"c.execute(sql, (user.last_login_at, name0))c.commit()
  4. 4. ORM 101 – and there was lightsession = Session()user = session.query(User).filter(name=name).one()user.last_login_at = time.time()session.commit()
  5. 5. The devil is in the details● Compound WHERE clauses, subqueries, outer joins, sql functions, ...● Eager/lazy loading● Support for legacy schemas● Inheritance● Conceptual integrity● Setup overhead● Database support
  6. 6. What SQLAlchemy is not
  7. 7. [Demolition photo]
  8. 8. Tough love?“Disproving the myth of the best database layer isthe one that makes the database invisible is aprimary philosophy of SA. If you dont want to dealwith SQL, then theres little point to using a[relational] database in the first place.”
  9. 9. Technical excellence● PK: multi-column is fine; mutable is fine; any data type is fine; doesnt have to be named “id” http://blogs.ittoolbox.com/database/soup/archives/primary-keyvil-part-i-73● Recognizes all database defaults instead of allowing a few special cases like “created_at”● Doesnt make up its own query language● No XML● Introspection or define-tables-in-Python● Session/unit-of-work based● Migrations
  10. 10. Supported databasesPostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess
  11. 11. Database dependence: a feature● Performance – Functions, partial indexes, bitmap indexes, partitioning, replication, ...● Features – Views, arrays, recursive joins, full-text searching, ...See also: http://powerpostgresql.com/Downloads/database_d
  12. 12. CachingBeaker integration is an example w/ the 0.6distribution (currently in beta)
  13. 13. Questions
  14. 14. Todays agenda: fundamentalsData Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRelationship lifecycleBackrefs
  15. 15. Agenda 2: More ORM detailsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
  16. 16. Agenda 3: Extensions and related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
  17. 17. Two ORM patternsActive RecordData Mapper
  18. 18. Active Record
  19. 19. Data Mapper
  20. 20. SQLAlchemy supports both● Declarative plugin for common simple situations● Full data mapper power when you need it
  21. 21. Tables for this tutorialusersaddressesordersorderitemskeywordsitemkeywords
  22. 22. Tablesusers = Table(users, metadata, Column(user_id, Integer, primary_key = True), Column(name, String(40)))users = Table(users, metadata, autoload=True)users = Table(users, metadata, autoload=True, Column(name, String(40), default=Jonathan))
  23. 23. Legacy columnsack = Table(ACK110030, metadata, Column(ITMNUMBER, Integer, primary_key=True, key=id), Column(MNFCENTERLC_I, Integer, ForeignKey(MFC43222.id), key=manufacturing_center_id), ...)
  24. 24. Setup from scratchengine = create_engine(sqlite:///:memory:, echo=True)metadata = MetaData()metadata.bind = engineSession = sessionmaker(bind=engine)from sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()
  25. 25. Quick setuphttp://old.utahpython.org/sqla2010/from tutorial_tables import *create()data()
  26. 26. Table + mapped class togetherclass Order(Base): __tablename__ = orders order_id = Column(Integer, primary_key=True) user_id = Column(Integer, ForeignKey(users.c.user_id)) description = Column(description, String(50)) isopen = Column(Integer, ColumnDefault(1))
  27. 27. Full data mapper patternorders = Table(orders, metadata, Column(order_id, Integer, ...), Column(user_id, Integer, ForeignKey(users.c.user_id)), Column(description, String(50)), Column(isopen, Integer, ColumnDefault(1)),class Order(object): passmapper(Order, orders)
  28. 28. The way we will do mappingclass Order(Base): __table__ = orders
  29. 29. Queryingsession = Session()q = session.query(Order)print q.all.get.first, .one
  30. 30. Query modificationq = session.query(Order).filter.filter_by.order_by [desc, asc].limit, .offset
  31. 31. Operators== >= <= > <~ | &not_ or_ and_in_between like startswith endswith
  32. 32. Some examplesq = session.query(Order)q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like(order%)).all()q.filter((Order.user_id==7) | (Order.isopen==1)).all()q.filter(or_(Order.user_id==7, Order.isopen==1)).all()
  33. 33. Slicing: limit/offset sugarq = session.query(Order)q.limit(1).offset(2)q[2:3]
  34. 34. Questions
  35. 35. Exercise● Map the orderitems table to an OrderItem class● Get a list of all OrderItems – Where they belong to order #3 – ... or the item name is “item 1” – ... ordered by item name(Now would be a good time to look attutorial_samples.py)
  36. 36. clear_mappers()
  37. 37. Creating, updatingo = Order()o.user_id = 7o.description = order 6session.add(o)o.order_id is Nonesession.commit()o.order_id == 6o.description = order Bsession.commit()session.delete(o)session.commit()
  38. 38. Scoped (“smart”) sessionsSession = scoped_session( sessionmaker(autoflush=True, autocommit=False))assert Session() == Session()
  39. 39. Scoped sessions 2Base = declarative_base(metadata=Session.metadata)class Order(Base): ...o = Order()o.user_id = 7o.description = order 6session.commit()
  40. 40. Direct updates, deletesSQL layer alert!orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_id==2).execute()
  41. 41. One-to-many relationsclass User(Base): orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first()print u.orders
  42. 42. Editing collectionso = Order(description=An order)u.orders.append(o)session.commit()
  43. 43. Why sessions are your friendsSome ORMs rely on explicit saveu = User.get(1)u.orders[0].description = An orderu.save() # not real SA code# doh! orders[0] was not saved!More convenient, less error-prone to let ORMtrack dirty objects
  44. 44. Identity mapRows with same PK get mapped to sameobject (per-session)Limited caching for get()Only for get()
  45. 45. Managing the identity mapsesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)expunge_all, expire_all
  46. 46. Questions
  47. 47. ExerciseLoad the user named jack (lowercase)Remove his first orderSave changes to the db
  48. 48. Fun with collectionsu = session.query(User).filter_by(name=jack).one()u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1)Order(order_id=1,user_id=None,...)
  49. 49. Two solutionso = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=uorder 1,isopen=0)session.delete(o)session.commit()>>> u.orders[Order(order_id=3,...), Order(order_id=5,...)] Why does it make sense for this to work differently than the previous example?
  50. 50. #2: delete-orphanclass User(Base): __table__ = users orders = relation(Order, cascade="all, delete-orphan", order_by=[Order.order_id])u = session.query(User).get(7)u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1) is NoneTrue
  51. 51. Questions
  52. 52. Exercisedef user_for_order(order): session = Session.object_session(order) return ?
  53. 53. Backrefsclass User(Base): __table__ = users orders = relation(Order, backref=user, order_by=[orders.c.order_id])o = session.query(Order).first()o.user
  54. 54. Thats it for fundamentalsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
  55. 55. ExerciseList all users who have open orders (isopen isnonzero)
  56. 56. A taste of advanced queryingAll orders with an open order:q = session.query(User).join(User.orders)q.filter(Order.isopen==1).all()
  57. 57. Selecting multiple classesWhat if we want the user and the order?(efficiently)q = session.query(User, Order) # cartesian join!q = q.join(User.orders)# or!q = q.filter(User.user_id==Order.user_id)u, o = q.filter(Order.isopen==1).first()
  58. 58. Dropping down to SQLsql = """select u.*from users uwhere u.user_id in ( select user_id from orders where isopen = 1)"""session.query(User).from_statement(sql)sql = """select u.*, o.*from users u join orders o on (u.user_id = o.user_id)where o.isopen = 1"""session.query(User, Order).from_statement(sql)
  59. 59. Dropping down a little lessq = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()
  60. 60. ExerciseIn a single query, select the users and orderswhere the order description is like order%
  61. 61. One to oneclass Address(Base): __table__ = addressesclass User(Base): orders = relation(Order, order_by=[orders.c.order_id]) address = relation(Address)How does SQLA know to treat these differently?
  62. 62. Many to manyclass Keyword(Base): __table__ = keywordsclass Item(Base): __table__ = orderitems keywords = relation(Keyword, secondary=itemkeywords)
  63. 63. Relation queriesuser = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order.user_id==user.user_id).all()
  64. 64. Relation queries 2q = session.query(Order)q.filter(Order.user.has(name=jack)).all()q.filter(Order.user.has((User.name==jack) | (User.user_id >= 9))).all()# q.filter(Order.user_id== select([users.c.user_id], users.c.name==jack)).all()# q.filter(Order.user_id== select([users.c.user_id], (users.c.name==jack) | (users.c.user_id >= 9))).all()
  65. 65. Relation queries 3q = session.query(User)q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any( Order.description.like(order%))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.isopen > 0))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.description.like(order%)))).all()
  66. 66. Relation queries 4Just need a raw EXISTS clause? Use .any() or.has() without extra parameters.
  67. 67. Relation queries 5Keyword = session.query(Keyword).filter_by(name=red).one()q = session.query(Item)q.filter(Item.keywords.contains(keyword)).all()# q.filter(Item.item_id.in_( select([itemkeywords.c.item_id], itemkeywords.c.keyword_id ==keyword.keyword_id))).all()
  68. 68. ExerciseRetrieve all users that do not have any orders.
  69. 69. Breathe EasyThat was our last exercise!
  70. 70. Eager loadingclass User(Base): orders = relation(Order, order_by=[orders.c.order_id], lazy=False)# orq = session.query(User).options(eagerload(orders))# also lazyload, noload
  71. 71. Transactions are simple● session: autocommit=False – this is the default – commit() / rollback() manually● autocommit=True – each flush() also commits
  72. 72. Other transaction features● begin_nested● manual transaction management at the Connection level
  73. 73. __init____init__ is for object creation, not loadinguse @reconstructorclass Order(Base): __table__ = orders def __init__(self): self.foo = [] @reconstructor def loaded(self): self.foo = []
  74. 74. Questions
  75. 75. Related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
  76. 76. Migrate# one-time setupmigrate create path/to/upgradescripts "comment"migrate manage dbmanage.py--repository=path/to/upgradescripts –url=db-connection-url./dbmanage.py version_control# repeat as necessary:./dbmanage.py script_sql sqlite# edit script./dbmanage.py upgrade
  77. 77. FormAlchemyorder1 = session.query(Order).first()from formalchemy import FieldSetfs = FieldSet(order1)print fs.render()
  78. 78. FormAlchemy, cont.from formalchemy import Gridorders = session.query(Order).all()g = Grid(Order, orders)print g.render()
  79. 79. SqlSoup>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)>>> db.users.filter(db.users.user_id < 10).all()[MappedUsers(user_id=7,name=jack),MappedUsers(user_id=8,name=ed),MappedUsers(user_id=9,name=fred)]>>> db.users.first()MappedUsers(user_id=7,name=jack)>>> _.ordersTraceback (most recent call last): File "<stdin>", line 1, in <module>AttributeError: MappedUsers object has no attributeorders
  80. 80. SqlSoup 2>>> db.users.relate(orders, db.orders)>>> db.users.first().orders[MappedOrders(...)]>>> db.users.filter(db.users.orders.any()).all()[MappedUsers(...)]
  81. 81. SqlSoup 3s = select([func.count(*)], users.c.user_id==orders.c.user_id, from_obj=[orders], scalar=True)s2 = select([users,s.label(order_count)]).alias(users_with_count)db.users_with_count = db.map(s2)
  82. 82. Elixir class Person(Entity): has_field(name, Unicode) acts_as_taggable() ... some_person_instance.add_tag(cool) ... cool_people = Person.get_by_tag(cool)http://cleverdevil.org/computing/52/
  83. 83. Resourcesirc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir
  84. 84. Final Questions

×