• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
PyCon 2010 SQLAlchemy tutorial

PyCon 2010 SQLAlchemy tutorial



Links to files are broken, sorry.

Links to files are broken, sorry.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.


11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • nice tutorial / overview
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    PyCon 2010 SQLAlchemy tutorial PyCon 2010 SQLAlchemy tutorial Presentation Transcript

    • Install● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5 1. easy_install, if needed ● wget http://peak.telecommunity.com/dist/ez_setup.py ● python ez_setup.py 2. easy_install sqlalchemy==0.5.8● http://old.utahpython.org/sqla2010/
    • Michael BayerMichael Bayer is a software architect in New York City and is the creator of SQLAlchemy.http://techspot.zzzeek.org/@zzzeek
    • ORM 101 – the bad old daysc = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c.fetchone())user.last_login_at = time.time()sql = "UPDATE users SET last_login_at = %s WHERE name = %s"c.execute(sql, (user.last_login_at, name0))c.commit()
    • ORM 101 – and there was lightsession = Session()user = session.query(User).filter(name=name).one()user.last_login_at = time.time()session.commit()
    • The devil is in the details● Compound WHERE clauses, subqueries, outer joins, sql functions, ...● Eager/lazy loading● Support for legacy schemas● Inheritance● Conceptual integrity● Setup overhead● Database support
    • What SQLAlchemy is not
    • [Demolition photo]
    • Tough love?“Disproving the myth of the best database layer isthe one that makes the database invisible is aprimary philosophy of SA. If you dont want to dealwith SQL, then theres little point to using a[relational] database in the first place.”
    • Technical excellence● PK: multi-column is fine; mutable is fine; any data type is fine; doesnt have to be named “id” http://blogs.ittoolbox.com/database/soup/archives/primary-keyvil-part-i-73● Recognizes all database defaults instead of allowing a few special cases like “created_at”● Doesnt make up its own query language● No XML● Introspection or define-tables-in-Python● Session/unit-of-work based● Migrations
    • Supported databasesPostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess
    • Database dependence: a feature● Performance – Functions, partial indexes, bitmap indexes, partitioning, replication, ...● Features – Views, arrays, recursive joins, full-text searching, ...See also: http://powerpostgresql.com/Downloads/database_d
    • CachingBeaker integration is an example w/ the 0.6distribution (currently in beta)
    • Questions
    • Todays agenda: fundamentalsData Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRelationship lifecycleBackrefs
    • Agenda 2: More ORM detailsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
    • Agenda 3: Extensions and related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
    • Two ORM patternsActive RecordData Mapper
    • Active Record
    • Data Mapper
    • SQLAlchemy supports both● Declarative plugin for common simple situations● Full data mapper power when you need it
    • Tables for this tutorialusersaddressesordersorderitemskeywordsitemkeywords
    • Tablesusers = Table(users, metadata, Column(user_id, Integer, primary_key = True), Column(name, String(40)))users = Table(users, metadata, autoload=True)users = Table(users, metadata, autoload=True, Column(name, String(40), default=Jonathan))
    • Legacy columnsack = Table(ACK110030, metadata, Column(ITMNUMBER, Integer, primary_key=True, key=id), Column(MNFCENTERLC_I, Integer, ForeignKey(MFC43222.id), key=manufacturing_center_id), ...)
    • Setup from scratchengine = create_engine(sqlite:///:memory:, echo=True)metadata = MetaData()metadata.bind = engineSession = sessionmaker(bind=engine)from sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()
    • Quick setuphttp://old.utahpython.org/sqla2010/from tutorial_tables import *create()data()
    • Table + mapped class togetherclass Order(Base): __tablename__ = orders order_id = Column(Integer, primary_key=True) user_id = Column(Integer, ForeignKey(users.c.user_id)) description = Column(description, String(50)) isopen = Column(Integer, ColumnDefault(1))
    • Full data mapper patternorders = Table(orders, metadata, Column(order_id, Integer, ...), Column(user_id, Integer, ForeignKey(users.c.user_id)), Column(description, String(50)), Column(isopen, Integer, ColumnDefault(1)),class Order(object): passmapper(Order, orders)
    • The way we will do mappingclass Order(Base): __table__ = orders
    • Queryingsession = Session()q = session.query(Order)print q.all.get.first, .one
    • Query modificationq = session.query(Order).filter.filter_by.order_by [desc, asc].limit, .offset
    • Operators== >= <= > <~ | &not_ or_ and_in_between like startswith endswith
    • Some examplesq = session.query(Order)q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like(order%)).all()q.filter((Order.user_id==7) | (Order.isopen==1)).all()q.filter(or_(Order.user_id==7, Order.isopen==1)).all()
    • Slicing: limit/offset sugarq = session.query(Order)q.limit(1).offset(2)q[2:3]
    • Questions
    • Exercise● Map the orderitems table to an OrderItem class● Get a list of all OrderItems – Where they belong to order #3 – ... or the item name is “item 1” – ... ordered by item name(Now would be a good time to look attutorial_samples.py)
    • clear_mappers()
    • Creating, updatingo = Order()o.user_id = 7o.description = order 6session.add(o)o.order_id is Nonesession.commit()o.order_id == 6o.description = order Bsession.commit()session.delete(o)session.commit()
    • Scoped (“smart”) sessionsSession = scoped_session( sessionmaker(autoflush=True, autocommit=False))assert Session() == Session()
    • Scoped sessions 2Base = declarative_base(metadata=Session.metadata)class Order(Base): ...o = Order()o.user_id = 7o.description = order 6session.commit()
    • Direct updates, deletesSQL layer alert!orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_id==2).execute()
    • One-to-many relationsclass User(Base): orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first()print u.orders
    • Editing collectionso = Order(description=An order)u.orders.append(o)session.commit()
    • Why sessions are your friendsSome ORMs rely on explicit saveu = User.get(1)u.orders[0].description = An orderu.save() # not real SA code# doh! orders[0] was not saved!More convenient, less error-prone to let ORMtrack dirty objects
    • Identity mapRows with same PK get mapped to sameobject (per-session)Limited caching for get()Only for get()
    • Managing the identity mapsesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)expunge_all, expire_all
    • Questions
    • ExerciseLoad the user named jack (lowercase)Remove his first orderSave changes to the db
    • Fun with collectionsu = session.query(User).filter_by(name=jack).one()u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1)Order(order_id=1,user_id=None,...)
    • Two solutionso = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=uorder 1,isopen=0)session.delete(o)session.commit()>>> u.orders[Order(order_id=3,...), Order(order_id=5,...)] Why does it make sense for this to work differently than the previous example?
    • #2: delete-orphanclass User(Base): __table__ = users orders = relation(Order, cascade="all, delete-orphan", order_by=[Order.order_id])u = session.query(User).get(7)u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1) is NoneTrue
    • Questions
    • Exercisedef user_for_order(order): session = Session.object_session(order) return ?
    • Backrefsclass User(Base): __table__ = users orders = relation(Order, backref=user, order_by=[orders.c.order_id])o = session.query(Order).first()o.user
    • Thats it for fundamentalsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
    • ExerciseList all users who have open orders (isopen isnonzero)
    • A taste of advanced queryingAll orders with an open order:q = session.query(User).join(User.orders)q.filter(Order.isopen==1).all()
    • Selecting multiple classesWhat if we want the user and the order?(efficiently)q = session.query(User, Order) # cartesian join!q = q.join(User.orders)# or!q = q.filter(User.user_id==Order.user_id)u, o = q.filter(Order.isopen==1).first()
    • Dropping down to SQLsql = """select u.*from users uwhere u.user_id in ( select user_id from orders where isopen = 1)"""session.query(User).from_statement(sql)sql = """select u.*, o.*from users u join orders o on (u.user_id = o.user_id)where o.isopen = 1"""session.query(User, Order).from_statement(sql)
    • Dropping down a little lessq = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()
    • ExerciseIn a single query, select the users and orderswhere the order description is like order%
    • One to oneclass Address(Base): __table__ = addressesclass User(Base): orders = relation(Order, order_by=[orders.c.order_id]) address = relation(Address)How does SQLA know to treat these differently?
    • Many to manyclass Keyword(Base): __table__ = keywordsclass Item(Base): __table__ = orderitems keywords = relation(Keyword, secondary=itemkeywords)
    • Relation queriesuser = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order.user_id==user.user_id).all()
    • Relation queries 2q = session.query(Order)q.filter(Order.user.has(name=jack)).all()q.filter(Order.user.has((User.name==jack) | (User.user_id >= 9))).all()# q.filter(Order.user_id== select([users.c.user_id], users.c.name==jack)).all()# q.filter(Order.user_id== select([users.c.user_id], (users.c.name==jack) | (users.c.user_id >= 9))).all()
    • Relation queries 3q = session.query(User)q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any( Order.description.like(order%))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.isopen > 0))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.description.like(order%)))).all()
    • Relation queries 4Just need a raw EXISTS clause? Use .any() or.has() without extra parameters.
    • Relation queries 5Keyword = session.query(Keyword).filter_by(name=red).one()q = session.query(Item)q.filter(Item.keywords.contains(keyword)).all()# q.filter(Item.item_id.in_( select([itemkeywords.c.item_id], itemkeywords.c.keyword_id ==keyword.keyword_id))).all()
    • ExerciseRetrieve all users that do not have any orders.
    • Breathe EasyThat was our last exercise!
    • Eager loadingclass User(Base): orders = relation(Order, order_by=[orders.c.order_id], lazy=False)# orq = session.query(User).options(eagerload(orders))# also lazyload, noload
    • Transactions are simple● session: autocommit=False – this is the default – commit() / rollback() manually● autocommit=True – each flush() also commits
    • Other transaction features● begin_nested● manual transaction management at the Connection level
    • __init____init__ is for object creation, not loadinguse @reconstructorclass Order(Base): __table__ = orders def __init__(self): self.foo = [] @reconstructor def loaded(self): self.foo = []
    • Questions
    • Related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
    • Migrate# one-time setupmigrate create path/to/upgradescripts "comment"migrate manage dbmanage.py--repository=path/to/upgradescripts –url=db-connection-url./dbmanage.py version_control# repeat as necessary:./dbmanage.py script_sql sqlite# edit script./dbmanage.py upgrade
    • FormAlchemyorder1 = session.query(Order).first()from formalchemy import FieldSetfs = FieldSet(order1)print fs.render()
    • FormAlchemy, cont.from formalchemy import Gridorders = session.query(Order).all()g = Grid(Order, orders)print g.render()
    • SqlSoup>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)>>> db.users.filter(db.users.user_id < 10).all()[MappedUsers(user_id=7,name=jack),MappedUsers(user_id=8,name=ed),MappedUsers(user_id=9,name=fred)]>>> db.users.first()MappedUsers(user_id=7,name=jack)>>> _.ordersTraceback (most recent call last): File "<stdin>", line 1, in <module>AttributeError: MappedUsers object has no attributeorders
    • SqlSoup 2>>> db.users.relate(orders, db.orders)>>> db.users.first().orders[MappedOrders(...)]>>> db.users.filter(db.users.orders.any()).all()[MappedUsers(...)]
    • SqlSoup 3s = select([func.count(*)], users.c.user_id==orders.c.user_id, from_obj=[orders], scalar=True)s2 = select([users,s.label(order_count)]).alias(users_with_count)db.users_with_count = db.map(s2)
    • Elixir class Person(Entity): has_field(name, Unicode) acts_as_taggable() ... some_person_instance.add_tag(cool) ... cool_people = Person.get_by_tag(cool)http://cleverdevil.org/computing/52/
    • Resourcesirc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir
    • Final Questions