Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PyCon 2010 SQLAlchemy tutorial

9,423 views

Published on

Links to files are broken, sorry.

Published in: Technology
  • DOWNLOAD FULL BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • nice tutorial / overview
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

PyCon 2010 SQLAlchemy tutorial

  1. 1. Install● Python 2.5 or 2.6 (or pysqlite for 2.4)● SQLAlchemy 0.5 1. easy_install, if needed ● wget http://peak.telecommunity.com/dist/ez_setup.py ● python ez_setup.py 2. easy_install sqlalchemy==0.5.8● http://old.utahpython.org/sqla2010/
  2. 2. Michael BayerMichael Bayer is a software architect in New York City and is the creator of SQLAlchemy.http://techspot.zzzeek.org/@zzzeek
  3. 3. ORM 101 – the bad old daysc = db.cursor()sql = "SELECT * FROM users WHERE name = %s"c.execute(sql, (name,))user = User(*c.fetchone())user.last_login_at = time.time()sql = "UPDATE users SET last_login_at = %s WHERE name = %s"c.execute(sql, (user.last_login_at, name0))c.commit()
  4. 4. ORM 101 – and there was lightsession = Session()user = session.query(User).filter(name=name).one()user.last_login_at = time.time()session.commit()
  5. 5. The devil is in the details● Compound WHERE clauses, subqueries, outer joins, sql functions, ...● Eager/lazy loading● Support for legacy schemas● Inheritance● Conceptual integrity● Setup overhead● Database support
  6. 6. What SQLAlchemy is not
  7. 7. [Demolition photo]
  8. 8. Tough love?“Disproving the myth of the best database layer isthe one that makes the database invisible is aprimary philosophy of SA. If you dont want to dealwith SQL, then theres little point to using a[relational] database in the first place.”
  9. 9. Technical excellence● PK: multi-column is fine; mutable is fine; any data type is fine; doesnt have to be named “id” http://blogs.ittoolbox.com/database/soup/archives/primary-keyvil-part-i-73● Recognizes all database defaults instead of allowing a few special cases like “created_at”● Doesnt make up its own query language● No XML● Introspection or define-tables-in-Python● Session/unit-of-work based● Migrations
  10. 10. Supported databasesPostgreSQLMySQLSQLiteFirebirdOracleMSSQLSybaseDB2InformixSAPDBMSAccess
  11. 11. Database dependence: a feature● Performance – Functions, partial indexes, bitmap indexes, partitioning, replication, ...● Features – Views, arrays, recursive joins, full-text searching, ...See also: http://powerpostgresql.com/Downloads/database_d
  12. 12. CachingBeaker integration is an example w/ the 0.6distribution (currently in beta)
  13. 13. Questions
  14. 14. Todays agenda: fundamentalsData Mapper vs Active RecordSA FundamentalsMapping basicsQueriesSessions & the identity mapRelationship lifecycleBackrefs
  15. 15. Agenda 2: More ORM detailsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
  16. 16. Agenda 3: Extensions and related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
  17. 17. Two ORM patternsActive RecordData Mapper
  18. 18. Active Record
  19. 19. Data Mapper
  20. 20. SQLAlchemy supports both● Declarative plugin for common simple situations● Full data mapper power when you need it
  21. 21. Tables for this tutorialusersaddressesordersorderitemskeywordsitemkeywords
  22. 22. Tablesusers = Table(users, metadata, Column(user_id, Integer, primary_key = True), Column(name, String(40)))users = Table(users, metadata, autoload=True)users = Table(users, metadata, autoload=True, Column(name, String(40), default=Jonathan))
  23. 23. Legacy columnsack = Table(ACK110030, metadata, Column(ITMNUMBER, Integer, primary_key=True, key=id), Column(MNFCENTERLC_I, Integer, ForeignKey(MFC43222.id), key=manufacturing_center_id), ...)
  24. 24. Setup from scratchengine = create_engine(sqlite:///:memory:, echo=True)metadata = MetaData()metadata.bind = engineSession = sessionmaker(bind=engine)from sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()
  25. 25. Quick setuphttp://old.utahpython.org/sqla2010/from tutorial_tables import *create()data()
  26. 26. Table + mapped class togetherclass Order(Base): __tablename__ = orders order_id = Column(Integer, primary_key=True) user_id = Column(Integer, ForeignKey(users.c.user_id)) description = Column(description, String(50)) isopen = Column(Integer, ColumnDefault(1))
  27. 27. Full data mapper patternorders = Table(orders, metadata, Column(order_id, Integer, ...), Column(user_id, Integer, ForeignKey(users.c.user_id)), Column(description, String(50)), Column(isopen, Integer, ColumnDefault(1)),class Order(object): passmapper(Order, orders)
  28. 28. The way we will do mappingclass Order(Base): __table__ = orders
  29. 29. Queryingsession = Session()q = session.query(Order)print q.all.get.first, .one
  30. 30. Query modificationq = session.query(Order).filter.filter_by.order_by [desc, asc].limit, .offset
  31. 31. Operators== >= <= > <~ | &not_ or_ and_in_between like startswith endswith
  32. 32. Some examplesq = session.query(Order)q.filter_by(user_id=7).order_by(Order.isopen).first()q.filter(Order.description.like(order%)).all()q.filter((Order.user_id==7) | (Order.isopen==1)).all()q.filter(or_(Order.user_id==7, Order.isopen==1)).all()
  33. 33. Slicing: limit/offset sugarq = session.query(Order)q.limit(1).offset(2)q[2:3]
  34. 34. Questions
  35. 35. Exercise● Map the orderitems table to an OrderItem class● Get a list of all OrderItems – Where they belong to order #3 – ... or the item name is “item 1” – ... ordered by item name(Now would be a good time to look attutorial_samples.py)
  36. 36. clear_mappers()
  37. 37. Creating, updatingo = Order()o.user_id = 7o.description = order 6session.add(o)o.order_id is Nonesession.commit()o.order_id == 6o.description = order Bsession.commit()session.delete(o)session.commit()
  38. 38. Scoped (“smart”) sessionsSession = scoped_session( sessionmaker(autoflush=True, autocommit=False))assert Session() == Session()
  39. 39. Scoped sessions 2Base = declarative_base(metadata=Session.metadata)class Order(Base): ...o = Order()o.user_id = 7o.description = order 6session.commit()
  40. 40. Direct updates, deletesSQL layer alert!orders.update(orders.c.order_id==2).execute(isopen=1)orders.delete(orders.c.order_id==2).execute()
  41. 41. One-to-many relationsclass User(Base): orders = relation(Order, order_by=[Order.order_id])u = session.query(User).first()print u.orders
  42. 42. Editing collectionso = Order(description=An order)u.orders.append(o)session.commit()
  43. 43. Why sessions are your friendsSome ORMs rely on explicit saveu = User.get(1)u.orders[0].description = An orderu.save() # not real SA code# doh! orders[0] was not saved!More convenient, less error-prone to let ORMtrack dirty objects
  44. 44. Identity mapRows with same PK get mapped to sameobject (per-session)Limited caching for get()Only for get()
  45. 45. Managing the identity mapsesion.query(cls).populate_existing()session.expire(obj)session.refresh(obj)session.expunge(obj)expunge_all, expire_all
  46. 46. Questions
  47. 47. ExerciseLoad the user named jack (lowercase)Remove his first orderSave changes to the db
  48. 48. Fun with collectionsu = session.query(User).filter_by(name=jack).one()u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1)Order(order_id=1,user_id=None,...)
  49. 49. Two solutionso = u.orders[0]>>> oOrder(order_id=1,user_id=7,description=uorder 1,isopen=0)session.delete(o)session.commit()>>> u.orders[Order(order_id=3,...), Order(order_id=5,...)] Why does it make sense for this to work differently than the previous example?
  50. 50. #2: delete-orphanclass User(Base): __table__ = users orders = relation(Order, cascade="all, delete-orphan", order_by=[Order.order_id])u = session.query(User).get(7)u.orders = u.orders[1:]session.commit()>>> session.query(Order).get(1) is NoneTrue
  51. 51. Questions
  52. 52. Exercisedef user_for_order(order): session = Session.object_session(order) return ?
  53. 53. Backrefsclass User(Base): __table__ = users orders = relation(Order, backref=user, order_by=[orders.c.order_id])o = session.query(Order).first()o.user
  54. 54. Thats it for fundamentalsMulti-object queriesOne-to-oneMany-to-manyRelation queriesEager/lazy loadingTransactionsExtensions and related projects
  55. 55. ExerciseList all users who have open orders (isopen isnonzero)
  56. 56. A taste of advanced queryingAll orders with an open order:q = session.query(User).join(User.orders)q.filter(Order.isopen==1).all()
  57. 57. Selecting multiple classesWhat if we want the user and the order?(efficiently)q = session.query(User, Order) # cartesian join!q = q.join(User.orders)# or!q = q.filter(User.user_id==Order.user_id)u, o = q.filter(Order.isopen==1).first()
  58. 58. Dropping down to SQLsql = """select u.*from users uwhere u.user_id in ( select user_id from orders where isopen = 1)"""session.query(User).from_statement(sql)sql = """select u.*, o.*from users u join orders o on (u.user_id = o.user_id)where o.isopen = 1"""session.query(User, Order).from_statement(sql)
  59. 59. Dropping down a little lessq = session.query(User, Order)q = q.join(User.orders)q.filter("orders.isopen = 1").all()
  60. 60. ExerciseIn a single query, select the users and orderswhere the order description is like order%
  61. 61. One to oneclass Address(Base): __table__ = addressesclass User(Base): orders = relation(Order, order_by=[orders.c.order_id]) address = relation(Address)How does SQLA know to treat these differently?
  62. 62. Many to manyclass Keyword(Base): __table__ = keywordsclass Item(Base): __table__ = orderitems keywords = relation(Keyword, secondary=itemkeywords)
  63. 63. Relation queriesuser = session.query(User).get(7)q = session.query(Order)q.filter(Order.user==user).all()# q.filter(Order.user_id==user.user_id).all()
  64. 64. Relation queries 2q = session.query(Order)q.filter(Order.user.has(name=jack)).all()q.filter(Order.user.has((User.name==jack) | (User.user_id >= 9))).all()# q.filter(Order.user_id== select([users.c.user_id], users.c.name==jack)).all()# q.filter(Order.user_id== select([users.c.user_id], (users.c.name==jack) | (users.c.user_id >= 9))).all()
  65. 65. Relation queries 3q = session.query(User)q.filter(User.orders.any(Order.isopen > 0)).all()q.filter(User.orders.any( Order.description.like(order%))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.isopen > 0))).all()# q.filter(User.user_id.in_( select([orders.c.user_id], orders.c.description.like(order%)))).all()
  66. 66. Relation queries 4Just need a raw EXISTS clause? Use .any() or.has() without extra parameters.
  67. 67. Relation queries 5Keyword = session.query(Keyword).filter_by(name=red).one()q = session.query(Item)q.filter(Item.keywords.contains(keyword)).all()# q.filter(Item.item_id.in_( select([itemkeywords.c.item_id], itemkeywords.c.keyword_id ==keyword.keyword_id))).all()
  68. 68. ExerciseRetrieve all users that do not have any orders.
  69. 69. Breathe EasyThat was our last exercise!
  70. 70. Eager loadingclass User(Base): orders = relation(Order, order_by=[orders.c.order_id], lazy=False)# orq = session.query(User).options(eagerload(orders))# also lazyload, noload
  71. 71. Transactions are simple● session: autocommit=False – this is the default – commit() / rollback() manually● autocommit=True – each flush() also commits
  72. 72. Other transaction features● begin_nested● manual transaction management at the Connection level
  73. 73. __init____init__ is for object creation, not loadinguse @reconstructorclass Order(Base): __table__ = orders def __init__(self): self.foo = [] @reconstructor def loaded(self): self.foo = []
  74. 74. Questions
  75. 75. Related projectsMigrateFormAlchemySqlSoupElixirz3c.sqlalchemy
  76. 76. Migrate# one-time setupmigrate create path/to/upgradescripts "comment"migrate manage dbmanage.py--repository=path/to/upgradescripts –url=db-connection-url./dbmanage.py version_control# repeat as necessary:./dbmanage.py script_sql sqlite# edit script./dbmanage.py upgrade
  77. 77. FormAlchemyorder1 = session.query(Order).first()from formalchemy import FieldSetfs = FieldSet(order1)print fs.render()
  78. 78. FormAlchemy, cont.from formalchemy import Gridorders = session.query(Order).all()g = Grid(Order, orders)print g.render()
  79. 79. SqlSoup>>> from sqlalchemy.ext.sqlsoup import SqlSoup>>> db = SqlSoup(metadata)>>> db.users.filter(db.users.user_id < 10).all()[MappedUsers(user_id=7,name=jack),MappedUsers(user_id=8,name=ed),MappedUsers(user_id=9,name=fred)]>>> db.users.first()MappedUsers(user_id=7,name=jack)>>> _.ordersTraceback (most recent call last): File "<stdin>", line 1, in <module>AttributeError: MappedUsers object has no attributeorders
  80. 80. SqlSoup 2>>> db.users.relate(orders, db.orders)>>> db.users.first().orders[MappedOrders(...)]>>> db.users.filter(db.users.orders.any()).all()[MappedUsers(...)]
  81. 81. SqlSoup 3s = select([func.count(*)], users.c.user_id==orders.c.user_id, from_obj=[orders], scalar=True)s2 = select([users,s.label(order_count)]).alias(users_with_count)db.users_with_count = db.map(s2)
  82. 82. Elixir class Person(Entity): has_field(name, Unicode) acts_as_taggable() ... some_person_instance.add_tag(cool) ... cool_people = Person.get_by_tag(cool)http://cleverdevil.org/computing/52/
  83. 83. Resourcesirc://irc.freenode.net/#sqlalchemySQLA mailing listMigrate, FormAlchemy, Elixir
  84. 84. Final Questions

×