What makes Pony ORM different?
Fast object-relational mapper
which uses Python generators for
writing database queries
A Python generator
(p.name for p in product_list if p.price > 100)
A Python generator vs a SQL query
SELECT p.name
FROM Products p
WHERE p.price > 100
(p.name for p in product_list if p.pri...
(p.name for p in product_list if p.price > 100)
SELECT p.name
FROM Products p
WHERE p.price > 100
A Python generator vs a ...
(p.name for p in product_list if p.price > 100)
SELECT p.name
FROM Products p
WHERE p.price > 100
A Python generator vs a ...
(p.name for p in product_list if p.price > 100)
SELECT p.name
FROM Products p
WHERE p.price > 100
A Python generator vs a ...
The same query in Pony
SELECT p.name
FROM Products p
WHERE p.price > 100
select(p.name for p in Product if p.price > 100)
• Pony ORM
• Django
• SQL Alchemy
Query syntax comparison
Pony ORM:
select(p for p in Product
if p.name.startswith('A') and p.image is None
or p.added.year < 2014)
Query syntax com...
Django:
Product.objects.filter(
Q(name__startswith='A', image__isnull=True)
| Q(added__year__lt=2014))
Query syntax compar...
SQLAlchemy:
session.query(Product).filter(
(Product.name.startswith('A')
& (Product.image == None))
| (extract('year', Pro...
session.query(Product).filter(
(Product.name.startswith('A') & (Product.image == None))
| (extract('year', Product.added) ...
Query translation
select(p for p in Product
if p.name.startswith('A') and p.image is None
or p.added.year < 2014)
• Transl...
Building a query step by step
q = select(o for o in Order if o.customer.id == some_id)
q = q.filter(lambda o: o.state != '...
How Pony translates generator
expressions to SQL?
Python generator to SQL translation
1. Decompile bytecode and restore AST
2. Translate AST to ‘abstract SQL’
3. Translate ...
Python generator to SQL translation
1. Decompile bytecode and restore AST
2. Translate AST to ‘abstract SQL’
3. Translate ...
Bytecode decompilation
• Using the Visitor pattern
• Methods of the Visitor object correspond
the byte code commands
• Pon...
(a + b.c) in x.y
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE_OP in
Bytecode decompila...
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE_O...
Bytecode decompilation
(a + b.c) in x.y
> LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE...
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
> LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE...
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
> LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE_OP in
Stack
Getattr(Na...
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
> BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
COMPARE_OP in
Stack
Add(Name('...
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
> LOAD_FAST x
LOAD_ATTR y
COMPARE...
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
> LOAD_ATTR y
COMPARE...
Bytecode decompilation
(a + b.c) in x.y
LOAD_GLOBAL a
LOAD_FAST b
LOAD_ATTR c
BINARY_ADD
LOAD_FAST x
LOAD_ATTR y
> COMPARE...
Abstract Syntax Tree (AST)
a
in
+
.c
b
.y
x
(a + b.c) in x.y
Python generator to SQL translation
1. Decompile bytecode and restore AST
2. Translate AST to ‘abstract SQL’
3. Translate ...
What SQL it should be translated to?
a
in
+
.c
b
.y
x
(a + b.c) in x.y
It depends on variables types!
What SQL it should be translated to?
(a + b.c) in x.y
• If a and c are numbers, y is a collection
(? + "b"."c") IN (SELECT …)
• If a and c are strings, y is a collection
CONCAT...
• The result of translation depends on types
• If the translator analyzes node types by
itself, the logic becomes too comp...
• Encapsulates the node translation logic
• Generates the result of translation - ‘the
abstract SQL’
• Can combine itself ...
• StringAttrMonad
• StringParamMonad
• StringExprMonad
• StringConstMonad
• DatetimeAttrMonad
• DatetimeParamMonad
• Objec...
AST Translation
• Using the Visitor pattern
• Walk the tree in depth-first order
• Create monads when leaving each node
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
(a + b.c) in x.y
AST to SQL Translation
in
.y
x
+
a .c
b
StringParam
Monad
ObjectIter
Monad
StringAttr
Monad
StringExpr
Mo...
Abstract SQL
(a + b.c) in x.y
['LIKE', ['COLUMN', 't1', 'y'],
['CONCAT',
['VALUE', '%'], ['PARAM', 'p1'],
['COLUMN', 't2',...
Python generator to SQL translation
1. Decompile bytecode and restore AST
2. Translate AST to ‘abstract SQL’
3. Translate ...
Specific SQL dialects
['LIKE', ['COLUMN', 't1', 'y'],
['CONCAT',
['VALUE', '%'], ['PARAM', 'p1'],
['COLUMN', 't2', 'c'], [...
Other Pony ORM features
• Identity Map
• Automatic query optimization
• N+1 Query Problem solution
• Optimistic transactio...
Django ORM
s1 = Student.objects.get(pk=123)
print s1.name, s1.group.id
s2 = Student.objects.get(pk=456)
print s2.name, s2....
Django ORM
s1 = Student.objects.get(pk=123)
print s1.name, s1.group.id
s2 = Student.objects.get(pk=456)
print s2.name, s2....
Django ORM
s1 = Student.objects.get(pk=123)
print s1.name, s1.group.id
s2 = Student.objects.get(pk=456)
print s2.name, s2....
Django ORM
s1 = Student.objects.get(pk=123)
print s1.name, s1.group.id
s2 = Student.objects.get(pk=456)
print s2.name, s2....
Django ORM
s1 = Student.objects.get(pk=123)
print s1.name, s1.group.id
s2 = Student.objects.get(pk=456)
print s2.name, s2....
Pony ORM
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
Pony ORM – seeds, IdentityMap
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
St...
Pony ORM – seeds, IdentityMap
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
St...
Pony ORM – seeds, IdentityMap
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
St...
Pony ORM – seeds, IdentityMap
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
St...
Pony ORM – seeds, IdentityMap
s1 = Student[123]
print s1.name, s1.group.id
s2 = Student[456]
print s2.name, s2.group.id
St...
Solution for the N+1 Query Problem
orders = select(o for o in Order if o.total_price > 1000) 
.order_by(desc(Order.id)).pa...
Order 1
Order 3
Order 4
Order 7
Order 9
Customer 1
Customer 4
Customer 7
Solution for the N+1 Query Problem
Order 1
Order 3
Order 4
Order 7
Order 9
Customer 1
Customer 4
Customer 7
Solution for the N+1 Query Problem
One SQL query
Solution for the N+1 Query Problem
1
1
SELECT c.id, c.name, …
FROM “Customer” c
WHERE c.id IN (?, ?, ?)
orders = select(o ...
Automatic query optimization
select(c for c in Customer
if sum(c.orders.total_price) > 1000)
SELECT "c"."id", "c"."email",...
Transactions
def transfer_money(id1, id2, amount):
account1 = Account.objects.get(pk=id1)
if account1.amount < amount:
rai...
@transaction.atomic
def transfer_money(id1, id2, amount):
account1 = Account.objects.get(pk=id1)
if account1.amount < amou...
@transaction.atomic
def transfer_money(id1, id2, amount):
account1 = Account.objects 
.select_for_update.get(pk=id1)
if ac...
@db_session
def transfer_money(id1, id2, amount):
account1 = Account[id1]
if account1.amount < amount:
raise ValueError('N...
db_session
• Pony tracks which objects where changed
• No need to call save()
• Pony saves all updated objects in a single...
UPDATE Account
SET amount = :new_value
WHERE id = :id
AND amount = :old_value
Optimistic Locking
Optimistic Locking
• Pony tracks attributes which were read and
updated
• If object wasn’t locked using the for_update
met...
Entity-Relationship Diagram Editor
https://editor.ponyorm.com
Entity-Relationship Diagram Editor
https://editor.ponyorm.com
Entity-Relationship Diagram Editor
https://editor.ponyorm.com
Main Pony ORM features:
• Using generators for database queries
• Identity Map
• Solution for N+1 Query Problem
• Automati...
• Python 3
• Microsoft SQL Server support
• Improved documentation
• Migrations
• Ansync queries
Pony roadmap
• Site ponyorm.com
• Twitter @ponyorm
• Github github.com/ponyorm/pony
• ER-Diagram editor editor.ponyorm.com
• Installati...
How Pony ORM translates Python generators to SQL queries
Upcoming SlideShare
Loading in...5
×

How Pony ORM translates Python generators to SQL queries

6,106

Published on

Pony ORM is an Object-Relational Mapper implemented in Python. It uses an unusual approach for writing database queries using Python generators. Pony analyzes the abstract syntax tree of a generator and translates it to its SQL equivalent. The translation process consists of several non-trivial stages.
This talk was given at EuroPython 2014 and reveals the internal details of the translation process.

Published in: Software

How Pony ORM translates Python generators to SQL queries

  1. 1. Object-Relational Mapper Alexey Malashkevich @ponyorm
  2. 2. What makes Pony ORM different? Fast object-relational mapper which uses Python generators for writing database queries
  3. 3. A Python generator (p.name for p in product_list if p.price > 100)
  4. 4. A Python generator vs a SQL query SELECT p.name FROM Products p WHERE p.price > 100 (p.name for p in product_list if p.price > 100)
  5. 5. (p.name for p in product_list if p.price > 100) SELECT p.name FROM Products p WHERE p.price > 100 A Python generator vs a SQL query
  6. 6. (p.name for p in product_list if p.price > 100) SELECT p.name FROM Products p WHERE p.price > 100 A Python generator vs a SQL query
  7. 7. (p.name for p in product_list if p.price > 100) SELECT p.name FROM Products p WHERE p.price > 100 A Python generator vs a SQL query
  8. 8. The same query in Pony SELECT p.name FROM Products p WHERE p.price > 100 select(p.name for p in Product if p.price > 100)
  9. 9. • Pony ORM • Django • SQL Alchemy Query syntax comparison
  10. 10. Pony ORM: select(p for p in Product if p.name.startswith('A') and p.image is None or p.added.year < 2014) Query syntax comparison
  11. 11. Django: Product.objects.filter( Q(name__startswith='A', image__isnull=True) | Q(added__year__lt=2014)) Query syntax comparison
  12. 12. SQLAlchemy: session.query(Product).filter( (Product.name.startswith('A') & (Product.image == None)) | (extract('year', Product.added) < 2014)) Query syntax comparison
  13. 13. session.query(Product).filter( (Product.name.startswith('A') & (Product.image == None)) | (extract('year', Product.added) < 2014)) Query syntax comparison Product.objects.filter( Q(name__startswith='A', image__isnull=True) | Q(added__year__lt=2014)) select(p for p in Product if p.name.startswith('A') and p.image is None or p.added.year < 2014) Pony Django SQLAlchemy
  14. 14. Query translation select(p for p in Product if p.name.startswith('A') and p.image is None or p.added.year < 2014) • Translation from the bytecode is fast • The bytecode translation result is cached • The Python generator object is used as a cache key Python generator object
  15. 15. Building a query step by step q = select(o for o in Order if o.customer.id == some_id) q = q.filter(lambda o: o.state != 'DELIVERED') q = q.filter(lambda o: len(o.items) > 2) q = q.order_by(Order.date_created) q = q[10:20] SELECT "o"."id" FROM "Order" "o" LEFT JOIN "OrderItem" "orderitem-1" ON "o"."id" = "orderitem-1"."order" WHERE "o"."customer" = ? AND "o"."state" <> 'DELIVERED' GROUP BY "o"."id" HAVING COUNT("orderitem-1"."ROWID") > 2 ORDER BY "o"."date_created" LIMIT 10 OFFSET 10
  16. 16. How Pony translates generator expressions to SQL?
  17. 17. Python generator to SQL translation 1. Decompile bytecode and restore AST 2. Translate AST to ‘abstract SQL’ 3. Translate ‘abstract SQL’ to a specific SQL dialect
  18. 18. Python generator to SQL translation 1. Decompile bytecode and restore AST 2. Translate AST to ‘abstract SQL’ 3. Translate ‘abstract SQL’ to a concrete SQL dialect
  19. 19. Bytecode decompilation • Using the Visitor pattern • Methods of the Visitor object correspond the byte code commands • Pony keeps fragments of AST at the stack • Each method either adds a new part of AST or combines existing parts
  20. 20. (a + b.c) in x.y Bytecode decompilation
  21. 21. (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Bytecode decompilation
  22. 22. Bytecode decompilation (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack
  23. 23. Bytecode decompilation (a + b.c) in x.y > LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack Name('a')
  24. 24. Bytecode decompilation (a + b.c) in x.y LOAD_GLOBAL a > LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack Name('b') Name('a')
  25. 25. (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b > LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack Getattr(Name('b'), 'c') Name('a') Bytecode decompilation
  26. 26. (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c > BINARY_ADD LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack Add(Name('a'), Getattr(Name('b'), 'c')) Bytecode decompilation
  27. 27. Bytecode decompilation (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD > LOAD_FAST x LOAD_ATTR y COMPARE_OP in Stack Name('x') Add(Name('a'), Getattr(Name('b'), 'c'))
  28. 28. Bytecode decompilation (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x > LOAD_ATTR y COMPARE_OP in Stack Getattr(Name('x'), 'y') Add(Name('a'), Getattr(Name('b'), 'c'))
  29. 29. Bytecode decompilation (a + b.c) in x.y LOAD_GLOBAL a LOAD_FAST b LOAD_ATTR c BINARY_ADD LOAD_FAST x LOAD_ATTR y > COMPARE_OP in Stack Compare('in', Add(…), Getattr(…))
  30. 30. Abstract Syntax Tree (AST) a in + .c b .y x (a + b.c) in x.y
  31. 31. Python generator to SQL translation 1. Decompile bytecode and restore AST 2. Translate AST to ‘abstract SQL’ 3. Translate ‘abstract SQL’ to a concrete SQL dialect
  32. 32. What SQL it should be translated to? a in + .c b .y x (a + b.c) in x.y
  33. 33. It depends on variables types! What SQL it should be translated to? (a + b.c) in x.y
  34. 34. • If a and c are numbers, y is a collection (? + "b"."c") IN (SELECT …) • If a and c are strings, y is a collection CONCAT(?, "b"."c") IN (SELECT …) • If a, c and y are strings “x"."y" LIKE CONCAT('%', ?, "b"."c", '%') What SQL it should be translated to? (a + b.c) in x.y
  35. 35. • The result of translation depends on types • If the translator analyzes node types by itself, the logic becomes too complex • Pony uses Monads to keep it simple (a + b.c) in x.y AST to SQL Translation
  36. 36. • Encapsulates the node translation logic • Generates the result of translation - ‘the abstract SQL’ • Can combine itself with other monads The translator delegates the logic of translation to monads A Monad
  37. 37. • StringAttrMonad • StringParamMonad • StringExprMonad • StringConstMonad • DatetimeAttrMonad • DatetimeParamMonad • ObjectAttrMonad • CmpMonad • etc… Each monad defines a set of allowed operations and can translate itself into a part of resulting SQL query Monad types
  38. 38. AST Translation • Using the Visitor pattern • Walk the tree in depth-first order • Create monads when leaving each node
  39. 39. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b
  40. 40. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b
  41. 41. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad
  42. 42. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad
  43. 43. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad
  44. 44. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad
  45. 45. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad
  46. 46. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad
  47. 47. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad
  48. 48. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad
  49. 49. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad
  50. 50. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad
  51. 51. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad ObjectIter Monad
  52. 52. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad ObjectIter Monad
  53. 53. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad ObjectIter Monad StringAttr Monad
  54. 54. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad ObjectIter Monad StringAttr Monad
  55. 55. (a + b.c) in x.y AST to SQL Translation in .y x + a .c b StringParam Monad ObjectIter Monad StringAttr Monad StringExpr Monad ObjectIter Monad StringAttr Monad Cmp Monad
  56. 56. Abstract SQL (a + b.c) in x.y ['LIKE', ['COLUMN', 't1', 'y'], ['CONCAT', ['VALUE', '%'], ['PARAM', 'p1'], ['COLUMN', 't2', 'c'], ['VALUE', '%'] ] ] Allows to put aside the SQL dialect differences
  57. 57. Python generator to SQL translation 1. Decompile bytecode and restore AST 2. Translate AST to ‘abstract SQL’ 3. Translate ‘abstract SQL’ to a specific SQL dialect
  58. 58. Specific SQL dialects ['LIKE', ['COLUMN', 't1', 'y'], ['CONCAT', ['VALUE', '%'], ['PARAM', 'p1'], ['COLUMN', 't2', 'c'], ['VALUE', '%'] ] ] MySQL: `t1`.`y` LIKE CONCAT('%', ?, `t2`.`c`, '%') SQLite: "t1"."y" LIKE '%' || ? || "t2"."c" || '%'
  59. 59. Other Pony ORM features • Identity Map • Automatic query optimization • N+1 Query Problem solution • Optimistic transactions • Online ER Diagram Editor
  60. 60. Django ORM s1 = Student.objects.get(pk=123) print s1.name, s1.group.id s2 = Student.objects.get(pk=456) print s2.name, s2.group.id • How many SQL queries will be executed? • How many objects will be created?
  61. 61. Django ORM s1 = Student.objects.get(pk=123) print s1.name, s1.group.id s2 = Student.objects.get(pk=456) print s2.name, s2.group.id Student 123
  62. 62. Django ORM s1 = Student.objects.get(pk=123) print s1.name, s1.group.id s2 = Student.objects.get(pk=456) print s2.name, s2.group.id Student 123 Group 1
  63. 63. Django ORM s1 = Student.objects.get(pk=123) print s1.name, s1.group.id s2 = Student.objects.get(pk=456) print s2.name, s2.group.id Student 123 Student 456 Group 1
  64. 64. Django ORM s1 = Student.objects.get(pk=123) print s1.name, s1.group.id s2 = Student.objects.get(pk=456) print s2.name, s2.group.id Student 123 Student 456 Group 1 Group 1
  65. 65. Pony ORM s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id
  66. 66. Pony ORM – seeds, IdentityMap s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id Student 123 Group 1
  67. 67. Pony ORM – seeds, IdentityMap s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id Student 123 Group 1 seed
  68. 68. Pony ORM – seeds, IdentityMap s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id Student 123 Group 1 seed
  69. 69. Pony ORM – seeds, IdentityMap s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id Student 123 Student 456 Group 1 seed
  70. 70. Pony ORM – seeds, IdentityMap s1 = Student[123] print s1.name, s1.group.id s2 = Student[456] print s2.name, s2.group.id Student 123 Student 456 Group 1 seed
  71. 71. Solution for the N+1 Query Problem orders = select(o for o in Order if o.total_price > 1000) .order_by(desc(Order.id)).page(1, pagesize=5) for o in orders: print o.total_price, o.customer.name 1 SELECT o.id, o.total_price, o.customer_id,... FROM "Order" o WHERE o.total_price > 1000 ORDER BY o.id DESC LIMIT 5
  72. 72. Order 1 Order 3 Order 4 Order 7 Order 9 Customer 1 Customer 4 Customer 7 Solution for the N+1 Query Problem
  73. 73. Order 1 Order 3 Order 4 Order 7 Order 9 Customer 1 Customer 4 Customer 7 Solution for the N+1 Query Problem One SQL query
  74. 74. Solution for the N+1 Query Problem 1 1 SELECT c.id, c.name, … FROM “Customer” c WHERE c.id IN (?, ?, ?) orders = select(o for o in Order if o.total_price > 1000) .order_by(desc(Order.id)).page(1, pagesize=5) for o in orders: print o.total_price, o.customer.name SELECT o.id, o.total_price, o.customer_id,... FROM "Order" o WHERE o.total_price > 1000 ORDER BY o.id DESC LIMIT 5
  75. 75. Automatic query optimization select(c for c in Customer if sum(c.orders.total_price) > 1000) SELECT "c"."id", "c"."email", "c"."password", "c"."name", "c"."country", "c"."address" FROM "Customer" "c" WHERE ( SELECT coalesce(SUM("order-1"."total_price"), 0) FROM "Order" "order-1" WHERE "c"."id" = "order-1"."customer" ) > 1000 SELECT "c"."id" FROM "Customer" "c" LEFT JOIN "Order" "order-1" ON "c"."id" = "order-1"."customer" GROUP BY "c"."id" HAVING coalesce(SUM("order-1"."total_price"), 0) > 1000
  76. 76. Transactions def transfer_money(id1, id2, amount): account1 = Account.objects.get(pk=id1) if account1.amount < amount: raise ValueError('Not enough funds!') account2 = Account.object.get(pk=id2) account1.amount -= amount account1.save() account2.amount += amount account2.save() Django ORM
  77. 77. @transaction.atomic def transfer_money(id1, id2, amount): account1 = Account.objects.get(pk=id1) if account1.amount < amount: raise ValueError('Not enough funds!') account2 = Account.object.get(pk=id2) account1.amount -= amount account1.save() account2.amount += amount account2.save() Transactions Django ORM
  78. 78. @transaction.atomic def transfer_money(id1, id2, amount): account1 = Account.objects .select_for_update.get(pk=id1) if account1.amount < amount: raise ValueError('Not enough funds!') account2 = Account.objects .select_for_update.get(pk=id2) account1.amount -= amount account1.save() account2.amount += amount account2.save() Transactions Django ORM
  79. 79. @db_session def transfer_money(id1, id2, amount): account1 = Account[id1] if account1.amount < amount: raise ValueError('Not enough funds!') account1.amount -= amount Account[id2].amount += amount Transactions Pony ORM
  80. 80. db_session • Pony tracks which objects where changed • No need to call save() • Pony saves all updated objects in a single transaction automatically on leaving the db_session scope Transactions
  81. 81. UPDATE Account SET amount = :new_value WHERE id = :id AND amount = :old_value Optimistic Locking
  82. 82. Optimistic Locking • Pony tracks attributes which were read and updated • If object wasn’t locked using the for_update method, Pony uses the optimistic locking automatically
  83. 83. Entity-Relationship Diagram Editor https://editor.ponyorm.com
  84. 84. Entity-Relationship Diagram Editor https://editor.ponyorm.com
  85. 85. Entity-Relationship Diagram Editor https://editor.ponyorm.com
  86. 86. Main Pony ORM features: • Using generators for database queries • Identity Map • Solution for N+1 Query Problem • Automatic query optimization • Optimistic transactions • Online ER-diagram editor Wrapping up
  87. 87. • Python 3 • Microsoft SQL Server support • Improved documentation • Migrations • Ansync queries Pony roadmap
  88. 88. • Site ponyorm.com • Twitter @ponyorm • Github github.com/ponyorm/pony • ER-Diagram editor editor.ponyorm.com • Installation: pip install pony Thank you! Pony ORM

×