Proprietary and Confidential
Django ORM
Hi. I’m Ana.
Ana Trakhtman
Senior Full Stack
ana@tapingo.com
(972) 526-204-634
Proprietary and Confidential
What we will talk about
1) What is ORM?
2) Django Models
3) Querying in Django
4) Advanced querying in Django
5) Working with real production data and keeping it safe
So lets get started.
Proprietary and Confidential
What is ORM?
1. code library that automates the transfer of data stored in relational
databases tables into objects that are more commonly used in
application code
high-level abstraction upon a relational database
write Python code instead of SQL to create, read, update and delete
data and schemas in database
Object-relational mapper
How ORM looks like in Django
Django Models
1. A model is the single, definitive source of information about your data
2. maps to a single database table
3. Python class that subclasses django.db.models.Model.
4. Each attribute of the model represents a database field
5. Django gives you an automatically-generated database-access API
Let’s take a look at how a real model looks like
Querying in Django
Querying in Django
1. a model class represents a database table
2. an instance of that class represents a particular record in the
database table
3. QuerySet represents one or more record based on the lookup
Get single instance
Behind the scenes
• SELECT `orders_order`.`id`, `orders_order`.`is_mocked`, `orders_order`.`short_id`, `orders_order`.`owner_id`,
`orders_order`.`tender_id`, `orders_order`.`referral_order_id`, `orders_order`.`shop_id`, `orders_order`.`order_type_id`,
`orders_order`.`paytool_id`, `orders_order`.`creation_time`, AsText(`orders_order`.`location_owner`),
`orders_order`.`is_tip_available`, `orders_order`.`suggested_tip_amount`, `orders_order`.`suggested_tip_percent`,
`orders_order`.`suggested_tip_type`, `orders_order`.`has_suggestion`, `orders_order`.`clicked_suggestion`,
`orders_order`.`suggestion_number`, `orders_order`.`tender_style`, `orders_order`.`is_first_for_user_global`,
`orders_order`.`is_first_for_user_shop`, `orders_order`.`tapingo_credit`, `orders_order`.`merchant_credit`,
`orders_order`.`balance_before`, `orders_order`.`suggestion_id`, `orders_order`.`is_scheduled`,
`orders_order`.`estimated_print_time`, `orders_order`.`merchant_target_time`, `orders_order`.`organization_id`,
`orders_order`.`is_on_campus`, `orders_order`.`device_id`, `orders_order`.`owner_organization_id`,
`orders_order`.`is_shop_in_test_mode`, `orders_order`.`cancel_date`, `orders_order`.`is_valid_for_reporting`,
`orders_order`.`is_valid_for_billing`, `orders_order`.`not_valid_for_billing_reason`, `orders_order`.`is_valid_for_execution`,
`orders_order`.`is_selling_order`, `orders_order`.`user_response_time`, `orders_order`.`user_reported_ta`,
`orders_order`.`no_tax_on_service_fee`, `orders_order`.`is_picked_up_by_user`, `orders_order`.`reco_on_menu`,
`orders_order`.`current_equivalence`, `orders_order`.`is_odd` FROM `orders_order` WHERE `orders_order`.`id` = 1234
Get multiple instances
Actually, I want only completed orders and only from a certain
shop
What happened behind scenes
• SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN
`entities_profileentity` ON ( `orders_order`.`owner_id` =
`entities_profileentity`.`entity_ptr_id` ) WHERE `entities_profileentity`.`email` =
‘ana@tapingo.com'
• SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN
`entities_profileentity` ON ( `orders_order`.`owner_id` =
`entities_profileentity`.`entity_ptr_id` ) WHERE (`entities_profileentity`.`email` =
'ana@tapingo.com' AND `orders_order`.`state` = 'COMPLETED' AND
`orders_order`.`shop_id` = 14474)
What if I want NOT?
SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN
`entities_profileentity` ON ( `orders_order`.`owner_id` =
`entities_profileentity`.`entity_ptr_id` ) WHERE (`entities_profileentity`.`email` =
'ana@tapingo.com' AND NOT (`orders_order`.`state` = 'COMPLETED' AND
`orders_order`.`shop_id` = 14474))
How about OR?
Behind the scenes
• SELECT `orders_order`.`id`, …, T5.`id`, T5.`name`, T5.`profile_photo_id`, T5.`last_update`,
T5.`create_time`, `entities_profileentity`.`entity_ptr_id`, `entities_profileentity`.`user_id`,
`entities_profileentity`.`last_name`, `entities_profileentity`.`email`,
`entities_profileentity`.`birthdate`, `entities_profileentity`.`phone_number`,
`entities_profileentity`.`business_day`, `entities_profileentity`.`trial_notified`,
`entities_profileentity`.`is_waiter` FROM `orders_order` INNER JOIN `entities_profileentity` ON (
`orders_order`.`owner_id` = `entities_profileentity`.`entity_ptr_id` ) INNER JOIN
`entities_shopentity` ON ( `orders_order`.`shop_id` = `entities_shopentity`.`entity_ptr_id` ) INNER
JOIN `entities_entity` ON ( `entities_shopentity`.`entity_ptr_id` = `entities_entity`.`id` ) INNER JOIN
`entities_entity` T5 ON ( `entities_profileentity`.`entity_ptr_id` = T5.`id` ) WHERE
((`entities_profileentity`.`email` = 'ana@tapingo.com' OR `entities_profileentity`.`email` =
'nadav@tapingo.com') AND `entities_entity`.`name` LIKE '%gables%' AND `orders_order`.`state` =
'COMPLETED') ORDER BY `orders_order`.`execution_time` DESC LIMIT 1
What else is possible?
1. values/values_list - retrieve a sub set of fields
2. distinct - get only distinct instances based on specific field
3. reverse - reverse the order in which a queryset’s elements are returned
4. dates - Returns a QuerySet that evaluates to a list of datetime.date objects representing
all available dates of a particular kind within the contents of the QuerySet
Advanced Querying in Django
Annotating
Behind the scenes
SELECT `entities_entity`.`id`, `entities_entity`.`name`, `entities_shopentity`.`what_now_html_takeaway_id`,
`entities_shopentity`.`meal_hack`, `entities_shopentity`.`delivery_hack`, `entities_shopentity`.`dats`,
`entities_shopentity`.`mark_new_until`, `entities_shopentity`.`mark_new_service_type`,
`entities_shopentity`.`launch_date`, `entities_shopentity`.`target_orders`, `entities_shopentity`.`odd_pos_id`,
`entities_shopentity`.`for_waiters`, `entities_shopentity`.`is_partner`,
`entities_shopentity`.`is_supports_delivery`, `entities_shopentity`.`responsibilities`, COUNT(`orders_order`.`id`)
AS `num_orders` FROM `entities_shopentity` LEFT OUTER JOIN `orders_order` ON (
`entities_shopentity`.`entity_ptr_id` = `orders_order`.`shop_id` ) INNER JOIN `entities_entity` ON (
`entities_shopentity`.`entity_ptr_id` = `entities_entity`.`id` ) WHERE NOT (`entities_shopentity`.`is_deleted` = 1)
GROUP BY `entities_shopentity`.`entity_ptr_id` ORDER BY `num_orders` ASC LIMIT 1
Aggregation
Behind the scenes
SELECT AVG(`num_shops`) FROM (SELECT `billi_organization`.`id` AS Col1,
COUNT(`entities_shopentity`.`entity_ptr_id`) AS `num_shops` FROM `billi_organization` LEFT OUTER JOIN
`entities_shopentity` ON ( `billi_organization`.`id` = `entities_shopentity`.`organization_id` ) GROUP BY
`billi_organization`.`id` ORDER BY NULL) subquery
SELECT MAX(`num_shops`) FROM (SELECT `billi_organization`.`id` AS Col1,
COUNT(`entities_shopentity`.`entity_ptr_id`) AS `num_shops` FROM `billi_organization` LEFT OUTER JOIN
`entities_shopentity` ON ( `billi_organization`.`id` = `entities_shopentity`.`organization_id` ) GROUP BY
`billi_organization`.`id` ORDER BY NULL) subquery
Running Raw SQL
1. Run your exact SQL
2. Maps to the model in use
Working with production data
Gains
1. See real data
2. See data in near real-time
3. Be pro-active and relevant
Risks
1. Data analytics = heavy queries
2. Locking time sensitive tables
3. Overloading the DB, impacting overall system performance
Solution - use read replica
Question?

tapingo-django-orm

  • 1.
  • 2.
  • 3.
    Ana Trakhtman Senior FullStack ana@tapingo.com (972) 526-204-634 Proprietary and Confidential
  • 4.
    What we willtalk about 1) What is ORM? 2) Django Models 3) Querying in Django 4) Advanced querying in Django 5) Working with real production data and keeping it safe
  • 5.
    So lets getstarted. Proprietary and Confidential
  • 6.
  • 7.
    1. code librarythat automates the transfer of data stored in relational databases tables into objects that are more commonly used in application code high-level abstraction upon a relational database write Python code instead of SQL to create, read, update and delete data and schemas in database Object-relational mapper
  • 8.
    How ORM lookslike in Django
  • 9.
    Django Models 1. Amodel is the single, definitive source of information about your data 2. maps to a single database table 3. Python class that subclasses django.db.models.Model. 4. Each attribute of the model represents a database field 5. Django gives you an automatically-generated database-access API
  • 10.
    Let’s take alook at how a real model looks like
  • 11.
  • 12.
    Querying in Django 1.a model class represents a database table 2. an instance of that class represents a particular record in the database table 3. QuerySet represents one or more record based on the lookup
  • 13.
  • 14.
    Behind the scenes •SELECT `orders_order`.`id`, `orders_order`.`is_mocked`, `orders_order`.`short_id`, `orders_order`.`owner_id`, `orders_order`.`tender_id`, `orders_order`.`referral_order_id`, `orders_order`.`shop_id`, `orders_order`.`order_type_id`, `orders_order`.`paytool_id`, `orders_order`.`creation_time`, AsText(`orders_order`.`location_owner`), `orders_order`.`is_tip_available`, `orders_order`.`suggested_tip_amount`, `orders_order`.`suggested_tip_percent`, `orders_order`.`suggested_tip_type`, `orders_order`.`has_suggestion`, `orders_order`.`clicked_suggestion`, `orders_order`.`suggestion_number`, `orders_order`.`tender_style`, `orders_order`.`is_first_for_user_global`, `orders_order`.`is_first_for_user_shop`, `orders_order`.`tapingo_credit`, `orders_order`.`merchant_credit`, `orders_order`.`balance_before`, `orders_order`.`suggestion_id`, `orders_order`.`is_scheduled`, `orders_order`.`estimated_print_time`, `orders_order`.`merchant_target_time`, `orders_order`.`organization_id`, `orders_order`.`is_on_campus`, `orders_order`.`device_id`, `orders_order`.`owner_organization_id`, `orders_order`.`is_shop_in_test_mode`, `orders_order`.`cancel_date`, `orders_order`.`is_valid_for_reporting`, `orders_order`.`is_valid_for_billing`, `orders_order`.`not_valid_for_billing_reason`, `orders_order`.`is_valid_for_execution`, `orders_order`.`is_selling_order`, `orders_order`.`user_response_time`, `orders_order`.`user_reported_ta`, `orders_order`.`no_tax_on_service_fee`, `orders_order`.`is_picked_up_by_user`, `orders_order`.`reco_on_menu`, `orders_order`.`current_equivalence`, `orders_order`.`is_odd` FROM `orders_order` WHERE `orders_order`.`id` = 1234
  • 15.
    Get multiple instances Actually,I want only completed orders and only from a certain shop
  • 16.
    What happened behindscenes • SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN `entities_profileentity` ON ( `orders_order`.`owner_id` = `entities_profileentity`.`entity_ptr_id` ) WHERE `entities_profileentity`.`email` = ‘ana@tapingo.com' • SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN `entities_profileentity` ON ( `orders_order`.`owner_id` = `entities_profileentity`.`entity_ptr_id` ) WHERE (`entities_profileentity`.`email` = 'ana@tapingo.com' AND `orders_order`.`state` = 'COMPLETED' AND `orders_order`.`shop_id` = 14474)
  • 17.
    What if Iwant NOT? SELECT COUNT('*') AS `__count` FROM `orders_order` INNER JOIN `entities_profileentity` ON ( `orders_order`.`owner_id` = `entities_profileentity`.`entity_ptr_id` ) WHERE (`entities_profileentity`.`email` = 'ana@tapingo.com' AND NOT (`orders_order`.`state` = 'COMPLETED' AND `orders_order`.`shop_id` = 14474))
  • 18.
  • 19.
    Behind the scenes •SELECT `orders_order`.`id`, …, T5.`id`, T5.`name`, T5.`profile_photo_id`, T5.`last_update`, T5.`create_time`, `entities_profileentity`.`entity_ptr_id`, `entities_profileentity`.`user_id`, `entities_profileentity`.`last_name`, `entities_profileentity`.`email`, `entities_profileentity`.`birthdate`, `entities_profileentity`.`phone_number`, `entities_profileentity`.`business_day`, `entities_profileentity`.`trial_notified`, `entities_profileentity`.`is_waiter` FROM `orders_order` INNER JOIN `entities_profileentity` ON ( `orders_order`.`owner_id` = `entities_profileentity`.`entity_ptr_id` ) INNER JOIN `entities_shopentity` ON ( `orders_order`.`shop_id` = `entities_shopentity`.`entity_ptr_id` ) INNER JOIN `entities_entity` ON ( `entities_shopentity`.`entity_ptr_id` = `entities_entity`.`id` ) INNER JOIN `entities_entity` T5 ON ( `entities_profileentity`.`entity_ptr_id` = T5.`id` ) WHERE ((`entities_profileentity`.`email` = 'ana@tapingo.com' OR `entities_profileentity`.`email` = 'nadav@tapingo.com') AND `entities_entity`.`name` LIKE '%gables%' AND `orders_order`.`state` = 'COMPLETED') ORDER BY `orders_order`.`execution_time` DESC LIMIT 1
  • 20.
    What else ispossible? 1. values/values_list - retrieve a sub set of fields 2. distinct - get only distinct instances based on specific field 3. reverse - reverse the order in which a queryset’s elements are returned 4. dates - Returns a QuerySet that evaluates to a list of datetime.date objects representing all available dates of a particular kind within the contents of the QuerySet
  • 21.
  • 22.
  • 23.
    Behind the scenes SELECT`entities_entity`.`id`, `entities_entity`.`name`, `entities_shopentity`.`what_now_html_takeaway_id`, `entities_shopentity`.`meal_hack`, `entities_shopentity`.`delivery_hack`, `entities_shopentity`.`dats`, `entities_shopentity`.`mark_new_until`, `entities_shopentity`.`mark_new_service_type`, `entities_shopentity`.`launch_date`, `entities_shopentity`.`target_orders`, `entities_shopentity`.`odd_pos_id`, `entities_shopentity`.`for_waiters`, `entities_shopentity`.`is_partner`, `entities_shopentity`.`is_supports_delivery`, `entities_shopentity`.`responsibilities`, COUNT(`orders_order`.`id`) AS `num_orders` FROM `entities_shopentity` LEFT OUTER JOIN `orders_order` ON ( `entities_shopentity`.`entity_ptr_id` = `orders_order`.`shop_id` ) INNER JOIN `entities_entity` ON ( `entities_shopentity`.`entity_ptr_id` = `entities_entity`.`id` ) WHERE NOT (`entities_shopentity`.`is_deleted` = 1) GROUP BY `entities_shopentity`.`entity_ptr_id` ORDER BY `num_orders` ASC LIMIT 1
  • 24.
  • 25.
    Behind the scenes SELECTAVG(`num_shops`) FROM (SELECT `billi_organization`.`id` AS Col1, COUNT(`entities_shopentity`.`entity_ptr_id`) AS `num_shops` FROM `billi_organization` LEFT OUTER JOIN `entities_shopentity` ON ( `billi_organization`.`id` = `entities_shopentity`.`organization_id` ) GROUP BY `billi_organization`.`id` ORDER BY NULL) subquery SELECT MAX(`num_shops`) FROM (SELECT `billi_organization`.`id` AS Col1, COUNT(`entities_shopentity`.`entity_ptr_id`) AS `num_shops` FROM `billi_organization` LEFT OUTER JOIN `entities_shopentity` ON ( `billi_organization`.`id` = `entities_shopentity`.`organization_id` ) GROUP BY `billi_organization`.`id` ORDER BY NULL) subquery
  • 26.
    Running Raw SQL 1.Run your exact SQL 2. Maps to the model in use
  • 27.
  • 28.
    Gains 1. See realdata 2. See data in near real-time 3. Be pro-active and relevant
  • 29.
    Risks 1. Data analytics= heavy queries 2. Locking time sensitive tables 3. Overloading the DB, impacting overall system performance
  • 30.
    Solution - useread replica
  • 31.

Editor's Notes

  • #3 Introduce yourself and talk about why you are so excited to be part of the Tapingo team – a team that’s solving one of the most fundamental problems today: the way we buy things. Go around the room and talk about each person’s hopes and issues.
  • #7 Present this as a “we all know” not as “we’re telling you something you don’t know.” People should be nodding in agreement here.
  • #8 Like in any field, competition is a stone’s throw away. Or closer.
  • #9 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #10 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #11 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #12 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #13 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #14 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #15 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #16 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #17 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #18 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #19 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #20 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #21 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #22 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #23 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #24 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #25 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #26 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #27 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #28 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #29 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #30 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #31 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.
  • #32 Give examples of each. Spotify – no need to buy full albums, listen to exactly what you want. Uber – convenience is power. Amazon – no need to go to the store. UPS – even old-school companies leverage technology. You can track your package like never before.