Inside the Query Optimizer Webinar


Published on

The Salesforce Platform allows developers to build enterprise applications using Visualforce, Apex and SOQL. To ensure that your applications perform and scale as your business grows, you'll want to write efficient and selective queries. The query optimizer uses several algorithms to determine the best SQL to generate from your SOQL. Some factors involved in this process include multitenancy, metadata and indexes.

Watch this webinar to:

Get an overview of multitenancy and metadata
Understand how to write selective and scalable SOQL queries
Learn how the query optimizer converts SOQL to SQL
See examples of the performance impact of indexes
Find out how skinny tables work

Published in: Technology, Business
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Inside the Query Optimizer Webinar

  1. 1. Inside the Query OptimizerFrom’s Customer Centric Engineering – Technical Enablement team
  2. 2. Join the conversation: #forcewebinarSafe harborSafe harbor statement under the Private Securities Litigation Reform Act of 1995:This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertaintiesmaterialize or if any of the assumptions proves incorrect, the results of, inc. could differ materially from the resultsexpressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could bedeemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or otherfinancial items and any statements regarding strategies or plans of management for future operations, statements of belief, anystatements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering newfunctionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in ouroperating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome ofintellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which weoperate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, newreleases of our service and successful customer deployment, our limited history reselling products, and utilizationand selling to larger enterprise customers. Further information on potential factors that could affect the financial results of,inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and otherscontaining important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currentlyavailable and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions basedupon features that are currently available., inc. assumes no obligation and does not intend to update these forward-looking statements.
  3. 3. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarJohn TanArchitect Evangelist@johntansfdcJaikumar BathijaArchitect – DB Performance@Speakers
  4. 4. Join the conversation: #forcewebinarFollow Developer Force for the latest news@forcedotcom / #forcewebinarDeveloper Force groupDeveloper Force – Community+Developer Force – CommunityDeveloper Force
  5. 5. Join the conversation: #forcewebinarArchitect Core Resource page•  Featured content for architects•  Articles, papers, blog posts, events•  Follow us on TwitterUpdated weekly!
  6. 6. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarHave questions?§  We have an expert support team at the ready to answer your questionsduring the webinar.§  Ask your questions via the GoToWebinar Questions Pane.§  The speaker(s) will choose top questions to answer live at the end of thewebinar.§  Please post your questions as we go along!§  Only post your question once; we’ll get to it as we go down the list.
  7. 7. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarToday s Learning GoalAWARENESS
  8. 8. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarWhy are we here?SELECT IdFROM AccountWHERE Status__c != ‘Closed’ ANDRating = Null ANDCreatedDate > 2013-04-01Empower developers to write selective queries. Don’t worry we have lots of examples.
  9. 9. Join the conversation: #forcewebinarSelective FiltersReduces thenumber ofrecords in yourresult set.Leveragesindexes.Avoids full tablescans.
  10. 10. Join the conversation: #forcewebinarQuery Performance ImpactUser Experience(Visualforcepages, API,Reports,Listviews, etc).Governor Limits(Timeouts,ConcurrentRequest Limit,Concurrent APIlimit, etc).Large DataVolumes (LDV).
  11. 11. Join the conversation: #forcewebinarJoin the conversation: #forcewebinar
  12. 12. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarAgenda•  Design -•  Query Optimizer•  SOQL Examples•  Skinny Tables•  Other Performance Factors
  13. 13. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarQuery Optimizer
  14. 14. Join the conversation: #forcewebinarQuery execution
  15. 15. Join the conversation: #forcewebinarMultitenancy
  16. 16. Join the conversation: #forcewebinarBasic Algorithm•  Pre-Query engine.•  Chooses the most selective filter from the WHERE clause.•  Determine the best leading table/index to drive the query.
  17. 17. Join the conversation: #forcewebinarIndexing•  Standard index - is available out of the box and we have a whole bunch offields that are indexed on Standard and custom entities.•  Custom index – is created on-demand, based on performance analysis donepro-actively by salesforce team.•  What other fields are indexed – External Id fields, fields marked unique,foreign keys by way of lookup or master detail relationship.
  18. 18. Join the conversation: #forcewebinarStatistics•  Pre-computed Statistics§  Row count§  User visibility§  Custom index§  Owner row count
  19. 19. Join the conversation: #forcewebinarOptions considered by the Optimizer
  20. 20. Join the conversation: #forcewebinarThe numbers game•  Standard index will be considered only if the filter fetches < 30% of therecords for the first million records and less than 15% of the records after thefirst million records, up to 1M records. * The selectivity threshold is subjectto change.•  Custom index will be considered only if the filter fetches < 10% of the recordsfor the first million records and less than 5% of the records after the firstmillion records, up to 333,333 records. * The selectivity threshold issubject to change.
  21. 21. Join the conversation: #forcewebinarThe numbers game – Standard Index# of records First Threshold Second Threshold Final ThresholdUp to 1 million 30% of total N/A 30% of totalUp to 2 million 300,000 150,000 450,000Up to 3 million 300,000 300,000 600,000Up to 4 million 300,000 450,000 750,000Up to 5 million 300,000 600,000 900,000Above 5.6 million 300,000 700,000 1,000,000
  22. 22. Join the conversation: #forcewebinarThe numbers game – Custom Index# of records First Threshold Second Threshold Final ThresholdUp to 1 million 10% of total N/A 10% of totalUp to 2 million 100,000 50,000 150,000Up to 3 million 100,000 100,000 200,000Up to 4 million 100,000 150,000 250,000Up to 5 million 100,000 200,000 300,000Above 5.6 million 100,000 233,333 333,333
  23. 23. Join the conversation: #forcewebinarOther OptimizationsAND optimizations§  Composite Index Join - INTERSECTION of indexes should still meetselectivity threshold.OR optimizations§  Union - SUM of the filters should still meet selectivity threshold.sort optimizations§  an index aligns with our order by clause and the query has a row limit,we can use the index to find the first rows quickly and exit.
  24. 24. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarExamples
  25. 25. Join the conversation: #forcewebinarSchemaMyCase – 100,000 RecordsMyUser – 100,000 Records
  26. 26. Join the conversation: #forcewebinarReminder: Goal of Optimizer•  Generate efficient SQL•  Leverage an index to drive query•  Avoid full table scansQuery Optimizer cannot make up for non-selective filters. It willmake the best choice from the filters in your query.
  27. 27. Join the conversation: #forcewebinarSelectivitySELECT Id FROM MyCase__cWHERE Status__c = ‘Closed’ will be do a full table scanSELECT Id FROM MyCase__cWHERE Status__c = ‘New’ will use the indexIndexed Field Value # of Records Selective?Status Closed 96,500 NoStatus New 3,500 YesTotal # Records = 100,000Selectivity Threshold = 10,000
  28. 28. Join the conversation: #forcewebinarNot Equals / Not InCan’t use index because of not equalsSELECT Id FROM MyCase__cWHERE Priority__c != 3 will do a full table scanSELECT Id FROM MyCase__cWHERE Priority__c IN (1,2) will use the indexIndexed Value # of Records Selective?Priority 1 6,000 YesPriority 2 3,500 YesPriority 3 90,500 NoTotal # Records = 100,000Selectivity Threshold = 10,000
  29. 29. Join the conversation: #forcewebinarFormula FieldsField Type FormulaCaseType__c Formula(Text)CASE(MyUser__r.UserType__c,1,”Gold”,”Silver”)Can’t create an index on CaseType__c since this formula spans objectsIF MyUser__r.UserType__c has an index•  SELECT Id FROM MyCase__c WHERE MyUser__r.UserType__c = 1
  30. 30. Join the conversation: #forcewebinarFormula FieldsField TypeCaseTypeClone__c Text(255)Or avoid a join and create CaseTypeClone__c field and index it•  SELECT Id FROM MyCase__c WHERE CaseTypeClone__c = ‘Gold’
  31. 31. Join the conversation: #forcewebinarIndexed Field Value # of Records Selective?ClosedDate Non-Null 96,500 Yes for specific datesClosedDate Null 3,500 YesNullsCustomer Support will need to create a custom index that includes null records. Standard indexes by defaultinclude nulls.SELECT Id FROM MyCase__c WHERE ClosedDate__c = null will use the index # Records = 100,000Selectivity Threshold = 10,000
  32. 32. Join the conversation: #forcewebinarDate & Number RangeSELECT IdFROM MyCaseWHERE ClosedDate__c > 2013-01-01 AND ClosedDate__c < 2013-02-01Query Optimizer can detect only date and number ranges.
  33. 33. Join the conversation: #forcewebinarAND conditionsComposite Index JoinSELECT Id FROM MyUserWHERE FirstName__c = ‘Jane’ AND LastName__c = ‘Doe’ AND City__c = ‘San Francisco’Step 1 – Allow each index to still be considered if they return < 2X selectivity thresholdStep 2 – INTERSECTION of all indexes must meet *selectivity thresholdStep 3 – Use composite index join to drive query*If all indexes are standard indexes, use standard index selectivity threshold. Otherwise, use the custom index standard selectivitythreshold
  34. 34. Join the conversation: #forcewebinarAND conditionsComposite Index Join – MyUser object 100,000 records
  35. 35. Join the conversation: #forcewebinarAND conditionsComposite Index Join – MyUser object 100,000 records
  36. 36. Join the conversation: #forcewebinar2-column IndexFor this simple example, it makes more sense to have Customer Support create a 2-columnindex.
  37. 37. Join the conversation: #forcewebinarOR conditionsUnionSELECT Id FROM MyUserWHERE FirstName__c = ‘Jane’ OR LastName__c = ‘Doe’ OR City__c = ‘San Francisco’Step 1 – Each field must be indexed and meet selectivity thresholdStep 2 – ADDITION of all the indexes must meet *selectivity thresholdStep 3 – Use union to drive query*If all indexes are standard indexes, use standard index selectivity threshold. Otherwise, use the custom index standard selectivitythreshold
  38. 38. Join the conversation: #forcewebinarOR conditionsUnion – MyUser object 100,000 records
  39. 39. Join the conversation: #forcewebinarOR conditionsUsing SOSL may be a better option•  SELECT Id FROM Account WHERE PersonMobilePhone LIKE ‘%123’ – leading %wildcard as bad as full scan•  SELECT Id FROM Account WHERE PersonMobilePhone = ‘1234567890’ ORPersonHomePhone = ‘1234567890’ OR Phone = ‘1234567890’
  40. 40. Join the conversation: #forcewebinarRelationshipRelationshipSELECT Id FROM MyCase__cWHERE MyUser__r.JobType = 1 AND Priority__c = ‘Priority 1’Each index’s selectivity threshold is analyzed separately and the index with the lowerthreshold % is chosen.
  41. 41. Join the conversation: #forcewebinarSoft Deletes•  Records in the Recycle Bin with isDeleted = true•  DO NOT USE isDeleted = false as a filter•  Counted in pre-computed statistics•  Use hard delete option in Bulk API or Contact Customer Support
  42. 42. Join the conversation: #forcewebinarSort Optimization•  Number and Date fields only•  Limit Clause required•  Can make up for a non-selective filterSELECT Id FROM MyCase__cORDER BY CreatedDate LIMIT 10SELECT Id FROM MyCase__cWHERE CreatedDate > 2001-01-01ORDER BY CreatedDate LIMIT 10
  43. 43. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarReviewü  Selectivity thresholds determine if an index is consideredü  Not Equals filters will not leverage indexesü  Be careful filtering on Nullü  And conditions involve an INTERSECTION of indexesü  OR conditions involve an ADDITION of indexesü  ORDER BY with a LIMIT on an index can make up for non-selective filters
  44. 44. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarSharing
  45. 45. Join the conversation: #forcewebinarRecord Visibility•  Applies only to non-Admin users.•  Depending on your user profile, you may have visibility to few or largenumber of records.•  Sharing tables may drive query instead of index
  46. 46. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarSkinny Table
  47. 47. Join the conversation: #forcewebinarSkinny Table
  48. 48. Join the conversation: #forcewebinarSkinny Tables•  Single Object•  Maximum of 100 fields•  Not Aggregate/Summary. 1:1 record count between source object and skinny•  It is not a cross-object join•  Updates to source object automatically reflected in skinny•  Improved performance – minimal joins since fields are in one table
  49. 49. Join the conversation: #forcewebinarSkinny Table
  50. 50. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarWhen are Skinny Tables used?ü  After attempting to tune with custom indexesü  All fields selected and filtered must be in skinnyü will analyze and create
  51. 51. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarOther Performance Factors
  52. 52. Join the conversation: #forcewebinarPerformance FactorsSharing§  Test as a non-System Admin UserData Skews§  Avoid parent-child and ownership data skewsArchivingDatabase Caching§  Avoid relying on cache performance or attempting to warm the cache
  53. 53. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarKey Takeawaysü  Query Performance improves with indexesü  Use selective filters to reduce result setü  Query Optimizer chooses the best table/index to drive a queryü  Skinny Tables may help when indexing is exhausted
  54. 54. Join the conversation: #forcewebinarCheat Sheet:Indexed Fields & Search OptimizationCheat SheetDatabasehttp://developer.force.comQuery Optimization OverviewWhen building queries, list views, and reports, its best to create filter conditionsthat are selective so that scans the most appropriate rows in theobjects that your queries target. This best practice is especially important whenyour queries target "large objects," objects containing more than one millionrecords.When writing SOQL, consider using the following fields, which can make yourquery filter conditions more selective, and improve your query response timesand your databases overall performance.Selectivity OverviewSeveral things can affect the selectivity of a query filters conditions.Whether the field in the condition has an indexWhether the value in the condition is selective relative to the total number of records in theobject. These numbers determine the selectivity threshold, which the query optimizeruses to ensure that the most appropriate index, if any, drives each of your queries.Whether the operator in the condition permits the use of available indexesWhen writing your queries, remember the following selectivity conditions and tips.SOQLFields with Database IndexesPrimaryKeysForeignKeysAuditDatesCustomFieldsIdNameOwnerIdCreatedByIdLastModifiedByIdLookup fieldsMaster-detailrelationship fieldsCreatedDateLastActivityDateSystemModstampUnique fieldsExternal IDfieldsIndex Selectivity Conditions and ThresholdsUnary Condition:Standard IndexUnary Condition:Custom uses astandard index ifthe filter targets lessthan:30% of the firstmillion records15% of all recordsafter the first millionrecords1 million uses acustom index if thefilter targets lessthan:10% of the firstmillion records5% of all recordsafter the firstmillion records333,333 usesa compositeindex join if thefilter targets lessthan:Twice the indexselectivitythresholds foreach fieldThe indexselectivitythresholds forthe intersectionof those fieldsForce.comuses a unionif the filtertargets lessthan:The indexselectivitythresholds foreach fieldThe indexselectivitythresholds forthe sum ofthose fieldsForconditionsthat dontstart witha leadingwildcard,Force.comtests thefirst 100,000rows forselectivity.Query Optimization ResourcesIn addition to this cheat sheets previous sections, we recommend reading thefollowing related resources, which can help you retrieve the records you wantfrom a large volume of data—and do so quickly and efficiently.Best Practices for Deployments with Large Data Volumes (white paper) Apex Code Developers Guide (guide) Blogs: Engineering (blog posts)How to Improve Listview Performance (Salesforce Knowledge article)In the online help:» "Build Effective Filters"» "Getting the Most Out of Filter Logic"» "Improve Report Performance"Index Selectivity ExceptionsWhen you build a filter condition with the following operators, doesnt use an availableindex. Instead, it scans all records in the object to find the records that satisfy the condition. Feelfree to use these operators, but be sure to add selective filter conditions.The following filter operators» not equal to» contains» does not containWhen used with text and text fields, the following comparison operators» (<)» (>)» (<=)» (>=)Additionally, doesnt use available indexes when you use:Leading wildcardsNon-deterministic or cross-object formula fieldsSOSLFields with Search Indexes Search Selectivity TipsGeneral Sidebar Search and Advanced SearchBe as selective as possible. Forexample, use Michael*, not Mich*.Remember that Chatter feedsearches arent affected by the scopeof your search; Chatter feed searchresults include matches across allobjects.Search for the exact phrase with an advanced search.Limit scope by targeting:» Specific objects» Rows owned by the searcher» Rows within a division, when applicableSee "Search Overview" in the online help.GeneralName fieldsPhone fieldsText fieldsPicklist fieldsThese fields vary by object. See "Search Fields" in the online help.
  55. 55. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarUpcoming EventsApril 21-27, 2013Salesforce Mobile Developer WeekMay 8, 2013Summer ‘13 Release Developer Preview WebinarMay 9, 2013SOQL Best Practices CodeTalk
  56. 56. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarSurveyYour feedback is crucial to the success of our webinar programs.Thank you!*Look in the GoToWebinar chatwindow now for a hyperlink.
  57. 57. Join the conversation: #forcewebinarJoin the conversation: #forcewebinarJohn TanArchitect Evangelist@johntansfdcJaikumar BathijaArchitect – DB PerfomanceQ&A