Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moyez Dreamforce 2017 presentation on Large Data Volumes in Salesforce

453 views

Published on

As enterprises continue to push more or their data to the cloud, Salesforce has seen data volumes in its tenant orgs grow at an exponential rate. How do you manage such volumes efficiently? How do you build queries and reports that respond in a timely manner?

Published in: Data & Analytics
  • Looking For A Job? Positions available now. FT or PT. $10-$30/hr. No exp required. ■■■ http://t.cn/AieXipTS
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Moyez Dreamforce 2017 presentation on Large Data Volumes in Salesforce

  1. 1. Tools, Techniques and Solutions To Avoid A Big-Data Blowout In Your Org moyez@t.digital, @moyezthanawalla Moyez Thanawalla, President – Thanawalla Digital
  2. 2. Moyez Thanawalla President, Thanawalla Digital @MoyezThanawalla moyez@t.digital http://t.digital
  3. 3. Credit: https://www.entrepreneur.com/article/273561 Processing data at…breakneck speeds requires two technologies: a system that can handle developments as quickly as they appear and a data warehouse capable of working through each item once it arrives.
  4. 4. What Prompted Me To Speak About Large Data in Salesforce? AT&T Uverse: • Exponential Record Growth. • Expected to double in size next year • Slow queries, mostly relegated to overnight batch jobs • 48 hour turn-around to get leads allocated to dealers • Client need to react much, much faster (minutes instead of days) to business ad-hoc needs • Yes, Salesforce CAN go there
  5. 5. By [2020], our accumulated digital universe of data will grow from 4.4 zettabyets today to around 44 zettabytes, or 44 trillion gigabytes. Even on a logarithmic scale, data is growing at an exponential rate…
  6. 6. By [2020], our accumulated digital universe of data will grow from 4.4 zettabyets today to around 44 zettabytes, or 44 trillion gigabytes. Even on a logarithmic scale, data is growing at an exponential rate…
  7. 7. …And Salesforce Orgs are Leading The Way ”The truth is that as salesforce.com popularity has skyrocketed, so too has the size of databases underlying custom and standard app implementations on our cloud platforms. It might surprise you to learn that our team works regularly with customers that have large Force.com objects upwards of 10 million records.” Steve Bobrowski, Salesforce Customer Centric Engineering Group
  8. 8. Your Six Steps To Database Success Step 1. Understand What You Can Control…(and what you can’t) Step 2. Understand How your Data is Conceptualized Step 3. Understand and Leverage Indexes Step 4. Ask for Skinny Tables Step 5. Develop Metadata Tables Where Possible Step 6. With Lightning, Push Processing to Client-Side
  9. 9. Step 1. Understand What You Can Control…(and what you can’t) “As a customer, you also cannot optimize the SQL that underlie many application operations because it is generated by the system, not written by each tenant. “
  10. 10. …And Managing Large Volumes in Salesforce is Different.. Multitenancy and Metadata
  11. 11. Step 2. Understand How your Data is Conceptualized In Agile, the Class-diagrams of Domain Modelling, derived from the Use-Cases, have usually replaced Entity-Relationship modelling; but the need for planning has not diminished. We still need to understand the data and what it’s supposed to do and what are the best and safest ways to manage, store, and protect it. ….in other words…Are class-diagrams the enemy of database design?
  12. 12. Step 2. Understand How your Data is Conceptulized
  13. 13. Step 3. Understand and Leverage Indexes Salesforce supports custom indexes to speed up queries, and you can create custom indexes by contacting Salesforce Customer Support. On Most Objects… • RecordTypeId • Division • CreatedDate • Systemmodstamp • Name • Email (for contacts and leads) • Foreign key relationships • The unique Salesforce record ID. Salesforce also supports custom indexes on custom fields, Except for • multi-select picklists, • text areas (long), • text areas (rich), • non-deter. formula fields, • encrypted text fields. Declaring a field as an External ID causes an index to be created on that field; You can create External IDs only on the following fields. • Auto Number • Email • Number • Text
  14. 14. The Force.com Query Plan Tool
  15. 15. What Does The Query Optimizer Tell Me? If the cost for the table scan is lower than the index, and the query is timing out, you will need to perform further analysis on using other filters to improve selectivity, or, if you have another selective filter in that query that is not indexed but is a candidate for one.
  16. 16. What Is The Criteria for a Selective Query” Does Your Query Have and Index? • If the filter is on a standard field, it'll have an index if it is a primary key (Id, Name, OwnerId), a foreign key (CreatedById, LastModifiedById, lookup, master-detail relationship), and an audit field (CreatedDate, SystemModstamp). Custom fields will have an index if they have been marked as Unique or External Id • If the filter doesn't have an index, it won't be considered for optimization. • If the filter has an index, determine how many records it would return: For a standard index, the threshold is 30 percent of the first million targeted records and 15 percent of all records after that first million. In addition, the selectivity threshold for a standard index maxes out at 1 million total targeted records, which you could reach only if you had more than 5.6 million total records. For a custom index, the selectivity threshold is 10 percent of the first million targeted records and 5 percent all records after that first million. In addition, the selectivity threshold for a custom index maxes out at 333,333 targeted records, which you could reach only if you had more than 5.6 million records. If the filter exceeds the threshold,it won't be considered for optimization. If the filter doesn't exceed the threshold, this filter IS selective, and the query optimizer will consider it for optimization. • If the filter uses an operator that is not optimizable, it won’t be considered for optimization. The following type of operators are not optimizable: != , Leading %, null value comparisons,
  17. 17. How to Read the Results of the Query Plan?
  18. 18. How to Read the Results of the Query Plan?
  19. 19. How to Read the Results of the Query Plan?
  20. 20. How to Read the Results of the Query Plan?
  21. 21. How to Read the Results of the Query Plan?
  22. 22. How to Read the Results of the Query Plan?
  23. 23. Step 4. Ask for Skinny Tables Salesforce uses the concept of “Skinny Tables” to speed up queries by avoiding joins Characterisitics… • Must be enabled by Salesforce • Is a collection of frequently used fields • Records are kept in sync with the underlying table structure. • Contains both Standard and Custom fields. • Does not include soft-deleted records. • Ideal when your table size grows over a million records • The unique Salesforce record ID. Considerations… • Can be created on all custom objects… • but only on certain std objects., • Skinny tables can contain the following field types: • Checkbox, Date, Date/Time, Email, Number, Percent, Phone, Picklist, Multi-select Picklist, Text, Text Area, Text Area (long) and URL.
  24. 24. Step 5. Develop Metadata Tables Where Possible Can you infer aggregate abstractions in your data? If so, pull those away into a metadata table, and query, sort and report on *that* table instead.
  25. 25. Step 6. With Lightning, Push Processing to Client-Side If moving excel tables to Salesforce, where the user wants to ‘filter on the fly’ Consider doing a broad query against Salesforce, and loading the data into a Lightning Component (array or grid) where the user can further filter his data in an ‘excel’ manner.
  26. 26. Your Six Steps To Database Success Step 1. Understand What You Can Control…(and what you can’t) Step 2. Understand How your Data is Conceptualized Step 3. Understand and Leverage Indexes Step 4. Ask for Skinny Tables Step 5. Develop Metadata Tables Where Possible Step 6. With Lightning, Push Processing to Client-Side
  27. 27. Want To Know More? Salesforce Best Practices For Large Data Volume: • https://resources.docs.salesforce.com/sfdc/pdf/sal esforce_large_data_volumes_bp.pdf Trailhead: • https://trailhead.salesforce.com/en/modules/datab ase_basics_dotnet/units/writing_efficient_queries Query Plan Tool Details: • https://help.salesforce.com/articleView?id=000199 003&language=en_US&type=1

×