Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hyperbatch (LoteRapido) - Punta Dreamin' 2017

310 views

Published on

A Hyper-Fast Batchable Interface for Salesforce Presented in Punta del Este, Uruguay for Punta Dreamin' 2017

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Hyperbatch (LoteRapido) - Punta Dreamin' 2017

  1. 1. HyperBatch (LoteRapido) Daniel Peter Lead Applications Engineer, Kenandy Salesforce MVP Bay Area Salesforce Developer User Group Organizer 22x certified dan@danpeter.com @danieljpeter A Hyper-Fast Batchable Interface for Salesforce
  2. 2. Apex Database.Batchable
  3. 3. Why HyperBatch? ​Speed! ​Case study: Account / Contact Batches • Prerequisite: 121K Accounts already in the system • CreateContactsBatch: Creates 3 Contacts for each Account with a random “probability” field for each. 363k Contacts total. • UpdateAccountsBatch: For each Account, update the highest and lowest probability on the Account by querying the child Contacts. Get the overall highest and lowest probability across all the Accounts. • DeleteContactsBatch: Delete all of the Contacts in the system. Keep a running total of how many get deleted.
  4. 4. Why HyperBatch? ​Speed! ​CreateContactsBatch • Using traditional Apex Database.Batchable: 45 mins
  5. 5. Why HyperBatch? ​Speed! ​UpdateContactsBatch • Using traditional Apex Database.Batchable: 10 mins
  6. 6. Why HyperBatch? ​Speed! ​DeleteContactsBatch • Using traditional Apex Database.Batchable: 33 mins total (got row lock errors, had to run twice)
  7. 7. Why HyperBatch? ​Speed! ​CreateContactsHyperBatch • Using HyperBatch • 2 mins 12 seconds
  8. 8. Why HyperBatch? ​Speed! ​UpdateContactsHyperBatch • Using HyperBatch • 55 seconds
  9. 9. Why HyperBatch? ​Speed! ​DeleteContactsHyperBatch • Using HyperBatch • 3 mins 15 seconds • RowLock / retry • 9500 Batch Size
  10. 10. Why HyperBatch? ​Speed! Operation Database.Batchable HyperBatch Difference Percentage CreateContacts 45 1.4 43.6 3.2% UpdateContacts 10 0.9 9.1 9.9% DeleteContacts 33 3.25 29.8 10.9% Total 88 5.55 82.5 6.7%
  11. 11. Why HyperBatch? ​Speed! ​Summary • Running all 3 example batch jobs takes only 6 mins instead of 88 mins. • You save 82 mins. • It only takes 6.7% of the time!
  12. 12. Why HyperBatch? ​User Experience ​Traditional Apex Batch ​HyperBatch
  13. 13. Why HyperBatch? ​Concurrency ​Asynchronous • Apex Batch • 5 running per org • HyperBatch • Not subject to 50 job per transaction limit (each request is a new transaction) ​Synchronous • Apex Batch • N/A • HyperBatch • 10 long running per org, unlimited otherwise
  14. 14. Why HyperBatch? ​Concurrency
  15. 15. Why HyperBatch? ​Concurrency ​Row lock behavior • Apex Batch: default is a failed batch execution. Retry logic can be built, but it will likely exceed the transaction limits. • HyperBatch: row locks retry automatically until the transaction succeeds. Each re-attempt gets a new context!
  16. 16. How it Works ​Summary • HyperBatch interface that mimics the Database.Batchable interface. • Browser orchestration for selecting jobs and running them on-demand. • Lightning Design System, Visualforce (Lightning Components would be a data bottleneck). • AJAX toolkit for PK chunking the query locator. • Wrapping requests in unique identifiers for closed loop execution – JavaScript function binding. • Each execute can return some state of type Object, it can be anything. • Async • Parallel remote actions fire the qeueables for the batch executions methods. (Not serial!) • JavaScript polls for the status of the qeueables, waiting for them to complete. • Execute state is stored in a custom object, and a list of them is returned to the finish() method, then they are deleted. • Sync • Parallel remote actions fire the synchronous transactions. (Not serial!) • Execute state is returned synchronously and stored in JavaScript in the browser until they are all complete.
  17. 17. How it Works ​Interface
  18. 18. How it Works ​Asynchronous Architecture
  19. 19. How it Works ​Synchronous Architecture
  20. 20. How it Works ​Bulk API for Apex
  21. 21. synchronous vs. asynchronous governor limits Synchronous Asynchronous SOQL Queries 100 200 Heap Size 6MB 12MB CPU Time 10 seconds 60 seconds
  22. 22. Demo
  23. 23. New Features • Synchronous mode • Batch Size Control • Test Methods
  24. 24. Roadmap • Throttle requests (x number of open connections) • Enhance the user interface • Support custom iterators instead of just query locator • Support simple data operations like update a field or delete records without having to write Apex • Chunk in 2 dimensions: (Parent Id, then Record Id) to avoid row lock errors
  25. 25. Resources • Salesforce Developer Blogs • “Data Chunking Techniques for Massive Orgs“ by Daniel Peter (https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking- techniques-massive-orgs.html) • “French Touch Dreamin’ – The first community-led event in France” by Philippe Ozil (https://developer.salesforce.com/blogs/developer-relations/2016/11/french-touch- dreamin-first-community-led-event-france.html) • Presentation from Forcelandia 2016: “PK Chunking – Divide and conquer massive objects in Salesforce” (http://www.slideshare.net/danieljpeter/forcelandia-2016-pk-chunking) • GitHub repo: HyperBatch (https://github.com/danieljpeter/HyperBatch)
  26. 26. Additional Resources
  27. 27. Thank Y u
  28. 28. Q&A

×