SlideShare a Scribd company logo
HyperBatch
​Daniel PETER
​Lead Applications Engineer, Kenandy
​Salesforce MVP
​Bay Area Salesforce Developer User Group Organizer
​20x certified
​dan@danpeter.com
​@danieljpeter
A Hyper-Fast Batchable Interface for Salesforce
Apex Database.Batchable
​Case study: Account / Contact Batches
• Prerequisite: 121K Accounts already in the system
• CreateContactsBatch: Creates 3 Contacts for each Account with a random “probability” field for
each. 363k Contacts total.
• UpdateAccountsBatch: For each Account, update the highest and lowest probability on the Account
by querying the child Contacts. Get the overall highest and lowest probability across all the
Accounts.
• DeleteContactsBatch: Delete all of the Contacts in the system. Keep a running total of how many
get deleted.
Speed!
Why HyperBatch?
​CreateContactsBatch
​Using traditional Apex Database.Batchable: 45 mins
Speed!
Why HyperBatch?
​UpdateContactsBatch
​Using traditional Apex Database.Batchable: 10 mins
Speed!
Why HyperBatch?
​DeleteContactsBatch
​Using traditional Apex Database.Batchable: 33 mins total (got row lock errors, had to run twice)
Speed!
Why HyperBatch?
​CreateContactsHyperBatch
​Using HyperBatch: 2 mins 12 seconds
Speed!
Why HyperBatch?
​UpdateContactsHyperBatch
​Using HyperBatch: 38 seconds
Speed!
Why HyperBatch?
​DeleteContactsHyperBatch
​Using HyperBatch: 1 min, 11 seconds
Speed!
Why HyperBatch?
Speed!
Why HyperBatch?
Operation Database.Batchable HyperBatch Difference Percentage
CreateContacts 45 2.2 42.8 4.9%
UpdateContacts 10 0.6 9.4 6.3%
DeleteContacts 33 1.2 28 3.6%
Total 88 4 84 4.6%
​Summary
​Running all 3 example batch jobs takes only 4 mins instead of
88 mins.
​You save 84 mins.
​It only takes 4.6% of the time!
Speed!
Why HyperBatch?
User Experience
Why HyperBatch?
​Traditional Apex batch ​HyperBatch
Concurrency
Why HyperBatch?
​Max concurrent jobs processing
​Apex Batch: 5 per org
​HyperBatch: ? 20 / user?
Concurrency
Why HyperBatch?
​Row lock behavior
​Apex Batch: default is a failed batch execution. Retry logic can be built, but it will likely exceed the
transaction limits.
​HyperBatch: row locks retry automatically until the transaction succeed. Each re-attempt gets a new
context!
Summary
How it works
• HyperBatch interface that mimics the Database.Batchable interface.
• Browser orchestration for selecting jobs and running them on-demand.
• Lightning Design System, Visualforce (Lightning Components would be a data bottleneck).
• AJAX toolkit for PK chunking the query locator.
• Parallel remote actions fire the qeueables for the batch executions methods. (Not serial!)
• Wrapping requests in unique identifiers for closed loop execution – JavaScript function binding.
• JavaScript polls for the status of the qeueables, waiting for them to complete.
• Each execute can return some state of type Object, it can be anything. These are stored in a
custom object, and a list of them is returned to the finish() method, then they are deleted.
Interface
How it works
HyperBatch Architecture
Roadmap
• Enhance the user interface
• Test methods
• Support custom iterators instead of just query locator
• Support simple data operations like update a field or delete records without having to write Apex
• Chunk in 2 dimensions: (Parent Id, then Record Id) to avoid row lock errors
• Salesforce Developer Blogs: “Data Chunking Techniques for Massive Orgs“ by Daniel Peter
(https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking-techniques-
massive-orgs.html)
• Presentation from Forcelandia 2016: “PK Chunking – Divide and conquer massive objects in
Salesforce” (http://www.slideshare.net/danieljpeter/forcelandia-2016-pk-chunking)
• GitHub repo: HyperBatch (https://github.com/danieljpeter/HyperBatch)
Resources
Q & A
​Daniel PETER
​Lead Applications Engineer, Kenandy
​Salesforce MVP
​Bay Area Salesforce Developer User Group Organizer
​20x certified
​dan@danpeter.com
​@danieljpeter
Merci
​Daniel PETER
​Lead Applications Engineer, Kenandy
​Salesforce MVP
​Bay Area Salesforce Developer User Group Organizer
​20x certified
​dan@danpeter.com
​@danieljpeter

More Related Content

What's hot

Effective approaches to web application security
Effective approaches to web application security Effective approaches to web application security
Effective approaches to web application security
Zane Lackey
 
Scrum in 15 Minutes
Scrum in 15 MinutesScrum in 15 Minutes
Scrum in 15 Minutes
Serge Rehem
 
Scaled Agile Framework (SAFe) Roles and Meetings
Scaled Agile Framework (SAFe) Roles and MeetingsScaled Agile Framework (SAFe) Roles and Meetings
Scaled Agile Framework (SAFe) Roles and Meetings
Rob Betcher
 
einstein-cheatsheet.pdf
einstein-cheatsheet.pdfeinstein-cheatsheet.pdf
einstein-cheatsheet.pdf
experio1
 
A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation
WrangleConf
 
AGILE@DELOITTE AGILE LANDSCAPE v02
AGILE@DELOITTE AGILE LANDSCAPE v02AGILE@DELOITTE AGILE LANDSCAPE v02
AGILE@DELOITTE AGILE LANDSCAPE v02
Chris Webb
 
My Journey So Far
My Journey So FarMy Journey So Far
My Journey So Far
skipangel
 
Product Management with PayPal Mobile App Creator
Product Management with PayPal Mobile App CreatorProduct Management with PayPal Mobile App Creator
Product Management with PayPal Mobile App Creator
Product School
 
DevOps adoption in the enterprise
DevOps adoption in the enterpriseDevOps adoption in the enterprise
DevOps adoption in the enterprise
Sanjeev Sharma
 
Agile Methodology
Agile MethodologyAgile Methodology
Agile Methodology
Aciron Consulting
 
Agile Methodology ppt
Agile Methodology pptAgile Methodology ppt
Approaches to scaling agile v1.0
Approaches to scaling agile v1.0Approaches to scaling agile v1.0
Approaches to scaling agile v1.0
Srinath Ramakrishnan
 
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
confluent
 
Agile Scrum Methodology
Agile Scrum MethodologyAgile Scrum Methodology
Agile Scrum Methodology
Rajeev Misra
 
Testing at Spotify
Testing at SpotifyTesting at Spotify
Testing at Spotify
Andrii Dzynia
 
Introduction agile scrum methodology
Introduction agile scrum methodologyIntroduction agile scrum methodology
Introduction agile scrum methodology
Amit Verma
 
Introduction to Agile - Scrum, Kanban, and everything in between
Introduction to Agile - Scrum, Kanban, and everything in betweenIntroduction to Agile - Scrum, Kanban, and everything in between
Introduction to Agile - Scrum, Kanban, and everything in between
Pravin Kumar Singh, PMP, PSM
 
Einstein Next Best Action (NBA)
Einstein Next Best Action (NBA)Einstein Next Best Action (NBA)
Einstein Next Best Action (NBA)
Amit Chaudhary
 
What the Heck Is a Product Owner?
What the Heck Is a Product Owner?What the Heck Is a Product Owner?
What the Heck Is a Product Owner?
Ron Lichty
 
Sprint Report Template.pptx
Sprint Report Template.pptxSprint Report Template.pptx
Sprint Report Template.pptx
gary965038
 

What's hot (20)

Effective approaches to web application security
Effective approaches to web application security Effective approaches to web application security
Effective approaches to web application security
 
Scrum in 15 Minutes
Scrum in 15 MinutesScrum in 15 Minutes
Scrum in 15 Minutes
 
Scaled Agile Framework (SAFe) Roles and Meetings
Scaled Agile Framework (SAFe) Roles and MeetingsScaled Agile Framework (SAFe) Roles and Meetings
Scaled Agile Framework (SAFe) Roles and Meetings
 
einstein-cheatsheet.pdf
einstein-cheatsheet.pdfeinstein-cheatsheet.pdf
einstein-cheatsheet.pdf
 
A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation A/B Testing at Pinterest: Building a Culture of Experimentation
A/B Testing at Pinterest: Building a Culture of Experimentation
 
AGILE@DELOITTE AGILE LANDSCAPE v02
AGILE@DELOITTE AGILE LANDSCAPE v02AGILE@DELOITTE AGILE LANDSCAPE v02
AGILE@DELOITTE AGILE LANDSCAPE v02
 
My Journey So Far
My Journey So FarMy Journey So Far
My Journey So Far
 
Product Management with PayPal Mobile App Creator
Product Management with PayPal Mobile App CreatorProduct Management with PayPal Mobile App Creator
Product Management with PayPal Mobile App Creator
 
DevOps adoption in the enterprise
DevOps adoption in the enterpriseDevOps adoption in the enterprise
DevOps adoption in the enterprise
 
Agile Methodology
Agile MethodologyAgile Methodology
Agile Methodology
 
Agile Methodology ppt
Agile Methodology pptAgile Methodology ppt
Agile Methodology ppt
 
Approaches to scaling agile v1.0
Approaches to scaling agile v1.0Approaches to scaling agile v1.0
Approaches to scaling agile v1.0
 
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
A Tale of Two Data Centers: Kafka Streams Resiliency (Anna McDonald, Confluen...
 
Agile Scrum Methodology
Agile Scrum MethodologyAgile Scrum Methodology
Agile Scrum Methodology
 
Testing at Spotify
Testing at SpotifyTesting at Spotify
Testing at Spotify
 
Introduction agile scrum methodology
Introduction agile scrum methodologyIntroduction agile scrum methodology
Introduction agile scrum methodology
 
Introduction to Agile - Scrum, Kanban, and everything in between
Introduction to Agile - Scrum, Kanban, and everything in betweenIntroduction to Agile - Scrum, Kanban, and everything in between
Introduction to Agile - Scrum, Kanban, and everything in between
 
Einstein Next Best Action (NBA)
Einstein Next Best Action (NBA)Einstein Next Best Action (NBA)
Einstein Next Best Action (NBA)
 
What the Heck Is a Product Owner?
What the Heck Is a Product Owner?What the Heck Is a Product Owner?
What the Heck Is a Product Owner?
 
Sprint Report Template.pptx
Sprint Report Template.pptxSprint Report Template.pptx
Sprint Report Template.pptx
 

Viewers also liked

HyperBatch - Snowforce 2017
HyperBatch - Snowforce 2017HyperBatch - Snowforce 2017
HyperBatch - Snowforce 2017
Daniel Peter
 
LDS salesforce saturday
LDS  salesforce saturdayLDS  salesforce saturday
LDS salesforce saturday
Daniel Peter
 
Tree Traversal #SalesforceSaturday
Tree Traversal #SalesforceSaturdayTree Traversal #SalesforceSaturday
Tree Traversal #SalesforceSaturday
Daniel Peter
 
Hyperbatch (LoteRapido) - Punta Dreamin' 2017
Hyperbatch (LoteRapido) - Punta Dreamin' 2017Hyperbatch (LoteRapido) - Punta Dreamin' 2017
Hyperbatch (LoteRapido) - Punta Dreamin' 2017
Daniel Peter
 
Providence: rapid vulnerability prevention
Providence: rapid vulnerability preventionProvidence: rapid vulnerability prevention
Providence: rapid vulnerability prevention
Salesforce Engineering
 
Forcelandia 2016 PK Chunking
Forcelandia 2016 PK ChunkingForcelandia 2016 PK Chunking
Forcelandia 2016 PK Chunking
Daniel Peter
 
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
Vivek Chawla
 
Building a Single Page App with Lightning Components
Building a Single Page App with Lightning ComponentsBuilding a Single Page App with Lightning Components
Building a Single Page App with Lightning Components
Salesforce Developers
 
Design in Tech Report 2017
Design in Tech Report 2017Design in Tech Report 2017
Design in Tech Report 2017
John Maeda
 

Viewers also liked (9)

HyperBatch - Snowforce 2017
HyperBatch - Snowforce 2017HyperBatch - Snowforce 2017
HyperBatch - Snowforce 2017
 
LDS salesforce saturday
LDS  salesforce saturdayLDS  salesforce saturday
LDS salesforce saturday
 
Tree Traversal #SalesforceSaturday
Tree Traversal #SalesforceSaturdayTree Traversal #SalesforceSaturday
Tree Traversal #SalesforceSaturday
 
Hyperbatch (LoteRapido) - Punta Dreamin' 2017
Hyperbatch (LoteRapido) - Punta Dreamin' 2017Hyperbatch (LoteRapido) - Punta Dreamin' 2017
Hyperbatch (LoteRapido) - Punta Dreamin' 2017
 
Providence: rapid vulnerability prevention
Providence: rapid vulnerability preventionProvidence: rapid vulnerability prevention
Providence: rapid vulnerability prevention
 
Forcelandia 2016 PK Chunking
Forcelandia 2016 PK ChunkingForcelandia 2016 PK Chunking
Forcelandia 2016 PK Chunking
 
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
Build Great Triggers Quickly with STP (the Simple Trigger Pattern)
 
Building a Single Page App with Lightning Components
Building a Single Page App with Lightning ComponentsBuilding a Single Page App with Lightning Components
Building a Single Page App with Lightning Components
 
Design in Tech Report 2017
Design in Tech Report 2017Design in Tech Report 2017
Design in Tech Report 2017
 

Similar to HyperBatch

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
xlight
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
Roger Xia
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
liujianrong
 
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
confluent
 
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshootingTarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
Jovan Popovic
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
FoundationDB
 
Azure saturday pn 2018
Azure saturday pn 2018Azure saturday pn 2018
Azure saturday pn 2018
Marco Pozzan
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017
Jags Ramnarayan
 
Drinking from the Firehose - Real-time Metrics
Drinking from the Firehose - Real-time MetricsDrinking from the Firehose - Real-time Metrics
Drinking from the Firehose - Real-time Metrics
Samantha Quiñones
 
What's New in .Net 4.5
What's New in .Net 4.5What's New in .Net 4.5
What's New in .Net 4.5
Malam Team
 
Intro to Apache Apex - Next Gen Platform for Ingest and Transform
Intro to Apache Apex - Next Gen Platform for Ingest and TransformIntro to Apache Apex - Next Gen Platform for Ingest and Transform
Intro to Apache Apex - Next Gen Platform for Ingest and Transform
Apache Apex
 
Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)
Klas Berlič Fras
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
Malin Weiss
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
Speedment, Inc.
 
Databus - Abhishek Bhargava & Maheswaran Veluchamy - DevOps Bangalore Meetup...
Databus - Abhishek Bhargava &  Maheswaran Veluchamy - DevOps Bangalore Meetup...Databus - Abhishek Bhargava &  Maheswaran Veluchamy - DevOps Bangalore Meetup...
Databus - Abhishek Bhargava & Maheswaran Veluchamy - DevOps Bangalore Meetup...
DevOpsBangalore
 
Square Peg Round Hole: Serverless Solutions For Non-Serverless Problems
Square Peg Round Hole: Serverless Solutions For Non-Serverless ProblemsSquare Peg Round Hole: Serverless Solutions For Non-Serverless Problems
Square Peg Round Hole: Serverless Solutions For Non-Serverless Problems
Chase Douglas
 
Spring batch for large enterprises operations
Spring batch for large enterprises operations Spring batch for large enterprises operations
Spring batch for large enterprises operations
Ignasi González
 
Tuning Your SharePoint Environment
Tuning Your SharePoint EnvironmentTuning Your SharePoint Environment
Tuning Your SharePoint Environment
vmaximiuk
 

Similar to HyperBatch (20)

Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
Keep your Metadata Repository Current with Event-Driven Updates using CDC and...
 
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshootingTarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
Tarabica 2019 (Belgrade, Serbia) - SQL Server performance troubleshooting
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
 
Azure saturday pn 2018
Azure saturday pn 2018Azure saturday pn 2018
Azure saturday pn 2018
 
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...SnappyData, the Spark Database. A unified cluster for streaming, transactions...
SnappyData, the Spark Database. A unified cluster for streaming, transactions...
 
SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017SnappyData at Spark Summit 2017
SnappyData at Spark Summit 2017
 
Drinking from the Firehose - Real-time Metrics
Drinking from the Firehose - Real-time MetricsDrinking from the Firehose - Real-time Metrics
Drinking from the Firehose - Real-time Metrics
 
What's New in .Net 4.5
What's New in .Net 4.5What's New in .Net 4.5
What's New in .Net 4.5
 
Intro to Apache Apex - Next Gen Platform for Ingest and Transform
Intro to Apache Apex - Next Gen Platform for Ingest and TransformIntro to Apache Apex - Next Gen Platform for Ingest and Transform
Intro to Apache Apex - Next Gen Platform for Ingest and Transform
 
Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)Basic Application Performance Optimization Techniques (Backend)
Basic Application Performance Optimization Techniques (Backend)
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
 
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
 
Databus - Abhishek Bhargava & Maheswaran Veluchamy - DevOps Bangalore Meetup...
Databus - Abhishek Bhargava &  Maheswaran Veluchamy - DevOps Bangalore Meetup...Databus - Abhishek Bhargava &  Maheswaran Veluchamy - DevOps Bangalore Meetup...
Databus - Abhishek Bhargava & Maheswaran Veluchamy - DevOps Bangalore Meetup...
 
Square Peg Round Hole: Serverless Solutions For Non-Serverless Problems
Square Peg Round Hole: Serverless Solutions For Non-Serverless ProblemsSquare Peg Round Hole: Serverless Solutions For Non-Serverless Problems
Square Peg Round Hole: Serverless Solutions For Non-Serverless Problems
 
Spring batch for large enterprises operations
Spring batch for large enterprises operations Spring batch for large enterprises operations
Spring batch for large enterprises operations
 
Tuning Your SharePoint Environment
Tuning Your SharePoint EnvironmentTuning Your SharePoint Environment
Tuning Your SharePoint Environment
 

More from Daniel Peter

Salesforce Slack Demo Cactusforce 2022
Salesforce Slack Demo Cactusforce 2022Salesforce Slack Demo Cactusforce 2022
Salesforce Slack Demo Cactusforce 2022
Daniel Peter
 
Rules-based Record Generation with Custom Metadata Types
Rules-based Record Generation with Custom Metadata Types Rules-based Record Generation with Custom Metadata Types
Rules-based Record Generation with Custom Metadata Types
Daniel Peter
 
Save Millions of Clicks! Easily migrate complex schemas from SQL to Salesforce.
Save Millions of Clicks!  Easily migrate complex schemas from SQL to Salesforce.Save Millions of Clicks!  Easily migrate complex schemas from SQL to Salesforce.
Save Millions of Clicks! Easily migrate complex schemas from SQL to Salesforce.
Daniel Peter
 
No Refresh Needed
No Refresh NeededNo Refresh Needed
No Refresh Needed
Daniel Peter
 
Using Custom Permissions to Simplify Security
Using Custom Permissions to Simplify SecurityUsing Custom Permissions to Simplify Security
Using Custom Permissions to Simplify Security
Daniel Peter
 
DF Global Gathering PuneWIT
DF Global Gathering PuneWITDF Global Gathering PuneWIT
DF Global Gathering PuneWIT
Daniel Peter
 
Dreamforce Global Gathering Bangaluru 2017
Dreamforce Global Gathering Bangaluru 2017Dreamforce Global Gathering Bangaluru 2017
Dreamforce Global Gathering Bangaluru 2017
Daniel Peter
 
Blaze a Trail to Predictive Selling With Einstein Intent
Blaze a Trail to Predictive Selling With Einstein IntentBlaze a Trail to Predictive Selling With Einstein Intent
Blaze a Trail to Predictive Selling With Einstein Intent
Daniel Peter
 
PK chunking presentation from Tahoe Dreamin' 2016
PK chunking presentation from Tahoe Dreamin' 2016PK chunking presentation from Tahoe Dreamin' 2016
PK chunking presentation from Tahoe Dreamin' 2016
Daniel Peter
 
Lightning Reports - Dreamforce 2015
Lightning Reports - Dreamforce 2015Lightning Reports - Dreamforce 2015
Lightning Reports - Dreamforce 2015
Daniel Peter
 
Callout architecture
Callout architectureCallout architecture
Callout architecture
Daniel Peter
 

More from Daniel Peter (11)

Salesforce Slack Demo Cactusforce 2022
Salesforce Slack Demo Cactusforce 2022Salesforce Slack Demo Cactusforce 2022
Salesforce Slack Demo Cactusforce 2022
 
Rules-based Record Generation with Custom Metadata Types
Rules-based Record Generation with Custom Metadata Types Rules-based Record Generation with Custom Metadata Types
Rules-based Record Generation with Custom Metadata Types
 
Save Millions of Clicks! Easily migrate complex schemas from SQL to Salesforce.
Save Millions of Clicks!  Easily migrate complex schemas from SQL to Salesforce.Save Millions of Clicks!  Easily migrate complex schemas from SQL to Salesforce.
Save Millions of Clicks! Easily migrate complex schemas from SQL to Salesforce.
 
No Refresh Needed
No Refresh NeededNo Refresh Needed
No Refresh Needed
 
Using Custom Permissions to Simplify Security
Using Custom Permissions to Simplify SecurityUsing Custom Permissions to Simplify Security
Using Custom Permissions to Simplify Security
 
DF Global Gathering PuneWIT
DF Global Gathering PuneWITDF Global Gathering PuneWIT
DF Global Gathering PuneWIT
 
Dreamforce Global Gathering Bangaluru 2017
Dreamforce Global Gathering Bangaluru 2017Dreamforce Global Gathering Bangaluru 2017
Dreamforce Global Gathering Bangaluru 2017
 
Blaze a Trail to Predictive Selling With Einstein Intent
Blaze a Trail to Predictive Selling With Einstein IntentBlaze a Trail to Predictive Selling With Einstein Intent
Blaze a Trail to Predictive Selling With Einstein Intent
 
PK chunking presentation from Tahoe Dreamin' 2016
PK chunking presentation from Tahoe Dreamin' 2016PK chunking presentation from Tahoe Dreamin' 2016
PK chunking presentation from Tahoe Dreamin' 2016
 
Lightning Reports - Dreamforce 2015
Lightning Reports - Dreamforce 2015Lightning Reports - Dreamforce 2015
Lightning Reports - Dreamforce 2015
 
Callout architecture
Callout architectureCallout architecture
Callout architecture
 

Recently uploaded

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 

Recently uploaded (20)

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 

HyperBatch

  • 1. HyperBatch ​Daniel PETER ​Lead Applications Engineer, Kenandy ​Salesforce MVP ​Bay Area Salesforce Developer User Group Organizer ​20x certified ​dan@danpeter.com ​@danieljpeter A Hyper-Fast Batchable Interface for Salesforce
  • 3. ​Case study: Account / Contact Batches • Prerequisite: 121K Accounts already in the system • CreateContactsBatch: Creates 3 Contacts for each Account with a random “probability” field for each. 363k Contacts total. • UpdateAccountsBatch: For each Account, update the highest and lowest probability on the Account by querying the child Contacts. Get the overall highest and lowest probability across all the Accounts. • DeleteContactsBatch: Delete all of the Contacts in the system. Keep a running total of how many get deleted. Speed! Why HyperBatch?
  • 4. ​CreateContactsBatch ​Using traditional Apex Database.Batchable: 45 mins Speed! Why HyperBatch?
  • 5. ​UpdateContactsBatch ​Using traditional Apex Database.Batchable: 10 mins Speed! Why HyperBatch?
  • 6. ​DeleteContactsBatch ​Using traditional Apex Database.Batchable: 33 mins total (got row lock errors, had to run twice) Speed! Why HyperBatch?
  • 7. ​CreateContactsHyperBatch ​Using HyperBatch: 2 mins 12 seconds Speed! Why HyperBatch?
  • 9. ​DeleteContactsHyperBatch ​Using HyperBatch: 1 min, 11 seconds Speed! Why HyperBatch?
  • 10. Speed! Why HyperBatch? Operation Database.Batchable HyperBatch Difference Percentage CreateContacts 45 2.2 42.8 4.9% UpdateContacts 10 0.6 9.4 6.3% DeleteContacts 33 1.2 28 3.6% Total 88 4 84 4.6%
  • 11. ​Summary ​Running all 3 example batch jobs takes only 4 mins instead of 88 mins. ​You save 84 mins. ​It only takes 4.6% of the time! Speed! Why HyperBatch?
  • 13. Concurrency Why HyperBatch? ​Max concurrent jobs processing ​Apex Batch: 5 per org ​HyperBatch: ? 20 / user?
  • 14. Concurrency Why HyperBatch? ​Row lock behavior ​Apex Batch: default is a failed batch execution. Retry logic can be built, but it will likely exceed the transaction limits. ​HyperBatch: row locks retry automatically until the transaction succeed. Each re-attempt gets a new context!
  • 15. Summary How it works • HyperBatch interface that mimics the Database.Batchable interface. • Browser orchestration for selecting jobs and running them on-demand. • Lightning Design System, Visualforce (Lightning Components would be a data bottleneck). • AJAX toolkit for PK chunking the query locator. • Parallel remote actions fire the qeueables for the batch executions methods. (Not serial!) • Wrapping requests in unique identifiers for closed loop execution – JavaScript function binding. • JavaScript polls for the status of the qeueables, waiting for them to complete. • Each execute can return some state of type Object, it can be anything. These are stored in a custom object, and a list of them is returned to the finish() method, then they are deleted.
  • 18.
  • 19. Roadmap • Enhance the user interface • Test methods • Support custom iterators instead of just query locator • Support simple data operations like update a field or delete records without having to write Apex • Chunk in 2 dimensions: (Parent Id, then Record Id) to avoid row lock errors
  • 20. • Salesforce Developer Blogs: “Data Chunking Techniques for Massive Orgs“ by Daniel Peter (https://developer.salesforce.com/blogs/developer-relations/2015/11/pk-chunking-techniques- massive-orgs.html) • Presentation from Forcelandia 2016: “PK Chunking – Divide and conquer massive objects in Salesforce” (http://www.slideshare.net/danieljpeter/forcelandia-2016-pk-chunking) • GitHub repo: HyperBatch (https://github.com/danieljpeter/HyperBatch) Resources
  • 21. Q & A ​Daniel PETER ​Lead Applications Engineer, Kenandy ​Salesforce MVP ​Bay Area Salesforce Developer User Group Organizer ​20x certified ​dan@danpeter.com ​@danieljpeter
  • 22. Merci ​Daniel PETER ​Lead Applications Engineer, Kenandy ​Salesforce MVP ​Bay Area Salesforce Developer User Group Organizer ​20x certified ​dan@danpeter.com ​@danieljpeter