SlideShare a Scribd company logo
Demystify Salesforce
Bulk API
SalesforceCodex.com 1
SalesforceCodex.com 2
About Me
Dhanik Lal Sahni
Technical Architect
Conduent Business Solution
Certified Application Architect
Twitter : @dhaniksahni
Blog : salesforcecodex.com
SalesforceCodex.com 3
Session 15-Agenda
• In a given scenario, recommend when to use API-based
integrations, such as SOAP, REST, Bulk, Streaming, Canvas,
Workflow outbound, APEX (Callouts, @ future, etc.), and
Lightning Connect to achieve business requirements.
• Compare and contrast the advantages and drawbacks (design
trade-offs) of using API-based integrations such as SOAP, REST,
Bulk, Streaming, Canvas, Workflow outbound, APEX (Callouts,
@future, etc.), and Lightning Connect.
28%
17%
15%
10%
15%
10%
5%
Integration Architecture
Designer Exam
1 2 3 4 5 6 7
Salesforce Integration Capabilities:: 28%
SalesforceCodex.com 4
AGENDA
• What is Bulk API?
• When to use Bulk API?
• Bulk API Processing steps
• Bulk API 1.0 vs 2.0
• Best Practice for using Bulk API
• Bulk API Limit
• Demo
SalesforceCodex.com 5
WHAT IS BULK API?
 Bulk API is an asynchronous API to work with high volumes of data.
 Bulk API provides a programmatic option to quickly load your data into
Salesforce.
 Use Bulk API for more than 50K records.
Salesforce Bulk API
(v1.0/v2.0)
Data Salesforce Org
SalesforceCodex.com 6
WHEN TO USE BULK API?
1. Bulk API is based on REST principles and is optimized for loading or deleting
large sets of data. Example: You have new client and they want to move their
data from existing system to Salesforce.
2. Salesforce processes batches in the background. So there is no problem of
network speed.
3. Easily controlled using standard features (setup menu).
4. Easiest way to use Bulk API is to enable it for processing records in Data
Loader using CSV files.
SalesforceCodex.com 7
BULK API PROCESSING (V1.0)
Client Job Processing Server
Batch
Batch
Process job to
insert/update data in
org
Save Result to job
Result
Result
Result
Check Job
Status
Data
SalesforceCodex.com 8
BULK API PROCESSING STEPS
Create Job
Upload Data
Single Batch/Multiple
Batch
Close Job
Check Status
Login to
Salesforce
Get session token
Use token to hit Salesforce APIs
post
post
patch
put
get
SalesforceCodex.com 9
BULK API V1.0 VS V2.0
BULK API 1.0
BULK API 2.0
Batch need to create manually No need to create batch.
Limits on number of record, file size and batch Based on daily limit in 24 hr hour
Built on custom REST framework so not like other
REST API Integration
Built on standard REST API so similar to other REST
API integration
Support serial and parallel processing Support only parallel processing
SalesforceCodex.com 10
BULK API VS SOAP API
SOAP API
BULK API
SOAP API is optimized for real-time client
applications that update a few records at a time.
Bulk API is designed to make it simple to process
data from a few thousand to millions of records.
SOAP API retrieves data in batches of up to 2,000
records.
Bulk API allows batches of up to 10,000 records.
SOAP API is based on SOAP WSDL protocol. It uses
XML format.
Bulk API uses REST protocol and allows XML, JSON,
and CSV data format.
SOAP API is synchronous processing. Bulk API is asynchronous
*DataLoader is using SOAP API for bulk API.
SalesforceCodex.com 11
BULK API 2.0
1. Bulk API 2.0 provides a simple interface to quickly load large amounts of data
into your Salesforce org and to perform bulk queries on your org data.
2. Bulk API 2.0 supports all OAuth flows supported by other Salesforce REST APIs.
3. Bulk API 2.0 simplifies uploading large amounts of data by breaking the data
into batches automatically. All we have to do is upload a CSV file with record
data and check back when the results are ready.
4. Instead of limiting the amount of data uploaded daily by the quantity of jobs,
Bulk API 2.0 uses a limit of total records uploaded
SalesforceCodex.com 12
BEST PRACTISE FOR BULK API
Use Parallel Mode
1. This is default mode and process record very fast.
2. If parallel processing cause lock contention then organize records to
process it parallel or do serial processing.
Organize Batches to Minimize
Lock Contention
1. Organize record in such a manner so that it will not reference records
in another batch.
2. Put related records in same batch.
Minimize Number of Fields
1. Processing time is faster if there are fewer fields loaded for each record.
2. Foreign key, lookup relationship, and roll-up summary fields are more likely to
increase processing time.
Optimize Batch Size
1. Submit batches that process in less than 10 minutes.
2. Batch sizes should be adjusted based on processing times.
3. Start with 5000 records and adjust the batch size based on processing time. If it
takes more than five minutes to process a batch, it may be beneficial to reduce
the batch size.
SalesforceCodex.com 13
BULK API LIMIT
Item Bulk API Limit Bulk API 2.0 Limit
Maximum number of records uploaded per
24-hour rolling period
150,000,000 (15,000 batches x 10,000
records per batch maximum)
150,000,000
Maximum time before a batch is retried 10 minutes The API automatically handles retries. If you
receive a message that the API retried more
than 10 times, use a smaller upload file and
try again.
Maximum file size 10 MB 150 MB
Maximum number of characters in a field 32,000 The same
Maximum number of fields in a record 5,000 The same
Maximum number of characters in a record 400,000 The same
Maximum number of records in a batch 10,000 N/A
Maximum number of characters for all the
data in a batch
10,000,000 N/A
salesforcecodex.com 14
Share some inspiring words about
@salesforcecodex on twitter
salesforcecodex.com 15
Keep Exploring. Keep Learning.

More Related Content

What's hot

Maximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component PerformanceMaximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component Performance
Salesforce Developers
 
Introduction to MuleSoft
Introduction to MuleSoftIntroduction to MuleSoft
Introduction to MuleSoft
Alexandra N. Martinez
 
LWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura InteroperabilityLWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura Interoperability
Salesforce Developers
 
Introducing the Salesforce platform
Introducing the Salesforce platformIntroducing the Salesforce platform
Introducing the Salesforce platform
John Stevenson
 
Deep dive into Salesforce Connected App
Deep dive into Salesforce Connected AppDeep dive into Salesforce Connected App
Deep dive into Salesforce Connected App
Dhanik Sahni
 
Architect day 20181128 - Afternoon Session
Architect day 20181128 - Afternoon SessionArchitect day 20181128 - Afternoon Session
Architect day 20181128 - Afternoon Session
Salesforce - Sweden, Denmark, Norway
 
Integrating with salesforce
Integrating with salesforceIntegrating with salesforce
Integrating with salesforce
Mark Adcock
 
Secure Salesforce: External App Integrations
Secure Salesforce: External App IntegrationsSecure Salesforce: External App Integrations
Secure Salesforce: External App Integrations
Salesforce Developers
 
Introduction to the Salesforce Security Model
Introduction to the Salesforce Security ModelIntroduction to the Salesforce Security Model
Introduction to the Salesforce Security Model
Salesforce Developers
 
Deep Dive into Apex Triggers
Deep Dive into Apex TriggersDeep Dive into Apex Triggers
Deep Dive into Apex Triggers
Salesforce Developers
 
Introduction to apex code
Introduction to apex codeIntroduction to apex code
Introduction to apex codeEdwinOstos
 
How to Use Salesforce Platform Events to Help With Salesforce Limits
How to Use Salesforce Platform Events to Help With Salesforce LimitsHow to Use Salesforce Platform Events to Help With Salesforce Limits
How to Use Salesforce Platform Events to Help With Salesforce Limits
Roy Gilad
 
Two-Way Integration with Writable External Objects
Two-Way Integration with Writable External ObjectsTwo-Way Integration with Writable External Objects
Two-Way Integration with Writable External Objects
Salesforce Developers
 
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Edureka!
 
Live coding with LWC
Live coding with LWCLive coding with LWC
Live coding with LWC
Salesforce Developers
 
Salesforce Integration
Salesforce IntegrationSalesforce Integration
Salesforce Integration
Joshua Hoskins
 
Introduction to Apex for Developers
Introduction to Apex for DevelopersIntroduction to Apex for Developers
Introduction to Apex for Developers
Salesforce Developers
 
Apex Testing Best Practices
Apex Testing Best PracticesApex Testing Best Practices
Apex Testing Best Practices
Salesforce Developers
 
REST API in Salesforce
REST API in SalesforceREST API in Salesforce
REST API in Salesforce
Vivek Deepak
 
Lightning web components episode 2- work with salesforce data
Lightning web components   episode 2- work with salesforce dataLightning web components   episode 2- work with salesforce data
Lightning web components episode 2- work with salesforce data
Salesforce Developers
 

What's hot (20)

Maximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component PerformanceMaximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component Performance
 
Introduction to MuleSoft
Introduction to MuleSoftIntroduction to MuleSoft
Introduction to MuleSoft
 
LWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura InteroperabilityLWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura Interoperability
 
Introducing the Salesforce platform
Introducing the Salesforce platformIntroducing the Salesforce platform
Introducing the Salesforce platform
 
Deep dive into Salesforce Connected App
Deep dive into Salesforce Connected AppDeep dive into Salesforce Connected App
Deep dive into Salesforce Connected App
 
Architect day 20181128 - Afternoon Session
Architect day 20181128 - Afternoon SessionArchitect day 20181128 - Afternoon Session
Architect day 20181128 - Afternoon Session
 
Integrating with salesforce
Integrating with salesforceIntegrating with salesforce
Integrating with salesforce
 
Secure Salesforce: External App Integrations
Secure Salesforce: External App IntegrationsSecure Salesforce: External App Integrations
Secure Salesforce: External App Integrations
 
Introduction to the Salesforce Security Model
Introduction to the Salesforce Security ModelIntroduction to the Salesforce Security Model
Introduction to the Salesforce Security Model
 
Deep Dive into Apex Triggers
Deep Dive into Apex TriggersDeep Dive into Apex Triggers
Deep Dive into Apex Triggers
 
Introduction to apex code
Introduction to apex codeIntroduction to apex code
Introduction to apex code
 
How to Use Salesforce Platform Events to Help With Salesforce Limits
How to Use Salesforce Platform Events to Help With Salesforce LimitsHow to Use Salesforce Platform Events to Help With Salesforce Limits
How to Use Salesforce Platform Events to Help With Salesforce Limits
 
Two-Way Integration with Writable External Objects
Two-Way Integration with Writable External ObjectsTwo-Way Integration with Writable External Objects
Two-Way Integration with Writable External Objects
 
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
Introduction to Salesforce | Salesforce Tutorial for Beginners | Salesforce T...
 
Live coding with LWC
Live coding with LWCLive coding with LWC
Live coding with LWC
 
Salesforce Integration
Salesforce IntegrationSalesforce Integration
Salesforce Integration
 
Introduction to Apex for Developers
Introduction to Apex for DevelopersIntroduction to Apex for Developers
Introduction to Apex for Developers
 
Apex Testing Best Practices
Apex Testing Best PracticesApex Testing Best Practices
Apex Testing Best Practices
 
REST API in Salesforce
REST API in SalesforceREST API in Salesforce
REST API in Salesforce
 
Lightning web components episode 2- work with salesforce data
Lightning web components   episode 2- work with salesforce dataLightning web components   episode 2- work with salesforce data
Lightning web components episode 2- work with salesforce data
 

Similar to Demystify Salesforce Bulk API

SharePoint 2010 Boost your farm performance!
SharePoint 2010 Boost your farm performance!SharePoint 2010 Boost your farm performance!
SharePoint 2010 Boost your farm performance!
Brian Culver
 
REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)Sascha Wenninger
 
Alfresco benchmark report_bl100093
Alfresco benchmark report_bl100093Alfresco benchmark report_bl100093
Alfresco benchmark report_bl100093ECNU
 
A Deep Dive into SharePoint 2016 architecture and deployment
A Deep Dive into SharePoint 2016 architecture and deploymentA Deep Dive into SharePoint 2016 architecture and deployment
A Deep Dive into SharePoint 2016 architecture and deployment
SPC Adriatics
 
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
Kevin Lee
 
PHP on Windows 2008
PHP on Windows 2008PHP on Windows 2008
PHP on Windows 2008
jorke
 
Streaming in the Wild with Apache Flink
Streaming in the Wild with Apache FlinkStreaming in the Wild with Apache Flink
Streaming in the Wild with Apache Flink
Kostas Tzoumas
 
Building a Great Web API - Evan Cooke - QCON 2011
Building a Great Web API - Evan Cooke - QCON 2011Building a Great Web API - Evan Cooke - QCON 2011
Building a Great Web API - Evan Cooke - QCON 2011
Twilio Inc
 
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from TechlightningFrequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Arul ChristhuRaj Alphonse
 
Processing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptxProcessing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptx
Pankaj Goyal
 
Processing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptxProcessing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptx
PankajGoyal164048
 
Business Applications Integration In The Cloud
Business Applications Integration In The CloudBusiness Applications Integration In The Cloud
Business Applications Integration In The Cloud
Anna Brzezińska
 
The ultimate api checklist by Blendr.io
The ultimate api checklist by Blendr.ioThe ultimate api checklist by Blendr.io
The ultimate api checklist by Blendr.io
Blendr.io
 
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
Amazon Web Services
 
Hia 1691-using iib-to_support_api_economy
Hia 1691-using iib-to_support_api_economyHia 1691-using iib-to_support_api_economy
Hia 1691-using iib-to_support_api_economy
Andrew Coleman
 
In salesfroce there are 4 types of api
In salesfroce there are 4 types of apiIn salesfroce there are 4 types of api
In salesfroce there are 4 types of api
Devang Rana
 
Highly available and scalable architectures
Highly available and scalable architecturesHighly available and scalable architectures
Highly available and scalable architectures
Phil Wicklund
 
No More Hops Towards A Linearly Scalable Application Infrastructure
No More Hops Towards A Linearly Scalable Application InfrastructureNo More Hops Towards A Linearly Scalable Application Infrastructure
No More Hops Towards A Linearly Scalable Application InfrastructureConSanFrancisco123
 
What's New in .Net 4.5
What's New in .Net 4.5What's New in .Net 4.5
What's New in .Net 4.5
Malam Team
 
Streaming in the Wild with Apache Flink
Streaming in the Wild with Apache FlinkStreaming in the Wild with Apache Flink
Streaming in the Wild with Apache Flink
DataWorks Summit/Hadoop Summit
 

Similar to Demystify Salesforce Bulk API (20)

SharePoint 2010 Boost your farm performance!
SharePoint 2010 Boost your farm performance!SharePoint 2010 Boost your farm performance!
SharePoint 2010 Boost your farm performance!
 
REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)REST - What's It All About? (SAP TechEd 2012, CD110)
REST - What's It All About? (SAP TechEd 2012, CD110)
 
Alfresco benchmark report_bl100093
Alfresco benchmark report_bl100093Alfresco benchmark report_bl100093
Alfresco benchmark report_bl100093
 
A Deep Dive into SharePoint 2016 architecture and deployment
A Deep Dive into SharePoint 2016 architecture and deploymentA Deep Dive into SharePoint 2016 architecture and deployment
A Deep Dive into SharePoint 2016 architecture and deployment
 
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
Exchange of data over internet using web service(e.g., soap and rest) in SAS ...
 
PHP on Windows 2008
PHP on Windows 2008PHP on Windows 2008
PHP on Windows 2008
 
Streaming in the Wild with Apache Flink
Streaming in the Wild with Apache FlinkStreaming in the Wild with Apache Flink
Streaming in the Wild with Apache Flink
 
Building a Great Web API - Evan Cooke - QCON 2011
Building a Great Web API - Evan Cooke - QCON 2011Building a Great Web API - Evan Cooke - QCON 2011
Building a Great Web API - Evan Cooke - QCON 2011
 
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from TechlightningFrequently asked MuleSoft Interview Questions and Answers from Techlightning
Frequently asked MuleSoft Interview Questions and Answers from Techlightning
 
Processing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptxProcessing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptx
 
Processing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptxProcessing large volume of data with MuleSoft and salesforce.pptx
Processing large volume of data with MuleSoft and salesforce.pptx
 
Business Applications Integration In The Cloud
Business Applications Integration In The CloudBusiness Applications Integration In The Cloud
Business Applications Integration In The Cloud
 
The ultimate api checklist by Blendr.io
The ultimate api checklist by Blendr.ioThe ultimate api checklist by Blendr.io
The ultimate api checklist by Blendr.io
 
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
Don’t Sacrifice Performance for Security: Best Practices for Content Delivery
 
Hia 1691-using iib-to_support_api_economy
Hia 1691-using iib-to_support_api_economyHia 1691-using iib-to_support_api_economy
Hia 1691-using iib-to_support_api_economy
 
In salesfroce there are 4 types of api
In salesfroce there are 4 types of apiIn salesfroce there are 4 types of api
In salesfroce there are 4 types of api
 
Highly available and scalable architectures
Highly available and scalable architecturesHighly available and scalable architectures
Highly available and scalable architectures
 
No More Hops Towards A Linearly Scalable Application Infrastructure
No More Hops Towards A Linearly Scalable Application InfrastructureNo More Hops Towards A Linearly Scalable Application Infrastructure
No More Hops Towards A Linearly Scalable Application Infrastructure
 
What's New in .Net 4.5
What's New in .Net 4.5What's New in .Net 4.5
What's New in .Net 4.5
 
Streaming in the Wild with Apache Flink
Streaming in the Wild with Apache FlinkStreaming in the Wild with Apache Flink
Streaming in the Wild with Apache Flink
 

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 

Demystify Salesforce Bulk API

  • 2. SalesforceCodex.com 2 About Me Dhanik Lal Sahni Technical Architect Conduent Business Solution Certified Application Architect Twitter : @dhaniksahni Blog : salesforcecodex.com
  • 3. SalesforceCodex.com 3 Session 15-Agenda • In a given scenario, recommend when to use API-based integrations, such as SOAP, REST, Bulk, Streaming, Canvas, Workflow outbound, APEX (Callouts, @ future, etc.), and Lightning Connect to achieve business requirements. • Compare and contrast the advantages and drawbacks (design trade-offs) of using API-based integrations such as SOAP, REST, Bulk, Streaming, Canvas, Workflow outbound, APEX (Callouts, @future, etc.), and Lightning Connect. 28% 17% 15% 10% 15% 10% 5% Integration Architecture Designer Exam 1 2 3 4 5 6 7 Salesforce Integration Capabilities:: 28%
  • 4. SalesforceCodex.com 4 AGENDA • What is Bulk API? • When to use Bulk API? • Bulk API Processing steps • Bulk API 1.0 vs 2.0 • Best Practice for using Bulk API • Bulk API Limit • Demo
  • 5. SalesforceCodex.com 5 WHAT IS BULK API?  Bulk API is an asynchronous API to work with high volumes of data.  Bulk API provides a programmatic option to quickly load your data into Salesforce.  Use Bulk API for more than 50K records. Salesforce Bulk API (v1.0/v2.0) Data Salesforce Org
  • 6. SalesforceCodex.com 6 WHEN TO USE BULK API? 1. Bulk API is based on REST principles and is optimized for loading or deleting large sets of data. Example: You have new client and they want to move their data from existing system to Salesforce. 2. Salesforce processes batches in the background. So there is no problem of network speed. 3. Easily controlled using standard features (setup menu). 4. Easiest way to use Bulk API is to enable it for processing records in Data Loader using CSV files.
  • 7. SalesforceCodex.com 7 BULK API PROCESSING (V1.0) Client Job Processing Server Batch Batch Process job to insert/update data in org Save Result to job Result Result Result Check Job Status Data
  • 8. SalesforceCodex.com 8 BULK API PROCESSING STEPS Create Job Upload Data Single Batch/Multiple Batch Close Job Check Status Login to Salesforce Get session token Use token to hit Salesforce APIs post post patch put get
  • 9. SalesforceCodex.com 9 BULK API V1.0 VS V2.0 BULK API 1.0 BULK API 2.0 Batch need to create manually No need to create batch. Limits on number of record, file size and batch Based on daily limit in 24 hr hour Built on custom REST framework so not like other REST API Integration Built on standard REST API so similar to other REST API integration Support serial and parallel processing Support only parallel processing
  • 10. SalesforceCodex.com 10 BULK API VS SOAP API SOAP API BULK API SOAP API is optimized for real-time client applications that update a few records at a time. Bulk API is designed to make it simple to process data from a few thousand to millions of records. SOAP API retrieves data in batches of up to 2,000 records. Bulk API allows batches of up to 10,000 records. SOAP API is based on SOAP WSDL protocol. It uses XML format. Bulk API uses REST protocol and allows XML, JSON, and CSV data format. SOAP API is synchronous processing. Bulk API is asynchronous *DataLoader is using SOAP API for bulk API.
  • 11. SalesforceCodex.com 11 BULK API 2.0 1. Bulk API 2.0 provides a simple interface to quickly load large amounts of data into your Salesforce org and to perform bulk queries on your org data. 2. Bulk API 2.0 supports all OAuth flows supported by other Salesforce REST APIs. 3. Bulk API 2.0 simplifies uploading large amounts of data by breaking the data into batches automatically. All we have to do is upload a CSV file with record data and check back when the results are ready. 4. Instead of limiting the amount of data uploaded daily by the quantity of jobs, Bulk API 2.0 uses a limit of total records uploaded
  • 12. SalesforceCodex.com 12 BEST PRACTISE FOR BULK API Use Parallel Mode 1. This is default mode and process record very fast. 2. If parallel processing cause lock contention then organize records to process it parallel or do serial processing. Organize Batches to Minimize Lock Contention 1. Organize record in such a manner so that it will not reference records in another batch. 2. Put related records in same batch. Minimize Number of Fields 1. Processing time is faster if there are fewer fields loaded for each record. 2. Foreign key, lookup relationship, and roll-up summary fields are more likely to increase processing time. Optimize Batch Size 1. Submit batches that process in less than 10 minutes. 2. Batch sizes should be adjusted based on processing times. 3. Start with 5000 records and adjust the batch size based on processing time. If it takes more than five minutes to process a batch, it may be beneficial to reduce the batch size.
  • 13. SalesforceCodex.com 13 BULK API LIMIT Item Bulk API Limit Bulk API 2.0 Limit Maximum number of records uploaded per 24-hour rolling period 150,000,000 (15,000 batches x 10,000 records per batch maximum) 150,000,000 Maximum time before a batch is retried 10 minutes The API automatically handles retries. If you receive a message that the API retried more than 10 times, use a smaller upload file and try again. Maximum file size 10 MB 150 MB Maximum number of characters in a field 32,000 The same Maximum number of fields in a record 5,000 The same Maximum number of characters in a record 400,000 The same Maximum number of records in a batch 10,000 N/A Maximum number of characters for all the data in a batch 10,000,000 N/A
  • 14. salesforcecodex.com 14 Share some inspiring words about @salesforcecodex on twitter