SlideShare a Scribd company logo
Unlocking the Power of
Salesforce Integrations
with Confluent
Presented By:
Big Compass and Confluent
November 2020
1. Integration Pain Points With Salesforce
2. Ways to Integrate with Salesforce
3. Solving the Challenge of Unlocking Salesforce Data
4. Salesforce + Confluent
5. Use Cases: Extracting Data out of Salesforce with
Confluent
Agenda
● Consulting - specializing in
integrations and related technologies
● Confluent partners
● We build connections between
systems, applications, people, and
ideas
Big Compass
Salesforce Integrations
● Subscription tier limits
● API/Event limits
● Countless customizations
● Custom code/triggers to push data
out of Salesforce
● Unexpected API Security
Integration Pain
Points With Salesforce
● How: Create triggers (PushTopics,
Platform Events, CDC) in Salesforce to
send events to a target
● When to Use: Great for real-time or
event-driven use cases
● Skills: Deeper Salesforce knowledge
required to create events
● Complexity: Higher - Create trigger in
Salesforce and subscribe to the event
using a tool like Confluent
Pushing vs. Pulling Salesforce Data
Considerations when choosing whether to send events out of Salesforce or use Salesforce APIs to pull data out of Salesforce
Pushing Data
● How: Use Salesforce APIs to query
data on-demand
● When to Use: Great for batch or on-
demand use cases
● Skills: Some Salesforce knowledge is
useful but not required
● Complexity: Lower - Least number of
moving parts to the solution -
Salesforce API, consumer/connector,
target system
Pulling Data
Ways to Integrate with Salesforce
Integration Method Description Good For Not Suitable For
Batch Single event/request
for many objects in
Salesforce
Easiest way to
implement a solution
for non-real-time
solutions
Constantly changing
data
Real-Time Create individual
events for each
change that occur in
Salesforce
Real-time use cases
and handling events
individually
Less frequently
changing data
AppExchange Install out of the box
apps in Salesforce to
integrate with other
systems
Use cases that
integrate with
common systems
Apps that are not built
yet or use cases that
go beyond the scope
of connectors
Custom Salesforce UI Expose your
Salesforce data via
custom UI built in
Salesforce
Organizations who
are all-in on
Salesforce
Organizations who
need the Salesforce
data in another
system
● Necessary to power downstream
systems/applications
● Do you push data out of Salesforce,
pull data out of Salesforce, or both?
● Pre-built connectors like Confluent’s
Salesforce Connector can help speed
up integration
● Not a trivial task without pre-built
connectors!
○ Need to author custom code
Unlocking Salesforce
Data - Not a Trivial
Task!
Real-Time Salesforce
Events
Eventing Out of Salesforce
Event Type Description Advantages Disadvantages
Platform Events Custom pub/sub events 1. Highly customizable
2. Scalable
1. Requires Salesforce
knowledge
PushTopics PushTopic events track
field changes in Salesforce
records
1. Out of the box
PushTopics can
kickstart your
implementation
1. Lower ability to
customize the event
Change Data Capture Notification on changes on
an object
1. Easy to implement 1. Lower ability to
customize the event
Outbound Event Send SOAP message to
web service
1. Can be used to
event to a RESTful
or SOAP endpoint
1. Lower ability to
customize the event
2. Uses SOAP
Unlocking Salesforce Data
Publisher (Platform Event) Subscriber/Processing Target Systems
Consumer Applications
Use Case: Synchronizing SFDC Data
Publisher (Platform Event)
Subscriber/Processing
Real-Time Target
Data Storage Target
Consumer Applications
● Pre-built source and sink connector to
speed up integrations
● Once data is in Confluent, easy to use
pub/sub model to process and route
the data
● Joining from multiple sources
● Streaming data
● Duplicate detection
● Execute business logic
● Run SQL queries
● Detect events
● Much more than standard pub/sub
Salesforce
Integrations with
Confluent
● Need to enrich and transform data
from multiple sources before it is sent
to a target system
● Clean your data
● Enrich your data
● Stage your data
Use Case: Data Prep
● Gain valuable insight about your
customers from multiple data sources
● Enrich data from various sources
● Generate actionable insight about
your customers
● Predict customer actions in real-time
Use Case: 360
Customer View
● Synchronize data with a target system
to power a mobile application
● Two-way synchronization with source
and sink connectors
● Duplicate detection
● Business logic
● Real-time streaming
Use Case:
Synchronizing SFDC
Data
Use Cases In Action: SFDC Sync From Multiple
Sources
http PUT cp01.woolford.io:8083/connectors/sfdc-contact-updates/config <<< '
{
"confluent.topic.bootstrap.servers": "cp01.woolford.io:9092,cp02...",
"confluent.topic.replication.factor": "1",
"connector.class": "io.confluent.salesforce.SalesforcePushTopicSourceConnector",
"kafka.topic": "sfdc-contact-updates",
"name": "sfdc-contact-updates",
"salesforce.consumer.key": "********",
"salesforce.consumer.secret": "********",
"salesforce.initial.start": "all",
"salesforce.object": "Contact",
"salesforce.password": "cxI3o3R$****",
"salesforce.password.token": "iggX9UEPizeaizvpg02Vj****",
"salesforce.push.topic.name": "ContactUpdates",
"salesforce.username": "alex@woolford.io",
"tasks.max": "1"
}'
PushTopic pushTopic = new PushTopic();
pushTopic.Name = 'ContactUpdates';
pushTopic.Query = 'SELECT Id, [...] FROM Contact';
pushTopic.ApiVersion = 50.0;
pushTopic.NotifyForOperationCreate = true;
pushTopic.NotifyForOperationUpdate = true;
pushTopic.NotifyForOperationUndelete = true;
pushTopic.NotifyForOperationDelete = true;
pushTopic.NotifyForFields = 'All';
insert pushTopic;
mysql> use healthcare;
Database changed
mysql> select * from patient;
+----+-----------+------------+--------------+------------+
| id | firstname | lastname | enroll_start | enroll_end |
+----+-----------+------------+--------------+------------+
| 1 | Donald | Trump | 2019-04-14 | 2020-04-13 |
| 2 | Vladimir | Putin | 2019-05-20 | 2020-05-19 |
| 3 | Boris | Johnson | 2019-01-24 | 2020-01-24 |
| 4 | Mark | Zuckerberg | 2019-03-03 | 2020-03-02 |
| 5 | Benjamin | Netanyahu | 2019-08-24 | 2020-08-23 |
| 6 | Jair | Bolsonaro | 2019-09-19 | 2020-09-18 |
| 7 | Farty | McFartface | 2019-08-23 | 2020-08-22 |
+----+-----------+------------+--------------+------------+
7 rows in set (0.00 sec)
http PUT http://cp01.woolford.io:8083/connectors/mysql-cdc-healthcare/config <<< '
{
"name": "mysql-cdc-healthcare",
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
...
"include.schema.changes": "false",
"snapshot.mode": "when_needed",
"table.whitelist": "healthcare.patient",
"transforms": "unwrap,HoistField",
"transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
"transforms.HoistField.type": "org.apache.kafka.connect.transforms.ExtractField$Key",
"transforms.HoistField.field": "id"
}'
CREATE STREAM AILMENT (
id STRING,
ailment VARCHAR
) WITH (
kafka_topic='ailment',
value_format='JSON',
KEY='id'
);
CREATE TABLE PATIENT (
id STRING,
firstname VARCHAR,
lastname VARCHAR,
enroll_start BIGINT,
enroll_end BIGINT
) WITH (
kafka_topic='deepthought.healthcare.patient',
value_format='JSON',
KEY='id'
);
select
ailment.id as id,
ailment,
firstname,
lastname,
case
when ailment.rowtime > enroll_start * 86400 * 1000
and ailment.rowtime <= enroll_end * 86400 * 1000
then true
else false
end as covered
from ailment inner join patient
on ailment.id = patient.id
emit changes;
Questions?
Appendix
References
● Salesforce API Documentation
● Comparing PushTopics, Platform Events, and Change Data
Capture
● External Objects
● Confluent KSQL
● Kafka Transformations

More Related Content

What's hot

Building a Logical Data Fabric using Data Virtualization (ASEAN)
Building a Logical Data Fabric using Data Virtualization (ASEAN)Building a Logical Data Fabric using Data Virtualization (ASEAN)
Building a Logical Data Fabric using Data Virtualization (ASEAN)
Denodo
 
Pimcore Master Data Management PPT
Pimcore Master Data Management PPTPimcore Master Data Management PPT
Pimcore Master Data Management PPT
Manish Shewaramani
 
Event Driven Software Architecture Pattern
Event Driven Software Architecture PatternEvent Driven Software Architecture Pattern
Event Driven Software Architecture Pattern
jeetendra mandal
 
Azure SQL Database Managed Instance
Azure SQL Database Managed InstanceAzure SQL Database Managed Instance
Azure SQL Database Managed Instance
James Serra
 
OFSAA - BIGDATA - IBANK
OFSAA - BIGDATA - IBANKOFSAA - BIGDATA - IBANK
OFSAA - BIGDATA - IBANK
ibankuk
 
Master Data Management – Aligning Data, Process, and Governance
Master Data Management – Aligning Data, Process, and GovernanceMaster Data Management – Aligning Data, Process, and Governance
Master Data Management – Aligning Data, Process, and Governance
DATAVERSITY
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake Overview
James Serra
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
James Serra
 
Data Mesh 101
Data Mesh 101Data Mesh 101
Data Mesh 101
ChrisFord803185
 
Best Practices for Adopting Microsoft Dynamics 365
Best Practices for Adopting Microsoft Dynamics 365Best Practices for Adopting Microsoft Dynamics 365
Best Practices for Adopting Microsoft Dynamics 365
Precisely
 
Future Of DevOps Trends 2023
Future Of DevOps Trends 2023Future Of DevOps Trends 2023
Future Of DevOps Trends 2023
WeCode Inc
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft Azure
Precisely
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Cathrine Wilhelmsen
 
Effective Healthcare Data Governance Strategy Propels Data Transformation
Effective Healthcare Data Governance Strategy Propels Data TransformationEffective Healthcare Data Governance Strategy Propels Data Transformation
Effective Healthcare Data Governance Strategy Propels Data Transformation
Health Catalyst
 
Tackling Salesforce Technical Debt
Tackling Salesforce Technical DebtTackling Salesforce Technical Debt
Tackling Salesforce Technical Debt
panayaofficial
 
Mule : Building Blocks for Microservices
Mule : Building Blocks for MicroservicesMule : Building Blocks for Microservices
Mule : Building Blocks for Microservices
Anirudh Pandit
 
Siligong.Data - May 2021 - Transforming your analytics workflow with dbt
Siligong.Data - May 2021 - Transforming your analytics workflow with dbtSiligong.Data - May 2021 - Transforming your analytics workflow with dbt
Siligong.Data - May 2021 - Transforming your analytics workflow with dbt
Jon Su
 
Event Driven Architecture
Event Driven ArchitectureEvent Driven Architecture
Event Driven Architecture
Stefan Norberg
 
Microservices Patterns with GoldenGate
Microservices Patterns with GoldenGateMicroservices Patterns with GoldenGate
Microservices Patterns with GoldenGate
Jeffrey T. Pollock
 
Oracle Stream Analytics - Developer Introduction
Oracle Stream Analytics - Developer IntroductionOracle Stream Analytics - Developer Introduction
Oracle Stream Analytics - Developer Introduction
Jeffrey T. Pollock
 

What's hot (20)

Building a Logical Data Fabric using Data Virtualization (ASEAN)
Building a Logical Data Fabric using Data Virtualization (ASEAN)Building a Logical Data Fabric using Data Virtualization (ASEAN)
Building a Logical Data Fabric using Data Virtualization (ASEAN)
 
Pimcore Master Data Management PPT
Pimcore Master Data Management PPTPimcore Master Data Management PPT
Pimcore Master Data Management PPT
 
Event Driven Software Architecture Pattern
Event Driven Software Architecture PatternEvent Driven Software Architecture Pattern
Event Driven Software Architecture Pattern
 
Azure SQL Database Managed Instance
Azure SQL Database Managed InstanceAzure SQL Database Managed Instance
Azure SQL Database Managed Instance
 
OFSAA - BIGDATA - IBANK
OFSAA - BIGDATA - IBANKOFSAA - BIGDATA - IBANK
OFSAA - BIGDATA - IBANK
 
Master Data Management – Aligning Data, Process, and Governance
Master Data Management – Aligning Data, Process, and GovernanceMaster Data Management – Aligning Data, Process, and Governance
Master Data Management – Aligning Data, Process, and Governance
 
Data Lake Overview
Data Lake OverviewData Lake Overview
Data Lake Overview
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Data Mesh 101
Data Mesh 101Data Mesh 101
Data Mesh 101
 
Best Practices for Adopting Microsoft Dynamics 365
Best Practices for Adopting Microsoft Dynamics 365Best Practices for Adopting Microsoft Dynamics 365
Best Practices for Adopting Microsoft Dynamics 365
 
Future Of DevOps Trends 2023
Future Of DevOps Trends 2023Future Of DevOps Trends 2023
Future Of DevOps Trends 2023
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft Azure
 
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
Lessons Learned: Understanding Pipeline Pricing in Azure Data Factory and Azu...
 
Effective Healthcare Data Governance Strategy Propels Data Transformation
Effective Healthcare Data Governance Strategy Propels Data TransformationEffective Healthcare Data Governance Strategy Propels Data Transformation
Effective Healthcare Data Governance Strategy Propels Data Transformation
 
Tackling Salesforce Technical Debt
Tackling Salesforce Technical DebtTackling Salesforce Technical Debt
Tackling Salesforce Technical Debt
 
Mule : Building Blocks for Microservices
Mule : Building Blocks for MicroservicesMule : Building Blocks for Microservices
Mule : Building Blocks for Microservices
 
Siligong.Data - May 2021 - Transforming your analytics workflow with dbt
Siligong.Data - May 2021 - Transforming your analytics workflow with dbtSiligong.Data - May 2021 - Transforming your analytics workflow with dbt
Siligong.Data - May 2021 - Transforming your analytics workflow with dbt
 
Event Driven Architecture
Event Driven ArchitectureEvent Driven Architecture
Event Driven Architecture
 
Microservices Patterns with GoldenGate
Microservices Patterns with GoldenGateMicroservices Patterns with GoldenGate
Microservices Patterns with GoldenGate
 
Oracle Stream Analytics - Developer Introduction
Oracle Stream Analytics - Developer IntroductionOracle Stream Analytics - Developer Introduction
Oracle Stream Analytics - Developer Introduction
 

Similar to Unlocking the Power of Salesforce Integrations with Confluent

The Integrations Behind Connecting With Salesforce
The Integrations Behind Connecting With SalesforceThe Integrations Behind Connecting With Salesforce
The Integrations Behind Connecting With Salesforce
AaronLieberman5
 
Salesforce Integration Pattern Overview
Salesforce Integration Pattern OverviewSalesforce Integration Pattern Overview
Salesforce Integration Pattern Overview
Dhanik Sahni
 
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast IronIntegrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
Prolifics
 
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron ApplianceIntegrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Sandeep Chellingi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
SkyPlanner
 
Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015
Dreamforce
 
Salesforce and SAP Integration with Informatica Cloud
Salesforce and SAP Integration with Informatica CloudSalesforce and SAP Integration with Informatica Cloud
Salesforce and SAP Integration with Informatica Cloud
Darren Cunningham
 
Salesforce Lightning workshop
Salesforce Lightning workshopSalesforce Lightning workshop
Salesforce Lightning workshop
Shivanath Devinarayanan
 
Flow like a Rockstar @ SharePoint Saturday The Netherlands
Flow like a Rockstar @ SharePoint Saturday The NetherlandsFlow like a Rockstar @ SharePoint Saturday The Netherlands
Flow like a Rockstar @ SharePoint Saturday The Netherlands
Daniel Laskewitz
 
Lightning week - Paris DUG
Lightning week - Paris DUGLightning week - Paris DUG
Lightning week - Paris DUG
Paris Salesforce Developer Group
 
Lightning Developer Week - Bangalore Salesforce Developer Group
Lightning Developer Week - Bangalore Salesforce Developer GroupLightning Developer Week - Bangalore Salesforce Developer Group
Lightning Developer Week - Bangalore Salesforce Developer Group
Abhilash Kuntar
 
Salesforce Lightning workshop Hartford - 12 March
Salesforce Lightning workshop Hartford - 12 MarchSalesforce Lightning workshop Hartford - 12 March
Salesforce Lightning workshop Hartford - 12 March
Jitendra Zaa
 
Sydney MuleSoft Meetup #12 2020204
Sydney MuleSoft Meetup #12 2020204Sydney MuleSoft Meetup #12 2020204
Sydney MuleSoft Meetup #12 2020204
Royston Lobo
 
#TDXRecap India tour
#TDXRecap India tour#TDXRecap India tour
Salesforce Barcelona Admin Group, 26-Feb-2019
Salesforce Barcelona Admin Group, 26-Feb-2019Salesforce Barcelona Admin Group, 26-Feb-2019
Salesforce Barcelona Admin Group, 26-Feb-2019
Roger Borges Grilo
 
Bringing Your Back Office Data To Life with Salesforce Connect
Bringing Your Back Office Data To Life with Salesforce ConnectBringing Your Back Office Data To Life with Salesforce Connect
Bringing Your Back Office Data To Life with Salesforce Connect
Eugenio Roldán Romasanta
 
Control your world using the Salesforce1 Platform (IoT)
Control your world using the Salesforce1 Platform (IoT)Control your world using the Salesforce1 Platform (IoT)
Control your world using the Salesforce1 Platform (IoT)
InternetCreations
 
JDF18 - Connecting the customer success platform
JDF18 - Connecting the customer success platformJDF18 - Connecting the customer success platform
JDF18 - Connecting the customer success platform
Deepu Chacko
 
Lightning Connect: Lessons Learned
Lightning Connect: Lessons LearnedLightning Connect: Lessons Learned
Lightning Connect: Lessons Learned
Salesforce Developers
 
ISV Monthly Tech Enablement (August 2017)
ISV Monthly Tech Enablement (August 2017)ISV Monthly Tech Enablement (August 2017)
ISV Monthly Tech Enablement (August 2017)
Salesforce Partners
 

Similar to Unlocking the Power of Salesforce Integrations with Confluent (20)

The Integrations Behind Connecting With Salesforce
The Integrations Behind Connecting With SalesforceThe Integrations Behind Connecting With Salesforce
The Integrations Behind Connecting With Salesforce
 
Salesforce Integration Pattern Overview
Salesforce Integration Pattern OverviewSalesforce Integration Pattern Overview
Salesforce Integration Pattern Overview
 
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast IronIntegrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
Integrating Salesforce.com and Oracle ERP Using IBM WebSphere Cast Iron
 
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron ApplianceIntegrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015
 
Salesforce and SAP Integration with Informatica Cloud
Salesforce and SAP Integration with Informatica CloudSalesforce and SAP Integration with Informatica Cloud
Salesforce and SAP Integration with Informatica Cloud
 
Salesforce Lightning workshop
Salesforce Lightning workshopSalesforce Lightning workshop
Salesforce Lightning workshop
 
Flow like a Rockstar @ SharePoint Saturday The Netherlands
Flow like a Rockstar @ SharePoint Saturday The NetherlandsFlow like a Rockstar @ SharePoint Saturday The Netherlands
Flow like a Rockstar @ SharePoint Saturday The Netherlands
 
Lightning week - Paris DUG
Lightning week - Paris DUGLightning week - Paris DUG
Lightning week - Paris DUG
 
Lightning Developer Week - Bangalore Salesforce Developer Group
Lightning Developer Week - Bangalore Salesforce Developer GroupLightning Developer Week - Bangalore Salesforce Developer Group
Lightning Developer Week - Bangalore Salesforce Developer Group
 
Salesforce Lightning workshop Hartford - 12 March
Salesforce Lightning workshop Hartford - 12 MarchSalesforce Lightning workshop Hartford - 12 March
Salesforce Lightning workshop Hartford - 12 March
 
Sydney MuleSoft Meetup #12 2020204
Sydney MuleSoft Meetup #12 2020204Sydney MuleSoft Meetup #12 2020204
Sydney MuleSoft Meetup #12 2020204
 
#TDXRecap India tour
#TDXRecap India tour#TDXRecap India tour
#TDXRecap India tour
 
Salesforce Barcelona Admin Group, 26-Feb-2019
Salesforce Barcelona Admin Group, 26-Feb-2019Salesforce Barcelona Admin Group, 26-Feb-2019
Salesforce Barcelona Admin Group, 26-Feb-2019
 
Bringing Your Back Office Data To Life with Salesforce Connect
Bringing Your Back Office Data To Life with Salesforce ConnectBringing Your Back Office Data To Life with Salesforce Connect
Bringing Your Back Office Data To Life with Salesforce Connect
 
Control your world using the Salesforce1 Platform (IoT)
Control your world using the Salesforce1 Platform (IoT)Control your world using the Salesforce1 Platform (IoT)
Control your world using the Salesforce1 Platform (IoT)
 
JDF18 - Connecting the customer success platform
JDF18 - Connecting the customer success platformJDF18 - Connecting the customer success platform
JDF18 - Connecting the customer success platform
 
Lightning Connect: Lessons Learned
Lightning Connect: Lessons LearnedLightning Connect: Lessons Learned
Lightning Connect: Lessons Learned
 
ISV Monthly Tech Enablement (August 2017)
ISV Monthly Tech Enablement (August 2017)ISV Monthly Tech Enablement (August 2017)
ISV Monthly Tech Enablement (August 2017)
 

More from AaronLieberman5

Innovating on B2B Connectivity
Innovating on B2B ConnectivityInnovating on B2B Connectivity
Innovating on B2B Connectivity
AaronLieberman5
 
API Security - Everything You Need to Know To Protect Your APIs
API Security - Everything You Need to Know To Protect Your APIsAPI Security - Everything You Need to Know To Protect Your APIs
API Security - Everything You Need to Know To Protect Your APIs
AaronLieberman5
 
Integration Success with AWS and Boomi
Integration Success with AWS and BoomiIntegration Success with AWS and Boomi
Integration Success with AWS and Boomi
AaronLieberman5
 
Deep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed MicroservicesDeep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed Microservices
AaronLieberman5
 
Serverless & Serverless Devops: Scaling Together
Serverless & Serverless Devops: Scaling TogetherServerless & Serverless Devops: Scaling Together
Serverless & Serverless Devops: Scaling Together
AaronLieberman5
 
Deep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed MicroservicesDeep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed Microservices
AaronLieberman5
 
Extending The Power Of Anypoint Platform Using Anypoint Service Mesh
Extending The Power Of Anypoint Platform Using Anypoint Service MeshExtending The Power Of Anypoint Platform Using Anypoint Service Mesh
Extending The Power Of Anypoint Platform Using Anypoint Service Mesh
AaronLieberman5
 
Serverless Cloud Integrations Meetup: The Path Forward
Serverless Cloud Integrations Meetup: The Path ForwardServerless Cloud Integrations Meetup: The Path Forward
Serverless Cloud Integrations Meetup: The Path Forward
AaronLieberman5
 
Accelerate Your Development: CI/CD using AWS and Serverless
Accelerate Your Development: CI/CD using AWS and ServerlessAccelerate Your Development: CI/CD using AWS and Serverless
Accelerate Your Development: CI/CD using AWS and Serverless
AaronLieberman5
 
Serverless Cloud Integrations: The Path Forward
Serverless CloudIntegrations: The Path ForwardServerless CloudIntegrations: The Path Forward
Serverless Cloud Integrations: The Path Forward
AaronLieberman5
 
Layered API Security: What Hackers Don't Want You To Know
Layered API Security: What Hackers Don't Want You To KnowLayered API Security: What Hackers Don't Want You To Know
Layered API Security: What Hackers Don't Want You To Know
AaronLieberman5
 
Never Lose Data Again: Robust Integrations With MuleSoft
Never Lose Data Again: Robust Integrations With MuleSoftNever Lose Data Again: Robust Integrations With MuleSoft
Never Lose Data Again: Robust Integrations With MuleSoft
AaronLieberman5
 
Securing ap is oauth and fine grained access control
Securing ap is   oauth and fine grained access controlSecuring ap is   oauth and fine grained access control
Securing ap is oauth and fine grained access control
AaronLieberman5
 
What Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API SecurityWhat Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API Security
AaronLieberman5
 
Light Speed Integrations With Anypoint Flow Designer
Light Speed Integrations With Anypoint Flow DesignerLight Speed Integrations With Anypoint Flow Designer
Light Speed Integrations With Anypoint Flow Designer
AaronLieberman5
 
Sprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime FabricSprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime Fabric
AaronLieberman5
 
Sprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime FabricSprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime Fabric
AaronLieberman5
 
What Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API SecurityWhat Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API Security
AaronLieberman5
 
How to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
How to Expand Anypoint Platform's Capabilities by Developing Custom ConnectorsHow to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
How to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
AaronLieberman5
 
Connect the Dots: Logging and Custom Connectors
Connect the Dots: Logging and Custom ConnectorsConnect the Dots: Logging and Custom Connectors
Connect the Dots: Logging and Custom Connectors
AaronLieberman5
 

More from AaronLieberman5 (20)

Innovating on B2B Connectivity
Innovating on B2B ConnectivityInnovating on B2B Connectivity
Innovating on B2B Connectivity
 
API Security - Everything You Need to Know To Protect Your APIs
API Security - Everything You Need to Know To Protect Your APIsAPI Security - Everything You Need to Know To Protect Your APIs
API Security - Everything You Need to Know To Protect Your APIs
 
Integration Success with AWS and Boomi
Integration Success with AWS and BoomiIntegration Success with AWS and Boomi
Integration Success with AWS and Boomi
 
Deep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed MicroservicesDeep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed Microservices
 
Serverless & Serverless Devops: Scaling Together
Serverless & Serverless Devops: Scaling TogetherServerless & Serverless Devops: Scaling Together
Serverless & Serverless Devops: Scaling Together
 
Deep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed MicroservicesDeep Visibility: Logging From Distributed Microservices
Deep Visibility: Logging From Distributed Microservices
 
Extending The Power Of Anypoint Platform Using Anypoint Service Mesh
Extending The Power Of Anypoint Platform Using Anypoint Service MeshExtending The Power Of Anypoint Platform Using Anypoint Service Mesh
Extending The Power Of Anypoint Platform Using Anypoint Service Mesh
 
Serverless Cloud Integrations Meetup: The Path Forward
Serverless Cloud Integrations Meetup: The Path ForwardServerless Cloud Integrations Meetup: The Path Forward
Serverless Cloud Integrations Meetup: The Path Forward
 
Accelerate Your Development: CI/CD using AWS and Serverless
Accelerate Your Development: CI/CD using AWS and ServerlessAccelerate Your Development: CI/CD using AWS and Serverless
Accelerate Your Development: CI/CD using AWS and Serverless
 
Serverless Cloud Integrations: The Path Forward
Serverless CloudIntegrations: The Path ForwardServerless CloudIntegrations: The Path Forward
Serverless Cloud Integrations: The Path Forward
 
Layered API Security: What Hackers Don't Want You To Know
Layered API Security: What Hackers Don't Want You To KnowLayered API Security: What Hackers Don't Want You To Know
Layered API Security: What Hackers Don't Want You To Know
 
Never Lose Data Again: Robust Integrations With MuleSoft
Never Lose Data Again: Robust Integrations With MuleSoftNever Lose Data Again: Robust Integrations With MuleSoft
Never Lose Data Again: Robust Integrations With MuleSoft
 
Securing ap is oauth and fine grained access control
Securing ap is   oauth and fine grained access controlSecuring ap is   oauth and fine grained access control
Securing ap is oauth and fine grained access control
 
What Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API SecurityWhat Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API Security
 
Light Speed Integrations With Anypoint Flow Designer
Light Speed Integrations With Anypoint Flow DesignerLight Speed Integrations With Anypoint Flow Designer
Light Speed Integrations With Anypoint Flow Designer
 
Sprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime FabricSprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime Fabric
 
Sprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime FabricSprinting with Anypoint Runtime Fabric
Sprinting with Anypoint Runtime Fabric
 
What Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API SecurityWhat Hackers Don’t Want You To Know: How to Maximize Your API Security
What Hackers Don’t Want You To Know: How to Maximize Your API Security
 
How to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
How to Expand Anypoint Platform's Capabilities by Developing Custom ConnectorsHow to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
How to Expand Anypoint Platform's Capabilities by Developing Custom Connectors
 
Connect the Dots: Logging and Custom Connectors
Connect the Dots: Logging and Custom ConnectorsConnect the Dots: Logging and Custom Connectors
Connect the Dots: Logging and Custom Connectors
 

Recently uploaded

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 

Recently uploaded (20)

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 

Unlocking the Power of Salesforce Integrations with Confluent

  • 1. Unlocking the Power of Salesforce Integrations with Confluent Presented By: Big Compass and Confluent November 2020
  • 2. 1. Integration Pain Points With Salesforce 2. Ways to Integrate with Salesforce 3. Solving the Challenge of Unlocking Salesforce Data 4. Salesforce + Confluent 5. Use Cases: Extracting Data out of Salesforce with Confluent Agenda
  • 3. ● Consulting - specializing in integrations and related technologies ● Confluent partners ● We build connections between systems, applications, people, and ideas Big Compass
  • 5. ● Subscription tier limits ● API/Event limits ● Countless customizations ● Custom code/triggers to push data out of Salesforce ● Unexpected API Security Integration Pain Points With Salesforce
  • 6. ● How: Create triggers (PushTopics, Platform Events, CDC) in Salesforce to send events to a target ● When to Use: Great for real-time or event-driven use cases ● Skills: Deeper Salesforce knowledge required to create events ● Complexity: Higher - Create trigger in Salesforce and subscribe to the event using a tool like Confluent Pushing vs. Pulling Salesforce Data Considerations when choosing whether to send events out of Salesforce or use Salesforce APIs to pull data out of Salesforce Pushing Data ● How: Use Salesforce APIs to query data on-demand ● When to Use: Great for batch or on- demand use cases ● Skills: Some Salesforce knowledge is useful but not required ● Complexity: Lower - Least number of moving parts to the solution - Salesforce API, consumer/connector, target system Pulling Data
  • 7. Ways to Integrate with Salesforce Integration Method Description Good For Not Suitable For Batch Single event/request for many objects in Salesforce Easiest way to implement a solution for non-real-time solutions Constantly changing data Real-Time Create individual events for each change that occur in Salesforce Real-time use cases and handling events individually Less frequently changing data AppExchange Install out of the box apps in Salesforce to integrate with other systems Use cases that integrate with common systems Apps that are not built yet or use cases that go beyond the scope of connectors Custom Salesforce UI Expose your Salesforce data via custom UI built in Salesforce Organizations who are all-in on Salesforce Organizations who need the Salesforce data in another system
  • 8. ● Necessary to power downstream systems/applications ● Do you push data out of Salesforce, pull data out of Salesforce, or both? ● Pre-built connectors like Confluent’s Salesforce Connector can help speed up integration ● Not a trivial task without pre-built connectors! ○ Need to author custom code Unlocking Salesforce Data - Not a Trivial Task!
  • 10. Eventing Out of Salesforce Event Type Description Advantages Disadvantages Platform Events Custom pub/sub events 1. Highly customizable 2. Scalable 1. Requires Salesforce knowledge PushTopics PushTopic events track field changes in Salesforce records 1. Out of the box PushTopics can kickstart your implementation 1. Lower ability to customize the event Change Data Capture Notification on changes on an object 1. Easy to implement 1. Lower ability to customize the event Outbound Event Send SOAP message to web service 1. Can be used to event to a RESTful or SOAP endpoint 1. Lower ability to customize the event 2. Uses SOAP
  • 11. Unlocking Salesforce Data Publisher (Platform Event) Subscriber/Processing Target Systems Consumer Applications
  • 12. Use Case: Synchronizing SFDC Data Publisher (Platform Event) Subscriber/Processing Real-Time Target Data Storage Target Consumer Applications
  • 13. ● Pre-built source and sink connector to speed up integrations ● Once data is in Confluent, easy to use pub/sub model to process and route the data ● Joining from multiple sources ● Streaming data ● Duplicate detection ● Execute business logic ● Run SQL queries ● Detect events ● Much more than standard pub/sub Salesforce Integrations with Confluent
  • 14. ● Need to enrich and transform data from multiple sources before it is sent to a target system ● Clean your data ● Enrich your data ● Stage your data Use Case: Data Prep
  • 15. ● Gain valuable insight about your customers from multiple data sources ● Enrich data from various sources ● Generate actionable insight about your customers ● Predict customer actions in real-time Use Case: 360 Customer View
  • 16. ● Synchronize data with a target system to power a mobile application ● Two-way synchronization with source and sink connectors ● Duplicate detection ● Business logic ● Real-time streaming Use Case: Synchronizing SFDC Data
  • 17. Use Cases In Action: SFDC Sync From Multiple Sources
  • 18.
  • 19.
  • 20. http PUT cp01.woolford.io:8083/connectors/sfdc-contact-updates/config <<< ' { "confluent.topic.bootstrap.servers": "cp01.woolford.io:9092,cp02...", "confluent.topic.replication.factor": "1", "connector.class": "io.confluent.salesforce.SalesforcePushTopicSourceConnector", "kafka.topic": "sfdc-contact-updates", "name": "sfdc-contact-updates", "salesforce.consumer.key": "********", "salesforce.consumer.secret": "********", "salesforce.initial.start": "all", "salesforce.object": "Contact", "salesforce.password": "cxI3o3R$****", "salesforce.password.token": "iggX9UEPizeaizvpg02Vj****", "salesforce.push.topic.name": "ContactUpdates", "salesforce.username": "alex@woolford.io", "tasks.max": "1" }' PushTopic pushTopic = new PushTopic(); pushTopic.Name = 'ContactUpdates'; pushTopic.Query = 'SELECT Id, [...] FROM Contact'; pushTopic.ApiVersion = 50.0; pushTopic.NotifyForOperationCreate = true; pushTopic.NotifyForOperationUpdate = true; pushTopic.NotifyForOperationUndelete = true; pushTopic.NotifyForOperationDelete = true; pushTopic.NotifyForFields = 'All'; insert pushTopic;
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26. mysql> use healthcare; Database changed mysql> select * from patient; +----+-----------+------------+--------------+------------+ | id | firstname | lastname | enroll_start | enroll_end | +----+-----------+------------+--------------+------------+ | 1 | Donald | Trump | 2019-04-14 | 2020-04-13 | | 2 | Vladimir | Putin | 2019-05-20 | 2020-05-19 | | 3 | Boris | Johnson | 2019-01-24 | 2020-01-24 | | 4 | Mark | Zuckerberg | 2019-03-03 | 2020-03-02 | | 5 | Benjamin | Netanyahu | 2019-08-24 | 2020-08-23 | | 6 | Jair | Bolsonaro | 2019-09-19 | 2020-09-18 | | 7 | Farty | McFartface | 2019-08-23 | 2020-08-22 | +----+-----------+------------+--------------+------------+ 7 rows in set (0.00 sec)
  • 27. http PUT http://cp01.woolford.io:8083/connectors/mysql-cdc-healthcare/config <<< ' { "name": "mysql-cdc-healthcare", "connector.class": "io.debezium.connector.mysql.MySqlConnector", ... "include.schema.changes": "false", "snapshot.mode": "when_needed", "table.whitelist": "healthcare.patient", "transforms": "unwrap,HoistField", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.HoistField.type": "org.apache.kafka.connect.transforms.ExtractField$Key", "transforms.HoistField.field": "id" }'
  • 28.
  • 29. CREATE STREAM AILMENT ( id STRING, ailment VARCHAR ) WITH ( kafka_topic='ailment', value_format='JSON', KEY='id' ); CREATE TABLE PATIENT ( id STRING, firstname VARCHAR, lastname VARCHAR, enroll_start BIGINT, enroll_end BIGINT ) WITH ( kafka_topic='deepthought.healthcare.patient', value_format='JSON', KEY='id' );
  • 30. select ailment.id as id, ailment, firstname, lastname, case when ailment.rowtime > enroll_start * 86400 * 1000 and ailment.rowtime <= enroll_end * 86400 * 1000 then true else false end as covered from ailment inner join patient on ailment.id = patient.id emit changes;
  • 33. References ● Salesforce API Documentation ● Comparing PushTopics, Platform Events, and Change Data Capture ● External Objects ● Confluent KSQL ● Kafka Transformations

Editor's Notes

  1. No one talks about pain points of integrating with Salesforce! A Google search of “Salesforce integration pain points” returned nothing useful. The top hit was 3 pain points of user engagement with Salesforce, and then how Salesforce solves various pain points for an organization. So if we look at Salesforce from the perspective of integrations, we see some integration pain points Subscription tier limits Lack of features at lower subscription tiers Security - not industry-standard (unexpected) security model with Salesforce tokens For API requests This all leads to Data locked up in Salesforce and makes it more difficult than it should be to synchronize target systems with Salesforce
  2. External Objects are also a way to integrate external data with Salesforce https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_external_objects.htm
  3. Getting the data out of SFDC is not a trivial task! We have talked about ways to integrate with Salesforce, but you can already see that there are quite a few considerations for integrating with Salesforce It’s necessary so often to power downstream systems/applications Salesforce is acting as the enterprise’s data hub more often these days So first, you need to figure out if you pull data out of Salesforce, push data out of Salesforce, or do both in a hybrid model Confluent’s source and sink connectors can greatly help speed up Salesforce integrations It takes custom code otherwise to integrate with Salesforce So now we’ll talk more about real-time use cases because that is what so many organizations are moving towards
  4. Real-time events are the next trend. Gone are the days of batch processing. Real-time events play well with an event driven architecture and microservices based approach, both of which are great architectures for the modern enterprise And the more we can deliver real-time data, the better insights we can get, the quicker we can take action, and the more we can unlock the potential of the data running through our systems
  5. When you want to event out of Salesforce, you have some considerations to take. https://www.thecloudfountain.com/platform-events-vs-push-topic-vs-change-data-capture/ https://trineo.com/blog/2019/04/push-vs-cdc https://developer.salesforce.com/docs/atlas.en-us.integration_patterns_and_practices.meta/integration_patterns_and_practices/integ_pat_data_virtualization.htm Under the hood, these events use the Salesforce event bus for the most part Platform events Highly scalable and my recommended way to event out of SFDC Create using Apex code PushTopics Define using a SOQL query Events the results of the SOQL query and triggered based on changes to an object CDC Publishes new or changed records on an object Outbound event SOAP style eventing Been around for a long time
  6. Show: KStreams with queryable state Punctuate Processor API and event collation SFDC connector, both ways
  7. Pre-built source and sink connector to speed up integrations Speeds up your integration and makes it easy to get the data into a tool that unlocks processing potential Once data is in Confluent, easy to use pub/sub model to process and route the data Easily add new subscribers Want to scalably event out to a database, cold storage, Hadoop, and a consumer application? Hook into Kafka topics Joining from multiple sources Use SQL-like queries to join data from multiple sources for data enrichment Streaming data Your latency is only as good as your processing power With Confluent, reliably scale to handle the largest workloads How you will accomplish truly real-time use cases Duplicate detection Execute business logic Such as detect when a Salesforce lead needs to be converted to an opportunity based on JSON messages Run SQL queries To find trends and events Detect events Execute actions based on SQL queries Much more than just pub/sub
  8. Use Confluent as your data stage to prep your data for downstream systems
  9. Combine data from multiple data sources easily with Confluent using SQL queries on a stream Enrich data, and send it to your target system of choice with pre-build sink connectors
  10. How might all of those use cases be combined and look from an architecture perspective?
  11. Kafka vs traditional messaging systems: fan-out scale-out
  12. Add reference to talk where that describes how to make your own connector.
  13. List all the fields for a SFDC object: String SobjectApiName = 'Contact'; Map<String, Schema.SObjectType> schemaMap = Schema.getGlobalDescribe(); Map<String, Schema.SObjectField> fieldMap = schemaMap.get(SobjectApiName).getDescribe().fields.getMap(); for(String fieldName : fieldMap.keyset() ) { system.debug(fieldMap.get(fieldName) + '|' + fieldMap.get(fieldName).getDescribe().getType()); } In case it’s necessary to delete the PushTopic: List<PushTopic> pts = [SELECT Id FROM PushTopic WHERE Name = 'ContactUpdates']; Database.delete(pts);
  14. select ID_RESP_H, getGeoForIp(ID_RESP_H) from CONN emit changes;
  15. https://github.com/alexwoolford/snowplow-c360-mongo
  16. Source code: https://github.com/alexwoolford/confluent-healthcare-coverage-demo
  17. Message has a key, which is important because it will later be used to perform a join.
  18. https://github.com/alexwoolford/confluent-healthcare-coverage-demo INSERT INTO patient (firstname, lastname, enroll_start, enroll_end) VALUES ('Kim', 'Jong-Un', NOW(), DATE_ADD(NOW(), INTERVAL 1 YEAR)); kafka-console-consumer --bootstrap-server localhost:9092 --topic deepthought.healthcare.patient --from-beginning
  19. About binlog: whenever the master's database is modified, the change is written to a file, the so-called binary log, or binlog. The master has a thread, called the dump thread, that continuously reads the master's binlog and sends it to the slave. Mention: when_needed snapshot mode helps in the event the connector/DB can’t communicate for a while and it needs to “catch up” (i.e. not all the data is available in the binlog). a connector can have multiple SMT’s applied to it. In this case, the unwrap SMT is pulling out the data from the payload (removes all the schema), and the HoistField is taking the id and putting it in the key so it can later be joined.
  20. Mention: connect is a fault-tolerant framework. Note that there are two (or more) nodes. If a node were to fail, its tasks would resume on another node.
  21. Mention: Epoch days for dates, hence the multiplication to get epoch millis select ailment.id as id, ailment, firstname, lastname, ailment.rowtime as ailment_ts, enroll_start * 86400 * 1000 as enroll_start, enroll_end * 86400 * 1000 as enroll_end, case when ailment.rowtime > enroll_start * 86400 * 1000 and ailment.rowtime <= enroll_end * 86400 * 1000 then true else false end as covered from ailment inner join patient on ailment.id = patient.id emit changes;