High performing Salesforce Data Connectors
•Download as PPTX, PDF•
1 like•389 views
In this webinar, my colleague and I spoke about how enterprises can leverage Salesforce Data Connectors (ODBC, JDBC) in their enterprise tools such as: 1. Analytics tools: Tableau, Power BI, Excel, YellowFin, SAP Lumira, SAS and more 2. ETL Tools: Informatica Power Center, AWS Glue, Oracle Database Gateway, Linked Server 3. Big Data tools: Apache Sqoop, Apache Nifi and Apache Flume 4. Data Integration Tools: SSIS, MuleSoft, SAP Data Hub Here is the abstract: Named IDC’s #1 CRM provider for the fifth straight year, Salesforce plays a large role in empowering digital transformation efforts at companies across the world. Unsurprisingly, businesses increasingly want access to their Salesforce data for analytics and other mission-critical processes. Enterprises often turn to homegrown Salesforce connectors to deliver Salesforce data to other business apps. However, building a custom data connector for Salesforce isn’t easy—it can take months or even years. Join our upcoming webinar, where we dive into the different considerations for developing a Salesforce data connector. We’ll discuss: What is a Salesforce data connector? Which language should you pick—SQL, SOQL or SAQL? What goes into a Salesforce connector (metadata, SQL processing, throttling, etc.)? When should you build vs. buy? Plus, we’ll wrap things up with a demo of how to pull Salesforce data into Tableau. Watch the webinar today! Speakers: Nishanth Kadiyala, Senior Product Marketing Specialist, Progress Avadhoot Kulkarni, Principal Software Engineer, Progress
Recommended
More Related Content
What's hot
What's hot (19)
Similar to High performing Salesforce Data Connectors
Similar to High performing Salesforce Data Connectors (20)
Recently uploaded
Recently uploaded (20)
High performing Salesforce Data Connectors
- 1. High-Performing Salesforce Connectors Nishanth Kadiyala Product Marketing Manager Avadhoot Kulkarni Product Engineering, Product Owner
- 2. 2© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Audio Bridge Options & Question Submission
- 3. 3© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Agenda Overview of Salesforce Connector SQL vs SOQL vs SAQL Anatomy of a Salesforce Connector Build vs Buy Demo
- 4. 4© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Overview of Salesforce Connector
- 5. 5© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Salesforce is the #1 CRM Source 1: https://www.salesforce.com/campaign/worlds-number-one-CRM/ Source 2: https://expandedramblings.com/index.php/salesforce-statistics/ Source 3: https://www.businessinsider.com/salesforces-ceo-mark-benioff-is-hoping-to-double-sales-2019-1 1 FY18 Stats: • 150,000+ Customer • FY18 Revenue: $10.5B (25% YoY) Projections: • FY19 Revenue: $13B+ • FY 22 Revenue: $20B+ 2 3
- 6. 6© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Sample Salesforce Workflow Source: https://www.crmevangelist.com/2017-05-30_11-55-32/
- 7. 7© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Source: http://www.cubiccompass.com/blog
- 8. 8© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Built-in Reporting and Analytics Capabilities: • Reporting on sales pipeline • Providing a snapshot of sales and operational data • Quantifying customer inquiry requests Challenges: • Display limitations of rows of data in reports • Limited cross-object reporting capabilities • Difficulties in providing historical or contextual analysis for the organization Picture source: https://twitter.com/klipfolio/status/789215947890561024
- 9. 9© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Why Salesforce Data Connector? Enterprise Analytics ETL App IntegrationComplex Queries Data Integration Big Data
- 10. 10© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.Source: Ice Age
- 11. 11© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.Source: Ice Age
- 12. 12© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. SQL vs SOQL / SAQL
- 13. 13© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. SOQLSQL • SQL is a universal language • Supported by all popular analytics tools • Queries run on a relational view SAQL • SOQL is Salesforce’s proprietary query language • Supported only in the Salesforce ecosystem: Apex, SFDC REST API, Force.com IDE • Queries run on Salesforce objects • SAQL works only within Wave / Einstein analytics platform • Supported through Analytics REST APIs, dashboards, lenses • Queries run on analytics datasets
- 14. 14© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. SOQL capabilities SOQL is a SELECT-like query language for accessing objects Field selection Filters Limited joins Limited Group Bys Aggregates Order By
- 15. 15© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Filters SQL SOQL Multi-select picklists Use equivalent string functions …FROM Account WHERE MenuItem = ‘AAA’ …FROM Account WHERE MenuItem != ‘AAA’ …FROM Account WHERE MenuItem includes ‘AAA’ …FROM Account WHERE MenuItem excludes ‘AAA’ …FROM Account WHERE MenuItem = ‘AAA;BBB’ Allows specifying timestamps using escapes …FROM Account WHERE CreatedDate = {ts ‘2010-08-10 09:15:37’} …FROM Account WHERE CreatedDate = {ts ‘2010-08-10 09:15:37’} …FROM Account WHERE CreatedDate = 2010-08- 10T09:15:37
- 16. 16© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Joins SQL SOQL Joins across any tables or objects SQL allows JOINs of all kinds These examples use a foreign key SELECT * FROM Account A INNER JOIN Contacts C on A.ID = C.AccountID SELECT * FROM Account A FULL OUTER JOIN Contacts C on A.ID = C.AccountID SELECT * FROM Account A FULL OUTER JOIN Contacts C on A.ID = C.AccountID WHERE A.ID IS NULL or C.AccountID IS NULL This example joins on an arbitrary field SELECT * FROM Account A INNER JOIN Contacts C ON A.State AND C.State WHERE A.ZipCode = C.ZipCode SOQL only allows joins on foreign keys through relationship queries This is achieved through either dot notation following child-to-parent relationships: SELECT Contact.Name, Contact.Account.Name FROM Contact WHERE… Or through sub-queries when following parent-to-child relationships SELECT Account.Name, (SELECT Contact.FirstName, Contact.LastName FROM Account.Contacts) FROM Account WHERE…
- 17. 17© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Aggregates, Group By, Order By SQL SOQL Supports Aggregates, Group By, Order By SELECT SUM(Amount), FiscalQuarter, Probability FROM Opportunity GROUP BY FiscalQuarter, Probability Similar to SQL Unlimited Aggregation SELECT SUM(Amount), FiscalQuarter, Probability FROM Opportunity GROUP BY FiscalQuarter, Probability SOQL does not allow a query containing a Group By to return more than 2000 records. Add a Having or Where clause to limit the results. SELECT SUM(Amount), FiscalQuarter, Probability FROM Opportunity GROUP BY FiscalQuarter, Probability HAVING SUM(Amount) > 10000
- 18. 18© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Inserts, Updates, Deletes SQL SOQL Syntax for DML operations In Salesforce, Insert, Update and Delete operations are performed through APIs Writes in single operation SQL simplifies the process as a single query can modify the data For updates and deletes, a SOQL query must be used to identify the records to modify, so it’s a 2-step process Full CRUD support
- 19. 19© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Source: Ice Age
- 20. 20© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Anatomy of a Salesforce SQL Connector
- 21. 21© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Why SQL – Go Generic If you have app which need to connect to multiple sources • Flexibility • Scale Providing Salesforce connectivity to your customers • The kind of operations and queries scale very fast • No two use cases are exactly same
- 22. 22© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Popular options for SQL access to Salesforce Direct SQL Access Progress DataDirect ODBC and JDBC drivers for direct SQL connectivity [Download] bit.ly/SQL2SF app Indirect SQL Access Extract data using Salesforce data loader, or other third- party tools, into an intermediate RDBMS
- 23. 23© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. DataDirect architecture for direct SQL access to Salesforce Progress DataDirect ODBC/JDBC Connectors Schema Manager Your Application Salesforce.com Schema User Defined Schema Driver uses SOAP API Bulk API Metadata API
- 24. 24© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. SQL Engine • SQL Translation • Resultset Shaping • Local Processing Schema Manager • Normalization • Metadata Information • Local Tables Limits • Paging • Throttling • Idle Session timeouts • Error Mapping and handling Security • Encryptions • Authentications • Proxy Performance • Bulk vs Non- Bulk • Result Caching • Multi-threading Components of SQL Connector to Salesforce
- 25. 25© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Building Contextual Relational Mapping • All Complex field become child objects Winding • Complex field get promoted to become parent fields Unwinding • Convert to Comma separated values string Flattening • Too complex to normalize, map as JSON type JSONify
- 26. 26© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Exposing Metadata & Connector Capabilities ODBC • SQLTables • SQLColumns • SQLProcedure s (reports) • GetInfo • ... JDBC • getTables • getColumns • getProcedures • Database Metadata • ...
- 27. 27© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Supporting Local Objects Memory Tables • Stores temp results for transformation • Fast performance • Accessible for the same connection scope Disk Tables • Persisted Temp tables • Can be accessed across connections (same schema map) System Information • Cache Information • Usage Information (Sessions, API Calls) • Etc.
- 28. 28© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Adding SQL Engine Capabilities SQL Translation Optimizations Post Processing
- 29. 29© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Handling Limits & Exceptions Result Paging Session Re-auth API Throttling Error Code Mapping
- 30. 30© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Security Considerations Encryptions • TLS 1.2 Authentication • User ID • OAuth 2.0 • SSO Proxy • HTTP Forward Proxy
- 31. 31© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Performance Bulk vs Non-Bulk • Bulk is not always performant. As its name suggests its only for bulk job. Intelligence should be in the connector to decide what is the most optimal way to complete the user request. Result Caching • Cache results whenever required and fetch incrementally to further optimize fetch time Multi-threading • Multi-threading, in the cloud deployment world, multi-tenant multi-user application is a very common theme. • Connector can reduce the synchronization overhead to minimum
- 32. 32© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Build vs Buy
- 33. 33© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Faster time to Market
- 34. 34© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Day-1 Support Policy
- 35. 35© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Enterprise-ready Connectors Built by Experts
- 36. 36© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Security Vulnerability Policy PRIORITY* TIME GUIDELINE VERSION(S) High Risk (CVSS 8+ or industry equivalent) 30 days Active (i.e. latest shipping version) and all Supported versions Medium Risk (CVSS 5-to-8 or industry equivalent) 180 days Active (i.e. latest shipping version) Low Risk (CVSS 0-to-5 or industry equivalent) Next major release or best effort Active (i.e. latest shipping version) * Priority is established based on the current version of the Common Vulnerability Scoring System (CVSS), an open industry standard for assessing the severity of computer system security vulnerabilities. For additional information on this scoring system, refer to https://en.wikipedia.org/wiki/CVSS.
- 37. 37© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Trusted Partner for All Connectivity Challenges
- 38. 38© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. 96 of Fortune 100 Enterprises leverage our connectivity solutions ISVs distribute DataDirect drivers 350+ 10,000+ Enterprises rely on Progress for their connectivity Leading BI vendors embed DataDirect drivers. 8 of the 9 © 2018 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Users seamlessly access data daily enabled by DataDirect 1M+ 38 Mission: To give Enterprises and ISVs frictionless access to the information they need by harmonizing access to the world’s disparate data.
- 39. 39© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Expansive List of DataDirect Connectors Big Data/NoSQL • Apache Hadoop Hive • Cloudera • Hortonworks • MapR • EMR • BigInsights • Pivotal HAWQ • MongoDB • Cassandra • DataStax • Apache Spark Data Warehouses • Amazon Redshift • SAP Sybase IQ • Teradata • Pivotal Greenplum Relational • Aurora • Clipper • dBase • EnterpriseDB • IBM DB2 • IBM Informix • FoxPro • Microsoft Analytics Platform • Microsoft SQL Azure • MySQL • Oracle DB • Pervasive SQL • PostgreSQL • Progress OpenEdge • Progress Rollbase • SAP Sybase • SQL Server SaaS/Cloud • Google Analytics • IBM DashDB • IBM DB2 Hosted • Salesforce.com • FinancialForce • Veeva CRM • ServiceMAX • Any Force.com App • Azure SQL Data Warehouse • Microsoft Dynamics CRM • Oracle Eloqua • Oracle ADWC • Oracle Cloud Service • Oracle Sales Cloud • Oracle Service Cloud • Progress Rollbase • SugarCRM EDI/XML/Text • EDIFACT • EDIG@S • EANCOM • X12 • IATA • Healthcare EDI: X12, HIPAA, ICD-10, HL7 • Flat files: CSV, TSV, dBase, Clipper, Foxpro, Paradox • Text Files Custom Connectivity • Autonomous REST Connector • OpenAccess SDK • SequeLink
- 40. 40© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.Source: Ice Age
- 41. 41© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Demo
- 42. 42© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. 8.0 is 300% faster compared to 7.1 for 400K records. Scales better with large data. Better optimized for multi-threaded applications. Bulk Fetch : Throughput Comparision
- 43. 43© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Resources Downloads: https://www.progress.com/connectors/salesforce Tutorials: • ODBC – https://www.progress.com/blogs/sas-odbc-access-to-salesforce-sql-server-and-marketo – https://www.progress.com/blogs/python-connectivity-to-salesforce-eloqua-or-google • JDBC – https://www.progress.com/tutorials/jdbc/connect-to-salesforce-from-aws-lambda-using-jdbc-driver – https://www.progress.com/blogs/importing-salesforce-data-into-amazon-s3-via-jdbc-driver Case study: https://www.progress.com/customers/microstrategy You can email us at: nkadiyal@progress.com or akulkarn@progress.com
- 44. 44© 2019 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. QnA
Editor's Notes
- Welcome everyone! My name is Nishanth Kadiyala. I am a Product Marketing Manager at Progress and it’s my pleasure to introduce our latest Salesforce connectors today. I also have Avadhoot with me. Avadhoot, do you want to do a quick intro? Hi, I am Avadhoot Kulkarni. I am Product Owner at Progress and was involved in crafting the enhanced Salesforce connectors which we are introducing today. So a proud moment for me.
- Before we begin, a few housekeeping items: This webinar is being recorded. You will receive a link by email after the recording becomes available. When you joined the webinar, by default the audio bridge connected your device to the VoIP option. If you prefer to listen using your phone, select the “telephone” option and an operator will walk you through connecting to the bridge. We’ll hold Q&A after the presentation. So, I encourage you to submit your questions as we go through the Questions Tab on the right side of your screen. We’ll respond to as many as we can. If we run out of time before your question is answered, we will reach out to you afterwards. If you experience technical issues or need assistance, please enter a message and we will respond to you as soon as possible.
- Alright…Today, we will walk you through the typical journey of a product manager or an enterprise architect… We will answer the basic questions like Why salesforce connector…which options to pick from when you are building a salesforce connector…how to build one….why someone would buy instead of building it for themselves and finally we will demo our Salesforce connector
- So, lets get started…
- Salesforce is the world’s #1 CRM both by revenue as well as market share. According to their FY18 report, over 150000 enterprises are leveraging Salesforce in their organization. They had over $10B revenue which is like 25% YoY growth! As you can see in the picture to the right, Salesforce owns 1/5th of the market. And Salesforce is projected to hit the 20B mark in 2022! So, be assured that salesforce support requests will only continue to grow over the next few years!
- Sales representatives use Salesforce to effectively manage their customer relationships. For those of you who may have never used Salesforce, here is a quick example. Within Salesforce…Each company is mapped to an account….all related leads are saved as contacts…you can access product, pricing and account specific discount information…manage the entire sales cycle from this single portal. You can share contracts, track orders, process invoices, access helpdesk tickets and much more!
- And this is what it looks like in the backend. Salesforce saves data in the form of objects behind the scenes. These objects are the equivalents of a database table, and they are designed contextually to support the various workflows in Salesforce. While they resemble tables, they are not necessarily normalized….like I said they are primarily designed to support CRM operations….CRM stands for Customer Relationship Management As you can see, the objects are wildly interconnected and this ecosystem can be further expanded using what we call external objects. External objects are nothing but, external data sources that can be made to look like native salesforce objects using a tool called Salesforce Connect. I won’t get into the details, but that is how Salesforce has become the holy grail for sales reps.
- So, as you can imagine, large amount of data is being produced in Salesforce everyday and naturally the managers would like to get insights on this data. So, Salesforce offers built in reporting capabilities to help answer basic questions such as: how much did a sales rep or a team close in a given region or quarter….or something like how much pipeline did we close between march and august 2016. So..yeah….it can answer the basic questions….However, this module is not suitable for more complex analysis: Example: Trying to identify upsell opportunities based on service center calls and historical upgrade trends Or identifying enterprises similar to an existing account so that you can run targeted campaigns… Or if you are trying to get a more holistic view of the customer…Salesforce doesn’t have an easy way to run reporting on external data sources… In such cases you need custom development using the different Salesforce APIs.
- But…Salesforce’s SOAP and REST endpoints are built for simple integration needs… When you need raw data in larger chunks, REST and SOAP won’t be good enough, you will need a Salesforce Data Connector. A data connector is something that lets you query against the underlying datasets and makes it possible to move large amount of data. You can see here some of the use cases for a Salesforce data connector. Analytics tools require access to huge datasets in order to provide insights. Tools like Tableau, Cognos, OBIEE, SAS, SAP Data Hub and all others need a Salesforce data connector Similarly ETL tools such as Informatica Power Center, SSIS, Oracle Gateway, Linked Server, AWS Glue also need such a connector On the Big Data side I see customers using data connectors with tools like Apache Sqoop, Nifi and Flume So on and so forth, just remember that if your app needs a lot of data from other data sources or needs to run complex queries on external data sets, then data connectors is the right way to go about it.
- Without the data connector…this is what the current situation looks like. For those of you who did not recognize the Squirrel in the picture….this is from the movie series Ice Age. And that is Scrat, the squirrel which cannot get hold of the Acorn to the right no matter how hard it tries.
- And that is how developers exactly feel about Salesforce data….There is this rich data about customers inside the CRM application, but they cannot get access to it in the way they like to access data. By the way…this Scrat sequence is hilarious …if you haven’t watched it, I will recommend that you watch it later today.
- So, now…lets talk about how can you make salesforce data available to these developers in the format they prefer…. You could build a universal connector using the popular SQL language or you can use a Salesforce proprietary language. In this section, Avadhoot and I will compare these different options
- Beginning from the left…SQL is structured query language and most developers are familiar with it. ODBC and JDBC are the more popular standard SQL languages that are supported by most analytics tools. Tableau, PowerBI, Excel, TIBCO, SAS, SAP Lumira, OBIEE, Cognos and more…all of them support ODBC or JDBC. With SQL you run queries on top of a relational view. Moving to SOQL….SOQL stands for Salesforce Object Query Language. SOQL is Salesforce’s proprietary language that provides the means by which Salesforce can prevent queries from adversely affecting customers who rely on shared resources. It has been designed to help salesforce developers build complex reports and custom applications. SOQL can be accessed from Apex, from Salesforce REST API or Force.com IDE. With SOQL… queries run against Salesforce objects. Even external data can be accessed using Salesforce Connect…although there are several limitations to it. And the next one is SAQL…SAQL is Salesforce Analytics Query Language. Some of you may have heard of Wave or Einstein analytics or Analytics Cloud. They are all…more or less a BI tool that turns salesforce data into easy-to-understand, mobile-friendly reports. I’d say one of the major differences between Salesforce reports and this module is that, Einstein Analytics allows users to easily import external data. It even offers connectors for Dynamics CRM, NetSuite, RedShift, BigQuery and more. But the biggest roadblock is that SAQL doesn’t operate on the typical Salesforce Objects. It needs analytics datasets for running its queries. So, the developers are either constrained by the existing datasets available as part of this module or need they to create custom datasets. Since salesforce objects are much more straight forward, standardized and easy to deal with than custom datasets, enterprises choose SOQL more often than SAQL and sometimes a bit of both. So, today, we will look specifically at how SOQL compares to SQL.
- SOQL is basically a select only query language designed solely for accessing data from Salesforce objects. While it has been designed to look and feel like SQL, there are several limitations to it if we look closely at how the Joins, Group Bys and Filters operate. Avadhoot…Would you like to go through the details?
- SOQL is more reach in not tableau data handling when the relationship is clear. E.g. there are set of functions to look for specific items in the PickList. In SQL, the pickList type columns becomes a VARCHAR field and need to use standard SQL string function to process that information. Similarly, SOQL has syntax to travers the polymorphic relationship, In this example, 'What' is a field in 'Event' object with in itself have multiple fields which include 'Type'. SOQL syntax make it easy to traverse such relations. SQL need to stick with standard JOIN syntax to handle such situations. SQl can handle the Escape Syntaxes where SOQL can't.
- We wont be able to do a JOIN queries in SOQL if there is no parent-child relationship between the objects. But some analytical process might have a need to do that. SQL join syntax do not worry about object relationship.
- Both SQl and SQOL support aggregates but there are some limits on the result set side that one can have in SOQL.
- SOQL syntax is only for select. For all other operations we need to use APIs. Also, updates and delete are 2 step process. One need to do SOQL first to fetch the information about all the affected records using the same filters to get the IDs, and then execute Update/Delete on the rows with those IDs.
- Humm.... SOQL I need to learn....But, I know SQL, I can do it!
- Architecture – Where will the driver sit? Embeddable (ISVs) as well as easily deployable component (for DEUs)…Make sure they understand that this is not a SaaS offering. Its easily embeddable and distributable for ISVs which can be deployed on-premise (behind firewall) or in a cloud application. Direct End users - The solution can be deployed on any Windows/Linux system.
- Salesforce API/WSDL provides you some metadata information. There are some and some times different expectations from ODBC and JDBC specification side on how this metadata information should be returned to the user. Almost all the BI/ETL tools which use standard ODBC/JDBC connectivity rely on the metadata and driver capability information to work properly with them.
- Accessing Salesforce objects is one part, but the users will also need some local objects to provide more flexibility to store temp data. In memory for faster processing but also a way to persist it to share across multiple sessions. Also other critical information about the connector as system, diagnostics etc.
- SQL Translation – Parsing, validating the SQL Syntax and converting it to SOQL/APIs. Optimizations – Retrieve Reader where IN clause of ID columns, etc. Local Processing – Not all SQLs can be translated to SOQL. You need capability to generate the SOQL which would be most appropriate and then local processing of the result to finetune. e.g. JOIN between objects without a relation, one need to fetch both the object results separately and then join then in memory in connector. Disk Caching – We cannot hold millions of records in memory and/or do Local Processing without worrying about running out of memory. Can we?
- Result Paging – Salesforce will not return all the records at the same time. So need to implement a mechanism to process and ask for more. Salesforce session gets expired after certain idle period (configurable in Salesforce instance), if your connector can detect such expiration of session, re-connect and re-try, it reduced lot of overhead from the application using this connector. May not apply for REST, but there is a daily SOAP API limit per user by Salesforce. So API is currency. The connector should use it carefully, and do some throttling when a single request is trying to consume too much of it.
- Encryptions – Need to be aware and up to date with the different encryption requirements of Salesforce but also on the news on the security side. Proxy – Security teams do not want their servers exposed to internet directly, so proxy connections are basic necessity when one wants to connect to cloud service like Salesforce. Authentication – There will always a strong requirement of supporting another authentication mechanism, as there would someone from you customers/prospects base already using it.
- Bulk is not always performant. As its name suggests in only for bulk job. User do not have a way to guess how big a result would be for certain query, the intelligence should be in the connector to decide what is the most optimal way to complete the user request. Connector need to be intelligent to cache results whenever required and just do incremental fetches to further optimize the select time especially with the data sources hosted on cloud like Salesforce. Multi-threading, in the cloud deployment world, multi-tenant multi-user application is a very common theme. If your connector can reduce the synchronization overhead to minimum, it’s a great value to the user.
- That was a great deep-dive. Thank you, Avadhoot. So…now let’s say you have decided to use a Salesforce SQL Connector with your tool….but you are faced with the typical build vs buy dilemma. Sometimes organizations develop things in-house and usually, the reasons vary from Cost savings, to Control on timeline to tighter integration with their product. So, today, we will look at what you’d get out of partnering with Someone like Progress instead of building it internally
- Progress’ Salesforce Connector has all the bells and whistles that Avadhoot mentioned earlier. But When you look at cost of developing such a high quality and reliable connector, the time component is often neglected. I can bet that at any given time of the year, all of you have a list of new features and capabilities on your roadmap, but you do not enough time to get to all of them. With progress, you accelerate your time to market. With the respect to the speed with which we deliver new drivers. a product owner of a top BI vendor said the following: “We estimated it would have taken us at least six months to build a Salesforce API connector, and then we would have been responsible for maintaining the connector. We concluded that our time and resources were far better spent developing differentiating features for our own applications. We were able to decrease our time to market by 5 times using DataDirect versus building our own connector. And we are able to build a proof of concept within just hours to demonstrate the value to our customers.” It is not just that one vendor but…Over dozen BI vendors are leveraging our salesforce connector today. So, why build and maintain it, when you can just buy it a number of your peers.
- A big challenge with cloud data sources such as Salesforce is that they update their software very frequently. Salesforce for example has 3 to 4 releases every year. If you were to build your own connector, the maintenance costs can quickly tear the roof apart. But that is our business at Progress DataDirect. We live and breath connectivity. We have a day-1 support policy, which guarantees that our drivers will start getting certified with the latest APIs as soon as they become available. Many of our partners say that this is invaluable when they are dealing with the big data and cloud ecosystem that is constantly changing. In the case of Salesforce, they are at version 45 and our drivers are certified with version 44. We are planning to soon release hotfix a for version 45 as well.
- Next…it is the quality of our connectors….We have been building connectors for around 3 decades and even the Salesforce Connectors have been in the market for about a decade now. Through the years, we have incorporated a tested and perfected process, that allows us to churn out world class connectors at a maniacal pace. When I say our connectors are enterprise-ready, I mean that: 1. We support all the niche features of a driver that make it secure and high-performing. For example, we support connection pooling, SQL Upleveling, Bulk Load, security mechanisms like Kerberos, OAuth, NTLM and more 2. We create drivers that are compatible with a wide variety of platforms such as AIX, HP, Linux, Solaris and Windows. And finally 3. We run millions and millions of tests to ensure high quality and seamless compatibility with all the popular analytical tools out there.
- Moving on to the next one…With the deluge of data breaches in the recent past, it has become pertinent to react to security risks sooner than later. We have a security vulnerability policy in place to address such exact risks. Based on CVSS score, we categorize risks into high medium and low and address them with the appropriate measures with in 30 days…180 days or in the next major release. We are even part of the TSANET Multi-vendor channel support that gives us access to companies such as Microsoft, Oracle, TIBCO, SAS, SAP, IBM and other leading technology companies . So, as part of the TSANET, we all work together to address such security risks.
- And most importantly you get to rely on a single, trusted partner for all your connectivity needs. Progress’ award-winning technical support team is available round the clock all 7 days a week. I am proud to let you know that in our most recent customer satisfaction survey, a staggering 91% of DataDirect partners reported a positive satisfaction. In the words of Demuth, the Director of Partner Engineering at MicroStrategy: “Our clients are constantly asking for new data source support, and we know that DataDirect will quickly make an ODBC driver available. The Salesforce and Hive drivers are only the most recent in a long series of such innovations. They position us very favorably against the competition.” And it is not just MicroStrategy, we are the leaders in this space….
- 350+ ISVs including 8 of the 9 leading BI vendors and over 10000 enterprises rely on Progress DataDirect for their connectivity needs. We are the co-founders of ODBC and a key member of the JDBC expert group. Like I mentioned earlier…We have been in this space for over 30 years and we incessantly innovate to solve all connectivity challenges related to relational, big data and cloud systems.
- Here are a few popular data sources that we support. We support a wide variety of Hadoop distributions, We support MongoDB and Cassandra on the NoSQL side. We support all major relational sources and even their cloud-hosted versions. We support Salesforce, Eloqua, Oracle Service Cloud, Dynamics CRM Google Analytics and many more on the SaaS side. And a special call out for Autonomous REST Connector and OpenAccess SDK that make it extremely easy for developers to develop custom connectors. If you have a proprietary API, these products can help you SQL enable it with as less as 0 lines of code…
- And one last picture of Scrat. So, This is what partnering with Progress DataDirect will feel like to your internal and external developers. All the applications in the world can be made available to them with no major development effort. Now lets take a look at our latest Salesforce ODBC driver in action…
- Acknowledge our other analytics partners.
- This chart shows the performance gain you can have by using 8.0 with default options over 7.1.
- SOQL has ability to pick columns in the query, rather that’s the only things it can do. It does not have support for wildcard like * to denote select all columns.
- One to one…one to many….many to many