Real-time BO Universe to
Cloud Data Sources
Sumit Sarkar (@SAsInSumit)
Chief Data Evangelist, DataDirect
Experience with Data Connectivity for BI
Talk to BI communities across Oracle, SAP,
IBM, Microstrategy, Tableau, JasperSoft
and Qlikview.
Advocate for BI professionals at shows
across Dreamforce, Hadoop Strata and
MongoDBWorld
Contributor to TDWI, Odata.org, Oracle
Data Integration, Salesforce Developers,
Progress Data Connections, and
Microstrategy
DataDirect ODBC, JDBC, OData for Disruptive Data
Big Data/NoSQL
 Apache Hadoop Hive
 Cloudera
 Hortonworks
 MapR
 EMR
 Pivotal HAWQ
 MongoDB
 Cassandra
 SparkSQL
 Apache Solr*
Data Warehouses
 Amazon Redshift
 SAP Sybase IQ
 Teradata
 Oracle Exadata
 Pivotal Greenplum
Relational
 Oracle DB
 Microsoft SQL
Server
 IBM DB2 for I
 IBM DB2 for z/OS
 IBM DB2 for LUW
 MySQL
 MemSQL
 PostgreSQL
 IBM Informix
 SAP Sybase
 Pervasive SQL
 Progress OpenEdge
 Progress Rollbase
 Splice Machine*
 IBM DashDB*
SaaS/Cloud
 Salesforce.com
 Database.com
 FinancialForce
 Veeva CRM
 ServiceMAX
 Hubspot
 Marketo
 Microsoft Dynamics
CRM
 Microsoft SQL Azure
 Oracle Eloqua
 Oracle Service Cloud
 Google Analytics
 Netsuite*
 SQL over HTTPS
In-Memory
 MemSQL
 SAP HANA
 Oracle TImesTen*
 VoltDB*
Agenda
1- Introduction to SAP Business Objects Cloud
Universes
2- Architecture options for Cloud Universes
3- Best Practices and Lessons Learned
Goals
1- Understanding of a Cloud Universe
2- Be the thought leader on cloud data sources.
1- Introduction to SAP Business Objects Cloud
Universes
a. What is a Cloud Universe?
b. Common Cloud Data Sources
c. Common use cases in production
Introduction: What is a cloud Universe?
Introduction: Common cloud data sources for BOBJ
SaaS
Salesforce
Veeva CRM
NetSuite
ServiceNow
Cloud9
WorkDay
Tavant
Kinaxis Rapid Response
Cloud Databases
Amazon Redshift
SQL Server Azure
Hosted DBs
Introduction: Common Use Cases
•Salesforce reporting (native reporting inadequate)
•Migrating/Consolidating BI Platforms to Business
Objects
•Real-time data blending in MSU to supplement the
Data Warehouse with real-time Salesforce data
•Real-time Mobile Universe Web Intelligence
2- Architecture options for Cloud Universes
a. Real-time / Direct
b. Data Warehouse
c. Staging Database
d. Hybrid (Real-time and Data Warehouse)
e. Pros/Cons
Architecture: Real-time / Direct
UNIVERSE
Architecture: Staging Databases
UNIVERSE
Architecture: Data Warehouse
UNIVERSE
Architecture: Data Warehouse and real-time
UNIVERSE
Architecture: Pros/Cons
Real-time
Direct
Data
Warehouse
Staging
Database
DW and real-
time (MSU)
Self Service Y
Rapid
Development
Y
Real-time Y Y
360 view Y Y
Local
Connection
Y Y Y
Architecture: SaaS ODBC3 Universe Connection
IDT/UDT – 32-bit
App Server/BODS – 64-bit
3- Best Practices
a. SaaS data sources are not relational databases or MPP
warehouses (non-optimized joins)
b. How to handle authentication
c. Keeping up with the APIs
d. Real-time versus ETL (MSU and SSU)
e. Understand road map for new SaaS applications
Best Practices: SaaS APIs vs databases
• Determine if SaaS source has a query language
• What relationships are exposed between objects
• Capacity planning for larger in-memory operations
LESSONS LEARNED
Modeling Universe on top of unrelated objects from
any SaaS application with large data volumes will be
a challenge – not really different from RDBMS.
Best Practices: Authentication
• Salesforce shops typically setup a common BI user
• Single Sign-On requirements
LESSONS LEARNED
How to delegate BOBJ SSO to Salesforce SSO?
Best Practices: Keeping up with the APIs
• Find out how often APIs change for your SaaS source
• Schema management for new objects/fields
• Refresh schema?
• Understand API call limits for 24 hour period
LESSONS LEARNED
Salesforce API changes quarterly and requires updates to
connectors to support latest fields/objects. This is reason
native connector with BODS does not work well.
Best Practices: Real-time versus ETL
• Understand the performance of the APIs
• What data volumes are required?
LESSONS LEARNED
Pulling very large data volumes in activity and lead
records from Eloqua or Marketo for a real-time
Universe is not practical.
Best Practices: Know your data road map
• Demonstrate thought leadership by showing what SaaS
sources you can support.
• Understand the SaaS BI landscape by department to compare
contrast your services.
LESSONS LEARNED
Departments may not engage BOBJ group and
duplicate BI efforts further fragmenting the data
intelligence.
Resources
• Blog tutorial to create a Salesforce Universe:
https://blogs.datadirect.com/2012/05/sap-business-objects-
universe-to-salesforce-crm-database-com-force-com.html
• Blog tutorial to create a Marketing Universe:
https://blogs.datadirect.com/2014/01/sap-business-objects-
universe-marketing-data-eloqua-marketo.html
• Blog tutorial to integrate BO Data Services with Cloud Sources:
https://blogs.datadirect.com/2015/02/sap-bods-linux-
salesforce-com-netsuite.html
Love to hear from SAP BO
community!
www.linkedin.com/in/meetsumit
Sumit.sarkar@progress.com
@SAsInSumit
919-461-4284

Ibis 2015 final template

  • 1.
    Real-time BO Universeto Cloud Data Sources Sumit Sarkar (@SAsInSumit) Chief Data Evangelist, DataDirect
  • 2.
    Experience with DataConnectivity for BI Talk to BI communities across Oracle, SAP, IBM, Microstrategy, Tableau, JasperSoft and Qlikview. Advocate for BI professionals at shows across Dreamforce, Hadoop Strata and MongoDBWorld Contributor to TDWI, Odata.org, Oracle Data Integration, Salesforce Developers, Progress Data Connections, and Microstrategy
  • 3.
    DataDirect ODBC, JDBC,OData for Disruptive Data Big Data/NoSQL  Apache Hadoop Hive  Cloudera  Hortonworks  MapR  EMR  Pivotal HAWQ  MongoDB  Cassandra  SparkSQL  Apache Solr* Data Warehouses  Amazon Redshift  SAP Sybase IQ  Teradata  Oracle Exadata  Pivotal Greenplum Relational  Oracle DB  Microsoft SQL Server  IBM DB2 for I  IBM DB2 for z/OS  IBM DB2 for LUW  MySQL  MemSQL  PostgreSQL  IBM Informix  SAP Sybase  Pervasive SQL  Progress OpenEdge  Progress Rollbase  Splice Machine*  IBM DashDB* SaaS/Cloud  Salesforce.com  Database.com  FinancialForce  Veeva CRM  ServiceMAX  Hubspot  Marketo  Microsoft Dynamics CRM  Microsoft SQL Azure  Oracle Eloqua  Oracle Service Cloud  Google Analytics  Netsuite*  SQL over HTTPS In-Memory  MemSQL  SAP HANA  Oracle TImesTen*  VoltDB*
  • 4.
    Agenda 1- Introduction toSAP Business Objects Cloud Universes 2- Architecture options for Cloud Universes 3- Best Practices and Lessons Learned
  • 5.
    Goals 1- Understanding ofa Cloud Universe 2- Be the thought leader on cloud data sources.
  • 6.
    1- Introduction toSAP Business Objects Cloud Universes a. What is a Cloud Universe? b. Common Cloud Data Sources c. Common use cases in production
  • 7.
    Introduction: What isa cloud Universe?
  • 8.
    Introduction: Common clouddata sources for BOBJ SaaS Salesforce Veeva CRM NetSuite ServiceNow Cloud9 WorkDay Tavant Kinaxis Rapid Response Cloud Databases Amazon Redshift SQL Server Azure Hosted DBs
  • 9.
    Introduction: Common UseCases •Salesforce reporting (native reporting inadequate) •Migrating/Consolidating BI Platforms to Business Objects •Real-time data blending in MSU to supplement the Data Warehouse with real-time Salesforce data •Real-time Mobile Universe Web Intelligence
  • 10.
    2- Architecture optionsfor Cloud Universes a. Real-time / Direct b. Data Warehouse c. Staging Database d. Hybrid (Real-time and Data Warehouse) e. Pros/Cons
  • 11.
  • 12.
  • 13.
  • 14.
    Architecture: Data Warehouseand real-time UNIVERSE
  • 15.
    Architecture: Pros/Cons Real-time Direct Data Warehouse Staging Database DW andreal- time (MSU) Self Service Y Rapid Development Y Real-time Y Y 360 view Y Y Local Connection Y Y Y
  • 16.
    Architecture: SaaS ODBC3Universe Connection IDT/UDT – 32-bit App Server/BODS – 64-bit
  • 21.
    3- Best Practices a.SaaS data sources are not relational databases or MPP warehouses (non-optimized joins) b. How to handle authentication c. Keeping up with the APIs d. Real-time versus ETL (MSU and SSU) e. Understand road map for new SaaS applications
  • 22.
    Best Practices: SaaSAPIs vs databases • Determine if SaaS source has a query language • What relationships are exposed between objects • Capacity planning for larger in-memory operations LESSONS LEARNED Modeling Universe on top of unrelated objects from any SaaS application with large data volumes will be a challenge – not really different from RDBMS.
  • 23.
    Best Practices: Authentication •Salesforce shops typically setup a common BI user • Single Sign-On requirements LESSONS LEARNED How to delegate BOBJ SSO to Salesforce SSO?
  • 24.
    Best Practices: Keepingup with the APIs • Find out how often APIs change for your SaaS source • Schema management for new objects/fields • Refresh schema? • Understand API call limits for 24 hour period LESSONS LEARNED Salesforce API changes quarterly and requires updates to connectors to support latest fields/objects. This is reason native connector with BODS does not work well.
  • 25.
    Best Practices: Real-timeversus ETL • Understand the performance of the APIs • What data volumes are required? LESSONS LEARNED Pulling very large data volumes in activity and lead records from Eloqua or Marketo for a real-time Universe is not practical.
  • 26.
    Best Practices: Knowyour data road map • Demonstrate thought leadership by showing what SaaS sources you can support. • Understand the SaaS BI landscape by department to compare contrast your services. LESSONS LEARNED Departments may not engage BOBJ group and duplicate BI efforts further fragmenting the data intelligence.
  • 27.
    Resources • Blog tutorialto create a Salesforce Universe: https://blogs.datadirect.com/2012/05/sap-business-objects- universe-to-salesforce-crm-database-com-force-com.html • Blog tutorial to create a Marketing Universe: https://blogs.datadirect.com/2014/01/sap-business-objects- universe-marketing-data-eloqua-marketo.html • Blog tutorial to integrate BO Data Services with Cloud Sources: https://blogs.datadirect.com/2015/02/sap-bods-linux- salesforce-com-netsuite.html
  • 28.
    Love to hearfrom SAP BO community! www.linkedin.com/in/meetsumit Sumit.sarkar@progress.com @SAsInSumit 919-461-4284

Editor's Notes

  • #2 Focus is on open data industry standards Synopsis The explosion of Cloud Data Sources such as Salesforce.com, Google Analytics, Marketo, Eloqua, etc are disrupting BI infrastructure; and business sponsors are turning to one-off BI solutions further fragmenting reporting capabilities.  Learn best practices and common gotchas for SAP Business Objects shops looking to establish robust connectivity from Single or Multi Source Universes to cloud data; and take back control of future BI projects.  With the Universe, learn how to create a direct and real-time connection to all business systems across the enterprise and in the cloud.  Leverage existing skills and infrastructure to consume cloud data; and establish your group as the thought leader on cloud data sources.  Discussion is focused on cloud connectivity achieved with ODBC3 relational sources with third party ODBC drivers from leading vendors in the space.
  • #4 BOBJ and Crystal projects with NoSQL sources such as MongoDB
  • #8 https://help.sap.com/businessobject/product_guides/sbo41/en/sbo41_data_acs_en.pdf Pg 167
  • #9 No Marketing?? (Google Analytics, Eloqua, Marketo, Pardot)
  • #12 One version of the truth and “self service”. Stanford TDWI story Mobile Webi reports delivered to 600 ipads
  • #13 Helps supplement poor SaaS APIs not suitable for heavy duty workloads Easyl and Eloqua?
  • #14 BODS to Salesforce/NetSuite are common projects ETL platform Moving data into EDW or SAP HANA Starting to see Hadoop as well
  • #15 Progress Software does this: Sales Management Dashboards are pulled in real-time Business Review data is pulled from archives
  • #19 Create Project Create New Database Connection
  • #21 Create Data Foundation Layer Insert Tables and Joins into DFX layer Create Business Layer Create Folders, Classes and Objects (Dimension, Detail, Measure) in BLX layer Creating queries to test Publish to Repository
  • #22 Worked with 30+ SAP shops integrating cloud data sources over last 3 years.
  • #27 Birst, GoodData, Cloud9, Tableau
  • #28 Birst, GoodData, Cloud9, Tableau