• Save
Office 365 & SharePoint Online Data Integration
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Office 365 & SharePoint Online Data Integration

on

  • 3,235 views

The Cloud Connector for Microsoft SharePoint 2010 and Office 365 takes your local line-of-business data to your cloud-based or any other hosted SharePoint - easy to connect, secure, always up-to-date ...

The Cloud Connector for Microsoft SharePoint 2010 and Office 365 takes your local line-of-business data to your cloud-based or any other hosted SharePoint - easy to connect, secure, always up-to-date and bi-directional, if required. Content stored in almost any corporate data source, e.g. databases, files, ERP / CRM / ECMS or other legacy platforms and on-premise applications like Dynamics, SharePoint, Active Directory, SAP etc. is replicated to native SharePoint lists into the cloud via secure web services.
All features of native SharePoint lists are completely available for your external data. While the SharePoint Business Data Catalog (BDC), the SharePoint Business Data Connectivity Services (BCS) "External List" or the SQL Server Integration Services (SSIS) out-of-the-box are not made for cloud-based or sandboxed scenarios, the SharePoint Cloud Connector was specially developed to solve the specific challenges of remote hosted disconnected systems in the cloud without any direct access to corporate business data and very restricted feature customization capabilities.

Statistics

Views

Total Views
3,235
Views on SlideShare
3,231
Embed Views
4

Actions

Likes
0
Downloads
0
Comments
0

2 Embeds 4

http://www.linkedin.com 3
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Office 365 & SharePoint Online Data Integration Presentation Transcript

  • 1. Get in the Cloud – With your corporate data sources connected
    A data-integration strategy needs to be within the foundation of every cloud computing plan. This integration needs to be innate to the architecture; it can't be something you're reminded of at the rollout meeting.
    David Linthicum / Cloud Computing
    Providing consistent enterprise data in SharePoint can solve many issues with the cloud
    Cloud Connector
    www.layer2.de
  • 2. SharePoint in the Cloud – Key Customer Pain Points
    Data Islands with no Access to Corporate LOB Data
    Very restricted Feature Customization Capabilities
    Complexity and Costs of Data and Process Integration
    Quality, Consistency and Up-To-Datenessof Data
    Secure Data Access and Transfer
    Manual Data Acquisition, Data Redundancy
    Providing consistent enterprise data in SharePoint can solve many issues with the Cloud
    Cloud Connector
    www.layer2.de
  • 3. Cloud Connector for SharePoint 2010 – Key Features
    The Cloud Connector for SharePoint migrates or integrates your local line-of-business data to your cloud-based or any other hosted SharePoint - easy to connect, secure, always up-to-dateand bi-directional if required.
    Data stored in almost any corporate data source, e.g. databases, files, ERP/CRM/ECMS or other legacy platforms, SharePoint, Active Directory etc. is replicated to native SharePoint lists into the cloud, e.g. Microsoft Office 365, via secure Web Services.
    All features of native SharePoint lists are supported for your external data, e.g. RSS feeds, email notification and workflows in the SharePoint cloud, when remote corporate data changes.
    No programming or tools required. Setup in a few minutes guided by a wizard.
    No installation or customizing in the SharePoint cloud required.
    100% Microsoft SharePoint 2010 compatible – New Client Object Model CSOM APIused.
    Source data subsets and data aggregation to data destinations are supported.
    A connection is only required for data exchange – not for list usage. The SharePoint list acts as some kind of cache in Cloud. No direct access to external company data required (“push” – no “pull” strategy). The Cloud Connector service application runs where your data resides today - on any server or client inside your corporate network.
    Replicates corporate LOB data to native SharePoint lists in the cloud
    Cloud Connector
    www.layer2.de
  • 4. SharePoint Cloud Connector – Architecture
    Enterprise Data Sources
    Cloud Connector(s)
    Data Access Providers
    @ W7/W2K3/W2K8:
    …and other drivers
    - Connection Service- Connection Admin- Connection Wizard
    …and more applications
    Secure Data Transfer
    SharePoint 2010 Client Side Object Model (CSOM)
    …and more databases
    HTTP or HTTPS/SSL
    Windows, Claims-based (FBA) etc.
    …and more file types
    Operates inside the corporate network - with secure connection to the cloud
    Cloud Connector
    www.layer2.de
  • 5. SharePoint Cloud Connector – Data Sources
    You can connect almost any data source via ODBC, OLEDB or .NET based drivers to the SharePoint Cloud. If no driver is available, please export as XLS/CSV and connect to that file.
    29.08.2011
    5
    Almost any corporate data source can be connected
  • 6. Cloud Connector – Installation
    Register & download zip
    Start setup.msi
    Accept license agreement
    Select typical installation
    Accept changes to computer
    Press “Finish” to exit
    See next slides about what is installed
    8/29/2011
    6
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 7. Cloud Connector – Program Directory
    Select Layer2 Cloud Connector in Programs
    You can open the directory, where the connection definitions are managed as XML files.
    You can open the directories where the log files are saved as CSV.
    You can start previously defined connections manually (instead of using the windows service).
    You can start a Cookie Manager, that offers an additional (but not required) option to authenticate to Microsoft Office 365.
    Please use the provided links to uninstall or add / remove software features of Windows to do that – if required.
    8/29/2011
    7
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 8. Cloud Connector – Program Data Directory
    All connections are saved in the ‘Connections’ directory as XML data.
    The ‘History’ directory contains internal data to manage the connections, don’t touch it.
    If you use a licensed version, please place the license file in the ‘License’ directory. If no license file found, the connector works in Shareware mode with 25 items per list maximum.
    You will find the log files in the log directory. There are system log files for any general log entries and one log file per connection (named like the connection file itself) for more connection specific entries. Please consult these log files in case of any issues.
    A metabase is located in the metabase directory starting with V2.0. It contains information for bi-directional and improved one-way replication (without PK mapping).
    8/29/2011
    8
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 9. Cloud Connector – Connection Service
    Layer2 Cloud Connector Service is installed as a windows service.
    This service automatically executes enabled connections defined in the program data directory “connections”.
    Please stop this service to use manual synchronization provided in the program menu, especially for setup and testing of connections.
    8/29/2011
    9
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 10. Cloud Connector - How to start?
    Create a connection definition file.
    Prepare the data entities for source and destination.
    Define a connection between two entities in in the connection definition file.
    Start the connection manually from program menu to test.
    Check screen outputs, log files and destination list for possible issues to fix.
    Fix the issues and start connection manually again.
    Finally start the Layer2 Cloud Connector windows service to process your connection in background.
    8/29/2011
    10
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 11. Cloud Connector – Create a connection definition
    Create a connection definition file in the connections folder, e.g. myConnection.xml.
    Set interval in minutes, e.g. 60 for every hour.
    Set version to 1.1.
    Set enabled to true to allow executing. Don’t forget to set to enabled to false to exclude this connection from execution.
    Define two data entities to connect (see below).One of the entities must be SharePoint-based!
    Prepare a field mapping section (see below).
    Please note: Coding XML is a V1.0 limitation. There will be a GUI later on.
    8/29/2011
    11
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 12. Cloud Connector – Prepare your data source
    You have to give your data entity an appropriate name to refer to.
    Your data entity must have the type “source” to be used as data source in a one-way connection. The type “bi-directional” is not supported yet.
    You have to enter a valid provider, a data driver that is already installed on your system. See below for more details.
    You have to enter a valid connection string, that is specific for your provider. See below for more details.
    You have to enter a valid select statement to retrieve your data.See your data source documentation.
    Optionally you can enter a primary key, a column that has unique values. In most cases your data source will automatically provide this information. Please note, that there must be any primary key or one or more unique columns for the data source to connect.
    8/29/2011
    12
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 13. Cloud Connector – Prepare your data destination
    You have to give your data entity an appropriate name to refer to.
    Your data entity must have the type “destination” to be used as data destination in a one-way connection. The type “bi-directional” is not supported yet.
    You have to enter a valid provider, a data driver that is already installed on your system. See below for more details.
    You have to enter a valid connection string, that is specific for your provider. See below for more details.
    You don’t have to enter a valid select statement for the data destination.
    Optionally you can enter a primary key, a column that has unique values. In most cases your data source will automatically provide this information. Please note, that there must be any primary key or one or more unique columns for the data source to connect.
    Please note, that the data destination is not created automatically, it must already be existing. In case of SharePoint for example, please create a destination list and the appropriate fields. You can use any list type you want, e.g. contacts for any address-based information etc.
    8/29/2011
    13
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 14. Cloud Connector – Prepare field mapping
    You can map the fields / columns of the data source and the data destination to each other.
    There is an autoMapping = “true” attribute in the fieldMappings section. That will map fields with the same name automatically. Please note that auto mapping is false by default.
    You have to map the primary key of the data source to some unique field of the data destination (ProductID in this sample). This column is used for any insert, update, delete operations in case of data source content changes.
    Please note that the ProductName field is mapped to the SharePoint Title column here as a sample. In case of SharePoint the internal column names are used to map, not column titles.
    You can use isMapped = “false” to exclude fields from auto-mapping for any reason. Default is isMapped=“true”.
    See log files for a detailed mapping report (mapped and unmapped fields).
    8/29/2011
    14
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 15. Cloud Connector – Ready to check the connection!
    Start the connection manually from the program menu to test.
    Check screen outputs and log files for possible issues to fix.
    Fix the issues, save the file and start connection manually again.
    For common issues with certain data entities see below.
    8/29/2011
    15
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 16. Cloud Connector – Data Providers
    Data providers are installed drivers on your computer.
    You have to use the so called invariant name to refer to a provider in the configuration file as data entity property.
    Typically the following providers are used:
    The Layer2.SharePoint.Provider: This provider comes with the Cloud Connector to connect to SharePoint 2010 via CSOM (Client Side Object Model) using Web Services.
    System.Data.Odbc: The .Net Framework Data Provider for Odbcis used to connect via ODBC to virtually any data source (comes with OS).
    System.Data.OleDb: The .Net Framework Data Provider for OleDb used to connect via OLEDB to virtually any data source(comes with OS).
    System.Data.OracleClient. The .Net Framework Data Provider for Oracle used to connect to Oracle database (comes with Oracle client).
    System.Data.SqlClient. The .Net Framework Data Provider for SqlServer used to connect to Microsoft SQL Server (comes with OS).
    See your machine.config for other installed providers.
    You can use any 32 or 64 bit provider, depending on your hardware and OS.
    8/29/2011
    16
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 17. Cloud Connector – Connection Strings
    Virtually any data source has a connection string.
    See your provider documentation for more details.
    You will find lots of connection strings here: www.connectionstrings.com.
    See below for connection strings for some specific providers, especially about the Layer2 SharePoint Provider used to connect to any SharePoint 2010 based installation, including Office 365 and SharePoint Online.
    8/29/2011
    17
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 18. Cloud Connector – SQL queries
    You can use any SQL query your data providers supports.
    Use aliases to map fields with specific names, e.g.: “select ProductName as Title, * from myTable”
    Use where conditions to connect a data subset only, e.g. :“select * from myTable where cloud = true”
    You can use views or procedures as supported by provider.
    8/29/2011
    18
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 19. Cloud Connector – The Layer2 SharePoint Provider
    The Layer2 SharePoint Provider is used to connect to any SharePoint 2010 server including Office 365 or any other local or remote hosted SharePoint 2010 installation.
    The provider can be used as data source, destination or both. Items are inserted, updated and deleted as required on source changes.
    Only modified items are changed in the destination. Not mapped fields remain untouched.
    You can use the following attributes in the connection string:
    Data Source: URL to the data site (required).Example: https://www.mysite.com/sites/myspsite/
    Initial Catalog: The list name, title or GUID (required). Example: MyCloudList
    View: optional List view name, URL or GUID. Please use views whenever possible for best performance.
    SelectStatement: Not supported. Use a list view instead of - as data source or destination.
    Authentication: Authentication method. Examples: Windows | Anonymous | Office365 | Office365IEBasedDefault = Windows. See below for Office 365 special requirements.
    User Id: User name, optionally with domain. Password: User password. Required for Windows and Office365.
    If no user and password is given, the current user authentication is used. Please note, that in case of you are using the windows service, this is the service account.
    PrimaryKey: Not required, the SharePoint item ID is used by default.
    8/29/2011
    19
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 20. Cloud Connector – Office 365 / SharePoint Online
    The Layer2 SharePoint Provider can be used to connect to any Office 365 or SharePoint Online based SharePoint installation.
    You can use user id / password as requested in a browser-based login with authentication method “Office365” defined: Authentication=Office365
    Custom domains: Use Realm=https://myOriginalDomain.sharepoint.com in connection string with Authentication=Office 365 if your current data source url is not based on sharepoint.com (using aliases). Otherwise authentication will fail. Realm is available starting with V1.1.
    The connector alternatively can use cookie-based Microsoft Online accounts for authentication. Please use Authentication=Office365IEBased for this. No user / password must be given in this case.
    Please use the Layer2 Office 365 Cookie Manager (see below) to set, refresh or delete the required cookie for Office365IEBased login. Please keep in mind, that the cookie expires as set by Microsoft.
    Token-Service: Optionally you can address any specific token service: SecureTokenService=“Url-to-Service”. By default the current Microsoft Token Service is addressed.
    Login-URL: Optionally you can specify any URL to login with the given token. SignInUrl=“Url-to-login”. By default currently it is set to: “/_forms/default.aspx?wa=wsignin1.0“.
    BatchReadItems: Because of possible list-throttling issues you optionally can set a batch size to read lists in parts of maximum n items. Default: BatchReadItems=5000
    BatchWriteItems: Because of possible performance issues you optionally can set a write operations batch size to execute operations in parts of maximum n items. Default: BatchWriteItems=5000
    8/29/2011
    20
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 21. Cloud Connector – Layer2 Office 365 Cookie Manager
    Optionally please use the Cookie Manager to create a new cookie, refresh or remove existing cookies to login to Office 365 using an Microsoft Online account with the Cloud Connector with option Authentication=Office365IEBased.
    After successful login press Save to store the cookie for further usage.
    Please note that the preferred method to connect to Office 365 is:Authentication=“Office365” without using the Cookie Manager.
    8/29/2011
    21
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 22. Cloud Connector – Performance
    The Cloud Connector is build to manage large external data sources with SharePoint in the cloud.
    To optimize performance on data source side please use pre-defined views in case of SQL Server instead of complex SQL queries. Please minimize the number of columns (e.g. no “select * from myTable”).
    Please optimize network speed for connection (e.g. via Internet)
    Insert and update operations on SharePoint are the most time consuming part of the connection, for example the first initial insert from a database.E.g. transferring 100.000 SQL Server records with 10 fields to an empty SharePoint list in Office 365 takes 6 hours in our tests.Please use a manual connection for initial setup.
    The performance of update operations depends on amount of data changed. If only some records / items are changed in our 100.000 items list example, it takes some minutes only to update the list.
    Delete operations are very fast. Please note that there is no undelete / recycling for deleted SharePoint list items.
    Please use list views in case of SharePoint as data source and / or destination because of much better performance. Fewer columns are to manage in this case.
    8/29/2011
    22
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 23. Sample File to connect a SQL database to SharePoint
    <?xmlversion="1.0" encoding="utf-8" ?><connectionxmlns="http://www.layer2.de/schemas/cloudConnections" interval="60" enabled=„true“ version="1.2"><dataEntities>
    <dataEntityname="MyNorthwindDatabase"type="source"provider="System.Data.SqlClient"connectionString="Data Source=srvsql01; Initial Catalog=Northwind; User Id=MyID; Password=MyPassword“ selectStatement="Select * FromCategories"primaryKey="CategoryID"></dataEntity>
    <dataEntityname="MySharePoint“ type="destination“ provider="Layer2.SharePoint.Provider"connectionString="Data Source=http://MySharePoint/MySPSite/; Initial Catalog=MySPList; User Id=MySPID; Password=MySPPassword;"selectStatement="“></dataEntity>
    </dataEntities>
    <fieldMappingsautoMapping=“true“>
    <fieldMapping><fieldentity="MyNorthwindDatabase" name="CategoryID"/><fieldentity="MySharePoint" name="CategoryID" /></fieldMapping><fieldMapping><fieldentity="MyNorthwindDatabase" name="CategoryName"/><fieldentity="MySharePoint" name= "Title" /></fieldMapping></fieldMappings>
    </connection>
    8/29/2011
    23
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 24. Sample File to connect SharePoint 2010 to Office 365
    <?xmlversion="1.0" encoding="utf-8" ?>
    <connectionxmlns="http://www.layer2.de/schemas/cloudConnections"
    interval="10" enabled= “true“ version="1.2">
    <dataEntities>
    <dataEntityname="MySharePointSource“ type="source“ provider="Layer2.SharePoint.Provider"
    connectionString="Data Source=http://MySharePointServer/MySharePointSite/; Initial Catalog=Links; User Id=MyUserID; Password=MyPassword; view=myExport">
    </dataEntity>
    <dataEntityname="MyO365Destination“ type="destination“ Provider="Layer2.SharePoint.Provider"
    connectionString="Data Source=http://MyCloud.sharepoint.com/sites/MyCloudSite/; Initial Catalog=Links; Authentication=Office365; User Id=MyUserID@onmicrosoft.com; Password=MyPassword“ >
    </dataEntity>
    </dataEntities>
    <fieldMappingsautoMapping=“true“>
    <fieldMapping>
    <fieldentity="MySharePointSource" name="ID"/>
    <fieldentity="MySharePointDestination" name="SourceID" />
    </fieldMapping>
    <fieldMapping>
    <fieldentity="MySharePointSource" name="URL"/>
    <fieldentity="MySharePointDestination" name= " URL" />
    </fieldMapping>
    </fieldMappings>
    </connection>
    8/29/2011
    24
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 25. Bi-directional replication with V2.0
    With V2.0 bi-directional replication is available. Please simply remove the type (destination/source) from the data entity or set to "bidirectional" to activate this option. You can add some options for conflict resolution before the data entity sections, e.g.:<conflictResolution mode="warnAndContinue" />By default operation is stopped on replication conflict.
    You can add conflict resolution options on field level within the fieldMapping section, e.g.:
    <fieldMapping>
    <conflictResolution mode="winnerLoser">
    <winningEntity name="SQLSource" />
    </conflictResolution>
    <field entity="SQLSource" name="ProductName"/>
    <field entity="SharePointDestination" name="Title" />
    </fieldMapping>
    You don't have to map the primary keys anymore with Version 2.0. But still there must be primary keys to work with.
    8/29/2011
    25
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 26. Cloud Connector –SharePoint Issues and Workarounds
    Calendar items are not supported at this time. There is an pending issue with Microsoft CSOM to set the event end time.
    Item IDs are read-only with SharePoint. You cannot map any field to an ID of the data destination list. Please note that Item IDs will be different in source and destination list.
    Some lists are using self lookups to reference some parent item, e.g. in case of dependent tasks. That will not work by default, because the Item IDs are changed by SharePoint. It is planned to automatically correct this with an auto-remap feature.
    Lookups are done by ID. The provider transfers the content of the lookup field correctly. But the lookup will work as expected only if the item looked up has the same ID compared to the target list.
    Person / Group lookups are fully supported. But the referred persons or groups have to exists with the same ID in the destination system. This is usually not the case. It is planned to solve this issue with an auto-discover feature, to find the right account by login name.
    For field-mapping the internal column names are used with SharePoint. See log files for more information about currently mapped and unmapped fields.
    If you have more issues, please contact sales@layer2.de directly to solve.
    8/29/2011
    26
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 27. Whatelsecan I do withtheCloud Connector?
    Add columns to your list, that are not in the data source, to add additional information or files / attachments.
    Tag, rate and comment your external data using managed metadata.
    Use typed lists, e.g. contacts for contact data, events for calendar data.
    Aggregate content from several data sources in one data destination list.
    Transfer subsets of data source only, e.g. all contacts with cloud=true flag using views.
    Bring your data offline with Outlook or SharePoint Workspace.
    Save license costs with external CRM/ERP systems – pull the data to SharePoint.
    Start notifications and workflows on external data change.
    Include your external data into portal search.
    Your data source has no driver available? Export and use CSV-based file import.
    Don’t forget: It’s not only for the cloud – any SharePoint 2010 is supported.
    More? Contact sales@layer2.de
    29.08.2011
    27
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 28. About Layer2
    As a Microsoft Gold Certified Partner based in Hamburg / Germany, Layer2 is one of the leading providers of Microsoft SharePoint extensions in Europe with a major focus on Data Integration, Collaborative Knowledge Management and Social Networking.
    Hamburg
    GERMANY
    1991: Formation as “Cords und Jacobi, EDV-Beratung und IT-Dienstleistungen” in Hamburg / Germany
    2001: Renamed to Layer2 GmbH
    2003: Focusing on Microsoft technologies, particularly .NET and SharePoint
    2007: Market entry Austria and Switzerland
    2008: Microsoft Certified Partner
    2009: Microsoft Gold Certified Partner (renewed 2011)
    Nuremberg
    29.08.2011
    28
    Layer2 – The SharePoint Experts from Hamburg (Germany) – www.layer2.de/en/
  • 29. Who isusing Layer2 SharePoint Add-Ons
    29.08.2011
    29
    Layer2 – The SharePoint Experts from Hamburg / Germany
  • 30. What areyou waiting for?www.layer2.de
    You can download and use a completely FREE shareware version of the SharePoint Cloud Connector today at:
    http://www.layer2.de/en/products/Pages/Cloud-Connector-for-SharePoint-2010-Office365.aspx
    See FAQs or contact sales@layer2.de for more information. Please note, that there are more SharePoint features and web parts available on www.layer2.de/en/.
    Download
    8/29/2011
    Layer 2 GmbH | Eiffestr. 664b | D-20537 Hamburg | www.layer2.de/en/ | Sales@layer2de
    30