Your SlideShare is downloading. ×
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Data Federation/EII Uses And Abuses
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data Federation/EII Uses And Abuses

3,887

Published on

As the need for "right-time" information becomes more prevalent, organizations are looking to new information delivery methods such as Enterprise Information Integration (EII). It's more than a …

As the need for "right-time" information becomes more prevalent, organizations are looking to new information delivery methods such as Enterprise Information Integration (EII). It's more than a distributed query, but where do vendors over-promise and under-deliver? This presentation will explain EII, how it's different from other integration technologies, and the underlying mechanisms. The goal is to outline areas where EII is a good fit and areas where other tools may be more appropriate.

Published in: Technology
1 Comment
4 Likes
Statistics
Notes
  • Here is a discussion on what TRUE data virtualization is all about. Don't confuse this with simple, traditional data federation, which is a subset of data virtualization.

    http://tdwi.org/Articles/2011/06/15/Virtualization-and-Data-Integration-Issues.aspx#

    Some information on Informatica's data virtualization solution:
    http://www.informatica.com/Pages/data_virtualization_index.aspx
    http://www.informatica.com/products_services/data_services/Pages/index.aspx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,887
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
1
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Bio stuff: Background, been through some of these decisions
  • Transcript

    • 1. Enterprise Information Integration Technology: Architectures, Uses and Abuses Mark Madsen http://ThirdNature.net [email_address]
    • 2. Intro & Outline
      • Why we need new tools for on-demand integration
      • What are the alternatives?
      • EII definition and models
      • How EII tools work
      • Ways you can use and abuse the tools
      • Some usage criteria
    • 3. Problems Requiring New Approaches
      • The technical problems driving us to extend our integration tools:
        • Data currency
        • Data source diversity
        • Data type diversity
        • Delivery via different APIs
        • On-demand data consistency
        • Regulation and data security
    • 4. What We’ve Been Building
      • The original view:
      • Data warehouse as a system or application.
      • Cleansed historical data on a snapshot basis.
      • Physical storage of all the data to be accessed.
      • End user tools and infrastructure combined.
      SQL Warehouse Database ETL ODS Mart Databases Documents Flat Files XML Queues ERP Applications Source Environments Data Warehouse Clients Dashboards OLAP Productivity BAM/BPM Reporting DM Data Mining
    • 5.
      • The new view:
      • Data warehouse as a platform.
      • Access to both historical and current data.
      • Multiple storage methods, possibly distributed.
      • Multiple access methods.
      • Data usage decoupled from underlying platform.
      What We Need to Build Now SQL Warehouse Database ETL ? ? Mart ODS ? ? Content Store Databases Documents Flat Files XML Queues ERP Applications Source Environments Data Consumers Databases Dashboards OLAP Productivity BAM/BPM Reporting ETL Data Mining Applications
    • 6. EDR: Enterprise Data Replication EDR Server Order Entry CRM Fulfillment Inventory EDR EDR
    • 7. ETL: Extract, Transform and Load ETL Engine The good points Connectivity Transformation Read-write access Metadata The bad points Requires a physical target Latency Distributed query limitations Complexity Database Targets Databases Documents Flat Files XML Services ERP Applications Source Environments
    • 8. Enterprise Application Integration
      • Multiple models
      • Multiple implementations
      • Evolution toward open standards-based services and SOA
      Hub Point Bus
    • 9. EII: Enterprise Information Integration EII Server Databases Documents Flat Files XML Services ERP Applications Source Environments SQL SOAP WS-* REST File Virtual Models Consuming Environments Databases Dashboards OLAP Productivity BAM/BPM Reporting ETL ERP Applications
    • 10. EII Architecture: Process-based EII View APIs xform Publish an ETL process as a service – the output of the process is the “view” Approach some ETL vendors have for on-demand access Back-to-front model Applications Databases XML/Services
    • 11. EII Architecture: Model-based View is defined as a model over sources Most EII products work this way Front-to-back model Applications Databases XML/Services EII View APIs
    • 12. How EII Works 1. User’s client sends a request which goes to the EII server interface Applications Databases XML/Services EII Server EII View SQL
    • 13. How EII Works 2. The EII server parses the request and creates query fragments to send to the sources Applications Databases XML/Services EII Server EII View SQL
    • 14. How EII Works 3. Each source system receives its query fragment, processes it and returns the result Applications Databases XML/Services EII Server SQL
    • 15. How EII Works 4. EII server assembles the final result (which may mean additional processing) and the result is sent back to the client Applications Databases XML/Services EII Server EII View SQL
    • 16. How You Work With EII Tools EII Server
      • Define data sources
      • Define integrated view
      • Map source data into the integrated view
      • Publish view
      • Tune query*
      • Query via the published interfaces
    • 17. How You Work With EII Tools EII Server
      • Define data sources
      • Define integrated view
      • Map source data into the integrated view
      • Publish view
      • Tune query*
      • Query via the published interfaces
    • 18. How You Work With EII Tools
      • Define data sources
      • Define integrated view
      EII Server EII Virtual View
      • Define data sources
      • Define integrated view
      • Map source data into the integrated view
      • Publish view
      • Tune query*
      • Query via the published interfaces
    • 19. How You Work With EII Tools
      • Define data sources
      • Define integrated view
      • Map source data into the integrated view
      • Publish view
      • Query via the published interfaces
      • Tune query*
      EII Server EII Virtual View SQL SOAP WS-* REST File
    • 20. What Sorts of Problems Can We Solve?
      • Applications where you are being asked to provide any of the following:
        • More up-to-date data
        • Data in alternate formats
        • Data in non-relational formats
        • Data for portals, web sites or composite applications
    • 21. Adding On-demand BI Capabilities Source Environments Data Warehouse ETL Historical data Current data ODS BI Applications
    • 22. Drill-through to Source Detail Source Environments Data Warehouse ETL Drill-through accesses detail data via virtual tables in the warehouse ODS BI Applications
    • 23. Consolidated Views Data Mart Source Environments ODS ETL Data Mart EII Server ETL Data is available via the EII server, providing a virtual consolidation mart BI Applications
    • 24. Performance Management and BI Source Environments EII Server ETL Data Warehouse BI Applications ODS Dashboards BPM BAM Operational Reporting
    • 25. Compliance and Regulatory Reporting
      • Many compliance and reporting solutions deal with process, content and tracking documents.
      • Typical characteristics:
        • Historical data
        • Non-BI data
        • Multiple source applications
        • Documents and content
        • Needs vary over time
    • 26. Portals, Composite Applications and SOA
      • Typical Portal scenario:
      • Disparate data
      • Heterogeneous sources
      • Point integration
      • Minimal reuse
      Databases Documents Flat Files XML Services ERP Applications Source Environments
    • 27. Portals, Composite Applications and SOA
      • EII provides a “data services” layer
      • Standards-based interfaces
      • Single view of source data
      • Single point of integration
      • Reuse of data as a service
      Data Services (EII) Databases Documents Flat Files XML Services ERP Applications Source Environments
    • 28. Portals, Composite Applications and SOA Data Services (EII) Transaction Services (EAI) Databases Documents Flat Files XML Services ERP Applications Source Environments
    • 29. Rapid Prototyping
      • If you begin building a data services layer, it can make the prototyping of new data-intensive applications go much faster.
        • No need to physically model, extract and store data for a prototype application.
        • Simple to set up
      • Examples:
        • Adding a subject area
        • Adding a star schema
        • Adding data to a portal
    • 30. Just Remember It’s a Prototype
    • 31. Going Further: the Virtual Data Warehouse
    • 32. Other Questionable Ideas
      • Getting around pesky rules and procedures:
        • Like being direct query access cutoffs
        • Or security
        • Or development controls regarding data access
      • You can even treat an EII view like a data source for copying data…
    • 33. EII Isn’t a Miracle Cure
      • It can help you take shortcuts you shouldn’t.
      • Some other shortcomings you may face:
        • Metadata sharing and interoperability
        • Reliability (failure modes, not the tools)
        • Performance considerations
    • 34. Some Guidance: When Does EII Fit?
      • There are some basic criteria and tradeoffs to consider:
        • Data currency vs. latency
        • Diversity of data sources
        • Data cleansing & transformation
        • Predictability of performance
        • Access to the same data is needed via different interfaces
        • Non-relational sources
        • Frequency of access
        • Data volumes
        • And more…

    ×