Download File


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 13 13 5
  • 13 13 5
  • As you can see from this chart, we have 5 overarching technology categories with multiple solutions in each. ( Reading the descriptors below is optional for the presenter, based on the audience)   Data Management and integration, allows any data in the enterprise to be presented in the right format, at the right time, and in the right context.  These solutions help to ensure business continuity, effective enterprise data management, mobile integration and access, and optimal data warehousing.  Application integration delivers consistent transactional content while identifying relevant processes inside or outside the enterprise.  Sybase solutions in this area are quite robust. And ERP and CRM applications can be integrated through our Business Process Integration solutions. Sybase has also beat BEA and others in an application server shoot-out. By using an audited benchmark, our application server came in first with the fastest performance at the lowest cost.  Vertical Process Integration, creates Information Liquidity flow in vertical markets with specialized needs, such as the securities industry and government regulation.  These are pre-built models. In Financial Services, we can connect to just about any financial message including FIX engine and SWIFT connectivity, FIX over SWIFT and Gold-certified SWIFT message libraries, and recently announced packaged connectivity for VMUs, including OMGEO and GSTP. With our TradeForce Suite, we integrate transactions for STP. And, in the area of compliance, Sybase leads with solutions for HIPAA and complying with the USA PATRIOT Act. Anywhere integration helps deliver information from multiple sources to multiple users in wired, wireless or disconnected modes. Sybase is a leader in this category, with market-leading products for mobile database, mobile middleware and enterprise portals.  Development and management provide tools for you to manage metadata as a base across development products. Here Sybase is key in helping to reduce development time, with tools for data and process modeling, enterprise monitoring, and portlet and web development.
  • Download File

    1. 1. INT420 - Federated Database Solutions with Sybase Middleware <ul><li>Lynn Weatherwax </li></ul><ul><ul><li>Director of Engineering – Data Integration </li></ul></ul><ul><ul><li>Infrastructure Platform Group, Sybase </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>
    2. 2. Abstract Many companies today seek optimized ways to access structured and unstructured data in their enterprises, without extensive programming or costly database consolidation. Virtual databases with links to distributed heterogeneous sources enable applications to consistently access and combine data in new ways. Sybase solves this problem while supporting the optimized query capabilities and scalability of an enterprise class relational database, via ASE, Enterprise Connect Data Access, Enhanced Full Text Search, and ASE Web Services. Leverage SQL and XML standards to make your data work for you in your enterprise.
    3. 3. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Strategies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Summary and Resources </li></ul>
    4. 4. Enterprise Information Integration (EII) <ul><li>“ Enterprise information integration (EII) is seeing a new surge of interest in 2003, largely because its virtual and/or federated approach to data integration has potential benefits. </li></ul><ul><li>Compared to other data integration technologies, solutions based on a vendor’s EII tool tend to be quick to deploy, inexpensive in terms of licensing and very flexible since they’re based on a metadata-based view of enterprise data. </li></ul><ul><li>In addition, EII relies on distributed queries to effect data integration in near real-time, thus delivering fresher data than traditional batch methods. </li></ul><ul><li>Many companies are evaluating vendors’ EII products in 2003, so 2004 should see an upsurge in EII implementations.” </li></ul><ul><li>Philip Russom, IdeaByte </li></ul>
    5. 5. Factors Driving EII Today <ul><li>Changes in company structure </li></ul><ul><li>Increased competition, requiring better intelligence </li></ul><ul><li>Government regulations </li></ul><ul><ul><li>Patriot Act </li></ul></ul><ul><ul><li>Sarbanes-Oxley Act </li></ul></ul><ul><ul><li>HIPPA </li></ul></ul>
    6. 6. EII Challenge <ul><ul><li>Every enterprise has federated data, due to </li></ul></ul><ul><ul><li>Mergers and acquisitions </li></ul></ul><ul><ul><li>Legacy mainframe applications </li></ul></ul><ul><ul><li>Applications required to work with specific databases </li></ul></ul>XML DB2 UDB Sybase Microsoft Informix VSAM Oracle Adabas Documents
    7. 7. Option: Migrate All Data/Apps to One Database… <ul><li>Sometimes this is feasible, but often there are time and $$ restrictions… </li></ul><ul><li>And what about the next acquisition, or the next undiscovered information island? </li></ul>XML DB2 UDB My Centralized Database Microsoft Informix VSAM Oracle Adabas Documents
    8. 8. Enterprise Information Integration Requirements <ul><ul><li>Leverage existing hardware and software investment </li></ul></ul><ul><ul><li>Ensure secure access </li></ul></ul><ul><ul><li>Integrate legacy data and applications while they remain in place – short-term or long-term </li></ul></ul><ul><ul><li>Metadata transparently aggregates data, no programming required </li></ul></ul><ul><ul><li>Components that work together, forming a complete, end-to-end solution </li></ul></ul><ul><ul><li>Supported in standard client/server environments, yet fully e-business ready (supports XML, Java, etc.) </li></ul></ul>
    9. 9. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Strategies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Summary and Resources </li></ul>
    10. 10. Strategies for Integrating Federated Data <ul><li>Data Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>Messaging/Data Replication </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul>
    11. 11. Enterprise Connect Data Access <ul><li>Make all your enterprise data look like one database </li></ul>Any mainframe data Oracle MS SQL Server Sybase DB2 UDB for OS/390, NT/UNIX AS/400 Informix ODBC-enabled databases Documents/Files Custom Specialty Data Stores ASE/CIS Mainframe Connect Distributed Proxy Tables Specialty Data Store Toolkit Any Client Or Browser
    12. 12. Mainframe Connect - Extending the Mainframe Enterprise Connect VSAM IMS Datacom IDMS Adabas Mainframe Sybase ASE Replication Server Enterprise Portals, App Servers Direct Connect for OS/390 DB2 Mainframe Applications and Transactions Any Client Mainframe as a Client (Client Options) Access to any MF Data/Applications (Server Options and XJS/390) Dynamic Access to DB2 (DB2 UDB for OS/390 Option) “ Gatewayless” configuration
    13. 13. XJS/390 Enterprise Integrator ASE/CIS VSAM IMS Datacom IDMS Sequential Adabas Mainframe Soap, XML, HTML, and Scripts ASE Replication Server Enterprise Portals, App Servers Direct Connect for OS/390 DB2 Mainframe Applications, and Transactions Any Client Or Browser MQSeries MQSeries Access to MF Data and Applications (Server Options and XJS/390) CICS Programs, Data HTTP, MQ, Web Services IMS Programs, Data, HTTP, MQ Navigate CICS Transactions Integrate with MainframeCONNECT No data is too complex to web- or message- enable “ Gatewayless” configuration
    14. 14. Scenario – Customer Service <ul><li>Problem: Phone Service Two-Step </li></ul><ul><li>Taking 20+ minutes/call to help customer buy new options </li></ul><ul><li>Customer data from IMS mainframe “green screens” </li></ul><ul><li>Regional service data from distributed Informix databases </li></ul><ul><li>Goal: Faster Service </li></ul><ul><li>Present all data at once in easy-to-use GUI application so customer service operators can provide faster service </li></ul><ul><li>Must leverage existing application logic and data </li></ul>Informix Regional service data Mainframe IMS customer data
    15. 15. Solution – Customer Service <ul><li>OS/390 IMS customer data </li></ul><ul><li>Informix local telephone service data </li></ul>Enterprise Connect Mainframe Connect GUI Customer Service application. Solution: <2 minutes/call with GUI customer service application transparently accessing mainframe and distributed data Proxy Tables
    16. 16. Data Access – How It Works <ul><li>Set up connectivity to remote servers </li></ul><ul><li>Define remote servers and security in ASE-CIS </li></ul><ul><li>Automatically set up proxy tables mapped to remote heterogeneous tables (federated data) </li></ul><ul><li>Set up security as desired </li></ul><ul><li>Metadata retained within ASE-CIS </li></ul><ul><li>Now queries work with the proxy tables as if they were local </li></ul><ul><li>Create stored procedures, views for proxy tables as needed </li></ul><ul><li>Federate data is accessible transparently via SQL </li></ul>Enterprise Connect Mainframe Connect
    17. 17. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Technologies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Resources </li></ul>
    18. 18. Document Access – Full Text Search <ul><li>Store and index documents for quick search </li></ul><ul><li>PDF, Microsoft Word, many other formats supported </li></ul><ul><li>Ability to join database data with results of full text search </li></ul><ul><li>Automated indexing, backup, restore </li></ul>
    19. 19. ASE – Enhanced Full Text Search Option (EFTS) <ul><li>Full Text Search of Wide Variety of Documents </li></ul><ul><li>Combined Search Results with Data from Other Tables </li></ul>ASE/CIS Full Text Search Engine <ul><li>Documents stored as binary in user-created source tables with identity column for unique id </li></ul><ul><li>CIS index tables (proxy tables) mapped to text search engine collections </li></ul>Any Client Or Browser Collections (Directories with text indexes and unique identifiers) Supports over 200 formats including Microsoft Word, XML, SGML, PDF
    20. 20. Scenario – Resume Search <ul><li>Problem: Find Resumes with Certain Skills, Match with other Data on Potential Employees </li></ul><ul><li>Scan individual resume documents in various formats for candidate with appropriate skills </li></ul><ul><li>Once candidate is located, query ASE database containing other information on the candidate </li></ul><ul><li>Assemble information by hand in report for interviewers </li></ul>DB2 database with other candidate info Resume Documents
    21. 21. Solution – Resume Search ASE-CIS Resume Search Application <ul><li>Solution: </li></ul><ul><li>Single point of access for both documents and other data </li></ul><ul><li>One query locates matching documents based on relevance, joins this data w/ related federated data on candidates, presents complete info to end user </li></ul>EFTS Option for ASE ASE database with documents and other data on candidates Text indexes Collection
    22. 22. Enhanced File Text Search – How It Works <ul><li>Setup </li></ul><ul><li>Store documents in ASE as binary data </li></ul><ul><li>Set up EFTS remote server defined as server class “sds” </li></ul><ul><li>Run stored procedures that extract and index the text of each document into “collections” </li></ul><ul><li>Proxy tables w/in ASE-CIS mapped to indexes called “collections” </li></ul><ul><li>Each document has unique id in database </li></ul><ul><li>As documents are changed/updated, triggers track changes so indexes can be kept in sync </li></ul>
    23. 23. Enhanced File Text Search – How It Works <ul><li>Run-time </li></ul><ul><li>Query for particular text string is sent through proxy mapping to EFTS for query against collection </li></ul><ul><li>Document ids returned, based on relevance (degree of match with query) </li></ul><ul><li>If desired, join with other data </li></ul><ul><li>Return data to client application </li></ul><ul><li>Client application may then display lists of docs or actual doc content as desired </li></ul>
    24. 24. EFTS – Advanced Capabilities <ul><li>Topics </li></ul><ul><ul><li>Groupings of information that represents a concept or subject area. </li></ul></ul><ul><ul><li>Precompiled query made available for run-time searches by any user </li></ul></ul><ul><li>Clustering </li></ul><ul><ul><li>Group similar documents </li></ul></ul><ul><li>Query by Example </li></ul><ul><ul><li>Search for documents that have similar phrases or are similar to docs already indexes </li></ul></ul><ul><li>Custom Thesaurus </li></ul><ul><ul><li>Build a custom thesaurus with synonyms you define </li></ul></ul>
    25. 25. Enhanced Full Text Search – More Features <ul><li>Sample Text Load program </li></ul><ul><ul><li>Loads data into text or image columns into a source table </li></ul></ul><ul><ul><li>Delivered as source code </li></ul></ul><ul><li>Stored procedures to help automate </li></ul><ul><ul><li>EFTS configuration </li></ul></ul><ul><ul><li>Alteration of cluster capabilities </li></ul></ul><ul><ul><li>Backup and recover text indexes </li></ul></ul><ul><ul><li>Display information about a text index </li></ul></ul><ul><ul><li>Optimize indexes </li></ul></ul>
    26. 26. ASE 12.5 Content Management Option (External File System Support – EFS) <ul><li>File System Directory Access </li></ul><ul><li>SQL access to file system directories and their underlying files is enabled via proxy tables </li></ul><ul><li>Each directory entry is visible as a row in a proxy table </li></ul><ul><li>Regular file content is available as an image column in the proxy table </li></ul><ul><ul><ul><ul><ul><li>Combine with Full Text Search to index and search on documents external to ASE via ASE queries </li></ul></ul></ul></ul></ul>
    27. 27. File System Directory Access – How it Works <ul><li>Create a proxy table for the directory </li></ul><ul><ul><li>The directory pathname must reference directory visible to/searchable by ASE </li></ul></ul><ul><ul><li>Proxy table is created which maps column names to file attributes and content </li></ul></ul><ul><ul><li>Option to include subdirectories </li></ul></ul><ul><li>Attributes of directory contents converted to SQL data types </li></ul><ul><ul><li>Content of regular files returned via content column </li></ul></ul><ul><ul><li>Always treated as an image datatype </li></ul></ul><ul><li>SQL Commands supported </li></ul><ul><ul><li>Insert causes new file to be created in directory </li></ul></ul><ul><ul><li>Update allows filename change </li></ul></ul><ul><ul><li>Delete causes file to be deleted </li></ul></ul><ul><ul><li>Select returns columns as required </li></ul></ul><ul><ul><li>Readtext will retrieve contents of the file </li></ul></ul><ul><ul><li>Writetext will modify contents of the file </li></ul></ul><ul><li>File content may be queried with the select statement </li></ul>
    28. 28. File Access <ul><li>SQL access to individual files within a file system enabled via proxy tables </li></ul><ul><ul><li>Contents of file must be readable characters, </li></ul></ul><ul><ul><li>255 char max records delimited with newline </li></ul></ul><ul><li>Useful for exporting ASE data to other applications that cannot otherwise be accessed via CIS </li></ul><ul><li>SQL alternative to bcp out or bcp in </li></ul><ul><li>Create proxy_table command creates a proxy table with one column called “record”, 255 varchar </li></ul><ul><li>To specify column names and datatypes use the create [existing] table command </li></ul>
    29. 29. File Access – How It Works <ul><li>Example, to read individual records within the ASE error log file: </li></ul><ul><ul><ul><li>create proxy_table errorlog external file at &quot; /usr/sybase/ase12_5/install/errorlog &quot; </li></ul></ul></ul><ul><ul><ul><li>select record from errorlog where record like &quot; %server% &quot; </li></ul></ul></ul><ul><ul><li>This query will return all rows from the file that match the like pattern. Rows longer than 255 bytes will be truncated. To specify longer rows: </li></ul></ul><ul><ul><ul><li>create existing table errorlog </li></ul></ul></ul><ul><ul><ul><li>( </li></ul></ul></ul><ul><ul><ul><ul><li>record varchar(512) null </li></ul></ul></ul></ul><ul><ul><ul><li>) </li></ul></ul></ul><ul><ul><ul><li>external file at &quot; /usr/sybase/ase12_5/install/errorlog &quot; </li></ul></ul></ul><ul><ul><li>In this case records up to 512 bytes in length will be returned </li></ul></ul>
    30. 30. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Technologies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Resources </li></ul>
    31. 31. <ul><li>Support for Storing, Searching and Querying XML documents </li></ul><ul><li>Key functionalities: </li></ul><ul><ul><li>XQL to query XML documents </li></ul></ul><ul><ul><li>Store parsed XML documents in ASE </li></ul></ul><ul><ul><li>Query parsed XML documents </li></ul></ul><ul><ul><li>Enhanced Full-Text Search is XML-aware </li></ul></ul><ul><li>Benefits </li></ul><ul><ul><li>Simple: Java classes to extract contents </li></ul></ul><ul><ul><li>General: extraction can be generalized </li></ul></ul><ul><ul><li>Optimized: store once, query many times </li></ul></ul><ul><ul><li>Tight integration with SQL language </li></ul></ul>ASE 12.5 as an XML Server
    32. 32. New XML Features with ASE 12.5.1 <ul><li>Native XML Engine </li></ul><ul><li>Lets you more efficiently store, query, manage and exchange XML data in its native format </li></ul><ul><li>Based on W3C, XML and other open Internet standards </li></ul><ul><li>Integrates SQL and XML Data Management functionalities natively in the same server </li></ul><ul><ul><li>reduces cost of ownership </li></ul></ul><ul><ul><li>reduces number of moving parts in a complex IT infrastructure </li></ul></ul><ul><ul><li>provides a solid data management platform to exploit the power of XML while leveraging existing IT investments. </li></ul></ul>
    33. 33. ASE 12.5.1 Native XML Engine <ul><li>Native XML Processor Running in ASE </li></ul><ul><li>Implements XPath – a significant subset of XQuery language standard for querying XML data </li></ul><ul><li>Upwardly compatible w/ XQL provided w/ ASE 12.5 </li></ul><ul><li>Supports for xml extension of select statements </li></ul><ul><li>Map SQL statements to standard SQLX-XML </li></ul><ul><li>Java-based forsqlj functions allow mapping of SQLX-XML back to SQL script that recreates the data </li></ul>
    34. 34. More on 12.5.1 XML Features <ul><li>Native XML Support </li></ul><ul><li>Store and Retrieve well formed XML data </li></ul><ul><li>Schema independent </li></ul><ul><li>SQL and XML Duality </li></ul><ul><li>Complete interoperability between SQL and XML at language and storage level </li></ul><ul><li>XML Indexing </li></ul><ul><li>Renders high query performance </li></ul><ul><li>Self defined indexes – No user input necessary </li></ul><ul><li>XPATH & XQUERY Support </li></ul><ul><li>Granular results – Returns documents, document fragments, elements </li></ul><ul><li>Wild card support for complex queries </li></ul>
    35. 35. How Does this Relate to Federated Data? <ul><li>Combine ASE and XML Management option with ECDA to </li></ul><ul><li>Query XML documents located in heterogeneous databases </li></ul><ul><li>Use SQL to extract and join data from heterogeneous databases, and convert it automatically to XML format </li></ul>ASE/CIS + ECDA Distributed Heterogeneous Data Proxy Tables Any Client Or Browser SQL XML XML SQL
    36. 36. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Strategies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Summary and Resources </li></ul>
    37. 37. What are Web Services? <ul><li>Self-contained, modular applications that can be described, published, located, and invoked over a network, generally, the Web. </li></ul><ul><li>Generally associated with SOAP, WSDL and UDDI protocol specifications. </li></ul><ul><li>Enable loose coupling of independent applications </li></ul>
    38. 38. What can web services do? <ul><li>Allow applications to share data. </li></ul><ul><ul><li>Discrete units of code; each handles a limited set of tasks. </li></ul></ul><ul><ul><li>Based on XML, the universal language of Internet data exchange </li></ul></ul><ul><li>Can be called across platforms and operating systems, regardless of programming language. </li></ul>
    39. 39. What About Data Available from Web Services? <ul><li>ASE Web Services Option for ASE 12.5.1 </li></ul><ul><li>Expose ASE functionality (Stored procedures within ASE) as Web Services – ASE as Web Services Producer </li></ul><ul><li>Extend ASE to access external Web Services and use the results in the SQL constructs seamlessly – ASE as a Web Services Consumer </li></ul><ul><li>Application data (e.g. Peoplesoft) exposed as Web Services can participate in the Federated Data world </li></ul>
    40. 40. ASE Web Services Producer ASE Any SOAP Client Input: Soap Output: Soap Input: JDBC Output: JDBC ASE Web Services Producer
    41. 41. ASE Web Services Consumer ASE ASE Web Services Consumer SOAP Server 2. RPC Event 3. SOAP request 4. SOAP response 5. Results 1. select 6. Results
    42. 42. Agenda <ul><li>Enterprise Information Integration (EII) </li></ul><ul><li>Technologies for Integrating Federated Data </li></ul><ul><li>Federated Database Access </li></ul><ul><li>Document Access - Full Text Search </li></ul><ul><li>XML </li></ul><ul><li>Web Services </li></ul><ul><li>Summary and Resources </li></ul>
    43. 43. Sybase EII Middleware Complete Access to Federated Data <ul><li>Sybase EII Middleware </li></ul><ul><li>Enterprise Connect Data Access and Mainframe Connect </li></ul><ul><li>ASE Options for EFTS, Content Management, XML Management, Web Services </li></ul><ul><li>Benefits </li></ul><ul><li>Leverage existing hardware/software investment </li></ul><ul><li>Leverage existing IT skills </li></ul><ul><li>Leverage Sybase support for and experience with heterogeneous environments </li></ul>
    44. 44. The Big Picture How Today’s Sybase Solutions Enable Information Liquidity
    45. 45. For More Information <ul><li>Product Documentation - see Enterprise Connect Data Access, Mainframe Connect, ASE Component Integration Services, XML Management Option, Web Connect </li></ul><ul><li> </li></ul><ul><li>Product Information and White Papers </li></ul><ul><li> </li></ul><ul><li>Newsgroups </li></ul><ul><li>sybase.public.gateways </li></ul><ul><li>sybase.public.omni </li></ul>
    46. 46. Sybase Developer Network (SDN) <ul><li>Developer software, services, up-to-date technical information: </li></ul><ul><li>White papers and documentation </li></ul><ul><li>Collaboration with other developers and Sybase engineers </li></ul><ul><li>Code samples and beta programs </li></ul><ul><li>Technical recordings </li></ul><ul><li>Free software </li></ul><ul><li>Join today: or visit SDN at TechWave’s Technology Boardwalk </li></ul>
    47. 47. SDN Presents CodeXchange <ul><li>Forum for exchanging samples, tools, scripts, etc. </li></ul><ul><li>New features enable community collaboration </li></ul><ul><ul><li>Download samples created by Sybase or external users </li></ul></ul><ul><ul><li>Leverage contributions of others to extend Sybase tools </li></ul></ul><ul><ul><li>Contribute code or start your own collaborative / open source project with input from other product experts </li></ul></ul><ul><li>Any SDN member can participate </li></ul><ul><ul><li>Log in using your MySybase account via SDN </li></ul></ul><ul><li>Join the collaboration already underway </li></ul><ul><ul><li> or via SDN at </li></ul></ul>A New Benefit for Sybase Developers
    48. 48. Check Out These Related TechWave Sessions! <ul><li>ASE111 - Integration, XML, and Web Services--Why are They Important to a DBA? </li></ul><ul><li>ASE114 - Querying and Transforming XML with XQuery, XPath, and SQLX </li></ul><ul><li>INT404 - Integrating Adaptive Server Enterprise with Web Services </li></ul><ul><li>Boardwalk demos </li></ul>
    49. 49. INT420 Federated Database Solutions with Sybase Middleware <ul><li>Lynn Weatherwax </li></ul><ul><ul><li>Director of Engineering – Data Integration </li></ul></ul><ul><ul><li>Infrastructure Platform Group, Sybase </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>Q & A