Db trends final


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • As we discussed, the infrastructure that may be needed to provide a specific service or create a new business model can be very complex. This complexity can be evident both physically and logically. In addition, this infrastructure may change rapidly as the business changes. The first steps in managing a critical business service surround identifying the specific infrastructure that the service needs to run. Essentially, we need to map out the path that the business transactions travel along to better understand how to manage the service. (next)
  • Depending on what business model is being adopted, a company may be required to integrate into a “virtual” enterprise to deliver specific services or business models. For example: If an organization has acquired an EProcurement application such as Ariba, the “service” will extend into more than just the “in-house” enterprise. You would want to manage the supplier’s system (as it relates to the transaction). You would want to have information from the Shipping company, etc.. No longer can we only think in terms of the “enterprise” we must always look further to determine if the service in question requires integration to a virtual enterprise and determine where management makes sense. Now, we will present the BMC Software approach to Systems Management in this new Era. (next)
  • Internet commerce services fail many times because there are weak infrastructures supporting them. 1.) Unreliability - The best of the largest retail e-commerce sites achieve 99.4% availability (Keynote Systems, benchmarker). The worst sites achieve only 86.7% availability and costs retailers millions of dollars in revenues. 2.) Complexity - If something crashes, it can take IT personnel hours to figure out where the problem is - routers, servers, operating systems, middleware, etc. 3.) Fragile - Simple upgrades and bug fixes can leave systems in limbo for weeks while IT personnel figure our which configurations won't crash every few hours. 4.) Vulnerable - Infrastructure today still has weaknesses in which a single event can bring down an entire system. For example, standards on single machine types, like NT, leave whole web sites vulnerable to viruses.
  • Unplanned outages - unexpected problems. End up having to recover the environment: the longer the recovery, the longer the unplanned outage becomes. Point to customer : Industry is focused on unplanned outages on open systems and trying to minimize number of surprises in the environment. Most companies say that more outage time is spent on planned outages than on unplanned outages. Planned outages are driven by change in the environment. This is becoming a much larger issue for customers. If you could reduce the amount of time on the planned outages, you can have a significant impact on the amount of downtime. BMC is the only vendor in industry that focuses on both planned and unplanned outages, and can reduce the overall impact of outages from an application perspective for a customer.
  • Unplanned outages - driven by problems Element failure Performance degradation Capacity limitation Application logic error Transaction backout Data corruption Planned outages - driven by change Database maintenance Application migrations Configuration upgrade Data propagation Source of 70% and 30% is CIO feed back during Customer briefings and meetings.
  • There are two methods of accessing your DB2 databases using Java: JDBC SQLJ
  • XML is getting a lot of publicity these days. If you believe everything you read, then XML is going to solve all of our interoperability problems, completely replace SQL, and possibly even deliver world peace. Okay, that last one is an exaggeration, but you get the point. In actuality, XML stands for eXtensible Markup Language. The need for extensibility, structure, and validation is the basis for the evolution of the web towards XML. XML, like HTML, is based upon SGML (Standard Generalized Markup Language) which allows documents to be self-describing, through the specification of tag sets and the structural relationships between the tags. HTML is a small, specifically defined set of tags and attributes, enabling users to bypass the self-describing aspect for a document. XML, on the other hand, retains the key SGML advantage of self-description, while avoiding the complexity of full-blown SGML.
  • XML provides a standard for data interchange. By incorporating XML into DB2 you can more directly and quickly access the XML documents. With DB2 you can search and store entire XML documents using SQL. You also have the option of combining XML documents with traditional data stored in relational tables. When you store or compose a document you can invoke DBMS functions to trigger an event to automate the interchange of data between applications.
  • However, there are some problems with XML. For example, standard web browsers do not currently understand the descriptive tags. This problem will be alleviated in time as XML-capable web browsers come to market. Another problem with XML is not really the fault of XML, but of market hype. There is a lot of confusion surrounding XML in the industry. Some folks believe that XML will provide metadata where none currently exists or that XML will replace SQL as a data access method for relational data. Neither of these assertions are true. There is no way that any technology, XML included, can conjure up information that does not exist. Humans must create the metadata tags in XML for the data to be described. XML enables self-describing documents. It does not describe your data for you. And XML does not do what SQL does. Hence, XML cannot replace SQL. SQL is the standard access method for relational data. It is used to “tell” a relational DBMS what data is to be retrieved. XML is a document description language. It describes the contents of data. XML may be useful for defining databases, but not for accessing them.
  • Db trends final

    1. 1. Trends and Issues Impacting Database Management Systems Circa 2004Craig S. MullinsDirector, Technology Planning/BMC Software, Inc.http://www.craigsmullins.com
    2. 2. General Industry Trends• Complexity • Heterogeneity • Speed-to-Market • Rapid change • Buy versus Build • Consolidation • Follow the Leader• Lack of Resources • Skilled Technicians – A lot of PFCSKs and IROCs • Time – Already overworked • Budget
    3. 3. DBMS Industry Trends• Rapid DBMS Versioning• Enabling for the Internet• Online, real-time …• Java and .Net• XML• Multimedia• Procedural logic• ERP and CRM• Mo’ data! Mo’ data!• Integration
    4. 4. New DBMS Versions• Analysis of New Features – Check all Requirements • Hardware and Software• Planning the Upgrade – Impact to system, applications – Scheduling• Fallback Strategy• Migration Verification
    5. 5. DBMS Subsumes Functionality• XML• ETL and Propagation• OLAP• Multimedia• Objects• Logic/Code – triggers, UDFs, stored procedures
    6. 6. The Internet• From DBA to eDBA Internet-Age DBA Skills
    7. 7. It’s About Change...Source: Gartner Group
    8. 8. Increasingly Complex Enterprise Infrastructure
    9. 9. A Virtual, Extended Infrastructure
    10. 10. Internet Infrastructure Weaknesses Problem Symptom Effect Sporadic crashes Unplanned Unreliable for no apparent outages reason Operators do not Simple problems Complex understand how to result in long resolve problems outages Fragile IT mgrs. Must Long debugging debug innocuous cycles for new changes releases Systems must be Vulnerable Viruses and bugs rolled back to attack all systems clean backups at onceSource: Forrester Research
    11. 11. The Cost of an Outage
    12. 12. E-Business Applications Fail to Deliver Service Because...Unplanned outages - Unplanned Outages Planned and Planned outages - driven by problems hr. workdays x Staff Shortages - 18 driven by change x Short Implementation Times x Management x Application failure x Executive Demands for Web Presence x Maintenance x x Element Failure Scalability Management Tool x Migrations x Performance Keep Up With Own Scripts x Unable to x x Capacity limitsWeb Load/Growth Version Unpredictable x x Transaction Success of Site Management x Unknown Backouts x Propagation x Customized Environments x Need Flexibility of Management
    13. 13. Where is Application Downtime? 30% of• Planned vs. Unplanned Outages – Planned outages represents 70% of Outages 70% of application downtime. – Just 30% is due to unplanned outages and 50% of the unplanned downtime is due to problems during planned downtime.
    14. 14. Impact on eDBAs• Downtime is not tolerated – Downtime was never “good” but it was tolerated – No longer, though as we move from availability to e-vailability with intelligent techniques & solutions• Avoiding downtime with automated tools – “On the fly” operational tuning • ALTER SYSTEM - Oracle9i • SET SYSPARM - DB2 V7 – Monitor performance across multiple platforms – Redundant systems – HACMP, RAID, etc.
    15. 15. Example: Keeping Your Systems Up! Monitor SYSPLEX ARCHIVE FAILED? Action DB2A ADD LOG NOTIFY AMLChange ZPARM DB2B Monitor DB2C Action EDM POOL FAILED? LOGS LOGS LOGS + LOG
    16. 16. Example: Recover with no Down Time! LOGS LOGS LOGS Undo/Redo SQL Log Analysis DB2A DB2B No need to take the data Data off-line to run DB2C SQL against it!
    17. 17. Online and Real-time• The need for more and more availability drives online and real-time maintenance – The DBMS begins to allow for more changes to be made during normal operations – The DBMS begins to gather statistics and performance metrics during normal operations – ISVs deliver more online, real-time features and functionality that the DBMS does not yet deliver – Less manual-intervention required
    18. 18. Database Design and Web Time • When the Web is involved everything becomes “rush-rush” - do it now! • Don’t let database design suffer - take your time and do it right. • Apps are temporary but data is forever! – If you do not believe this, then consider: “How often has your organization re-entered or re- keyed data into a new database when the data already exists elsewhere?”
    19. 19. Prepare for Global Scope Who is accessing the database?  Internal and External users  Local, National, and International  Now versus then 24 x 7 x 365¼
    20. 20. Impact on the DBA  Where is the performance problem?  Most experts agree that 75% to 80% of performance problems in relational applications is caused by poor SQL or gateway HTML application code, but on the web . . . ISP XML init.ora ZPARMs connection CGI 3GL bridge/router/hub DB2 Java operating system Connect DNSASP hardware HTTP network cabling Java application code SQL SQL*Net applet database schema network software
    21. 21. Java’s Popularity is Skyrocketing Java Software Market ($U.S. Millions) 2000 1750 1500 1250 1000 750 500 250 0 1997 1998 1999 2000 2001 2002Source: IDC
    22. 22. Why Java?• Portability• Make web pages active and dynamic• Like C/C++ with a smaller footprint• Lower cost of change (DLLs)• Download changes from the web• But – slower - interpreted, not compiled
    23. 23. Java and Databases: Two Methods• JDBC – Enables Dynamic SQL from Java – Uses API (CLI)• SQLJ – Enables Static SQL for Java – Uses embedded SQL
    24. 24. Java Alphabet Soup• J2EE - Java 2 Enterprise Edition – Standard services and specifications for making Java highly available, secure, reliable, and scalable for enterprise adoption• EJB - Enterprise Java Beans – Components that contain the business logic for a J2EE application
    25. 25. Impact of Java on DBA• Application tuning – Must understand Java • To provide guidance during design reviews – Is the problem in the SQL or the application • How can you tune the application if you do not understand the language (Java)? – Optimizing SQL is not enough since it may be embedded in poor application code – Must understand the SQL techniques used • JDBC and SQLJ
    26. 26. Microsoft .NET• ... is a set of Microsoft technologies for connecting people, systems, and devices• ... allows Internet Servers to expose functions to any client named as .NET web services• … enables software to be delivered as a service over the web• … is designed to let many different services and systems interact
    27. 27. Java versus .Net• ...designed to enable • …designed to enable applications to be development in deployed on any multiple languages as platform as long as long as the application they are written in is deployed on Java Windows
    28. 28. The Rush to XML
    29. 29. What is XML?• XML stands for eXtensible Markup Language. – Like HTML, XML is based on SGML (Standard Generalized Markup Language) – HTML uses tags to describe the appearance of data on a page, whereas XML uses tags to describe the data itself, instead of its appearance. – Allows documents to be self-describing, through the specification of tag sets and the structural relationships between the tags.
    30. 30. XML is a Meta Language• XML is actually a meta language - a language used to define other languages. – These languages are collected in dictionaries called Document Type Definitions (DTDs). – The DTD stores definitions of tags for specific industries or fields of knowledge. So, the meaning of a tag must be defined in a DTD before it can be used. – The DTD for an XML document can be either part of the document or stored in an external file.
    31. 31. Sample XML DTD<!DOCTYPE CUSTOMER [<!ELEMENT CUST (first_name, middle_initial, last_name, company_name, street_address, city, state, zip_code, country*)><!ELEMENT first_name (#PCDATA)><!ELEMENT middle_initial (#PCDATA)><!ELEMENT last_name (#PCDATA)><!ELEMENT company_name (#PCDATA)><!ELEMENT street_address (#PCDATA)><!ELEMENT city (#PCDATA)><!ELEMENT state (#PCDATA)><!ELEMENT zip_code (#PCDATA)><!ELEMENT country (#PCDATA)>]
    32. 32. Sample XML<CUSTOMER><first_name>Craig</first_name><middle_initial>S.</middle_initial><last_name>Mullins</last_name><company_name>BMC Software, Inc.</company_name><street_address>2101 CityWest Blvd.</street_address><city>Houston</city><state>TX</state><zip_code>77042</zip_code><country>U.S.A.</country></CUSTOMER>
    33. 33. Why is XML Important?• XML is used for exchanging and sharing data – Inter- and intra-organization• XML may be used to define database structures – and vice versa DOCTYPE XML definition e DDL lorem ipsum . . . Generat Physical Database e XML Generat
    34. 34. XML to DBMSs Add XML functionality to the DBMS • Extender capabilities similar to IBM DB2 video, image, audio, and other multimedia data types • Combine UDT, UDFs, and triggers for functionalitys XML document stored in a column -or-s XML components stored as parts of multiple columns in multiple tabless Formulate XML documents from existing tabless Search XML documents text and sectionss XQuery capabilities
    35. 35. XML and Data Management• Middleware: Software called from your • Wrappers: Software that treats XML application to transfer data between XML documents as a source of relational data. documents and databases. For data- These products typically query XML centric applications. documents using SQL. For data-centric• XML-Enabled Databases: Databases applications. with extensions for transferring data • Content Management Systems: between XML documents and Applications built on top of native XML themselves. Primarily for data-centric databases and/or the file system for applications. content/document management. For• Native XML Databases: Databases that document-centric applications. store XML in "native" form. For data- and • XML Query Engines: Standalone engines document-centric applications. that can query XML documents. For• XML Servers: XML-aware J2EE servers, data- and document-centric applications. Web application servers, integration • XML Data Binding: Products that can engines, and custom servers. For data- bind XML documents to objects. Some of and document-centric applications. these can also store/retrieve objects from the database. For data-centric applications. Source: XML Database Products by Ronald Bourret http://www.rpbourret.com/xml/XMLDatabaseProds.htm
    36. 36. Impact of XML on DBA• Database definition – Perhaps a new way to create databases • DTDs to build a database schema • Create DTD from a database schema • Interface to data modeling tools• Data access – Database extenders – XML defines the data it contains• New XML DBMS products? – Tamino (Software AG) – Xperanto (IBM)• Over-enthusiasm!
    37. 37. Logic andthe DBMS
    38. 38. Triggers vs. Functions vs. ProcsUP I N DD Code Code Code S EA E LT Triggers R EE T T UDFs E function( ) if this then that else do this stuff return x Stored end Procedures → Code Code Code SQL
    39. 39. Procedural DBA Duties Stored Procedures Performance MonitorsDE DV E Admin. DB2E Triggers B ProcessL UO GP External Design Libraries Review Functions
    40. 40. Role of the Procedural DBA DBCO Impl (COMMIT in em entation Ensuring proc, write o r guide) Reuse n m stratio ini set) Des i CO Ad rder, proc DB r firing o Rev gn igge iews (tr EX An PLAI On Call a ly sis N Coding for DBCO Complex Abends Queries QL i n gS Debuggi ng Schema un SQL T R esolutionDBCO = Database Code Object
    41. 41. Non-Traditional Data
    42. 42. “Universal” Data Complex Compound Docs Graphics Video Design Data Spatial Data Images Temporal Data TextUnstructured Structured Audio Existing Databases Seismic Data Simple Source: Gartner Group
    43. 43. Multimedia = Big Databases Object Typical Size HD TV 200 MB/second Feature-length, high-resolution movie 5-6 GB High-resolution video 3 GB/hour Feature-length movie 2 GB Video 1 GB/hour Radiologic image 40-60 MB Color image 20-40 MB Large image 200 KB-3 MB Text 30-40 KB/page Check image 45 KB
    44. 44. Integration and Federation• DBMSs are adapting to “handle” more types of non-traditional data – Spreadsheets – Word documents – Presentations• How? – Integrate the data into the DBMS – Federate and manage the data “where it lies”
    45. 45. DBMS to Manage All Kinds of Data Federated DBMS • A federated approach allows the DBMS to manage data where it existsSpreadsheet
    46. 46. DBMS to Manage All Kinds of Data Or integration… • An alternate approach “sucks” the non-relational data into the DBMS to be managedSpreadsheet
    47. 47. Autonomic ComputingReal Time SelfStatistics Managing Intelligent Automation Virtual Reorg Database Correct and Wizards Notify
    48. 48. Data Keeps Growing
    49. 49. Phenomenal Data Growth• “Global 2000 companies double the amount of data they own every year, while the average dot-com’s data doubles every 90 days.” – Mike Ruettgers, CEO of EMC Corp., Oracle Open World 2001• “Inside IBM we talk about 10 times more connected people, 100 time more network speed, a 1000 times more devices and a million times more data.” – Lou Gerstner, CEO of IBM Corp., eBusiness Expo 2000• A recent research note from Giga Information Group estimates that there are about 201,000TB, or about 197 petabytes, on the planet. Of course, this is just an estimate that Giga deems to be accurate within an order of magnitude (that is, within a factor of 10).• McKinsey & Company reports a CAGR of 76% for data storage.
    50. 50. Relational Database SizeSource: Gartner Group
    51. 51. Database Size Issues• Technology enables larger databases• Web, multimedia, data warehousing, and data mining drive up database size• Disk drives increase in capacity but speed of access does not keep up with capacity increases• Cost of storage decreasing; so why not store more data? But... – What data do users need to store? – How long must it be maintained? – What are they willing to pay? 0010 001010100 101011101011 101011010010101 01001001101010111000011 11001010100100101000100101
    52. 52. The Database Environment • A lot of choices! – Vendor, platform, and architecture of DBMS MVS, OS/390, z/OS Enterprise Windows NT / 2000 / XP - Parallel Edition Unix Departmental AIX Personal Sun Solaris Mobile (PDA) HP-UX Linux others? Others (VSE, VMS, MPE, OS/400, etc.) Desktop OS Windows 98 / ME / XP Linux Mac?Adabas, Teradata, PostgreSQL, Supra, Compaq Non-Stop SQL, Ingres, IMS, IDMS, Datacom, Teradata, others...
    53. 53. Heterogeneity Plus! • The DBA’s knowledge must span the entire enterprise – Operating Systems – Networking Protocols – Programming Languages – Business Objectives – And so on…
    54. 54. Impact on the DBA• Unrealistic Expectations – Impossible to master everything • Education is the first thing cut! – Impossible to specialize in a heterogeneous shop• Reactive mode is encouraged – Even though proactive mode is optimal – Who looks for more problems when they don’t have enough time in the day to solve the problems of those complaining the loudest. – DBA uses YBWJ method
    55. 55. Worldwide Spending for ERP Packaged Software $30 $25 $20 Manufacturing Distribution $15 Human Resources Financial $10 $5 $0 1995 1996 1997 1998 1999 2000 2001 2002Source: IDC
    56. 56. The Computing “Platform” Evolution This is the current “Customer Service Apps” battleground, NOT the DBMS2000+ “Base” Applications SAP, Oracle, Baan, PeopleSoft The DBMS DB2, Oracle, SQL Server Network (LAN/WAN) TCP/IP, Novell, WinNT Communications SNA, TCP/IP, NFS The Operating System MVS, UNIX, WIN (NT,95) Proprietary (IBM+) to Open60’s Hardware (Wintel, HP/Intel)
    57. 57. The Data Continuum o n pti Perce ence eri Wisdom p Ex Knowledge Integration ext › Multiple points Cont Information Integration of knowledge › Systems Understanding › People › Knowledge Business Systems applied Data › Data in › Technology to Context store and report PeopleSimple› Numbers › Identify› Characters › Capture› Date › Maintain› TimeComplex› BLOBs› CLOBsVaried› Content Management
    58. 58. The Data Continuum Wisdom Knowledge Knowledge + Application Information Technology + Systems People Data Models Knowledge Data Data Dictionary Management RepositoryDBMSFilesSpreadsheetsGraphicsFoldersEtc.
    59. 59. The Data Continuum o n ptiAn Example: rce Pe ence eri Wisdom Exp Knowledge ext Cont InformationData
    60. 60. Other DBMS Market Trends• ODBMS – no longer a threat to become “mainstream” but…• “Post-Relational” DBMS? – XML DBMS – The Associative Model – Multivalue DBMS• In Memory DBMS
    61. 61. Enter the DBA• The job of database administration is getting increasingly more difficult as database technology rapidly advances adding new functionality, more options, and more complex and comp- licated capabilities...
    62. 62. The DBA is a “Jack of all Trades” OS/390 C++ Windows V$ TablesLinux Oracle SQL XML Java DB2 Informix applet Unix MQConnect DB2 DNS gateway HTML VTAM CGI TCP/IPISP connection ZPARMs 3GL database schema COBOLASP operating system Java HTTP hardware SQL Server network software bridge/router/hub application code VB network cabling CICS SQL*Net JCL
    63. 63. So What is Needed?• Intelligent automation of DBA tasks – because no one has all the skilled resources they need – frees up more DBA time They work hard so you don’t have tooooo… “Scrubbing Bubbles”Intelligent Automated
    64. 64. A DBA Control PanelDBA-focused on DBA tasks, not focused on systems managementtasks like framework products… but integrated with systemsmanagement functionality where it makes sense.
    65. 65. Contact Informationhttp://www.craigsmullins.com/dba_book.htm Craig S. Mullins Technology Planning Craig_Mullins@BMC.com http://www.craigsmullins.com http://www.bmc.com http://www.craigsmullins.com/cm-book.htm