Published on

Implementation of uc4 scheduling in a complex environment

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Conclusion: Standard slide set for intruction presentations. Detail: Use this as a slide pool and remove all slides that are not relevant to the individual prospect/customer/... Ensure that you also remove the respective items from the overview slides.
  • Conclusion UC4:global manages IT-based business processes across all applications, independent of the technical infrastructure. Outline Infrastructure has grown over time Number of different operating systems / hardware platforms Business process owners do not care about IT background They want the business process to work across all applications UC4:global is the broker between business processes and the IT infrastructure Across all applications it manages the IT-based part of business processes It supports all operating system platforms and integrates with solutions for business process management/monitoring In UC4:global such processes are displayed as a graphical workflow Just like in Visio Each process step can be on a different platform
  • Conclusion: The UC4:global architecture is technically superior to all others. It supports the highest number of applications and operating systems. With UC4:global all applications can be managed centrally. Outline: Central Server Nonstop availability Scalability Available on 6 platforms Central logic Central data storage Relational database Support of MS-SQL, Oracle and DB2 UC4.DialogClient Platform-independent Java GUI Applications Well-known applications are controlled directly All SAP solutions, PeopleSoft, Oracle Applications and Siebel Platforms UC4:global supports the highest number of platforms and applications UC4:global supports 3 mainframe operating systems All important operating systems are supported Via agents / executors processes are started and monitored All executors can restart processes For z/OS the product UC4.RestartPlus is needed due to the platform’s peculiarities Connections For comprehensive monitoring Framework integration for most important products Individual integration via standard interface and CallAPI Programming interface for all supported platforms and applications For integration of individually developed applications J2EE Integration of UC4:global in J2EE-environements via Resource Adapter Details: The graphic shows schematically the composition of the UC4 system. The core is the UC4.Server, the central control of each UC4 system. The Uc4.Server is nonstop available and freely scalable. This is shown in the graphic with 4 servers – in fact the number of servers and thus the performance of UC4:global is unlimited. The UC4.Server runs on 6 platforms – Windows, AIX, HP-UX, Solaris, Linux and Linux for zSeries. It is available for all important operating system and hardware platforms. All data is stored centrally in a relational database. In UC4:global no scheduling data is stored on local systems. Hence, UC4 data is highly available and easily protected from unauthorized access. All supported databases (MS-SQL, Oracle, DB2) can be implemented highly available. As database platforms are Windows, UNIX and via DB2 support also z/OS available. The users work with the UC4.DialogClient, a user-friendly, graphical interface. This platform-independent Java interface is the central access for all UC4:global users and administrators. UC4:global integrates important business applications as SAP, PeopleSoft Siebel or Oracle directly via the specific standard interfaces. Processes in these applications are controlled and monitored by UC4:global and can be integrated with processes in all other applications. On operating system level applications are controlled via UC4.Executors. These are thin agent programs with very low resource usage. UC4:global offers the broadest support of operating system platforms in the market. To integrate frameworks and central monitoring solutions UC4:global offers a set of integration solutions. Well-known frameworks like HP OpenView (NNM and Operations), Patrol or Tivoli are integrated via decided solutions. All other frameworks can be integrated via standard technologies like SNMP or e-mail. For integration of all other applications, especially for self developed solutions, UC4:global offers programming interfaces on all supported platforms in the most common programming languages. For J2EE environments UC4:global offers the UC4.ResourceAdapter, a solution according to standards to integrate job scheduling functionality.
  • Conclusion: UC4‘s central architecture is technically the most advanced in the market. It is the basis for technical and economical superiority of UC4. Outline: UC4:global has a central architecture Currently the most modern system architecture Central data storage All job scheduling data is stored centrally no need to synchronize local data thus avoiding inconsistencies failsafe data management and the best possible performance Central logic – performance only needed on decided systems (UC4.Server) Nonstop processing – reliability without 3rd party products Scalability – no performance bottlenecks One architecture for all platforms supports more operating systems than any other job scheduler Thin agents – no resource usage on productive systems Details: The technical advantages of UC4’s central architecture: The central composition of this architecture enables central implementation of system logic as well as central data storage. The centralization enables an easy, cost-efficient and highly available solution. The central scheduling engine, the UC4.Server, is freely scalable because of its process-oriented structure. It can be easily and highly available implemented without 3rd party products. Most job schedulers are dependent on cluster installations for this. Through central data storage UC4:global provides all scheduling data including the JCL and all job reports centrally. Thus, UC4:global users have complete overview of all processes enterprise-wide. The database can be implemented fail-safe – the database vendors offer high availability concepts. In a UC4 system there is only a database connection to the UC4.Server. This makes data traffic manageable and easy to plan. Further, the costs for database licenses can be kept low – especially compared to other systems, which need a database connection for every component. A further advantage of the central architecture are the thin agents, called UC4 Executors. They only need very few resources. The full performance can be used for the actual tasks of the computers. Job schedulers with distributed architecture need lots of performance on productive systems.
  • Conclusion For business-critical job scheduling operations the job scheduler must provide nonstop operations. Details There are three possible ways to implement a scheduling engine (server): Nonstop operations Multiple physical servers process the workload in parallel. If an application instance or a server fails, work continues without interruption. The remaining copies of the scheduling engine take over the failed process’ work. Reconnects are only necessary for agents that were connected to the failed application process / server, i.e. only a fraction of all job scheduling agents. This architecture not only provides nonstop availability (99.999% and higher), but also unlimited performance. The workload is distributed across all physical servers. The alleged weakness of this architecture is the performance consumption for synchronizing the distributed application processes. In fact, resource usage is much lower than in a failover concept, where some expensive hardware stands around idle. Standby/failover A single threaded application, with a standby copy of the application running on an idle server. This approach results in double hardware costs (always one non-productive server). A failover takes some time, as all system components (especially the agents) must reconnect to the new server. In larger environments this may cause several minutes of downtime until the job scheduler is operational again. This architecture delivers very limited performance – the overall performance is limited by the capacity of 1 single CPU. Schedulers with standby architecture: Tidal Enterprise Scheduler, Appworx, Cybermation dSeries (Espresso), Standalone A single threaded applications runs on one server. If this single application process or the hardware fails, operations come to a complete standstill. Not suited for enterprise job scheduling. The only way to use this software architecture for production is installing it within an expensive cluster solution. This architecture delivers very limited performance – the overall performance is limited by the capacity of 1 single CPU. For Enterprise Automation, only nonstop operations is an option UC4:global is the only solution that offers all three options without the need to use third party software.
  • Conclusion Multi-client capability provides big savings. Outline If you have multiple customers Internal or external There are two ways to provide them with automation Standard way A separate scheduler for each customer Separate hardware Separate licenses for operating system, database, etc. Parallel update cycles UC4:global way One installed scheduler One database, ... Separated clients (logical scheduler partitions) for each customer Details “ Standard” scheduler 3 customers –> 3 job schedulers -> 3 databases Maintenance, updates, … for 3 systems Example Redwood uses Freudenberg IT as their main reference customer in the outsourcing industry The success story states that Freudenberg runs 25(!) job scheduling systems UC4:global – multi-client capable N customers -> 1 job scheduler, 1 database Especially valuable for IT service providers,… well, most IT departments are service providers with SLAs to keep With UC4:global, Freudenberg IT could eliminate 24 job scheduling systems (including DB, etc.)… Benefits 1 job scheduling system for any number of customers, business units, departments, … Savings for OS and DB licenses Save job scheduling licenses Simplify system administration
  • Conclusion Reliable data center operations require a process for change management in place. An enterprise job scheduler must provide appropriate features to support change management. UC4:global provides comprehensive functionality to support any change management process. Details In complex environments a simple change can have effects on many other applications and systems. For example: changing one job in a cross-platform job plan, may affect the runtime of all other jobs, resulting in missing SLAs. Change management for job scheduling is therefore essential. Any change must be evaluated for effects (and possible side effects) before being applied to productive systems. The entire change process must be documented to ensure that it has been performed correctly. Change management with UC4:global The key to change management with UC4:global is its multi-client capability. Development, Quality Assurance and productive environments can be set up in completely separated clients Only if QA releases a change after thorough testing, a change is transferred to the productive system UC4:global provides a full audit trail over the entire change process, including the transport between clients Benefits Reliable operations – “accidental” side effects are avoided Cost savings Integrated change management at no extra costs No parallel hardware and software infrastructure for development, QA and production necessary Full audit trail for all changes and transports
  • Conclusion Enterprise Automation requires a Managed File Transfer for exchanging data across the various platforms. Outline To manage IT-based processes independent of the IT infrastructure, transferring files between different platforms is a necessary tasks. UC4:global provides an integrated Managed File Transfer No separate product necessary License included with job scheduling Security UC4:global‘s integrated managed file transfers provides state-of-the-art security features Encryption: transfers are encrypted, using standard algorithms Network ports are configurable Unlike standard ftp, UC4:global does not use any standard port Port scans needed to find UC4 services – detected by any intrusion detection solution Password management Passwords for UC4 FT are securely stored in Login objects No cleartext password needed/used Performance The UC4 MFT can perform parallel file transfers Use available bandwith Accellerate transfers Integrated file compression For networks with low bandwith Save costs No parallel infrastructure Normal implementation: job scheduling agent + ftp server + ftp client => 3 x installation, 3 x maintenance, multiple network connections UC4:global: 1 Executor for job scheduling and file transfer Full audit trail SOX-compliant logging of all file transfer activities Flexibility External file transfer solution (usual appr. 3% of overall transfer volume) can also be managed (via normal jobs)
  • Conclusion UC4:global easily fulfills all base requirements for job scheduling. Outline The base requirement for job scheduling are fulfilled by quite some vendors, although in different depth and quality. Time and calendar based scheduling Start tasks at certain times or days E.g. daily backup, every day at 10 p.m.; monthly financial report at the last workday of the month at 11 p.m., … Event-driven process management Cross platform interdependencies Application integration User friendly GUI
  • Conclusion For end to end automation (= Enterprise Automation) basic job scheduling functionality is not sufficient. Outline Depending on the infrastructure in place, a number of features is required to implement Enterprise Automation Nonstop operations One centralized automation solution -> must not fail Scalability and throughput The automation solution must be able to grow with growing business demands. Full audit trail Quick problem resolution and compliance with legislation such as the Sarbanes-Oxley act requires detailed logging mechanisms. Multi client capability Managing multiple customers / business units without the need for multiple schedulers. Change Management Whether you have implemented ITIL or some other form of IT Governance – the scheduler must provide functionality for change management. Integrated Managed File Transfer A solution for managed file exchange is essential to ensure processing enterprise-wide. Object oriented design Process elements are typically used repeatedly – to avoid replicating them over and over again, process definition must make use of object-oriented technology. Support for virtualized environments With the increasing use of virtualization technology such as VMware, the automation solution must be able to work with dynamically changing infrastructures. Runtime control Automatic reactions on extensive or too short runtimes ensure business continuity. Time zone support Companies with locations in more than one time zone need time zone support to avoid typical user mistakes.
  • Conclusion Re-usable job objects save work time and ensure a full audit trail. Outline UC4:global is designed completely object-oriented (and this has nothing to do with programming methods). Working with UC4:global means working with objects Everything in UC4:global is an object (jobs, file transfers, but also users, user groups, etc.) Objects can be re-used and combined in multiple ways E.g. Users inherit their properties from user groups Job plans can be nested with any depth Job objects can be re-used any number of times, in any context Triggered by a schedule Started by y predecessor within a job plan Triggered by an event Example: a job for cleaning the directory for temporary files on Unix 1 scheduled daily run at 10 p.m. + event-driven run based on file system capacity 100 Unix servers With UC4:global: 1 job object is sufficient for all 200 occasions Other schedulers: 200 jobs necessary Only 1 job must be maintained/modified No mass changes in DB necessary (as with non-OO schedulers) Full audit trail (including modification history) available only with OO schedulers (mass changes in the DB cannot be recorded by the scheduler)
  • Conclusion Reliable enterprise automation needs reliable runtime management. Outline Everything is great, if processes run like planned Processes finish well within the service level agreements (SLAs) (Unfortunately complex, enterprise-wide processes sometimes provide undesired results...) Job Scheduling WITHOUT runtime control Short runtime If, for example, the first job runs very short (e.g. Because input data is still missing) Without runtime control, the negative result can be seen only after the entire job plan (chain, network, ...) has finished After troubleshooting, the entire network must be restarted Lots of time is wasted, SLAs are possibly violated Exceeded runtime A job runs significantly longer than usual (e.g. Because a good business day caused more data than usual) Without runtime control you learn about this exceeded runtime only when SLAs are violated (provided SLA monitoring is in place) UC4:global Enterprise Automation with runtime control UC4:global provides an integrated runtime control React automatically, if the runtime is X % of average/expected runtime E.g.: if runtime is 50% or less of normal runtime or exceeds 150% A job that normally runs 1h: if its finished in 30min or less, or takes more than 1.5h -> automatically start a job / notify administrator Short runtime Missing input data causes a very short job run. UC4:global automatically notifies the operator, using the powerful CallOperator object The job plan is interrupted immediately, unnecessary processing is stopped. The operator accesses the documentation of the short running job object He waits until the input data has arrived and Restarts the job with UC4:global‘s integrated restart management (which is available for all supported platforms) Early notification and integrated restart management save time -> SLA violations can be avoided Exceeded runtime After 90 minutes a UC4.CallOperator informs the operator about the exceeded runtime The job continues to run (more input data than usual is not an error, just a problem) The operator has now plenty of time to work on possible solutions, such as... Allocate more resources to successive processing (in virtualized environments) Skip certain processing parts today Inform users about today‘s delay well in time, i.e. be proactive instead of reactive If certain jobs or processes regularly exceed the runtime... ...UC4:global provides also critical path management with JobSolve to remedy regular runtime issues. Benefits of UC4:global‘s runtime control Quick reaction – avoid SLA violations Increase reliability Keep track of runtime problems without constantly monitoring manually.
  • Conclusion: A full audit trail is a necessity, particularly for publicly traded companies. Details: Most companies, including all publicly traded companies, must pass audits of their financial statements, including all processes leading to those statements. Enterprise Job Scheduling means managing all IT-based business processes, having significant effects on a company’s financial data. Searching for records of particular processes (i.e. jobs, job nets) causes tremendous effort and costs. If it takes three weeks to find a particular record on a backup tape, it delays the audited annual financial statement by three weeks. For publicly traded companies this usually means falling stock prices – a CEO’s nightmare in times of stockholder value, which he’ll share with all responsible managers. UC4:global is the only job scheduler providing a full audit trail out of the box: It collects detailed statistical processing information, The sysout a job produces is stored as job report The integrated version management tracks the complete modification history (see screenshot) Tools for archiving this information and an archive browser are supplied at no extra cost Benefits: No extra costs for collecting, storing and searching processing data Find any record within minutes Save costs for gathering auditing information Ensure quick publication of audited annual financial statement

    1. 1. Enterprise Automation with UC4:global <ul><li>Operations LCM </li></ul>
    2. 2. Enterprise Automation
    3. 3. UC4:global - Supported Systems
    4. 4. UC4:global - Architecture <ul><li>For all scheduling data </li></ul><ul><li>Secure </li></ul><ul><li>Consistent </li></ul><ul><li>Dynamic </li></ul><ul><li>Full audit trail </li></ul>Central Database <ul><li>Non-stop operations </li></ul><ul><li>Scalability </li></ul>Central Server Architecture <ul><li>For all platforms </li></ul><ul><li>Highest number of supported systems </li></ul>One Architecture
    5. 5. Job scheduling operations <ul><li>Standby / failover </li></ul><ul><ul><li>ACTIVE / ACTIVE server </li></ul></ul><ul><ul><li>Little Downtime for Agent/GUI reconnect </li></ul></ul>Situation LCM
    6. 6. Multi client capability – situation LCM
    7. 7. Change Management <ul><li>UC4:global – multi-client capability </li></ul><ul><ul><li>Separate clients for development, QA, production </li></ul></ul><ul><ul><li>One scheduler installation </li></ul></ul><ul><ul><li>Controlled transfer via sophisticated transport mechanism or by export and import xml </li></ul></ul><ul><ul><li>Full audit trail </li></ul></ul>
    8. 8. Integrated Managed File Transfer <ul><li>Secure </li></ul><ul><ul><li>Encryption, </li></ul></ul><ul><ul><li>Configurable ports </li></ul></ul><ul><ul><li>Password management </li></ul></ul><ul><li>Performance </li></ul><ul><ul><li>Parallel transfers, compression, … </li></ul></ul><ul><li>Cost-effective </li></ul><ul><ul><li>No parallel infrastructure </li></ul></ul><ul><ul><li>Transparency – full audit trail </li></ul></ul><ul><li>Flexibility </li></ul><ul><ul><li>Standard ftp products integrated </li></ul></ul>
    9. 9. Job Scheduling <ul><li>Time and calendar based scheduling </li></ul><ul><li>Event-driven process management </li></ul><ul><li>Cross platform interdependencies </li></ul><ul><li>Application integration </li></ul><ul><li>User friendly GUI </li></ul>OVERVIEW
    10. 10. Enterprise Automation <ul><li>Nonstop operations </li></ul><ul><li>Full audit trail </li></ul><ul><li>Multi client capability (= different environments) </li></ul><ul><li>Change Management </li></ul><ul><li>Integrated Managed File Transfer </li></ul><ul><li>Re-usability – Object oriented design </li></ul><ul><li>Dynamic Workload Management </li></ul><ul><li>Runtime control </li></ul>
    11. 11. Re-Usability <ul><li>Job object = re-usable task </li></ul><ul><ul><li>Triggered by events, schedule, job plan, … </li></ul></ul><ul><li>Save operating costs </li></ul><ul><ul><li>Simple maintenance </li></ul></ul><ul><ul><li>Full audit trail </li></ul></ul>
    12. 12. Dynamic Workload Management <ul><li>Flexible usage of servers </li></ul><ul><li>Intelligent workload balancing </li></ul><ul><li>Dynamic provisioning and resource allocation </li></ul>
    13. 13. Runtime control
    14. 14. Status migration jobs -> UC4 Stand van zaken op 11/06/2009 VHS
    15. 15. Refac
    16. 16. Monitoring jobs with HP Operations Center (HPOV) <ul><li>Job failed = returncode ne 0 -> event ovo </li></ul>Via uc4 scripting in de batch job wordt de API van HPOV aangesproken om een event aan te maken in HPOV. We gaan hier de returncode van de batch gaan controleren. Belangrijk is natuurlijk dat de batchen de correcte returncode teruggeven aan UC4.
    17. 17. Live Demo UC4 Demo by operations lcm