Oracle OpenWorld 2009 Session S311197
Jedi Masters Reveal
Oracle Application Integration Architecture (AIA) Foundation Pack Best Practices
Building Process Integrations
Handwritten Text Recognition for manuscripts and early printed texts
Oracle OpenWorld 2009 AIA Best Practices
1.
2.
3. Jedi Masters Reveal: Foundation Pack Best Practices Building Process Integrations Rajesh Raheja Senior Director, AIA Development October 13, 2009 Ivan Lazarov Director, AIA Product Management
4. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
5.
6. Integration Project Phases <Insert Picture Here> Project Planning Requirements, Architecture and Design Extensions or new Development End to End Integration Testing Implementation and Rollout 5% 35% 20% 35% 5% Typical Integration Project Phases and Time Spent
12. Register Integration Flows for Service Discovery Business Service Repository (OER in AIA 3.0) “ You can’t manage what you can’t see.” --- Forrester Research 1 2 3 4 5 Siebel UI (new/updated Billing Mgmt screens) CustomerPartyEBS (Requestor ABCSs) CRM AIA BRM (Provider ABCSs) GetAccountBalanceSiebelABCS QueryCustomerListPortal ProvImplABCS BRM Web Service (JCA Adapter) BRM API (Opcode)
13.
14. Use Case: Research Service Interoperability UCM Provider ABCS Customer Party EBS UCM Provider Service: SyncCustomerPartyListUCMProvABCSImpl Transformation: XfromUCMPersonToSyncCustomerPartyListResponse.xsl New Requestor ABCS
15. Solution: Research Service Interoperability XSL Mapping Analyzer (XMAN) HTML Report English Translation: Currency Code maps to Person’s Income or Organization’s Account. Lookup values are stored in the CURRENCY_CODE table.
16. Use Case: Validate Service Interoperability for Reuse Customer Party EBS CRMOD ABCS Siebel ABCS E-Biz ABCS JDE ABCS AccountSite/UsageCode PartyLocation/Type “ Address Usage” Attribute Mapping, XREF and DVM Inconsistency
17. Solution: Validate Service Interoperability for Reuse XSL Mapping Analyzer (XMAN) CSV Reports English Translation: CRMOD ignores DUNS Inquiry Identifier. EBiz ignores a Contact’s phone/fax numbers.
18. Solution: Validate Service Interoperability for Reuse XSL Mapping Analyzer (XMAN) CSV Reports English Translation: CRMOD ignores Accounts (uses Party). Only BRM and Siebel care about Billing Profile.
19. Follow the AIA Programming Model for Services AIA Integration Developers Guide
20. Ensure Adherence to Architecture Standards SOA Validation Checklists (Architecture and SOA Component Level)
21.
22. Ensure Adherence to AIA Standards PIP Auditor with Extensible, Categorized and Prioritized Test Cases
24. Ensure Adherence to AIA Standards PIP Auditor –Analyze Co-Deployment between every PIP combination
25. Ensure Adherence to AIA Standards Implementation Governance with Continuous Integration Builds
26. Oracle PIP Development Governance Structure AIA Foundation Pack and Process Integration Pack Projects Oracle Executive Management (Apps Unlimited, Fusion Apps) Integration Steering Committee Technology Governance Content Governance EBO Steering Committee Solution Engineering Architecture Steering Committee PIP Advisory Board Content Standards (CCTS, UN/CEFACT), Horizontal and Vertical Industry Standards Business Process Modeling Standards EBO Working Groups Service Co-Existence and Process Re-Use Change Management Automation Workgroups Technology Standards (WS -*) New Technology Product Adoption Integration Prototypes Source Control, Versioning and Build Processes Hosted Mapping Reports Hosted Audit Automation Solution Engineering Tools and Utilities Customer and Internal Implementation Guidance
27.
28.
29.
30.
31. Don’t Ignore the End User Oracle BPA Publisher – Process Flows Oracle Tutor – Desk Manuals & Procedures Oracle User Productivity Kit – Transactional Training & Performance Support
32.
33. Thursday: Leave with Answers on Oracle AIA… See you next year! Date/Time Location Title Speaker 10/15/2009 , Thursday @ 9:00 A.M. Moscone West (L3), Room 3008 Increase Agility, Better Understand Your Business with Reference Process Models Oracle AIA Product Executives 10/15/2009 , Thursday @ 9:00 A.M. Building Web Services for JD Edwards Enterprise One and Integrating with Oracle SOA Suite Oracle Product Executive 10/15/2009 , Thursday @ 10:30 a.m. Moscone West (L3), Room 3008 10 Things You MUST Know About Oracle Application Integration Architecture Oracle AIA Product Executive 10/15/2009 , Thursday @ 12:00 P.M. Moscone West (L3), Room 3008 Take the Pain Out of SOA Governance Oracle AIA Product Executive 10/15/2009 , Thursday @ 1:30 P.M. Moscone West (L3), Room 3008 Cloud Computing: Integration as a Service Oracle AIA Product Executive 10/15/2009 , Thursday @ 3:00 P.M. Moscone West (L3), Room 3008 Jumpstart your Apps Integration with Best of Breed Reference Architecture Partner Session: Infosys
34. For More Information www.oracle.com/aia http://bit.ly/aiawiki http://blogs.oracle.com/aia http://bit.ly/aiaforum OracleAIA Application Integration Architecture http://www.oracle.com/technology/products/applications/aia/index.html
Talking Points - Topic is AIA Foundation Pack Best Practices when building process integrations -Introduce the Jedi Masters Question: How many have implemented or are in the process of implementing AIA or FMW SOA?
Talking Points [Not Familiar] FP = integration fwk…sits on top of FMW…with pre-built content…speeds development of loose coupled process integrations Contains reference process models, canonical business objects and services engineered to the right level of granularity Comprehensive reference architecture…prescriptive programming model and governance tools and productivity utilities [Not Familiar] Most AIA customers adopting FP do so to transform their IT into a SOA Lot of lessons learned from these as well as internal PIP development Proven to reduce implementation risk and increase productivity AIA consolidated learning into best practices + supporting tools for adoption Think about how these apply these to your implementation To get benefits in your SOA project
Talking Points Best practices around different project phases -Integration projects differ from other development projects in that they are front and back loaded -You are implementing an end-to-end business process so major chunk of effort is upfront in functional analysis and design -Yet, the apps lack common semantics… have technical interoperability issues…requiring a lot more testing in the end -This has some impact on the way you plan for these projects. [Invite speaker]
Talking Points -With front and back loaded projects, waterfall may seem right, but we strongly recommend an iterative approach “iterfall” -For large scale integration projects, iterative approach provides visibility which reduces impl risk -Few things to keep in mind: PLAN * DEPENDENCIES * LEAN DOCS * HARDENING SPRINTS * CONTINUOUS INTG Question: Are you doing agile or waterfall?
Talking Points -Most integration work is related to your business processes, so you want this knowledge in-house -Certain skill sets are a must, get existing architects trained on the new technologies -Ramp up on AIA/SOA using training, docs and public resources (blogs/wikis) -CRP can really help get hands-on with the tech and understand OOTB capabilities
Talking Points -Follow any proven SDLC methodology but remember to include some key AIA/SOA tasks -Bake in time for architecture, design and go-live reviews -Some items would obviously differ from how Oracle does PIP development (highlighted) – solution, governance, rollout
Talking Points -Spreadsheet *template* to help plan implementation effort with past experience providing weighting per task e.g. BPEL -Weights can be tweaked, so use appropriately in context of larger project plan
Talking Points -Major requirements typically have impact on the final solution, so consider the impact and plan appropriately
Talking Points -SOA governance…process of visibility and control over service artifacts…ensure service meets specs for reuse, interop etc -Universal best practice to publish services to repository to get visibility (you can’t manage what you can’t see) -To publish: Analyze business processes, break down flows, register each service in the repository – do same for searching -Once you register your services, you would expect to be done with governance. Unfortunately that is not the case.
Talking Points -Repository itself is not enough for governance. SOA and Repository have been around for some time. -Yet, survey done shows that while everyone agrees to governance, 2/3 don’t practice it. Why? -In my experience, reason is that governance needs to be part of SDLC and automated, productive for developers -In addition, it should also focus more on service interoperability in addition to service discovery. Two real use cases. 176 responses for the survey from customers of Software AG.
Talking Points -You are responsible for synching Customer info to the MDM/UCM system -Repository shows UCM connector for CustomerParty as well as façade EBS interface – is this enough to use the services? -Probably, if it was a simple “GetStockQuote” service with primitive attributes…EAI models are complex…optional attributes -To use the service, you need one critical piece of info…MAPPING…from EBO attributes to UCM as well as XREF/DVM -AIA has a feature called XMAN - an extension to the repository that gives you this information
Talking Points -The HTML report shows you source and target attribute mapping along with DVM and XREF usage information -Here you can see that Currency Code when mapped to Income for Person and Account for Organization Demo for Use Case 1: Explain XMAN and show slicing and dicing based on dimensions. Navigate to http://stadf45.us.oracle.com:7777/AIAReports/ RV2.5 -> Core Apps -> UCM -> CustomerPartyEBO … Provider … SyncCustomerPartyListUCMProvABCSImpl … XfromUCMPersonToSyncCustomerPartyListResponse.xsl Demo for Use Case 2: Show Customization legend at the bottom using already opened page. Demo for Use Case 3: Download CSV and open in MS Excel. Show cleaned up report CustomerPartyEBOMapping_v2.xls. Filter the First column (EBM) for StateName and show the potential DVM to use. Demo for Use Case 4: Show Consolidated CSV and switch to CreateCustomerParty.xlsx – show pivot table analysis and that blank columns include potential issues.
Talking Points -Complex example: two SOA projects developed separately, used different mapping for Address Usage -Because SOA and repository was used, you are asked to reuse CRMOD and Ebiz connectors for a third SOA project -Mapping inconsistency means that you cannot pass Address Usage, hence reuse is limited unless you change all connectors -Real use case from AIA 2.3 discovered very late during QA – how can we find this earlier? XMAN to the rescue again
Talking Points -XMAN CSV reports contain mapping info from all integrations, so you can slice-n-dice e.g. pivot tables -E.g. pivot table showing which apps use what EBM attributes. Non usage is blank/red. Usage has a number -EBO attribute having usage in one app and non-usage in another is potential interop issue between those two apps -E.g. CRMOD-EBiz issue solved; but DUNS and Fax/Phone may be an issue if biz process needs them in the integration Demo for Use Case 1: Explain XMAN and show slicing and dicing based on dimensions. Navigate to http://stadf45.us.oracle.com:7777/AIAReports/ RV2.5 -> Core Apps -> UCM -> CustomerPartyEBO … Provider … SyncCustomerPartyListUCMProvABCSImpl … XfromUCMPersonToSyncCustomerPartyListResponse.xsl Demo for Use Case 2: Show Customization legend at the bottom using already opened page. Demo for Use Case 3: Download CSV and open in MS Excel. Show cleaned up report CustomerPartyEBOMapping_v2.xls. Filter the First column (EBM) for StateName and show the potential DVM to use. Demo for Use Case 4: Show Consolidated CSV and switch to CreateCustomerParty.xlsx – show pivot table analysis and that blank columns include potential issues.
Talking Points -Pivot tables for XREF show mapping at component level e.g. CRMOD ignores accounts, BRM and Siebel looks at billing etc.
Talking Points Technical governance…comprehensive reference architecture and a detailed programming model with the IDG Specifies how APIs should behave, what patterns services should follow and naming conventions Ensures services are standards based, consistent, extensible and technically interoperable – reducing risk Guide is close to 400 pages (27 pages for naming conventions!) – what about productivity?
Talking Points -Check for adherence to architecture standards using AIA validation checklist during all reviews throughout project lifecycle -What about code - would you not write a single line of code till you read 400 pages of doc? Of course not, use Artifact Gen
Talking Points Jumpstart standards compliant service development with code generator Give it the service specification, which EBO, which App, what MEP etc and it generates code outline Automates repeatable, mundane tasks e.g. naming conventions, hooks for error handling and extension points etc Saves time!!! Why Need for enforcing architectural governance policies Adhering to programming models Adhering to naming conventions / rules Need to enforce programming consistencies across projects Automatic Interface (WSDL) creation for ABCS (requestor as well as provider) Code Generation for Inbound & Outbound Service interaction related tasks Invocation of Enterprise Business Services Invocation of Application Services Error Handling Code snippet Code generation for plumbing tasks CAVS Enabling Extension Enabling – induction of injection points EBM Header Population Namespace declarations Automatic Interface (WSDL) creation for ABCS (requestor as well as provider) Code Generation for Inbound & Outbound Service interaction related tasks Invocation of Enterprise Business Services Invocation of Application Services Error Handling Code snippet Code generation for plumbing tasks CAVS Enabling Extension Enabling – induction of injection points EBM Header Population Namespace declarations Automatic code generation for runtime discovery of end points (multiple instances support) Optional code generation for transmission of authorization details Configuration Settings based on Design Patterns EBM Header transformation scripts and transformation for the root element Configuration file properties for the ABCS Will NOT generate Business Message Transformation Automatic code generation for runtime discovery of end points (multiple instances support) Configuration Settings based on Design Patterns EBM Header transformation scripts and transformation for the root Business Message element Configuration file properties for the ABCS Error handling fault policy file Will NOT generate Complete Business Message Transformation
Talking Points -Even with generators, extensions or new flows still need to be checked for standard compliance -AIA has a feature called PIP Auditor that scans source code to check for adherence to standards -Standards codified in extensible, categorized and prioritized XML test cases (110+ OOTB with 90% coverage) -This is used by Oracle for our own PIP development and audits 65k+ artifacts every day! (incl. 15k Industry PIPs)
Talking Points This is the compliance report for a given PIP which gives you statistics based on category and priority The detailed violation report goes down to line/column level so that you can fix them easily
Talking Points Auditor allows you to plan for service co-deployment on a single FMW instance For every PIP combination, it shows shared artifacts between PIPs that may or may not cause issues
Talking Points Of course, all these reports (content and technical) mean nothing if you don’t execute them frequently Oracle PIP development runs these reports on a daily basis for all of our 30 PIPs in development in AIA 2.5 It is monitored closely by the program managers in their weekly status reports as one of the release milestones
Talking Points -Talking about program managers, governance is not about just the technology, but also about supporting organization -SOA warrants a governance structure…Oracle AIA adheres to one driven from the top with experts for content/technology -Benefits come to you in the form of strongly governed pre-defined content, saving you time and effort
Talking Points -In summary, service discovery is not enough, service interoperability is key and only a repository is not enough -Productive tools are important to embed governance into SDLC. Oracle tools are proven by Oracle and used by Oracle -AIA gives you the benefit of a full blown organizational governance structure, so that you don’t have to build it
Talking Points -Develop a policy for end to end lifecycle management including builds, installs and patching
Talking Points -Define desired production topology upfront (even large customers fail to do this) -This can impact not only your environment planning, but the SOA Suite install as well as build/deploy scripts
Talking Points -Do integration testing using a test harness and simulator, which can help with env availability and regression tests. -This shows an example of using the AIA CAVS (you can implement this using other technologies such as iTKO Lisa) -Allows you to test only the integration component or bypass everything except the provider for unit testing The figure depicts the effect of setting the “RoutetoCAVS” property. The red and blue colors represent the effect of changing the property values at the provider or the requestor end.
Talking Points -Finally, too often we think only about the system-centric deployment of process, not the user-centric deployment. -Tools like Oracle Tutor and Oracle UPK help in deploying policy, procedures and training directly to end users.
Talking Points -AIA provides you with best practices and supporting tools for your SOA projects -Download the tools from the AIA Wiki and subscribe to our blog or ask questions on the Forum -Best practices help you increase productivity and reduce your implementation risk AIA OTN Page: http://www.oracle.com/technology/products/applications/aia/index.html Blog: http://blogs.oracle.com/aia Wiki: http://wiki.oracle.com/page/Application+Integration+Architecture or http://bit.ly/aiawiki Forum: http://www.bit.ly/aiaforum Twitter: http://www.twitter.com/OracleAIA
Talking Points -Follow any structured SDLC methodology but remember to include some key AIA/SOA tasks -Bake in time for architecture, design and go-live reviews -Some items would obviously differ from how Oracle does PIP development (highlighted) – solution, governance, rollout
Inability to quantify the ROI for SOA was cited as one of the top inhibitors for SOA adoption - Source: Software AG, Best Practices for SOA Governance User Survey, Summer 2008
Talking Points -Here is how Oracle follows the methodology (internal dev) -The meetings already happen and are repurposed for integration objectives -Strict technical governance around source control, dir structure, naming etc
Transactions should honor transaction commit boundaries and use Adapter services if needed. XSD standards e.g. E.g. Collection types for CRUD Lists of entities
Reuse existing connector services wherever available Develop connector services to be PIP and application version agnostic Use SOAP/ESB bindings to make connectors transport neutral Ensure connectors abstract requestor and provider systems Support multiple endpoint application instances with a single connector Avoid concurrent multiple versions of connector services Use Asynchronous Message Exchange Patterns for scalability Use Resequencer pattern to guarantee FIFO delivery Use ODI for Bulk Data Loads or extremely high performance needs Design for transaction support using adapters and defining boundaries
Talking Points -Finally, SOA governance is not only about content and technology, but also about organizational structure -This is the virtual SOA governance structure that aligns people and processes spanning across different org. -Business rationalization and Project portfolio management on the executive level -Domain experts staffs the various sub committees, both content governance and technology/architectural governance.
CRMOD – JDE1 vs CRMOP – EBIZ has mapping differences for the same EBO. Identified mapping differences CustomerParty, EBIZ to Siebel Order to Cash CRM OD to JDE E1 Lead to Order Identified connector issue on Sync operation Customer Party Sync loop at CustomerParty level (CRM OD) vs CustomerPartyAccount level (Ebiz) Possible Root Cause for issue(s) Inconsistent interpretation by each team Currently there isn’t a mechanism for detail review of mappings as each PIP team maps to complex EBOs such as CustomerParty Opportunity: Ensuring EBO adoption consistency As Ebiz, CRM OD and JDE E1 PIPs talk to PeopleSoft, Retail and other PIPs, there is an opportunity to collaborate on mappings already in place or happening in parallel to ensure consistent mappings This is necessary for true leverage of connectors developed.
Talking Points For upgrades, knowing what custom mappings have been done can be very useful for customers to identify if the upgrade will cause any functional issues. Here the tool can be used to identify the custom extensions in blue color in the HTML report. The CSV report can then be analyzed to find duplicate EBM mappings that also contain extended records.
Talking Points Viewing the html mapping report is fine if you are developing only one connector, but if you plan on adding more than one connector, then it is highly advisable to map the attributes of all the connectors at the same time to ensure semantic consistency across all connectors. To this extent, the tool produces Excel compatible csv files that can help you with that effort. You can start with putting all mappings in one spreadsheet during Functional Design phase, or merge all EBO related mappings to look it up as the source of XREF/DVM information for any given EBM attribute. In case no Oracle PIP mapping exist, utilities are provided that can flatten the EBO XSD schema (or an entire EOL directory) into CSV files, so that it can jumpstart the mapping effort.