Oracle integration cloud service (ICS) best practices learned from the field (OOW17)

Oracle Integration Cloud Service
Best Practices learned from the field

2Copyright © Capgemini & AMIS 2017. All Rights Reserved
Oracle Integration Cloud Service Best Practices learned from the field - Oracle Open World 2017
Introduction
Presenter Introduction
 Phil Wilkins & Robert van Mölken
 Co-authors of the book:
Implementing Oracle Integration Cloud
Agenda
 Best usage scenarios (ICS, SOACS or JCS…)
 Best ways to map large messages
 Using on-premises agents & how to monitor
more effectively
 Alternate approach integrating an Oracle DB
 Config Management & CI/CD with ICS
 Use JavaScript function to extend integrations
 Other Considerations
Addition resources
 https://oracle-integration.cloud
• Addresses new features
• Advanced applications of ICS
• Anything we couldn’t fit in the book
Insights into ICS based
upon implementing real-
world use cases

3 Membership Tiers
• Oracle ACE Director
• Oracle ACE
• Oracle ACE Associate
bit.ly/OracleACEProgram
500+ Technical
Experts Helping
Peers Globally
Connect:
Nominate yourself or someone you know: acenomination.oracle.com
@oracleace
Facebook.com/oracleaces
oracle-ace_ww@oracle.com

Best Usage Scenarios

When is ICS the best choice?
Low volume / small
scale, simple, DIY ok
JCS or ACCS + Event Hub
Modula Design or even all out
Microservice
OSSI
Allow DIY ‘integration’, user
centric processes e.g. Eloqua to
dropbox
Technical Orchestration,
Adherence to published
standards required, complex
end of orchestration needs,
need to tune workloads
Lift & Shift
– just moving workload
ICS
Orchestration with some
complexity, cloud centric
integration, LoB solutions,
PCS  API  ICS (map,
orchestrate)  end system
Green field?
Special cases
- Java beans for
technical logic
NOT
orchestration
Pricing model
- Pricing models can
push you up or down
- Higher volume more
likely to push you up
If we can avoid crossing the
divisions then we’re cost
effective
No. Occurrence of Use Cases
Complexityofworkload
high volume / complex Microservice & performance sensitivity …
- high elasticity needed at run
- execution speeds need to be optimal
- more discipline to avoid tight coupling
SOA CS

ICS & PaaS Conceptual Capability Overlap
Data Integration Applications Integration Process Integration
Application Development
SOACS
(inc MFT)
PCS
APIPCS
JCS
ACCS
ODICS
Golden
Gate
Event Hub
(A)VBCS
MCS
Development /
Application
Centric
Data Centric
Process
Driven
Event
Triggered
Integration
Key
ICS
OMCS S
S
I

Integration Cloud Position
Data Integration Applications Integration Process Integration
Application Development
SOACS
(inc MFT)
PCS
APIPCS
JCS
ACCS
ODICS
Golden
Gate
Event Hub
(A)VBCS
MCS
Development /
Application
Centric
Data Centric
Process
Driven
Event
Triggered
Integration
Key
ICS
OMCS S
S
I

ICS or SOA CS or SSI or OMCS or JCS or ACCS or …
 ICS + JCS or ACCS for Java beans to realize logic operations e.g. …
 Ability to use Atom feeds direction difficult
 Abstract complex logic into small beans
 MFT + Post Processor to turn into a stream of events
 Don’t recommend massive iteration in Orchestration
 Event stream – likely to be simpler
 If remain batch centric consider ODI CS
 Pub / Sub
 to overcome solution volatility
• As system availability & connectivity is more volatile – you can’t suspend upstream processes so easily as they aren’t
yours to suspend
• Systems not in your control so can’t humanly manage
 Need to filter – then need OMCS
 Lookup as a means to hold config parameters e.g. toggle suppression behaviour
 You don’t want upstream bulk load processes sending middleware into a frenzy e.g. system priming

Large Mappings

Large mappings can be a pain to edit in ICS UI
 Business Objects exposed by Adapters (e.g. Salesforce) can contain 100,000 + elements
 ICS mapper UI has a limited buffer cache, only loading elements partially
 Can take some effort to find correct element. Need a lot of scrolling!
 Searching an element is on alphabetic order and only one result at a time
How can we improve this experience?
 Detach mapping UI to extend buffer cache by larger chunks
 Use filter to find element(s)
 Expand on specific node to hide all other siblings / parents
 Expert mode: export transformation and edit offline

Detach mapping UI to extend buffer cache by larger chunks
 Limit of 50 elements in cache
 Limit of 100.001 elements on screen

Detach mapping UI to extend buffer cache by larger chunks
Scrolls faster than embedded table, but this is
still slow if you don’t know where to look!
If you found element to map then click on
detach again to embed the table to original size

Use filter to find element(s)

Expand on specific node to hide all other siblings / parents

Expert mode: export transformation and edit offline
Once exported, changed and imported in ICS
integration. Mappings can’t be changed in ICS UI
Notice that target only shows elements
belonging to CaseComment, no extensions

Agents

Use of ICS Agents - Observations
 Clarity of role for Executable Agent vs Connection Agent
 Exe Agent naming can be confusing, sometimes called ICS on-premises
 Clarity regarding what happens within the executable agent vs in the cloud
 Some hesitation given ‘black box’ to be deployed into the environment
 Where security is only perimeter based / no internal segmentation / security in depth is not implemented
 Clarity of what can and can’t pass back to the cloud and how
 Need to promote mind set regarding patch maintenance –
 Not patching agent versions could mean they stop working with cloud
 ICS roadmap to provide need to move towards self patching processes?
 Need for maintenance windows agents are currently single instance so can’t run rolling patch processes
 Execution agent monitoring
 Cloud monitoring is linked to the heartbeat – not log shipping

Comparison of Connection & Execution Agent Anatomies
Cloud
On-Premises or Other Cloud
Connection Agent Deployment
Source data
obtained via
JCA based
connections
Acquired
data sent to
svr
Connection
Agent
Managed
Svr
Phone home to manage
status and configuration Data channelled from
source to cloud
Client system e.g.
Secured Web API
Cloud Server
Managed
Svr 1
Managed
Svr 2
Admin Svr
Config &
Transient Data Cloud Server
Managed
Svr 1
Managed
Svr 2
Admin Svr
Config &
Transient Data
Phone home
reporting only exe
agent status
Execution Agent
Managed
Svr
Admin Svr
Config &
Transient Data
Execution Agent Deployment
Is realised by packaged
WebLogic server
Is realised by packaged
WebLogic server
Connections such
as WSDL & REST
can work on-
premises
Single admin server
including Integration
Designer and managed
server to execute
Configuration
data & transient
data during
execution
Multiple
servers running
to provide
resilience
Admin Server provides UI.
Managed Svr executes
integrations
Multiple
servers running
to provide
resilience
Admin Server provides UI.
Managed Svr executes
integrations
Client system
e.g. DB
Client system e.g.
Secured Web API files
File connectivity
available

More Effective Monitoring
 ICS provides basic capabilities around looking at logs and tuning what is logged
 Logs are rolling and depending upon activity may only last a couple of days
 Need to harvest logs into monitoring solution e.g. Oracle Log analytics
 Oracle Log analytics will need an IaaS node to deploy collector to
 Would be good if Oracle could deploy to all platforms Log Analytics configured but switched off
 Then just configure in cloud UI Analytics instance to use to enable
 For Oracle Log analytics solution – deploy IaaS (compute) node with agent that then connects and harvests
information
 For Agents – monitoring is down to you!
 A-Team created a simple Integration & scripts to allow simple monitoring

Alternate approach integrating an Oracle DB

Alternate approach integrating an Oracle DB
 ICS provides adapters for Oracle RDBMS, MySQL, DB2 and SQLServer database
 One downside: both need the on-premises Agent, no direct cloud connections
Are there any alternatives?
 ICS can call RESTful services without use of addition software, so …
 Create RESTful APEX services
• Use Oracle Application Express's SQL Workshop tool
• Supports both SQL and PL/SQL in queries
 Expose database using ORDS (REST Data services)
• Automatically generate REST APIs for individual tables and views
• Manually generate more advance APIs (e.g. stored procedures, nested tables)

RESTful APEX services

Expose database using ORDS (REST Data services)
So I just use APEX RESTful services then?
Do you want to choose own programming language without client drivers or proprietary networking?
 Use ORDS instead:
 Java EE application, runs on app server or standalone (Jetty)
 Access the database same way as other external services
 JSON all-the-way for easy integration
 Automatically generate basic CRUD APIs
 Generate advanced APIs (e.g. stored procedures, nested data in multiple tables)
 Can be developed using SQL Developer
 Can expose RDBMS, JSON Document Store and NoSQL

Continuous Integration – Continuous Delivery

Goal of Config Mgmt & Continuous Integration with ICS
Integrate
Test
Deploy
Configure
abandon config e.g. changes to lookups,
source prev. version
Copy base line versions
and promote to new
environment
Revert integration
e.g. doesn’t work
Latest changes
fail test -
rollback
Deploy lookup data, record
of connection values
Deploy Stub settings/test
data – execute tests
Regular snapshots to
allow rollback
Regular snapshots to
allow rollback
Track versions been
deployed to
environments
Track versions been
deployed to
environments

Configuration Management and CI/CD
 ICS has versioning NOT configuration management for integrations
 Try to use versioning for config mgmt. of every change & you will get swamped by integrations
 No automated means to provide automated comparison
 Consider CI style processes for promotion from dev to prod – as you would for SOA, you should with
applications (ConfigSnapshot)
 Fortunately ICS API can help …
 APIs to exporting & importing
curl -X GET -u jstein:welcome1 -H “Content-Type:application/json” -o ./TestProject.iar
https://integration.us.oraclecloud.com/icsapis/v1/integrations/TEST/01.00.0000/export
curl -X PUT -u jstein:welcome1 -H "Content-Type:application/json" -F file=
@"TestProject.iar;type=application/octet-stream"
https://integration.us.oraclecloud.com/icsapis/v1/integrations/import

Configuration Management and CI/CD – REPL Style
 REPL is a Python tool built by the A-Team to
make config management tasks easier than
CURL scripts
 Just install Python, plus dependencies
 Create a config file then …
 Interactive commands to perform processes e.g.
• Import/export
• (de)activate integrations
• Change connections
 Paste interactive commands into a text file
and then run as a script!

Configuration Management and CI/CD
- Simple use of
GIT/SVN to manually
check in/out and
release tag
- Use .iar files
- Scripts to allow
differences to be
identified within an iar
- Nightly checkpoint
process scripted
- Test setups included
- Choreograph tests
- Automation of
promotion process
(include activation of
integrations deployed)

JavaScript

Use JavaScript function to extend integrations
 New capability since 17.2.5 for JavaScript functions (Libraries)
 Functions running server-side, so no browser/client-side capabilities/APIs
 Functions can’t be created within ICS, but need to be uploaded
 as part of a library in both JS and JAR file (collection of JS files) formats.
 Use functions to perform mathematical calculations or data manipulations
 Functions can call other functions within the same JavaScript library
 Functions can currently only be used in Orchestrations using the Callout activity

For example a date-utils library

Libraries can be used in Orchestrations through a callout

Conclusion

Other Considerations
 Adapter complexity – not hiding need for Base64 etc
 Adapter – challenge of HCM deprecating part of the adapter as underlying API changes
 Be information rich – filter on the target not the source
 Exploit API Definition/Management tools such as API Platform
 Understand maintenance windows and their impact.
Get the Book – Visit The Website!

Phil Wilkins
Senior Consultant
Capgemini UK
Phil.Wilkins@capgemini.com
@PhilAtCapgemini
Robert van Mölken
Senior Consultant
AMIS (NL)
robert.van.molken@amis.nl
@robertvanmolken
https://oracle-integration.cloud
Future
titles…

Oracle integration cloud service (ICS) best practices learned from the field (OOW17)

More Related Content

What's hot

Viewers also liked

Similar to Oracle integration cloud service (ICS) best practices learned from the field (OOW17)

More from Phil Wilkins

Recently uploaded

Oracle integration cloud service (ICS) best practices learned from the field (OOW17)