Application Engine ETL
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Application Engine ETL

on

  • 1,853 views

Using the PeopleTools delivered with PeopleSoft, create a fully configurable ETL tool to connect PeopleSoft with external databases.

Using the PeopleTools delivered with PeopleSoft, create a fully configurable ETL tool to connect PeopleSoft with external databases.

Statistics

Views

Total Views
1,853
Views on SlideShare
1,852
Embed Views
1

Actions

Likes
0
Downloads
35
Comments
0

1 Embed 1

http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • We create an application engine program that leverages the PeopleSoft record object to dynamically create insert statements. These statements are executed against the external database through a  JDBC connection which is instantiated in PeopleCode.

Application Engine ETL Presentation Transcript

  • 1. Application Engine ETL – Direct access to external databases from PeopleSoft made easy
    Session #08714
    Ocotber 6, 2011
  • 2. Your Presenters
    David Sexton
    PeopleSoft ver. 5.01
    Senior Systems Analyst
    “Application Engine ETL – Direct access to external databases from PeopleSoft made easy”, presentation at HEUG Alliance ’11, March, 2011.
    “Justifying a Data Governance Initiative” presentation at HEUG Alliance ’11, with Lumen Data, March, 2011.
    “Justifying a Data Governance Initiative” presentation at Oracle Open World ’10, with Lumen Data, September, 2010.
    “Putting the Service into SOA – Consuming WSDLs in PeopleSoft”, presentation at HEUG Alliance ‘09, Anaheim, March, 2009.
    “Web Services for PSQuery”, presentation at Oracle Open World, September, 2008.
    “Web Services for PSQuery”, presented at HEUG Alliance ‘08, March, 2008.
    “Extending Applications with Java”, presented at HEUG Alliance ‘07, March, 2007.
    “Real World Web Services Integration from PeopleSoft”, presented at HEUG Alliance ‘07, March, 2007.
  • 3. University of Utah
    +
    =
    28,000 students
    18,000 employees
    Joining PAC 12
    2002 Olympic Opening and Closing Ceremonies
    Tier 1 Research Instition
    Medical School and Hospital
    30 minutes from….
  • 4. PeopleSoft Timeline
    Sep 97 : SA Campus Community / Admissions / Recruitment
    Jul 98 : FS Purchasing / Accounts Payable / General Ledger
    Oct 98 : Human Resources
    Apr 99 : Go live HE Upgrade 7.0 to 7.5 / Time & Attendance (KRONOS)
    May 99 : HR Base Benefits / Payroll
    Apr 00 : HE HR/SA Upgrade 7.5 to 7.6 / SA Financial Aid
    May 00 : SA Student Financials
    Feb 01 : FS Upgrade 6.0 to 7.02
    Oct 02 : HE HR/SA Upgrade 7.6 to 8.0 / Enterprise Portal 8.4
    Dec 03 : Upgrade 8.4 to 8.8 Enterprise Portal
    July 04 : FS Upgrade 7.02 to 8.4
    Mar 06 : HE HR/SA Upgrade 8.9
    Feb 10 : FS Upgrade 8.4 to 9.0
    Oct 10 : HR/SA Upgrade 9.0
    PeopleTools 8.50.12
    We have been working with PeopleSoft for awhile
  • 5. !!!!!! Disclaimer !!!!!!
    This presentation is based on our experience
    This presentation contains techniques only endorsed by us, and not necessarily by our employers
    Hopefully this presentation encourages you to share your experience at next year’s Openworld conference.
    If you have questions, please ask!
  • 6. Overview
    What is App Eng ETL ?
    Why we developed it
    Why not SOA/ Web Services
    Advanced Use Case
    What is required
    DEMO
    Where you can get it
  • 7. App Eng ETL – what is it
    We create an application engine program that leverages
    the PeopleSoft record object to dynamically create insert statements.
    These statements are executed against the external database through a  JDBC connection which is instantiated in PeopleCode.
  • 8. App Eng ETL – what is it
    Transformthe data in the PeopleSoft record/view
    Extractthe data from PeopleSoft using the SQL and Record objects
    Loadthe data using a JDBC connection to the external databse
  • 9. App Eng ETL Why ?
    Started MDM/Data governance/ Integration project Nov 2009
    Session Justifying a Data Governance Initiative: Get Approval the First Time #29312 Monday Mar 28 4:30 – 5:30
    People want Data !
    The more the better
    We wanted something that was
    Easy to maintain/reusable
    Easy to consume
  • 10. App Eng ETL Why not Web Services?
    SOA requires public services, data sets your organization has declared public
    https://wiki.doit.wisc.edu/confluence/display/EBOS/Repository+Home
    We’re not there yet
    Each data request must be approved individually
    Each integration is point to point
    Consuming web services (wsdl) is not a common skill set (on our campus)
    People are excited about it, but can they use it today?
    Thank You to Jim Phelps at Wisconsin
  • 11. App Eng ETL Why not Web Services?
    • Web Services are best suited where the data request is
    • 12. high frequency
    • 13. And the response data is
    • 14. low bandwidth
    • 15. Good use of web services
  • App Eng ETL Why ?
    We do
    We do
    • Build an SQR
    • 16. Creates a file
    • 17. Get permissions on the destination server
    • 18. SFTP the File
    • 19. Create View
    • 20. Configure – 1 page
    • 21. Run a process
    Destination does
    Nothing – data shows up in their data base
    Destination does
    Custom process to load the file into their database
  • 22. App Eng ETL Configure
  • 23. App Eng ETL Run
  • 24. App Eng ETL Positives
    • All integration objects are managed/show on compare reports
    • 25. SQL is in a view – provides a layer or abstration, should not be impacted by upgrades
    • 26. All integrations are easily visible
    • 27. Low cost per additional integration
    • 28. No additional coding, configuration only
    • 29. Less moving parts
    • 30. Customers LOVE it !
  • App Eng ETL what is required ?
    Server name where external DB is, DB sid
    Jeff.Robbins.awesome
    testschema
    Username/Password for the external DB
    /PSHOME/Appserv/Classes directory contains
    driver manager (“java.sql.DriverManager”)
    The MySql driver (“com.mysql.jdbc.Driver”)
  • 31. App Eng ETL what is required ?
    Record/View in Peoplesoft
    containing the data to be integrated, all fields will be sent
    A record on the external database with the same structure as the PeopleSoft record/view
    field names, field types, field lengths of the source and target table must match EXACTLY
    Generate the DDL from the PeopleSoft record/view, send to external database
  • 32. Demo configure and run
  • 33. WHAT ABOUT SECURITY?
    Gee Dave, this sounds great, but is the transmission of the data secure?
    Muliple options to secure the tranmission
    Oracle Wallet
    Talk to your Sales Rep
    Encyrption options at the JDBC level
    Source and destination must have Oracle Wallet
    Other Vendors supply JDBC drivers that have encrytpion capabilities
    Customs options we have investigated
    Custom Java using ports and SSL
    SSH tunneling with JDBC*
    http://www.ibm.com/developerworks/data/library/techarticle/dm-0312lurie/index.html
  • 34. SSH Tunnel
  • 35. Key Code Pieces
    SSH script
    # Referenced in UU_DATA_PUSH application engine. Used for
    # integrating PeopleSoft data to remote systems.  This script
    # sets up a secure tunnel to a remote host to connect to a
    # remote database securely.
     
    local_port=$1
    remote_host=$2
    remote_database_host=$3
    remote_database_port=$4
     
    ssh -f -g -L $local_port:$remote_database_host:$remote_database_port $remote_host sleep 60
  • 36. Key Code Pieces
    Call SSH script from PeopleCode
    &script_path = &path | "ssh_tunneling.sh";
    &local_port = &properties.get(UU_DP_RQST_AET.URL | ".local_port");
    &remote_host = &properties.get(UU_DP_RQST_AET.URL | ".remote_host");
    &remote_database_host = &properties.get(UU_DP_RQST_AET.URL | ".remote_database_host");
    &remote_database_port = &properties.get(UU_DP_RQST_AET.URL | ".remote_database_port");
    &shell_command = "/usr/bin/ksh -c """ | &script_path | " " | &local_port | " " | &remote_host | " " | &remote_database_host | " " | &remote_database_port | """";
    &processId = Exec(&shell_command, %Exec_Asynchronous + %FilePath_Absolute);
  • 37. Questions?
  • 38. Contacts
    David Sexton
    Senior Systems Analyst
    University Information Technology
    University of Utah
    riderxr200@yahoo.com