Application Engine ETL – Direct access to external databases from PeopleSoft made easy<br />Session #08714<br />Ocotber 6,...
Your Presenters<br /> David Sexton<br />PeopleSoft  ver. 5.01<br />Senior Systems Analyst<br />“Application Engine ETL – D...
University of Utah<br />+<br />=<br />28,000 students<br />18,000 employees <br /> Joining PAC 12<br /> 2002 Olympic Openi...
PeopleSoft Timeline<br />Sep 97 : SA Campus Community / Admissions / Recruitment<br />Jul   98 : FS Purchasing / Accounts ...
!!!!!! Disclaimer !!!!!!<br />This presentation is based on our experience <br />This presentation contains techniques onl...
Overview<br />What is App Eng ETL ?<br />Why we developed it<br />Why not SOA/ Web Services<br />Advanced Use Case<br />Wh...
App Eng ETL – what is it<br />We create an application engine program that leverages <br />the PeopleSoft record object to...
App Eng ETL – what is it<br />Transformthe data in the PeopleSoft record/view<br />Extractthe data from PeopleSoft using t...
App Eng ETL Why ?<br />Started MDM/Data governance/ Integration project Nov 2009<br />Session  Justifying a Data Governanc...
App Eng ETL Why not Web Services?<br />SOA requires public services, data sets your organization has declared public<br />...
App Eng ETL Why not Web Services?<br /><ul><li>Web Services are best suited where the data request is
high frequency
And the response data is
low bandwidth
Good use of web services</li></li></ul><li>App Eng ETL Why ?<br />We do<br />We do<br /><ul><li> Build an SQR
 Creates a file
 Get permissions on the destination server
 SFTP the File
 Create View
 Configure – 1 page
 Run a process</li></ul>Destination does<br />Nothing – data shows up in their data base<br />Destination does<br />Custom...
App Eng ETL Configure<br />
App Eng ETL Run<br />
App Eng ETL Positives<br /><ul><li>All integration objects are managed/show on compare reports
Upcoming SlideShare
Loading in …5
×

Application Engine ETL

2,014 views

Published on

Using the PeopleTools delivered with PeopleSoft, create a fully configurable ETL tool to connect PeopleSoft with external databases.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,014
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
44
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • We create an application engine program that leverages the PeopleSoft record object to dynamically create insert statements. These statements are executed against the external database through a  JDBC connection which is instantiated in PeopleCode.
  • Application Engine ETL

    1. 1. Application Engine ETL – Direct access to external databases from PeopleSoft made easy<br />Session #08714<br />Ocotber 6, 2011<br />
    2. 2. Your Presenters<br /> David Sexton<br />PeopleSoft ver. 5.01<br />Senior Systems Analyst<br />“Application Engine ETL – Direct access to external databases from PeopleSoft made easy”, presentation at HEUG Alliance ’11, March, 2011. <br />“Justifying a Data Governance Initiative” presentation at HEUG Alliance ’11, with Lumen Data, March, 2011. <br />“Justifying a Data Governance Initiative” presentation at Oracle Open World ’10, with Lumen Data, September, 2010. <br />“Putting the Service into SOA – Consuming WSDLs in PeopleSoft”, presentation at HEUG Alliance ‘09, Anaheim, March, 2009. <br />“Web Services for PSQuery”, presentation at Oracle Open World, September, 2008. <br />“Web Services for PSQuery”, presented at HEUG Alliance ‘08, March, 2008. <br />“Extending Applications with Java”, presented at HEUG Alliance ‘07, March, 2007.<br />“Real World Web Services Integration from PeopleSoft”, presented at HEUG Alliance ‘07, March, 2007.<br />
    3. 3. University of Utah<br />+<br />=<br />28,000 students<br />18,000 employees <br /> Joining PAC 12<br /> 2002 Olympic Opening and Closing Ceremonies <br />Tier 1 Research Instition<br /> Medical School and Hospital<br />30 minutes from….<br />
    4. 4. PeopleSoft Timeline<br />Sep 97 : SA Campus Community / Admissions / Recruitment<br />Jul 98 : FS Purchasing / Accounts Payable / General Ledger <br />Oct 98 : Human Resources <br />Apr 99 : Go live HE Upgrade 7.0 to 7.5 / Time & Attendance (KRONOS)<br />May 99 : HR Base Benefits / Payroll<br />Apr 00 : HE HR/SA Upgrade 7.5 to 7.6 / SA Financial Aid<br />May 00 : SA Student Financials<br />Feb 01 : FS Upgrade 6.0 to 7.02<br />Oct 02 : HE HR/SA Upgrade 7.6 to 8.0 / Enterprise Portal 8.4<br />Dec 03 : Upgrade 8.4 to 8.8 Enterprise Portal <br />July 04 : FS Upgrade 7.02 to 8.4<br />Mar 06 : HE HR/SA Upgrade 8.9<br />Feb 10 : FS Upgrade 8.4 to 9.0<br />Oct 10 : HR/SA Upgrade 9.0<br />PeopleTools 8.50.12<br />We have been working with PeopleSoft for awhile<br />
    5. 5. !!!!!! Disclaimer !!!!!!<br />This presentation is based on our experience <br />This presentation contains techniques only endorsed by us, and not necessarily by our employers<br />Hopefully this presentation encourages you to share your experience at next year’s Openworld conference.<br />If you have questions, please ask!<br />
    6. 6. Overview<br />What is App Eng ETL ?<br />Why we developed it<br />Why not SOA/ Web Services<br />Advanced Use Case<br />What is required<br />DEMO<br />Where you can get it<br />
    7. 7. App Eng ETL – what is it<br />We create an application engine program that leverages <br />the PeopleSoft record object to dynamically create insert statements. <br />These statements are executed against the external database through a  JDBC connection which is instantiated in PeopleCode. <br />
    8. 8. App Eng ETL – what is it<br />Transformthe data in the PeopleSoft record/view<br />Extractthe data from PeopleSoft using the SQL and Record objects<br />Loadthe data using a JDBC connection to the external databse<br />
    9. 9. App Eng ETL Why ?<br />Started MDM/Data governance/ Integration project Nov 2009<br />Session Justifying a Data Governance Initiative: Get Approval the First Time #29312 Monday Mar 28 4:30 – 5:30 <br />People want Data !<br />The more the better<br />We wanted something that was<br />Easy to maintain/reusable<br />Easy to consume<br />
    10. 10. App Eng ETL Why not Web Services?<br />SOA requires public services, data sets your organization has declared public<br />https://wiki.doit.wisc.edu/confluence/display/EBOS/Repository+Home<br />We’re not there yet<br />Each data request must be approved individually<br />Each integration is point to point<br />Consuming web services (wsdl) is not a common skill set (on our campus)<br />People are excited about it, but can they use it today?<br />Thank You to Jim Phelps at Wisconsin<br />
    11. 11. App Eng ETL Why not Web Services?<br /><ul><li>Web Services are best suited where the data request is
    12. 12. high frequency
    13. 13. And the response data is
    14. 14. low bandwidth
    15. 15. Good use of web services</li></li></ul><li>App Eng ETL Why ?<br />We do<br />We do<br /><ul><li> Build an SQR
    16. 16. Creates a file
    17. 17. Get permissions on the destination server
    18. 18. SFTP the File
    19. 19. Create View
    20. 20. Configure – 1 page
    21. 21. Run a process</li></ul>Destination does<br />Nothing – data shows up in their data base<br />Destination does<br />Custom process to load the file into their database<br />
    22. 22. App Eng ETL Configure<br />
    23. 23. App Eng ETL Run<br />
    24. 24. App Eng ETL Positives<br /><ul><li>All integration objects are managed/show on compare reports
    25. 25. SQL is in a view – provides a layer or abstration, should not be impacted by upgrades
    26. 26. All integrations are easily visible
    27. 27. Low cost per additional integration
    28. 28. No additional coding, configuration only
    29. 29. Less moving parts
    30. 30. Customers LOVE it !</li></li></ul><li>App Eng ETL what is required ?<br />Server name where external DB is, DB sid<br />Jeff.Robbins.awesome<br />testschema<br />Username/Password for the external DB<br />/PSHOME/Appserv/Classes directory contains<br />driver manager (“java.sql.DriverManager”)<br />The MySql driver (“com.mysql.jdbc.Driver”)<br />
    31. 31. App Eng ETL what is required ?<br />Record/View in Peoplesoft <br />containing the data to be integrated, all fields will be sent<br />A record on the external database with the same structure as the PeopleSoft record/view<br />field names, field types, field lengths of the source and target table must match EXACTLY<br />Generate the DDL from the PeopleSoft record/view, send to external database<br />
    32. 32. Demo configure and run<br />
    33. 33. WHAT ABOUT SECURITY?<br />Gee Dave, this sounds great, but is the transmission of the data secure?<br />Muliple options to secure the tranmission<br />Oracle Wallet<br />Talk to your Sales Rep<br />Encyrption options at the JDBC level<br />Source and destination must have Oracle Wallet<br />Other Vendors supply JDBC drivers that have encrytpion capabilities<br />Customs options we have investigated<br />Custom Java using ports and SSL<br />SSH tunneling with JDBC*<br />http://www.ibm.com/developerworks/data/library/techarticle/dm-0312lurie/index.html<br />
    34. 34. SSH Tunnel<br />
    35. 35. Key Code Pieces<br />SSH script <br /># Referenced in UU_DATA_PUSH application engine. Used for<br /># integrating PeopleSoft data to remote systems.  This script<br /># sets up a secure tunnel to a remote host to connect to a<br /># remote database securely.<br /> <br />local_port=$1<br />remote_host=$2<br />remote_database_host=$3<br />remote_database_port=$4<br /> <br />ssh -f -g -L $local_port:$remote_database_host:$remote_database_port $remote_host sleep 60<br />
    36. 36. Key Code Pieces<br />Call SSH script from PeopleCode<br />&script_path = &path | "ssh_tunneling.sh";<br />&local_port = &properties.get(UU_DP_RQST_AET.URL | ".local_port");<br />&remote_host = &properties.get(UU_DP_RQST_AET.URL | ".remote_host");<br />&remote_database_host = &properties.get(UU_DP_RQST_AET.URL | ".remote_database_host");<br />&remote_database_port = &properties.get(UU_DP_RQST_AET.URL | ".remote_database_port");<br />&shell_command = "/usr/bin/ksh -c """ | &script_path | " " | &local_port | " " | &remote_host | " " | &remote_database_host | " " | &remote_database_port | """";<br />&processId = Exec(&shell_command, %Exec_Asynchronous + %FilePath_Absolute);<br />
    37. 37. Questions?<br />
    38. 38. Contacts<br />David Sexton<br />Senior Systems Analyst<br />University Information Technology<br />University of Utah<br />riderxr200@yahoo.com<br />

    ×