Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automating Banner Financial Aid Dataload - San Jacinto College

1,127 views

Published on

Automating Banner Financial Aid Dataload - San Jacinto College

  • Be the first to comment

  • Be the first to like this

Automating Banner Financial Aid Dataload - San Jacinto College

  1. 1. Complete Automation of the Impossible, Financial Aid Dataload. November 2, 2015 Mark Deaver
  2. 2. 3 Property of Automic Software. All rights reserved Agenda + Background + Benefits + Requirements + Financial Aid Dataload Automic Example
  3. 3. 4 Property of Automic Software. All rights reserved • The Financial Aid Office would manually download ISIR files via EdConnect. • Financial Aid Office downloaded files for a given aid year once a week. • ISIR Files are merged into one file. • The concatenated file is transferred to the dataload directory. • The financial aid staff manually processed RCPTPxx, RCPMTCH and RCRTPxx Banner jobs. Background
  4. 4. 5 Property of Automic Software. All rights reserved • Increased productivity. • No manual intervention required. • Daily processing of student aid record files. • Streamlined processing ensures: – No Duplication – No Omission of files • Notification of all automated processes. • Output can be immediately emailed as text files or PDF files. Benefits to Complete Automation.
  5. 5. 6 Property of Automic Software. All rights reserved • The Applications Manager internal user id will need read/write access to various directories on the server. • TDClient – Department of Education mainframe communication software. – Does not replace EdConnect. – Method to transfer and receive files via a mainframe or server. – Proper Setup is key – • Password Initialization • Secfile Initialization • Advanced Scripting knowledge. • Automic Applications Manager. • A Documented Process. Requirements
  6. 6. 7 Property of Automic Software. All rights reserved • The Applications Manager internal owner user id will need read/write access to the following directories: • TDClient directory • Automic directory • Dataload directory • Various jobs within Applications Manager require access to various directories. • Dataload process transfers files to and from various directories. Requirements – Applications Manager server level access
  7. 7. 8 Property of Automic Software. All rights reserved • TDClient – Mainframe communication software for sending/receiving files via the SAIG mailbox network. – Uses existing TG number and password. – Software needs to be installed on the same server as the Application Manager. – EdConnect is still used to download non-automated files. – Software is provided free of charge by the Department of Education. – Instructions to install are provided on the Department of Education website. Requirements - TDClient
  8. 8. 9 Property of Automic Software. All rights reserved • Applications Manager can process different types of script files. • Applications Manager can process PERL (pl), SQL and SHELL scripts, etc. • Script files can handle specific functions and greatly enhance automation. • Jobs within Applications Manager can process a script file as it would any other normal job. Requirements – Advanced Scripting Knowledge
  9. 9. 10 Property of Automic Software. All rights reserved • Applications Manager – Automic automation software. Requirements – Applications Manager
  10. 10. 11 Property of Automic Software. All rights reserved • A documented process is a must. • A detailed step by step process that explains the steps, the jobs and the parameters. • A documented process will need the following: – A static set of parameters. – A list of sequential jobs to be accomplished. – List of requested output. – List of email recipients. – List of email verbiage. Requirements – Documented Process
  11. 11. 12 Property of Automic Software. All rights reserved Financial Aid Dataload – Automation Example using Applications Manager.
  12. 12. 13 Property of Automic Software. All rights reserved • The PERL script for the TDClient job is simple yet complex. • The script will be executed by Applications Manager and use parameters in calling job to build a transmit or receive subscript. • The PERL script will process the parameters and execute the subscript. • The subscript is a series of specific commands that tell TDClient what to do with the incoming or outgoing file. • The commands that are required to transmit or receive a file are located in the SAIG Host Communication Guide provided by the Department of Education. • Script is stored in the EXEC directory on the server of installed instance of Applications Manager. PERL script for the TDClient job.
  13. 13. 14 Property of Automic Software. All rights reserved • Example of full receive command: – tdclientc network=saigportal ftpuserid=TGxxxxx reset “transfer=(name=yourname receive=./path/to/the/file/to/receive.txt receiveuserid=TGxxxxx receiveclass=xxxxxxxx)” • The receive command explained: – Starts the FTP tdclientc process. – Network to log onto. – User ID (password is stored during the initialization) – Message Class to receive – Path PERL script for the TDClient job – Cont.
  14. 14. 15 Property of Automic Software. All rights reserved • The TDClient Job will call the TDClient script. • The script is executed using the parameters within the job. • The TDClient job has the following parameters. – Reset Option: Defaulted to Y – Transmission Command: T(transmit) or R(receive) – Path & File Name: The download path. – Message Class: Message class for filetype. – TG Destination Number: TG destination Number. Job for the TDClient Script.
  15. 15. 16 Property of Automic Software. All rights reserved Job for the TDClient Script – Cont.
  16. 16. 17 Property of Automic Software. All rights reserved Job for the TDClient Script – Cont.
  17. 17. 18 Property of Automic Software. All rights reserved • A PERL script was written to issue one command with path & file name. • The PERL script will strip out the header and trailer records. • The Header and Trailer records are annotated with special characters that are explained in the SAIGHostCommunicationGuide.pdf provided from the TDClient documentation. – Header Records are defined as O*N05 – Trailer Records are defined as O*N95 PERL script to Remove Header and Trailer Records.
  18. 18. 19 Property of Automic Software. All rights reserved • All files downloaded through TDClient will have a header and trailer record. • Banner will not process or understand these records. • Applications Manager will call the script using the parameters configured in job • The script will remove the header and trailer records. • The Remove Header job will have the following parameters. – Path & File Name: The file path. Job to remove Header and Trailer Records.
  19. 19. 20 Property of Automic Software. All rights reserved Job to remove Header and Trailer Records – Cont.
  20. 20. 21 Property of Automic Software. All rights reserved Job to remove Header and Trailer Records – Cont.
  21. 21. 22 Property of Automic Software. All rights reserved • The Dataload process has three email notifications. • Three different job statuses will trigger email notifications. – Aborted – Notification that a job has aborted. – Finished – Notification for a job that has finished. – Completed – Similar to finished however sends a different email. Notification Development.
  22. 22. 23 Property of Automic Software. All rights reserved Notification Development.
  23. 23. 24 Property of Automic Software. All rights reserved Notification Development. • Abort Notification uses the Generic Abort Message.
  24. 24. 25 Property of Automic Software. All rights reserved Notification Development. • Finish Notification uses custom text to notify for no files.
  25. 25. 26 Property of Automic Software. All rights reserved Notification Development. • Completed Notification uses custom text to notify for completion of job.
  26. 26. 27 Property of Automic Software. All rights reserved • Process Flow Level Prompts – Variables stored at the top level and referenced throughout each component. • The Financial Aid Dataload process flow has the following components: – TDCLIENT_PROCESS – REMOVE_HEADER – COPY_FILE – RCPTPxx – RCPMTCH – RCRTPxx – WPAALTD Financial Aid Dataload Process flow.
  27. 27. 28 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. - Prompts
  28. 28. 29 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. - Prompts • The Financial Aid Dataload process flow uses substitution variable stored under the prompts tab. • These prompts are assigned to following values: – #1 = Aid Year Code Default value: 1516 – #2 = File Name Default value: igco16op.dat – #3 = Message Class Default Value: IGCO16OP • The shortcut prompts are used throughout the process in each job to reference the value. • Changing the prompts are easier to manage as opposed to changing all non-referenced values in each component.
  29. 29. 30 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – TDClient Component. • The first job in the Dataload Process calls the TDCLIENT_PROCESS. • The TDClient script is executed and use the follow parameters: – Reset Option: Y – Transmission Command: R – Path & File Name: /u02/TDC/TDAccess3.2/incoming/{#2} – #2 substitution variable is referenced and will substitute igco16op.dat – Message Class: {#3} – #3 substitution variable is referenced and will substitute IGCO16OP – TG Destination Number: TGXXXXX. • This script will download all IGCO16OP files to the incoming directory on the server where the TDClient is installed.
  30. 30. 31 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – TDClient Component.
  31. 31. 32 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – TDClient Component. • The TDClient process has a condition that will send an email notification to the financial aid staff if there was no files downloaded.
  32. 32. 33 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – TDClient Component. • Questions regarding the downloading by Message class. – What about multiple IGCO16OP files stored on the SAIG mailbox? • Example: IGCO16OP.dat IGCO16OP.001 • All files with the associated Message Class are downloaded and merged into one file.
  33. 33. 34 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – REMOVE_HEADER Component. • The second job in the Dataload Process is to call the REMOVE_HEADER. • The Remove Header script will be executed and supply the following parameters to the script: – Path & File Name: /u02/TDC/TDAccess3.2/incoming/{#2} – #2 substitution variable is referenced and will substitute igco16op.dat. • The process will strip the header and trailer records from the file. • Example of Header and Trailer record: O*N05TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx, data in file data is file O*N95TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx,
  34. 34. 35 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow.
  35. 35. 36 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – REMOVE_HEADER Component.
  36. 36. 37 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – REMOVE_HEADER Component. • The REMOVE_HEADER has conditions on the component. • The condition will cancel the entire process if no file exist.
  37. 37. 38 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – REMOVE_HEADER Component. • Questions regarding multiple files with multiple header and trailer records. – What about multiple header and trailer records? • Example: O*N05TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx, data in file data is file O*N95TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx, O*N05TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx, data in file data is file O*N95TGxxxxx ,CLS=MSGCLASS,XXX,BAT=xxxxxxxxxxxxxxxxxxxxxxxxxx, • The script will remove all instances of header and trailer records. • The final file will resemble one complete larger file.
  38. 38. 39 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – COPY_FILE Component. • The COPY_FILE component will copy the file from one directory to another. • The COPY_FILE has two parameters – File Source & File Name. – File Destination & File Name. • This instance will copy the file incoming file from the /u02/TDC/TDAccess3.2/incoming directory to /u02/sct/banner/dataload/. • The job will also rename the file to the required naming convention for Banner usage, example {#1}esar.tap. • #1 substitution variable is used to reference the 1516 value.
  39. 39. 40 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – COPY_FILE Component.
  40. 40. 41 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – COPY_FILE Component. • The COPY_FILE has a condition to make an archived copy of the incoming file. • The condition will run a HOST command to copy the file to the archive directory.
  41. 41. 42 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPTPxx Component. • The RCPTPxx is a job in dataload process that loads the incoming ISIR records into the Temporary tables. • The RCPTPxx job can store the job parameters internally without the need to save a parameter set. • RCPTPxx has two conditions assigned to the job. • The first condition runs a host command to remove the {#1}esar.tap file from the dataload directory after the process is finished. • The second condition sends an email notification is the job aborts.
  42. 42. 43 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPTPxx Component.
  43. 43. 44 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPTPxx Component.
  44. 44. 45 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPTPxx Component.
  45. 45. 46 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPMTCH Component. • The RCPMTCH is the next job in the dataload process and attempts to match each temp record with an actual record in the Banner database. • The RCPMTCH job can store the job parameter internally without the need to save a parameter set. • RCPMTCH has two conditions assigned to the job. • The first condition sends an email notification if the job aborts. • The second condition sends an email notification for the job completion. • The email notification attaches all output in the email notification for completion.
  46. 46. 47 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPMTCH Component.
  47. 47. 48 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPMTCH Component.
  48. 48. 49 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPMTCH Component.
  49. 49. 50 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCPMTCH Component.
  50. 50. 51 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCRTPxx Component. • The RCRTPxx is the job that loads the matched records to the Banner database and loads unmatched records to a suspense table. • The RCRTPxx job can store the job parameter internally without the need to save a parameter set. • RCRTPxx has two conditions assigned to the job. • The first condition sends an email notification if the job aborts. • The second condition sends an email notification for the job completion. • The email notification attaches all output in the email notification for completion.
  51. 51. 52 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCRTPxx Component.
  52. 52. 53 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCRTPxx Component.
  53. 53. 54 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCRTPxx Component.
  54. 54. 55 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – RCRTPxx Component.
  55. 55. 56 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – WPPALTD Component. • The WPPALTD is a custom built Banner process that loads the actual SSN into the alternate ID. • The WPPALTD job can store the job parameter internally without the need to save a parameter set. • WPPALTD has no parameters assigned. • WPPALTD has one condition assigned to the job. • The condition sends an email notification if the job aborts. • No other notifications are necessary because no output is generated.
  56. 56. 57 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – WPPALTD Component.
  57. 57. 58 Property of Automic Software. All rights reserved Financial Aid Dataload Process flow. – WPPALTD Component.
  58. 58. 59 Property of Automic Software. All rights reserved Conclusion. • Can the complete automation of dataload be done? • YES. • The process is duplicated for other ISIR data files. – IGSA16OP – IGSG16OP – IDSA16OP • The only time ISIR files are manually processed during the initial load during new aid year setup, the remainder of the year the dataload process is 100% automated.
  59. 59. 60 Property of Automic Software. All rights reserved Conclusion. • Transitioning from year to year is simple. • Requires no human interaction, unless job aborts. • Return on Investment: – Daily import of ISIR files covering all available aid years, instead of being done weekly. – Freed up time for the Financial Aid staff to focus on other processes. – 416 yearly man hours were saved by automating this dataload process, which amounts to around $10,000 a year saved from one process. – The process has been automating dataload going on 4 years. – I have created over 50 process flows are currently being automated through Automic Applications Manager.
  60. 60. 61 Property of Automic Software. All rights reserved Thank you.

×