Reaching Out From PL/SQL (OPP 2010)

2,490 views

Published on

PL/SQL applications do not live on an island - any longer. Increasingly, applications need relate to the rest of the world. Either to make themselves and the services they provide accessible to external parties - that may not speak PL/SQL at all - or to access information or enlist help from external services.

Fortunately, PL/SQL can do much more than invoke other PL/SQL applications or execute SQL. PL/SQL - sometimes in conjunctions with other components in the Oracle RDBMS - provides many inbound and outbound channels for such interactions. This session discusses and demonstrates a number of channels - when and why to use them and how to use them.

The presentation discusses how to publish data to consumers via HTTP, using both XMLDB and the Embedded PL/SQL Gateway - for example to deliver HTML, XML or RSS or to provide REST-style (web)services that are much in demand. The session also discusses the importance of email as a vehicle for human-application interaction, demonstrating how to send and how to act on received emails. An important topic is how to engage in queue based interactions (for example with a SOA infrastructure) and it concludes with how through utl_http or XMLDB and (simple) middleware, the world of SOA, REST and even the internet is ours as well. It includes a demo on 'chatting from the database' (database triggers that send out IM alerts to human agents).

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,490
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • By reaching out I help ensure that the (Oracle) database keeps its rightful place in the IT landscape of organizations ( as well as the database developers)
  • The job is scheduled to run every minuteIt calls upon the Java Stored Procedure to retrieve the newly arrived email-messages – in a specific accountFor each message, an email event is enqueue in the AQ Queue. The new email-messages are subsequently removed.A queue listener has been registered on the queue with email eventsEvery email (event) is dequeued and processedThe email can contain query statements – in some form – that the processor knows how to handleThe reply to the query is then sent in an email message that forms the response to the original email messageNote: other listeners could dequeue the email events and process them in other waysIncoming messages from other sources could be retrieved and processed in a similar way as email messages (VOIP, SMS, IM/Chat, Twitter)The database can both send and retrieve messages from such serversNote: Java is much better equipped than plain PL/SQL (that really only has utl_http, utl_tcp, utl_smtp) for customized, flexible, advanced interaction with a wide range of infrastructures and technologies.
  • Screenshot:Frank sends email to Maggie – with a query on EmployeesAfter some time, a response is sent to this particular email – by the queue listener using the JSP to send(list of employee data, corresponding with “query”)
  • Write functional JavaFor example in JDeveloperUsing the full richness of the IDE and of the Java Language (and libraries)Wrap with PL/SQL Map Java & PL/SQL data types Load into DatabaseCustom classes and libraries (JAR-files)Set appropriate privilegesInvoke JSP through a normal PL/SQL interface
  • Challenge:Link email sent out to response received (some identification is required)Structure of the answer: how to recognize the decision and comment
  • Java is much better equipped than plain PL/SQL (that really only has utl_http, utl_tcp, utl_smtp) for customized, flexible, advanced interaction with a wide range of infrastructures and technologies.Java libraries can help to generate PDF, create and/or read Word and Excel etc.Through JMS – external Java applications can be accessed in a decoupled way. For example: Complex Event Processor
  • http://technology.amis.nl/blog/1201/publishing-data-manipulation-as-an-rss-feed-using-oracle-mod_plsql-and-flashback
  • http://technology.amis.nl/blog/?p=2498
  • 1.8.3.1 Continuous Query Notification EnhancementsContinuous Query Notification has been enhanced to provide notifications when the result set for a query changes on the server. These notifications can be registered at the row-level, rather than just at the table-level. Thus, alerts will only be sent when changes to the specific rows selected have been modified.Applications, such as Radio Frequency Identification (RFID), can now receive notifications directly from the database rather than build this functionality into the application.
  • Mention:11gR1 (11.1.0.7) and 11gR2 external table pre-processor: http://www.oracle.com/technetwork/database/xtables11g2009-preproc-1005-134225.pdfdbms_scheduler File Watcher
  • Mention:11gR1 (11.1.0.7) and 11gR2 external table pre-processor: http://www.oracle.com/technetwork/database/xtables11g2009-preproc-1005-134225.pdfhttp://database.itags.org/oracle/88815/ external jobs
  • Java is much better equipped than plain PL/SQL (that really only has utl_http, utl_tcp, utl_smtp) for customized, flexible, advanced interaction with a wide range of infrastructures and technologies.Java libraries can help to generate PDF, create and/or read Word and Excel etc.Through JMS – external Java applications can be accessed in a decoupled way. For example: Complex Event Processor
  • http://technology.amis.nl/blog/?p=2498
  • http://technology.amis.nl/blog/?p=2498
  • http://www.liberidu.com/blog/?p=244http://tardate.blogspot.com/2007/08/first-tests-of-11g-native-web-services.htmlhttp://download.oracle.com/docs/cd/B28359_01/appdev.111/b28369/xdb_web_services.htm#ADXDB3900
  • Reaching Out From PL/SQL (OPP 2010)

    1. 1. Reaching Out from PL/SQL Engaging with the world
    2. 2. Do not ask what your country can do for you Ask what you can do for your country?!
    3. 3. By all means, ask what the world around you can mean for your PL/SQL application and Also Ask what you (through PL/SQL & the Database) can do for your “environment”?!
    4. 4. Database (SQL & PL/SQL)
    5. 5. Blogger Presenter Author
    6. 6. Database (SQL & PL/SQL) SOA
    7. 7. Database (SQL & PL/SQL) • Channel • Initiative • Frequency • Format • Synchronous vs Asynchronous • Volume
    8. 8. Database (SQL & PL/SQL) Email User Interface Chat (IM) File (Excel, PDF,…) RSS Queue REST Web Servic e File, O/S JDBC, SQL Net, DB Link
    9. 9. Database (SQL & PL/SQL) Email User Interface Chat (IM) File (Excel, PDF,…) RSS Queue REST Web Servic e File, O/S JDBC, SQL Net, DB Link
    10. 10. The Human Side of Things
    11. 11. Challenges
    12. 12. Q
    13. 13. Demo steps • Java to receive email • Load Java and Expose as JSP • Create AQ • Create Job & Schedule that calls JSP and puts results on AQ • Create AQ listener –That sends email with response
    14. 14. 1 • Write Functional Java 2 • Wrap with PL/SQL 3 • Load into Database 4 • Set appropriate Privs 5 • Invoke JSP via PL/SQL Interface
    15. 15. Database Asking Questions
    16. 16. Other scenario with database as initiator of the dialog • An Update of an Employee’s Salary takes place • After Update Row level trigger fires – creates job to send out an email to have someone approve the salary raise – Sets the status of the salary to ‘awaiting approval’ • Email is sent to approver • Approvers sends an answer – Decision and comments • Answer is received & processed by database – Status of salary record is updated
    17. 17. dbms_epg (mod_plsql) http XMLDB http ftp webdav utl_mailutl_file external table
    18. 18. Expose recent HRM Data Changes• Several parties are interested in all changes in the area of Human Resources – We have received a request to make an overview of all recent changes available, preferably in XML format
    19. 19. Flashback Versions • Retrieve all states each record has been in – Every transaction that touched a row left a version of it – Pseudocolumns: xid, operation, starttime, endtime
    20. 20. Example of Employee Versions
    21. 21. Expose HRM Changes in XML document• Create package HRM_CHANGE_HISTORY – Containing procedure EMP_HISTORY that writes an XML document to htp.p with Employee change records • Using DBMS_EPG – Create DAD hrmhistory on path /hrm/* – Authorize DAD hrmhistory for schema SCOTT • Access URL http://localhost:2100/hrm/scott. hrm_change_history. emp_history J(2)EE Application Server Service http://host/../emp_history
    22. 22. Pull EMP change overview from browser
    23. 23. Publish Changes in Pretty Format• A simple XSLT style sheet can be applied in the hrm_change_history package – To transform XML data to an XHTML human readable format
    24. 24. Expose EMP Changes as RSS feed • RSS feed is an HTTP response in a predefined XML format (ATOM) • DBMS_EPG can be used to produce RSS feeds
    25. 25. Pull model: DBMS_EPG/Web Toolkit • Pull model means create on request – Always the most fresh data – Potentially large load on the database as for every request the XML document is recreated – Document is retrieved through a download by an HTTP client • Alternative: push (publish) model – Periodically create the XML document – Make it available to various clients • On the file system, on the web J(2)EE Application Server Service XDB Folder WEBDAV FTP HTTP PL/SQL SQL
    26. 26. Periodically publish XML Report • Create XML DB Folder – Is also a ‘directory page’ via HTTP – And a WEBDAV folder – And an FTP directory • Schedule Job to Create Resource in the XDB Folder – Based on XMLType: the result of a query that collects the desired data • To access: – Go to URL & open in browser – Map Network Drive via WEBDAV – Open FTP session J(2)EE Application Server Service XDB Folder WEBDAV FTP HTTP PL/SQL SQL
    27. 27. Create XML Resource DECLARE CURSOR xmlcur IS SELECT xmlelement("EMPHISTORY", ... l_xml_dir VARCHAR2(30) := '/public/hrmData'; l_doc_name VARCHAR2(30) := 'EmpChangeHistory.xml'; l_emp xmltype; l_res boolean BEGIN IF dbms_xdb.CreateFolder(l_xml_dir) THEN ... some ACL setting up END IF; open xmlcur; fetch xmlcur into l_emp; close xmlcur; l_res := dbms_xdb.createResource ( abspath=> l_xml_dir||l_doc_name , data=> l_emp ); COMMIT; end;
    28. 28. HRM Report via WEBDAV, SQL & HTTP
    29. 29. Viewing&Editing XDB Resource in Excel
    30. 30. Uploading data from file to database
    31. 31. APEX The Ultimate Database User Interface dbms_epg (mod_plsql)
    32. 32. Database Reaching Out to Human partners • DBMS_EPG (or good old mod_plsql) – Expose HTML and RSS & Process HTTP requests • XMLDB – Expose and allow manipulation of resources through HTTP, FTP and WEBDAV • Stored Java Procedures to – Interact with Mail Server, Chat; leverage – Process human file formats (PDF, Excel, images, …)
    33. 33. Machinations
    34. 34. Database (SQL & PL/SQL) Email Chat (IM) RSS Queue REST Web Servic e File, O/S JDBC, SQL Net, DB Link dbms_epg (mod_plsql) http XMLDB http ftp webdav utl_mailutl_file external table utl_http (httpuritype ) http JDBC/ODBC OCI, DB Link utl_tcp Advanced Queuing AQ jms native dbws
    35. 35. Tables & Business Rules Views PL/SQL “Services” Oracle Forms Forms DWH External Partners Web Application Email Server Java based SQL* Loader Modern Architecture and the role of the database
    36. 36. Tables & Business Rules Views PL/SQL “Services” Oracle Forms Forms External Partners Web Application Email Server Java based (extranet) Web Application Internal Users (outside intranet/firewall) “Mobile Enabled” ADF 11g ADF 11g Enterprise Service and Process Bus (SOA Suite 11g: BPEL, Mediator, Human Task, Business Rules, Technology Adapters) file s UCM (Document Management) DWH Operational Dashboard APEX
    37. 37. Tables & Business Rules Views PL/SQL “Services” Oracle Forms Forms External Partners Web Application Email Server Java based (extranet) Web Application Internal Users (outside intranet/firewall) “Mobile Enabled” ADF 11g ADF 11g Enterprise Service and Process Bus (SOA Suite 11g: BPEL, Mediator, Human Task, Business Rules, Technology Adapters) file s UCM (Document Management) DWH Operational Dashboard APEX
    38. 38. Be a good host • Provide services –Views for easy querying –Stored Procedure based APIs • For data retrieval and data manipulation –Queues as way-in (publish) as well as way-out (allow subscription) for events –Query Result Change Notification
    39. 39. API Services for consumers • Views – for encapsulation of (legacy) model, multi-table join, collection integration, SQL hiding • PL/SQL – for encapsulation, cursor based data retrieval, DML hiding, table decoupling View
    40. 40. Views hiding data model USERS PERSONS EMAIL_ ADDRESSES EMAIL_TYPE USERS
    41. 41. Providing a ‘business object’ API • DML API: still the View – aided by Instead Of trigger • Insert of one new row in USERS_VW can actually be four new records – USER, PERSON, EMAIL_TYPE EMAIL_ADDRESS USERS PERSONS EMAIL_ ADDRESSES EMAIL_TYPE USERS * * * *
    42. 42. FIY Data Service API: returning cursors • Fetch it Yourself! • Cursor is opened to be fetched from – Data is gathered and pre-processed before being returned – SQL is wrapped • And can be constructed at run-time – Access privileges are taken care of – Cursors can be handled elegantly in most client languages and technologies • For example Java: JDBC ResultSet Prompt Prompt Prompt Prompt Prompt Prompt Prompt Prompt Prompt A ct ie A ct ie HRM_API CURSOR
    43. 43. FIY Data Service API for Employees • Package HRM_API – Function get_managers – Return SYS_REFCURSOR HRM_AP I EMPDEPT
    44. 44. FIY Data Service API for Employees HRM_AP I EMPDEPT create or replace package body hrm_api as function get_managers return SYS_REFCURSOR is c_mgr sys_refcursor; begin open c_mgr for select empno , ename , sal , hiredate , deptno from emp where job = 'MANAGER' ; return c_mgr; end get_managers; end hrm_api;
    45. 45. Using FIY HRM API from PL/SQL HRM_AP I EMPDEPT declare l_mgr sys_refcursor:= hrm_api.get_managers; l_id number(10); l_name varchar2(25); l_salary number(8,2); l_hiredate date; l_deptno number(10); begin LOOP FETCH l_mgr INTO l_id, l_name, l_salary, l_hiredate, l_deptno; EXIT WHEN l_mgr%NOTFOUND; dbms_output.put_line ( 'Manager '||l_name ||' started on '||l_hiredate ||' and makes $'||l_salary ); END LOOP; CLOSE l_mgr; end;
    46. 46. Cursor can be for Dynamic SQL function get_employees_on_condition ( p_condition in varchar2 ) return SYS_REFCURSOR is c_emp sys_refcursor; begin open c_emp for ' select empno , ename , sal , hiredate , deptno from emp where '||p_condition ; return c_emp; end get_employees_on_condition; declare c_emp sys_refcursor:= hrm_api.get_employees_on_condition ( p_condition => 'job=''CLERK'' and sal < 1200' ); l_id number(10); ...
    47. 47. Cursor can be created inside SQL• The SQL CURSOR expression returns sys_refcursor based on query passed in – Cursor expressions can be nested • Cursor can even fetch data from a Collection (returned for example by a PL/SQL function) • CURSOR expression can be fetched into PL/SQL variable of type SYS_REFCURSOR • Note: SQL*Plus knows how to deal with cursor-types results – It automatically opens them and fetches records
    48. 48. The Super Host Query Result Change Notification
    49. 49. Query Result Change Notification• Continuous Query Notification: – Send an event when the result set for a query changes – Background process calls PL/SQL Handler or Java Listener or OCI client when the commit has occurred – Event contains rowid of changed rows • Used for: – Refreshing specific data caches (middle tier, global context) – (custom) Replication PL/SQL Java Listener
    50. 50. Implementing QRCN in PL/SQL • Set up the privileges for the user – CHANGE NOTIFICATION and EXECUTE on DBMS_CQ_NOTIFICATION • Create the Callback Handler procedure • Create a new QRCN registration • Add query (or queries) to the registration – The queries for which a change in the result set should trigger a call to the Callback Handler • Make sure job_queue_processes parameter is set to a nonzero value
    51. 51. QRCN Callback Handler • Signature is crucial! • The CHNF$_DESC type contains a lot information – Including operation, queries and rowids CREATE OR REPLACE PROCEDURE change_notification_callback ( ntfnds IN SYS.CHNF$_DESC ) IS ...
    52. 52. Registration of QCRN interest DECLARE reginfo cq_notification$_reg_info; l_cursor SYS_REFCURSOR; regid NUMBER; qosflags NUMBER; BEGIN qosflags := DBMS_CQ_NOTIFICATION.QOS_QUERY+ DBMS_CQ_NOTIFICATION.QOS_ROWIDS; /* 1. Construct registration information. 'change_notification_callback' is name of notification handler. QOS_QUERY specifies result-set-change notifications. */ reginfo := cq_notification$_reg_info ( 'scott.change_notification_callback' , qosflags , 0 );
    53. 53. Registration of QCRN interest (2)/* 2. Create registration */ regid := DBMS_CQ_NOTIFICATION.NEW_REG_START(reginfo); OPEN l_cursor FOR SELECT mgr FROM emp WHERE empno = 7902; CLOSE l_cursor; DBMS_CQ_NOTIFICATION.REG_END; END; ...
    54. 54. To look for the registered QCRNs • Inspect Data Dictionary View user_cq_notification_queries SELECT queryid , regid , to_char(querytext) FROM user_cq_notification_queries
    55. 55. To see the QRCN in action… • Perform a DML statement that will apply changes that modify the Query Result • Look for whatever effect the Callback Handler should have – Message to the middle tier via UTL_HTTP – Event on Advanced Queue – Audit Table updated – Asynchronous validations/calculations performed – Some custom-built form of replication – …
    56. 56. More Interaction from & to the database • Files into the database • Database to leverage O/S • Stored Java for non-human interactions • WebServices – publish and consume • Engage Middle Tier as ‘agent’
    57. 57. File Processing into the Database • Files can be –uploaded through dbms_epg –put on FTP/WEBDAV through XMLDB –read by utl_file or dbms_lob (from O/S) and by utl_http (from intranet/internet) –exposed into the database via External Tables –retrieved by Java Stored Procedure (e.g. from email attachment or remote FTP server)
    58. 58. Interaction from Database with O/S • Database can interact with operating system through –external jobs (through dbms_scheduler) –the External Table Preprocessor –utl_file (only for some file manipulation) –Java Stored Procedures –(C-based) External Procedures
    59. 59. EJB JMS Email Libraries for: • Graphical Manipulation • Translation, • Calculation, • Encryption, • … Communication Protocols: http, ftp, tcp ip, sms, grid, JDBC, JDBC-ODBC, .Net, … Operating Syste & File System
    60. 60. The database can ask the middle tier to access and provide services • Get information from (or to) services on the intranet or internet • Publish/Send information to the internet (RSS, email, chat) • Can tell the middle tier – Interesting events – Alert about (im)pending issues, attempted rule violations, … – The outcome of batch calculations • DB vs. Middle Tier: Security, Functionality, Load Balance WebLogic Server 11g SOA Suite 11g AQ Web App
    61. 61. • Database can call a middle tier service using utl_http – Passing in the destination and the message content • This Service sends it onwards to the Google Talk Server WebLogic Server 11g Service CHAT_ MGR Service Happy Chatting – IM from the Database
    62. 62. Database actively informs on events WebLogic Server 11g Service CHAT_ MGR Service EMP
    63. 63. Chat Manager package create or replace package body chat_mgr as c_im_destination CONSTANT varchar2(100):= 'im_account'; c_im_service_base_url CONSTANT varchar2(100):= 'http://host/service?'; c_amp constant varchar2(5) := chr(38); procedure send_im_message( p_msg in varchar2) is l_response varchar2(32000); begin l_response:= utl_http.request ( c_im_service_base_url ||'imDestination='||c_im_destination ||c_amp||'msgBody='||replace(p_msg,' ', '%20') , null ); end send_im_message; end chat_mgr;
    64. 64. Emp Informer create or replace trigger mananager_sal_update_watcher before insert or update of sal on emp for each row when (new.job ='MANAGER' and new.sal > 4000 and new.sal > nvl(old.sal,0)) declare l_msg varchar2(32000); begin l_msg:= 'Selfenrichment ' ||'Salary of '||:new.ename ||' ( manager in Department '||:new.deptno||') ' ||'is changed to '||:new.sal ; chat_mgr.send_im_message ( p_msg => l_msg ); end;
    65. 65. RESTful WebServices • Simple, stateless, light weight, http-based message exchange protocol – Use standard get, put, post, delete operations – Use URL to pass resource reference • Small, easy-to-understand messages • Typically XML or JSON based • Not using SOAP, or the WS*-stack • Often used directly from User Interface – (AJAX enabled) JavaScript
    66. 66. RESTful architecture RESTful PL/SQL API exposed through dbms_epg
    67. 67. RESTful resource navigation
    68. 68. Publish REST-ful HTTP API – directly on top of the RDBMS
    69. 69. Publish REST-ful HTTP API – directly on top of the RDBMS
    70. 70. Consuming REST-ful services • Invoking a REST-ful service is simply making an HTTP request to some URL • Can be done through: – utl_http – httpuritype – Stored Java • Result can be processed as just a string or as XML, JSON or other format – Libraries can help with special formats
    71. 71. Calling ‘formal’ WS* and SOAP based Web Services • Invoking a SOAP WS* service is making an HTTP request to some URL • Can be done through: – utl_http – Stored Java • Request and Response are both SOAP-wrapped XML messages – Using XMLDB functionality including XML Type these can easily by composed, transformed, validated and parsed
    72. 72. (Expose) Native WebServices • Publish and PL/SQL Package as externally available WebService – WSDL is auto-generated – SOAP calls over http and https are supported
    73. 73. Summary

    ×