TIBCO Spotfire is a comprehensive software platform that allows customers to access, analyze and create dynamic report(s) on their data.
A report that is created/opened using Spotfire is referred to as an Analysis Document and when saved is given an extension of “ .DXP ”.
This DXP file not only contains a series of metadata information, but it also contains references to the data itself and to various other components being part of the document, such as pages, filtering, bookmarks etc.
TIBCO Spotfire Architecture TIBCO Spotfire Automation Services Spotfire Automation Services provides a platform and tools for automating TIBCO Spotfire without user interaction and visible user interface .
TIBCO Spotfire Developer has a complete set of application programming interfaces (APIs) for not only configuring and automating the platform, but extending it with entirely new capabilities as well .
The SDK provides required Spotfire developer resources: the Spotfire Extension Project Template, development assemblies, example projects, and the Package Builder application wrapping the extensions for deployment
A Spotfire extension is the smallest functional unit added to the platform. It is developed in Visual Studio® and is included in a Spotfire add-in, enabling versioning, licensing, deployment and loading.
TIBCO Spotfire APIs & Integration with PP (PP Custom Extension) Pipeline Pilot
Automation Services - Terminology Task An Automation Services Task is the building block for creating a complex Job. Automation Services comes with a number of predefined tasks such as Apply Bookmark or Send Email , but since the platform is extendible, additional tasks may be supplied by other vendors or by end user’s IT organization. Job An Automation Services Job is a collection of Tasks to be executed in a sequential manner. A Job is stored in a Job File. Job File A Job File is a human-readable XML defining a Job. Job Builder The Automation Services Job Builder is an application that runs within Spotfire Professional and provides an easy-to-use interface for creating and editing Jobs
Spotfire Automation Services - Architecture The Job is normally created in the Job Builder and saved as an XML Job File . This file is then sent to the Automation Services Web Service for execution - from the Job Builder window (for testing), - or using the provided ClientJobSender tool, - or by your own custom tool. For each Job in XML format sent to the Automation Services Web Service for execution, a Job Executor process is started. This internal Job Executor logs on to a Spotfire Analytics Server using credentials stored in the Spotfire.Dxp.Automation.Launcher.exe.config file, executes the tasks one by one and then exits when a task fails or all tasks have completed successfully The PP Custom Task will be used when creating a Job and saving the Job file . The ClientJobSender tool will be used by Pipeline Pilot to send the Job file to the Automation Services Web Service . (Pipeline Pilot) ( PP Custom Ext. )
Automation Services – Scheduling a Job To programmatically activate a Job file, Spotfire has provided an application that excepts two arguments. The first parameter is the URL to the web server where the Automation Services is installed and the second argument is the path to the Job file, consisting of the custom task(s). (Spotfire.Dxp.Automation.ClientJobSender.exe) For example: First Argument: http://localhost/SpotfireAutomation/JobExecutor.asmx Second Argument: pathToJobFile/jobfilename.xml "c:tempdxpdataSpotfire.Dxp.Automation.ClientJobSender.exe" "http://laptopdayton/SpotfireAutomation/JobExecutor.asmx" "c:tempdxpdataDXPRequestJob.xml"
Solution – Develop Communication Message Using Spotfire provided ClientJobSender tool.
An XML formatted message will be defined to enable Pipeline Pilot to specify details for the type of DXP file to generate or the updates to make to an existing DXP file.
The PP Custom Task will also be aware of this custom developed XML formatted message and will be translate them into APIs calls to create the DXP file dynamically.
Spotfire Automation Services PP Custom Task Capable of creating a new DXP file or modifying an exiting DXP by using the Spotfire APIs Pipeline Pilot Protocol Capable of activating the PP Custom Task that is part of the Spotfire Automation Services.
Solution – Transmitting / Receiving Messages Spotfire Automation Services PP Custom Task Pipeline Pilot Protocol Activate the PP Custom Task job using Spotfire provided ClientJobSender tool.
Pipeline Pilot formats and places a XML formatted message requesting a DXP file. The details for the type of DXP file to create, the data file to use and other required information is stored in this file.
Pipeline Pilot activates the PP Custom Task to process it’s request for a DXP file.
The PP Custom Task retrieves the DXPRequest.xml file, interprets the instructions from Pile Pilot and using the APIs create/modify a DXP file.
The resultant DXP is stored in the Results folder.
Pipeline Pilot retrieves the resulting DXP file.
REQUEST FOLDER Accessible to both Spotfire and Pipeline Pilot. DXPRequest.xml 1 3 1 2 3 2 RESULTS FOLDER Accessible to both Spotfire and Pipeline Pilot. filename.dxp 5 4 4 5
Implementing the Solution – DXP Request Message format
DXP Request Message:
An XML formatted message structure will be defined for Pipeline Pilot to request customized DXP files from the Custom Spotfire task.
Both Pipeline Pilot and the Custom Spotfire Task will need to be aware of the message structure.
Some form of communication mechanism is required between Pipeline Pilot and the Spotfire Custom Task. One option is to use a common folder or folders to store the requests and the results. Pipeline Pilot and the Spotfire App would each monitor these folder(s) periodically or when activated.
Two sample Analysis Request files were created and stored in the DXPdata folder. The contents of one of the files is shown below. (This request file is for the demo only and the format of the final message is still to be developed.)
The custom task is hard coded to recognize that dxpRequest code of 2 is for a scatter plot and that the data source is the file specified in plotDataFile node and that the resultant DXP file is to be stored as specified in the dxpResultFile node.
Using these simple components the following protocol was created.
Proof-Of-Concept Demo Execution Sequence: 1) Create and Submit Analysis Request (file) On activating the protocol, a user is requested to selected one of the two predefine analysis request files. The selected request file is copied to the DXPData folder and renamed to dxprequest.xml – This is the name of a file that the custom task on Spotfire will look for.
Once the request file is copied to the DXPRequests folder, a another component activates the Automation Services Job on Spotfire to process this request. (there are other options for activating the Automation Services Job besides the method used in this demo.)
<dxpRequest>2</dxpRequest> A Request Type of 2 was implied to mean a Scatter Plot <xDataColumnName>Cost</xDataColumnName> Column name of the x data values
<yDataColumnName>Sales</yDataColumnName> Column name for the y data values
<plotDataFile>c:tempdxpdataSalesData.txt</plotDataFile> Location of the source data
<dxpResultFile>c:tempdxpresultsSalesDataScatterPlot.dxp</dxpResultFile> location for the resulting analysis document.
Proof-Of-Concept Demo Cont. When the Automation Services Job is activated, it executes the code of the custom task and also provides access to the running TIBCO Spotfire instance. A document opened in a running instance of TIBCO Spotfire is referred to as an Analysis Document. The document not only contains a series of metadata information, but it also contains references to the data itself, and to various other components being part of the document, such as pages, filtering, bookmarks etc. Since a data file was specified, the custom task uses the Open method of the Analysis Application and the specified data file. We now have a default analysis document that can be customized with the requested visualization. In this request file the requested visualization is for a scatter plot and so the available method(s) are used for adding a scatter plot visualization to the analysis document using the data file and the X and Y data column names. The analysis document is then saved using the name/location specified in the request file.
Proof-Of-Concept Demo Execution Sequence: 4) Receive Requested Analysis (.DXP File) Once control is returned, the resulting analysis file will have been stored in the DXPResults folder by the custom task that was executed by the Spotfire Automations Services Job.
And finally the analysis file is opened by a component executing the following command line
Note: The following plot was created using the default template provided by Spotfire’s API. The Filters; The Page and the Details-on-Demand were preconfigured but could be customized. In this case the data file and name of the X and Y columns were provided.