An approach to build high-performance OPC-XML  DA server system based on MONGODB and Comet                         Xiaopin...
items, it is sent immediately to the clients. In this mode, the        repeat the procedure of requesting to establish a c...
In this mode, after the client subscribes to request, the           C. Cache and history data storageserver and client est...
Figure 6. Data processing module activity diagram             has fast response speed, no major change in performance and ...
Upcoming SlideShare
Loading in …5

An approach to build high performance opc-xml da server system based on mongodb and comet


Published on

J Gabriel Lima -

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

An approach to build high performance opc-xml da server system based on mongodb and comet

  1. 1. An approach to build high-performance OPC-XML DA server system based on MONGODB and Comet Xiaoping Dai Qiyu Xie Department of Computer Science Department of Computer Science Anhui University of Technology Anhui University of Technology Ma’anshan, China Ma’anshan, China xieqiyu@gmail.comAbstract—A simple implementation for OPC XML-DA has been SubscriptionPolledRefresh and SubscriptionCancel, and theachieved by the traditional XML-DA server. However, little has basic operating mechanism is shown in Figure 1. As can bebeen attracted to the design and improvement of the capability. seen from Figure 1 that the pseudo callback mechanism isAn OPC-XML DA server system of high performance is designed based on a query (polled-pull). In order to simulate a callbackin this paper. We use .net asynchronous framework to design better, a high-level subscription mechanism[5] was alsoasynchronous subscription mechanism which is based on the designed, the operating mechanism of that shown in Figure 2.Comet. New caching mechanism is designed by non-relationaldatabase such as MongoDB. Historical data storage is achievedbased on the raw data compression method. Through a set oftesting, we found that this system greatly improve the abilityunder high concurrency and high volume of data. Keywords-high performance; opc-xml; asynchronoussubscription; web service I. INTRODUCTION Modern process control systems face increasing challenges,and are required to integrate the different types of underlyingcontrol networks for different regions, to realize internet-basedreal-time management. But the data standards used in thecontrol systems in the different area are not uniform, therefore, Figure 1. Basic subscription mechanismthe control systems are difficult to be integrated. Using OPC(OLE for Process Control, Process Control Object Linking andEmbedding) technology, these problems can be solved. Hold Time Wait TimeHowever, OPC DA technology is based on COM/DCOM, andis deficient in cross-platform and connectivity. OPC XML-DAtechnology eliminates the OPC DAs defects. In this paper, wedesign XML-DA server system with new asynchronoussubscription mechanism, caching mechanisms, and historicaldata storage mechanism. The system performs well with high minimumconcurrency and high volume of data. response time II. OPC XML KEY TECHNOLOGIES maximum time for service to responseA. OPC XML Specification OPC Foundation released OPC XML-DA 1.0 Figure 2. Advanced Subscription Mechanismspecification[1] in July 2003, for the OPC data access to bedescribed as a Web service[2]. XML-DA specification defines Under the advanced subscription mechanism, the servereight types of Web services, using XML-based SOAP immediately enter a Holdtime after receiving a request for aprotocol[3]. Table 1 describes the OPC XML-DA Web service. SubscriptionPolledRefresh, the Holdtime is used for waiting for data to change. Only after the Holdtime ending, if the valueB. The design of traditional OPC XML-DA servers has changed the request is return to the client, so that the 1) Subscription mechanism number of communications can be reduced. After the OPC XML-DA created a pseudo-callback mechanism Holdtime, if there is no change in the value then enter the(pseudo-callback)[4],and its basic services are Subscription, Waittime.During the Waittime, if there are changes in the value This research is supported by Natural Science Foundation of AnHui Education Bureau of China. (No. KJ2009A007Z) 978-1-4244-8165-1/11/$26.00 ©2011 IEEE 2881
  2. 2. items, it is sent immediately to the clients. In this mode, the repeat the procedure of requesting to establish a connection,clients dont need to poll, but still waiting. The setted minimum waiting for data, closing the connection process.value of maximum waiting time is the servers maximum 2) Cache and history data storage rarely mentioned: Forresponse time, while server will increase the burden of waiting. the caching mechanism, OPC XML protocol has only outlined 2) Data Management Optimization a function of data cache, no mention of how to design caching When the clients request to the server, they and the server mechanism and how to store. The storage of historical data isenter a loosely coupled relationship. Clients can request that done by the relational database, but, the amount of XML-DAsome of the properties are set to optimize the relationship data per day grows in the level of GB, and relational databasebetween the server and the clients and increase server can not complete such a massive task of real-time data data. These properties include: III. IMPROVEMENT AND IMPLEMENTATION OF OPC XML- RequestedSamplingRate DA SERVER EnableBuffering DeadBand Regarding the traditional OPC XML-DA server design By RequestedSamplingRate property, the clients can flaws, this paper made the following improvements:recommend a suitable SamplingRate (sampling time) to server 1) Subscription mechanism: this paper designedaccording to their refresh rate. In the case of multiple clients, asynchronous subscription mechanism based on .net, toby setting EnableBuffering property to true, the server can saveall the changed values which are collected within the actual release processing threads of the server-side in time and tosampling time into one buffer,then sends to the clients achieve the long connection between client and server basedaccording to the client sampling time, to avoid unnecessary on comet. In order to overcome the excessive backgrounddata transmission.DeadBand attribute is used to determine if threads caused by long connecting, and excessive resourcethe value has changed, and only the values changed exceed consumption, net interface is extended, therefore, the clientDeadBand are taken as changed. request is no longer assign to thread directly, but is added to a thread queue in background thread pool through the hashC. Traditional OPC XML-DA servers Architecture algorithm, for a thread to service multiple requests. OPC XML-DA servers architechture[6] can be classified as 2) Cache and history data storage: the non-relationalservice monitoring module, the data processing modules, OPC database mongoDB is used to store real-time and historicalDA interface module. Its functions are: data. Server subscribe for information to all lower nodes. The 1) Service monitoring modules: Monitor OPC XML-DA information is stored in the memory buffer, and updated whenclients SOAP request, receive data from data buffer; start the the node value changes. Historical data is compressed raw data,corresponding data processing module to process the received which stores only the updated node, and publish the completed XML document through the B. Subscription mechanismHTTP protocol. OPC XML-DA uses a pseudo-callback mechanism. Call 2) Data Management Optimization: Responsible for the back the subscribe process is expensive for the system and notinterpretation of the received XML format SOAP message, and very good to reduce the amount and frequency of dataconversion between XML and OPC , and conversion between exchange. Combine .net platform features and Comet principle,the data specification and service types of XML document and this paper have achieved the server push subscriptionOPC interface protocol , and to complete the assembly , split, mechanism in web service, whose operating mechanism isand resolution of XML documents and OPC data formats shown in Figure 3.according to communication requirements. 3) OPC DA interface module: To achieve OPC DA client,and complete access to data on the underlying device. OPCclient defines the transfer of information and access methodsbetween the business layer and between OPC server, andcomplete data exchange with OPC server.D. Flaws in traditional OPC XML-DA serverTraditional design XML-DA server has two major flaws: 1) Subscription mechanism: OPC XML-DA subscriptionis the pseudo callback mechanism. The connection betweenclient and server are low coupling, cross-platform, Non-permanent, and can not provide a callback mechanism. Thesubscription mechanism is still based on the clients repeated Figure 3. Comet subscription mechanismpolling. Every time, in order to get data, the client needs to 2882
  3. 3. In this mode, after the client subscribes to request, the C. Cache and history data storageserver and client establish connection. The server find the data- For high-performance XML-DA servers, the rational useupdate request then return the data immediately. If the data of caching mechanism will significantly improve systemupdate still do not arrive when time out, then return time out performance. In this paper, non-relational database mongoDBmark to the client. After receives the timeout mark from the is used to build data module including cache and history dataserver, client automatically connect to the server. After the storage.client cancels the subscription, the server stops the connectionwith the client. MongoDB[7] is an open sourced, storage-for-document database. MongoDB is easy to store data with object types, With the asynchronous subscription, the requested data are including documents-insert objects and number group, and suitreturned only if the data is updated, therefore, the amount and for a large number of data to insert real-timely, update andfrequency of data exchange are significantly reduced. And query, and have the ability to copy and high scalability thebecause that a reasonable timeout is set, connection times are application programme needed to do real-time data storage.less than traditional XML-DA subscription. However, when ahigh number of concurrent clients happening, because that the In this paper, buffer memory for real-time data isserver is assigned a process for each request thread. established, and all the node information is subscribed from OPC DA server through OPC DA client interface. For the first Therefore, we have achieved and expanded on .Net subscription, all node information saved as a document; laterIAsyncHttpHandler interface based on asynchronous IO when the data is updated, the updated data is stored as acommunication. The so-called asynchronous IO is that the document ,therefore, the compression storage of raw data isthread which the server assigned can be released back to the achieved. The storage format of nodes is a Bson (Binary-thread pool to serve other requests. The request sended by the encoded Serialization Object Notation) document, as shown inclient is monitored by self-defined programme and processed Figure BeginProcessRequest method, and join this request to thethread queue, the thread handle their own requests in the queuecircularly, then return the contents of the response to the clientby the EndProcessRequest way. When a request reachestimeout, the thread will remove the request. The operatingmechanism is shown in Figure 4. Figure 5. mongoDB nodes information stored in the document Data module handling mechanism shown in Figure 6: 1) When the client sends a request, mongoDB memory buffer is searched. If there is no cache, OPC DA client Figure 4. .Net asynchronous HTTP connection mechanism subscribe all the nodes information from the DA server. 2) If there is cache, the node information user requests is The key code is as follows: extracted from the cache and return. public class CometAsyncHandler : IHttpAsyncHandler{ 3) Whenever the DA server push to send data the memory buffer is updated. At same time only the latest node BeginProcessRequest(HttpContext ct, AsyncCallback cb, information store as a document.object extraData){ // Generate a an asynchronous return variable from contextCometResult result = new CometResult(ct, cb, extraData); // Add the request into a thread queue the thread poolCometThreadPool.QueueRequest(new CometRequest(this));} EndProcessRequest(IAsyncResult result){ // Write stream to the response and return to the clientserializer.WriteObject(result..OutputStream,result.Response); result.HttpContext.Response.End();}} 2883
  4. 4. Figure 6. Data processing module activity diagram has fast response speed, no major change in performance and keep a more stable service. Test data prove that the new serverThe key code is as follows: has outstanding performance in a large data flow and highPublic class DataHelper {// Data processing class concurrency. Public Array GetData(list){ TABLE I. THE COMPARISON OF TEST RESULTS // Get the latest data of specified node Node / Server the the the the if(NodeInfoCacheHelper.GetCurrent()!=null) { request type fastest slowest average number of response response response requests // If the memory buffer is not empty (ms) (ms) (ms) being interrupted return TransferData(NodeCacheHelper.GetCur(),list) } 1000/ Traditional 390 440 410 0 // Extract request data from the buffer 30 server 1000/ new server 320 410 330 0 else{ 30 1000/ Traditional 480 800 640 30 Array array = NodeInfoCacheHelper.BeginGetData(); 300 server // Start to subscribe all nodes and to return the latest data. 1000/ new server 370 500 440 2 300 return TransferData(array,list);}}} 5000/ Traditional 440 550 470 2 30 server 5000/ new server 330 520 400 0 IV. INTEGRATION TEST 30 5000/ Traditional 470 900 760 40A. Test platform and the method 300 server 5000/ new server 550 650 600 3 The testing platform: 300 OPC DA SERVER: wincc server and Intel Pentium DCPU2.8GHz win2003 operating system V. SUMMARY OPC XML-DA standard is a current development trend XML DA SERVER: Intel Pentium D CPU2.8GHz win2003 for OPC and is a good method to achieve long-range and cross-server operating system platform data exchange. In this paper, a high-performance OPC CLIENT: Intel Pentium D CPU2.8GHz, winxp operating XML-DA server is built, which has achieved effective real-system time management to the underlying control system, and improved the industrial on-site data access flexibility, and Test method:To perform 4 groups of reading and increased remote access to data from the on-site, and offered ansubscription, namely 1000 nodes / 30 requests, 1000 nodes / effective solution for industrial real-time control. It has great300 requests, 50 nodes / 30 requests, 5000 nodes / 300 significance for improving the on-site industrial management.requests. The request frequency is 1s, server refresh rate is 1s.Read operation tests the fastest response time, the slowestresponse time, average response time. Subscription operation REFERENCEStests server resource consumption and the number of requeststo be interrupted. [1] OPC Foundation.OPC XML-DA specification,version 1.01 [EB/OL].(2004-12-18). Test Results N=KEY&CI=283&CU=20. Test results are shown in Table 2. In the case of small [2] Peng Jin,Wang Jin Biu, “A OPC Specification-- OPC XML-DA”number of requests (30), with the number of nodes increasing metallurgical automation,vol. 28(1), pp. 6-9,2004.from 1000 to 5000, no significant change happened in response [3] SHONNARD A. Understanding SOAP [EB/OL].[2009-12-20].time, and the response is quick. Therefore, it is confirmed that increase in the number of nodes will not have much impact [4] Yuan xiaofang, Wang dong, “Subscribe to the realization of theon both the traditional server and the new server, and they both mechanism analysis in OPC” Computer Engineering and Applications, vol. 45(1), pp. 89-91,2009.have a good frequency response. But once a large number of [5] Liu li, Wang wenhai., “Realize the remote OPC real-time monitoringconcurrent requests (300) happen, the node number increases of research and design based on XML-DA” Control and Instrumentsfrom 1000 to 5000, the traditional server, which has too many in Chemical Industry, vol.34(2), pp. 32-35, 2007.threads, will seriously slow down the response speed. When [6] Kang lingyan Wang jinlin and Zuo zejun, “Process of informationthe performance dramatically change, there are many requests integration method based on the OPC-XML” Computer Engineering,are interrupted. With the same conditions, the new server still vol.35(10), pp. 283-285, 2009. [7] MongoDB. 2884