1. SOP - Master Page 1
Standard Operating Procedures
Version 02
SOP Sample for LinkedIn Display Purposes. No content included.
Author: Dan Schwartz Approved: 03-September-2013
Approved By:
Managing Director: Mike Kreutzjans
Managing Director: Dan May
Director of Operations: Dan Schwartz
2. SOP - Master Page 2
REVISIONS
The following revisions were applied to the original version:
Revision Date Version No. Author Description
02-Sep-2008 1.00 Dan Schwartz Created initial template
07-Oct-2008 1.01 Dan Schwartz
Updated template & creation of first version of the
table of contents.
08-Oct-2008 1.02 Dan Schwartz
Incorporated additional content sent from Ron
Lindstrom into the outline.
03-Dec-2008 1.03 Dan Schwartz Updated the PDU and IDC Developer Notices sections.
26-Dec-2008 1.04 Dan Schwartz
Completed initial versions of sections:
1 – Overview
2 – Scope
3 – About SpryWare
4 – Service Level Agreement (SLA)
29-Dec-2008 1.05 Dan Schwartz
Section & Change:
4 – SLA Updated to match current SLA
7 – Data & Hosting – Updated
8 – Customer Service Guide – Updated
9 – Release Management – Passed ownership of this
section to Ron Lindstrom.
13-Jan-2009 1.06 Dan Schwartz
Reworked the general outline. Too many updates to
document. Doc now ready for distribution to Ron,
Terry & Tony. Specific procedures can now be added.
13-Jan-2009 1.07 Dan Schwartz
Updates following a high level review with Ron.
Document distributed to Ron, Terry, Tony, Dan M, Mike
K and SharePoint.
21-Jan-2009 1.08 Dan Schwartz
Wrote Cisco Systems VPN section.
Client Side MIS.XML
PDU Outlet Configuration
12-May-2009 1.09 Dan Schwartz
Server Side MIS.XML
Remote Access
o Assent
o Base-2
o BNY
o Flow Trading
o Hold Brothers
o OptionsXpress
o Sterling
o UNX
MISVerify
25-Oct-2009 1.10 Dan Schwartz
MIS Upgrade Procedures
New Employee Procedures
Added CRM
Added Infrastructure Section
3. SOP - Master Page 3
02-Aug-2010 1.11 Dan Schwartz
Reconfigured & updated Email/SharePoint/CRM
under Intermedia
Removed content covered in stand alone docs
Added Bugzilla to the Toolsets section.
12-Jan-2011 1.12 Dan Schwartz
Server Decommission Procedures
Iron Mountain
03-Sep-2013 02 Dan S. Updated Org Chart
4. SOP - Master Page 4
TABLE OF CONTENTS
1. Overview.............................................................................................................................................10
2. Supplementary Documentation..........................................................................................................11
2.1 Stand Alone Document Directory...............................................................................................11
2.1.1 SpryWare Technical Overview ............................................................................................11
2.1.2 SOP – Active Directory ........................................................................................................11
2.1.3 SOP – Backup and Restore Procedures...............................................................................11
2.1.4 SOP – BGInfo.......................................................................................................................11
2.1.5 SOP – Bugzilla......................................................................................................................11
2.1.6 SOP – Client Remote Access Procedures ............................................................................11
2.1.7 SOP – CRM User Manual.....................................................................................................11
2.1.8 SOP – Data Editing...............................................................................................................11
2.1.9 SOP – DMS Prep..................................................................................................................11
2.1.10 SOP – DMS Linux MIS Build Procedures..............................................................................11
2.1.11 SOP – DMS Windows MIS Build Procedures.......................................................................11
2.1.12 SOP – Entitlement Procedures............................................................................................11
2.1.13 SOP – ESXi QuickGuide........................................................................................................11
2.1.14 SOP – General Office...........................................................................................................11
2.1.15 SOP – HDS ...........................................................................................................................11
2.1.16 SOP – IDC Developer Notices..............................................................................................11
2.1.17 SOP – Intel to SloarFlare NIC Upgrade Procedures.............................................................11
2.1.18 SOP – Iron Mountain...........................................................................................................11
2.1.19 SOP – Nagios Manual..........................................................................................................11
2.1.20 SOP – New Client Procedures .............................................................................................11
2.1.21 SOP – New Employee Procedures.......................................................................................11
2.1.22 SOP – SharePoint ................................................................................................................11
2.1.23 SOP – SLA & Customer Service Guide.................................................................................11
2.1.24 SOP – VPN Guide.................................................................................................................11
3. Scope...................................................................................................................................................12
4. About SpryWare..................................................................................................................................13
4.1 Locations.....................................................................................................................................13
4.2 Telephone ...................................................................................................................................13
4.3 Email............................................................................................................................................13
4.4 Hours of Operation .....................................................................................................................13
4.4.1 Holiday Schedule – NYSE, Canada & London Stock Exchange ............................................14
4.5 Org Chart.....................................................................................................................................15
4.6 Products ......................................................................................................................................15
4.6.1 The Direct Market Server (DMS).........................................................................................15
4.6.2 HDS......................................................................................................................................17
5. Service Level Agreement.....................................................................................................................18
5. SOP - Master Page 5
5.1 Scope...........................................................................................................................................18
5.2 Notification and Credits..............................................................................................................18
5.3 Exclusions....................................................................................................................................18
5.4 Credit Types ................................................................................................................................18
5.5 Service Availability Metrics.........................................................................................................19
5.5.1 Service Point Problem Resolution Time..............................................................................19
5.5.2 Request for Reasons for Outage Guarantee.......................................................................19
5.5.3 De Facto Outages................................................................................................................20
5.6 Applicability.................................................................................................................................20
6. Customer Service Guide......................................................................................................................21
6.1 Overview.....................................................................................................................................21
6.2 Network Monitoring ...................................................................................................................21
6.3 Hours of Operation .....................................................................................................................21
6.4 Email Support..............................................................................................................................22
6.5 Phone Support ............................................................................................................................22
6.6 Problem Severity, Communication and Response Definitions ...................................................22
6.6.1 Definitions...........................................................................................................................22
7. Spryware Website...............................................................................................................................23
8. Toolsets...............................................................................................................................................24
8.1 Active Directory ..........................................................................................................................24
8.2 Intermedia...................................................................................................................................25
8.2.1 Overview .............................................................................................................................25
8.2.2 Email – Exchange/Outlook..................................................................................................25
8.2.3 SharePoint...........................................................................................................................25
8.2.4 CRM.....................................................................................................................................29
8.3 SalesForce ...................................................................................................................................29
8.3.1 Overview .............................................................................................................................29
8.4 CVS ..............................................................................................................................................30
8.4.1 Overview .............................................................................................................................30
8.4.2 Process & Procedures .........................................................................................................30
8.5 Nagios..........................................................................................................................................30
8.6 BGINFO........................................................................................................................................31
8.7 MISVerify.....................................................................................................................................31
8.8 VPN..............................................................................................................................................32
8.9 Bugzilla ........................................................................................................................................32
9. Direct Market Server (DMS)................................................................................................................33
9.1 Overview.....................................................................................................................................33
9.2 Technical Requirements..............................................................................................................33
9.2.1 Hardware.............................................................................................................................33
9.2.2 Operating System................................................................................................................33
9.2.3 Network ..............................................................................................................................33
6. SOP - Master Page 6
9.3 MIS Build Procedures..................................................................................................................33
9.4 MIS.XML......................................................................................................................................34
9.4.1 Client Side ...........................................................................................................................35
9.4.2 Server Side (Data Server) ....................................................................................................36
9.4.3 Server Side (Proxy)..............................................................................................................37
9.5 MIS Decommission Procedures ..................................................................................................39
9.6 Utilities........................................................................................................................................40
9.6.1 Tableviewer.........................................................................................................................40
9.6.2 MDUMP...............................................................................................................................40
9.6.3 Tail.......................................................................................................................................40
9.6.4 MISTOP/MISTART ...............................................................................................................40
9.6.5 Snapshot..............................................................................................................................40
9.7 Process & Procedures .................................................................................................................41
9.7.1 SpryWare Upgrade..............................................................................................................41
9.7.2 Data Editing.........................................................................................................................41
9.7.3 Operating System Patches ..................................................................................................43
9.7.4 Conflation............................................................................................................................43
9.7.5 Arbitration...........................................................................................................................43
9.7.6 Front End/Back End/Proxy/MIG..........................................................................................44
9.7.7 MIS Redundancy/Fail Over..................................................................................................44
9.7.8 Server Detail/BGInfo...........................................................................................................44
9.7.9 Backup & Restore Procedures.............................................................................................44
9.7.10 External RAID Arrays...........................................................................................................44
9.7.11 Hardware Upgrades............................................................................................................44
9.7.12 Hardware Failures...............................................................................................................44
9.8 Troubleshooting Guide ...............................................................................................................45
9.8.1 Client Side Logging..............................................................................................................46
10. Infrastructure......................................................................................................................................48
10.1 ADM ............................................................................................................................................48
10.2 ESXi..............................................................................................................................................48
10.3 NAS..............................................................................................................................................48
10.4 Dell MD1000 ...............................................................................................................................48
11. Market Data........................................................................................................................................49
11.1 Overview.....................................................................................................................................49
11.2 SharePoint Market Data Calendar ..............................................................................................49
11.3 Market Data Calendar Sources ...................................................................................................50
11.3.1 NASDAQ OMX .....................................................................................................................50
11.3.2 Clearnet Calendar ...............................................................................................................51
11.3.3 Interactive Data Corporation (IDC) Developer Notices.......................................................51
12. SpryWare Hosted Data Centers ..........................................................................................................52
12.1 Overview.....................................................................................................................................52
7. SOP - Master Page 7
12.2 Redundancy ................................................................................................................................52
12.3 Disaster Recovery........................................................................................................................52
12.4 Support........................................................................................................................................52
12.5 Customer Care Area....................................................................................................................53
12.6 Site Access...................................................................................................................................53
12.7 Data Center Infrastructure..........................................................................................................53
12.7.1 Overview .............................................................................................................................53
12.7.2 Physical Security..................................................................................................................53
12.7.3 Network Operations Center (NOC).....................................................................................53
12.7.4 Power Systems & Distributed Redundancy ........................................................................54
12.7.5 Customer Power .................................................................................................................54
12.7.6 Network (Internet & Carrier Detail)....................................................................................54
12.7.7 Environmental Controls ......................................................................................................54
12.7.8 Fire Protection.....................................................................................................................55
12.7.9 Flood Control.......................................................................................................................55
12.7.10 Earthquake Preparedness...............................................................................................55
12.8 SpryWare Infrastructure.............................................................................................................56
12.8.1 Overview .............................................................................................................................56
12.8.2 PDU .....................................................................................................................................56
12.8.3 KVM.....................................................................................................................................56
12.8.4 Dell DRAC ............................................................................................................................56
12.8.5 Network ..............................................................................................................................56
12.8.6 Cabling.................................................................................................................................56
12.8.7 Monitoring ..........................................................................................................................56
12.9 Policies & Procedures .................................................................................................................56
12.9.1 New Cabinet Build Out........................................................................................................56
12.9.2 Cable a New MIS .................................................................................................................57
12.9.3 PDU Outlet Configuration ...................................................................................................58
12.9.4 Cisco 4948 Installation & Configuration .............................................................................59
12.9.5 Cisco 4948 Port Configuration ............................................................................................59
12.9.6 Dell DRAC ............................................................................................................................59
12.9.7 Chicago Equinix – Specific Procedures................................................................................59
12.9.8 New York SIAC – Specific Procedures .................................................................................62
13. Client Support .....................................................................................................................................65
13.1 Overview.....................................................................................................................................65
13.2 System Monitoring......................................................................................................................65
13.3 Remote Access............................................................................................................................66
14. Communication & Training.................................................................................................................67
14.1 Reporting.....................................................................................................................................67
14.1.1 Build Lists ............................................................................................................................67
14.1.2 Release Notes......................................................................................................................67
8. SOP - Master Page 8
14.1.3 Defect/Bug Reports.............................................................................................................67
14.1.4 Status Reports.....................................................................................................................67
14.2 Status Meetings ..........................................................................................................................67
14.3 Training Documentation .............................................................................................................67
14.3.1 KnowledgeQuest.................................................................................................................67
15. Technical Operations ..........................................................................................................................68
15.1 Overview.....................................................................................................................................68
15.2 System Monitoring......................................................................................................................68
15.2.1 Server Availability & Performance......................................................................................68
15.2.2 Application Software...........................................................................................................68
15.2.3 Website...............................................................................................................................68
15.3 Problem Management................................................................................................................68
15.3.1 Overview .............................................................................................................................68
15.4 Problem Description & Resolution..............................................................................................69
15.4.1 ALERT !!! mail.spryware.local ques is x items (ADM1) .......................................................69
15.5 Change Management..................................................................................................................70
15.5.1 Overview .............................................................................................................................70
15.5.2 Customer Request...............................................................................................................70
15.5.3 Internal Changes .................................................................................................................70
15.5.4 Policy & Procedures ............................................................................................................70
15.6 Process Management .................................................................................................................71
15.6.1 Holiday Procedures.............................................................................................................71
15.6.2 Upgrade Candidate Identification.......................................................................................71
15.6.3 Client Communications.......................................................................................................71
15.6.4 Application of Patches ........................................................................................................71
15.6.5 Prine....................................................................................................................................71
15.6.6 ADM1-3...............................................................................................................................71
15.6.7 NAS......................................................................................................................................71
15.7 Inventory Control........................................................................................................................71
15.8 Contingency Planning..................................................................................................................71
15.9 Issue Tracking..............................................................................................................................71
15.9.1 Defect – Functionally Correct, Inelegant Implementation .................................................72
15.9.2 Bug – Functionally Incorrect or Non-Functional.................................................................72
16. Customer Support...............................................................................................................................73
17. Quality Assurance ...............................................................................................................................74
17.1 Scope Definition..........................................................................................................................74
17.2 OK to Develop .............................................................................................................................74
17.3 Unit Testing.................................................................................................................................74
17.4 Push to QA ..................................................................................................................................74
17.5 Quality Control Testing ...............................................................................................................74
17.6 Push to Beta (Controlled Production Release) ...........................................................................74
10. SOP - Master Page 10
1. OVERVIEW
The SpryWare Standard Operating Procedures (SOP) document outlines the primary support
processes required for the operation of the company’s technical infrastructure. This includes the
offices located in Chicago & New York, the two data centers as well as support for the servers
located at customer sites.
This document will provide a framework for the continued documentation of operating procedures
as service offerings grow and the environment expands.
This document is broken down into multiple sections. These sections will document:
Site Locations and Contact Information
Service Level Agreement (SLA)
Process & Procedures
Hosted Data Center Model
Toolsets available for running the business
Communication & Training
SpryWare Market Information Server Solution Detail
Preferred Vendor List
This document does not include the Employee Handbook, SDK, etc. However, these documents can
be found on the SpryWare SharePoint site.
11. SOP - Master Page 11
2. SUPPLEMENTARY DOCUMENTATION
This Master document begins with a Table of Contents detailing the layout of the SpryWare
Standard Operating Procedures. Each subsequent topic is then covered in the sections that follow.
In some cases, the subject matter requires a lengthy write up; too much detail for this single Master
document. In other cases, the subject matter will be referenced frequently as part of day to day
operations. To address this, separate/stand alone documents are included as part of the overall SOP
documentation effort. In each case these stand alone documents are referenced in the master
document. Each stand alone document is titled using the following format:
SOP – Subject Matter Description v##
2.1 STAND ALONE DOCUMENT DIRECTORY
2.1.1 SpryWare Technical Overview
2.1.2 SOP – Active Directory
2.1.3 SOP – Backup and Restore Procedures
2.1.4 SOP – BGInfo
2.1.5 SOP – Bugzilla
2.1.6 SOP – Client Remote Access Procedures
2.1.7 SOP – CRM User Manual
2.1.8 SOP – Data Editing
2.1.9 SOP – DMS Prep
2.1.10 SOP – DMS Linux MIS Build Procedures
2.1.11 SOP – DMS Windows MIS Build Procedures
2.1.12 SOP – Entitlement Procedures
2.1.13 SOP – ESXi QuickGuide
2.1.14 SOP – General Office
2.1.15 SOP – HDS
2.1.16 SOP – IDC Developer Notices
2.1.17 SOP – Intel to SloarFlare NIC Upgrade Procedures
2.1.18 SOP – Iron Mountain
2.1.19 SOP – Nagios Manual
2.1.20 SOP – New Client Procedures
2.1.21 SOP – New Employee Procedures
2.1.22 SOP – SharePoint
2.1.23 SOP – SLA & Customer Service Guide
2.1.24 SOP – VPN Guide
12. SOP - Master Page 12
3. SCOPE
This document details all aspects associated with the support of SpryWare’s technical infrastructure
ranging from the company internal functions, like the telephone system and Microsoft Exchange, all
the way to the customer facing Market Information Server (MIS) and associated networking.
The document begins with a general overview of SpryWare highlighting information that can be
found in the About Us section of the SpryWare corporate website located at:
http://www.spryware.com
The next section documents the Service Level Agreement (SLA). The SLA records a common
understanding about services, priorities, responsibilities, guarantees and warranties. Each area of
the service scope defines the level of service SpryWare is committed to achieving.
A detailed overview of the various toolsets SpryWare relies on follows. These toolsets include, but
are not limited to Exchange, SharePoint, CRM, Nagios-based Network & Event Monitoring and
SalesForce.
Also included as in-scope for this document are sections detailing Technical Infrastructure, Release
Management, Quality Assurance and Customer Support. Processes & Procedures are documented
along with a list of approved vendors.
Areas outside the scope of this document include the Employee Handbook, SDK, etc. However,
these documents can be found on the SpryWare SharePoint site located at:
http://spryware.hostpilot.com
13. SOP - Master Page 13
4. ABOUT SPRYWARE
As detailed in the About Us section of the company website, SpryWare provides standardized
market data via direct exchange feeds. SpryWare’s flagship product, The Market Information Server
(MIS), represents the latest and fastest in the low-latency, DMA marketplace. Working in a niche
field, the company is 100% focused on delivering market data, faster.
4.1 LOCATIONS
SpryWare operates out of two locations situated in the heart of the Chicago and New York financial
districts. The corporate headquarters are located at the Chicago location. The office addresses are
listed below:
Chicago
One North LaSalle Street
Suite 4300
Chicago, IL 60602
New York
One Liberty Plaza
23rd
Floor
New York, NY 10006
4.2 TELEPHONE
Corporate HQ Chicago IL: (312) 922-7779
Chicago Fax: (312) 922-5158
New York Office: (212) 201-5515
4.3 EMAIL
Customer Support: Support@SpryWare.com
Sales: Sales@SpryWare.com
4.4 HOURS OF OPERATION
The SpryWare offices are staffed during standard business hours, 7:00 AM CT to 5:00 PM CT on all
market trading business days, as defined each year by the New York Stock Exchange (NYSE). The
2012 through 2014 Holiday Schedules are shown in the following section.
14. SOP - Master Page 14
4.4.1 Holiday Schedule – NYSE, Canada & London Stock Exchange
NYSE 2012 2013 2014
New Year's Day January 2 January 1 January 1
Martin Luther King, Jr. Day January 16 January 21 January 20
Washington's Birthday* February 20 February 18 February 17
Good Friday April 6 March 29 April 18
Memorial Day May 28 May 27 May 26
Independence Day July 4 July 4 July 4
Labor Day September 3 September 2 September 1
Thanksgiving Day November 22 November 28 7
Christmas December 25 December 25 December 25
TSX 2012 2013 2014
New Year's Day January 2 January 1 January 1
Family Day February 20 February 18 February 17
Good Friday April 6 March 29 April 18
Victoria Day May 21 May 20 May 19
Canada Day July 2 July 1 July 1
Civic Holiday August 6 August 5 August 4
Labour Day September 3 September 2 September 1
Thanksgiving Day October 8 October 14 October 13
Christmas Day December 25 December 25 December 25
Boxing Day December 26 December 26 December 26
LSE 2012 2013
New Year January 3 January 1
Good Friday April 6 March 26
Easter Monday April 9 April 1
May Day May 7 May 6
Spring June 4 March 27
Queen’s Diamond Jubilee June 5
Summer August 27 August 26
Christmas Day (Observed) December 25 December 25
Boxing Day (Observed) December 26 December 26
15. SOP - Master Page 15
4.5 ORG CHART
SpryWare was founded by Mike Kreutzjans and Dan May in 2004. Both Mike and Dan continue to
play in integral part in the day-to-day running of SpryWare and function as the senior members of
the SpryWare team. A copy of the company Organizational Chart, current as of November 23, 2012,
is shown below. An updated copy of the Org Chart can be found in the Shared Documents section of
the SpryWare SharePoint site.
Directors
Daniel May & Mike Kreutzjans
VP Sales & Marketing
Paul Zinone
Regional Sales Director
Dan Curry
Director of Development
Al Pintoy
Director of
Operations & Support
Dan Schwartz
Business
Operations Manager
Agata Lipinski
Regional Sales Director
Morgan Roberts
Software Eng.
John Dibling
Software Eng.
Danny Bordelon
Software Eng.
Inna Brodkin
Operations Manager
Terry Guder
Quality Assurance
Manager
Ron Lindstrom
Director of
Client Service
Keith Tylke
Director of
East Coast OPS
Tony Padovano
Regional Sales Director
Adam Growick
Operations Support
Specialist
Tim Perkin
Operations Support
Specialist
Dan Kim
4.6 PRODUCTS
4.6.1 The Direct Market Server (DMS)
The SpryWare Direct Market Server (DMS) ™ is a fully managed ticker plant designed to deliver
direct market data with virtually no latency. The DMS™ connects directly to exchange feeds while
normalizing and distributing the data to clients in under 100 microseconds. To achieve this high
throughput, we focus on multithreaded application development making full use of the processing
capacity of modern server class machines.
Low latency market data delivery is the primary goal of the DMS™ but we also include many value
added features. Below is an overview of configurable features included in our MIS ticker plant
technology.
Time Series Database – Can be configured to set time intervals for historic data. Includes
the ability to pick select only data of interest.
Tick Database – By default the MIS stores every trade and quote, but the MIS can be
configured to store any tick database to fit your needs.
Historical Data – Includes 15 years of Equity historical data
16. SOP - Master Page 16
Historic Data Retrieval and Aggregation – The API we provide with the MIS is highly
configurable. For historical data retrieval we use the FAST Protocol with High Speed
Indexing.
Database Management – The ticker plant is fully managed and updated in real-time. For
example, Corporate action data is updated throughout the trading day rather than after the
market close.
Data Validation and Cleansing – The MIS dynamically cleans erroneous data by running
each bid, ask and trade from the exchange through a validation algorithm.
Exchange Contract and Roll-Over Management – The MIS automatically handles
management of expirations and roll-over information for futures and options and strike
prices are dynamically updated.
Fundamental Data – The Fundamental data tables allow for extensive cross referencing
including access to instruments by CUSIP, ISIN and SEDOL.
Calculated Fields – The MIS comes preconfigured with real-time calculations like Highs,
Lows, and Cumulative Volume and many other important fields that are not provided
directly by each exchange.
Data Blending and Linking – DepthPlus allows clients multiple views and sorted levels of the
order book. DepthPlus sorts, aggregates and provides all levels of book depth. In addition
the client-side API is extremely configurable for a wide assortment of “views’ of the data.
User Authentication and Usage Reporting – Allows for permission controls down to the
end-user level.
17. SOP - Master Page 17
4.6.2 HDS
In addition to SpryWare’s flagship product, The Market Information Server (MIS), SpryWare is
offering HDS. SpryWare HDS is a complete archive of every trade and quote on all major equities,
options and futures.
The HDS archive is stored in the FIX Adapted for STreaming (FAST) format. The FAST
Protocol, sponsored by FIX Protocol Ltd. and co-authored by SpryWare, is the highly compact market
data standard that radically reduces the bandwidth required to transmit ever-increasing real-time
market data message rates. By leveraging field encoding, implicit tagging, and serialization, FAST
reduces bandwidth consumption by up to 90%.
The SpryWare MIS™ processes raw exchange data in various formats and normalizes them using
FAST. This FAST feed is then databased and archived. In applying the protocol to mass storage of
tick-by-tick data, SpryWare is able to make available very large amounts of tick data in files
that are a fraction of the size of our competitors. These large amounts of data are then made
immediately available to our clients and retrieved with ease due to their size.
Complex blackbox and algorithmic trading strategies that require tick-by-tick data combined
with rapidly rising market data volumes make data storage and retrieval a major concern. SpryWare
HDS allows traders to pursue these strategies with ease.
SpryWare HDS Tick Data is currently available for:
OPRA AMEX CBOE Futures CFE
NASDAQ UTP (UTDF & UQDF) Eq SIAC – All regional exchanges CBOE RLC & ITC
NASDAQ OMDF PBOT Indices NYMEX
NASDAQ TDDS Pink Sheets COMEX
NASDAQ BBDS CBOT ITC Kansas City Board of Trade ITC
NASDAQ NIDS CBOE Direct Minneapolis Grain Exchange ITC
NASDAQ MFDS CBOE One Chicago Winnipeg Commodity Exchange
NYSE CBOE Options on Futures Dow Jones Indices
18. SOP - Master Page 18
5. SERVICE LEVEL AGREEMENT
5.1 SCOPE
This Service Level Agreement (SLA) applies only to Customer’s service performance directly relating
to SpryWare’s Service. The scope of this SLA does not include, under any circumstances, any server
on the Internet, customer premise equipment (“CPE”) or local access service. The interpretation of
this SLA is, at all times, intended to be subject to and consistent with the Spryware Master Licensing
and Service Agreement (“Master Agreement”).
5.2 NOTIFICATION AND CREDITS
Service-impacting conditions can be reported either by Customer or by Spryware. At the time
Customer believes that a service impacting condition has occurred, Customer must initiate a trouble
ticket by contacting Spryware Customer Service in accordance with the method of contact set forth
in the Spryware [Customer Service Guide]. If the Spryware Customer Service believes that a service
impacting condition has occurred, it will open a trouble ticket on Customer’s behalf. Once a ticket
has been opened, Spryware will notify Customer of the situation and initiate diagnostic testing and
fault isolation activities to determine the nature of the service condition. A credit will be applied to
Customer’s first monthly invoice subsequent to Spryware’s determination that a service impacting
condition that lies within the scope of coverage has occurred.
5.3 EXCLUSIONS
The scope of this SLA covers only those events isolated to the areas specified in the Scope section
above. SLA does not apply to scheduled or Customer-requested service interruptions. Outages will
only be measured only in accordance with the sections below. Under no circumstances will any
tests (for example PING tests) performed by Customer, its vendors or partners be recognized by
Spryware as a valid measurable criterion of violation length, quality or type for the purposes of
establishing a service credit hereunder. Failure of Customer to inform Spryware of changes to
Customer’s technical contacts lists may result in the denial of credits under this SLA. The SLA
objectives contained herein apply only to SpryWare’s Customers; they do not apply to customers of
Customer. Under no circumstances will credits be given for outages involving: (a) trouble tickets
associated with new installations (b) trouble tickets erroneously opened by Customer; (c) a circuit
release required by Customer for testing; (d) trouble tickets opened by Customer for service
monitoring purposes only; or (e) trouble tickets related to Customer maintenance, configurations,
negligence, accidents or omissions.; or (f) events beyond SpryWare’s reasonable control, including,
but not limited by, the transmission failure of exchanges and third party information providers.
5.4 CREDIT TYPES
SpryWare offers two types of service credits under this SLA. Proactive credits are automatically
processed by the appropriate Spryware team member, and are credited to Customer’s monthly
invoice following SpryWare’s determination that the event lies within the scope of coverage.
19. SOP - Master Page 19
Customer is not required to request proactive credits. However, if Customer feels that Spryware has
not applied a credit correctly, Customer is able to request these credits via their Spryware Sales
Team anytime within thirty (30) business days of the resolution of the event in question. Non-
proactive service credits are not applied automatically to Customer’s monthly invoice. Non-
proactive credits must be requested in writing from Customer’s Spryware Sales Team within thirty
(30) business days of the resolution of the event for which the credit is being requested. Spryware
will apply any non-proactive credits due to Customer on Customer’s [monthly invoice] following
their request and SpryWare’s determination that the event lies within the scope of coverage.
5.5 SERVICE AVAILABILITY METRICS
5.5.1 Service Point Problem Resolution Time
Spryware will use commercially reasonable efforts to ensure that eligible outages to an area within
the Service Point and affecting Customer’s specific resource will be resolved as soon as reasonably
possible and, in any event, in less than ten (10) minutes. Failure to meet this objective may make
Customer eligible for a service credit to be applied to Customer’s next monthly invoice for Services.
5.5.1.1 Remedies and Credits
The Service Point Problem Resolution Time Guarantee is determined and calculated on a per-
occurrence basis, commencing upon Spryware Customer Service’s initial awareness of an outage
and ending when the service has been restored. Customer’s service credit is a prorated amount
derived from the length of the service outage and Customer’s Monthly Recurring License Fee
(“MRLF”) for the affected service. The processing of any Customer credit will be done by the
appropriate Spryware team member, who will apply the credit parameters as specified below.
Length of Outage Amount of Credit Credit Type
Less than 10 minutes None
Proactive
10 minutes to 30 minutes 2 days prorated MRLF
31 minutes to 60 minutes 4 days prorated MRLF
61 minutes to 6 hours 6 days prorated MRLF
More than 6 hours 4 weeks prorated MRLF
Second qualifying outage in
same calendar month
Two times the amount listed
above for the respective
second qualifying outage
5.5.2 Request for Reasons for Outage Guarantee
Spryware will use commercially available means to reasonably ensure that Customer will be sent a
formal written Reason for Outage (RFO) on events isolated to SpryWare’s Service Point as quickly as
possible, and in any event less than five (5) business days from the date of Customer’s request.
RFO’s extend beyond SpryWare’s standard notifications given for all events and can contain
additional details regarding an event. This service point specific guarantee excludes backbone
problems and any other services provided by SpryWare’s vendors, partners, or affiliates. Failure to
20. SOP - Master Page 20
meet this objective may make Customer eligible for a service credit to be applied to Customer’s next
monthly invoice for Services.
5.5.3 De Facto Outages
Spryware recognizes that time is of the essence with respect to the timely transmission of accurate
data, without which the Customer cannot conduct business. Spryware shall endeavor to cure any
material defects in data quality, completeness, and/or transmission speed, within a reasonable
period of time as such defects occur. In the event that such defects occur on a consistent or on-
going basis, Customer has the right to terminate the Master Agreement with thirty (30) days written
notice and without further liability or obligation to the Customer. This section contemplates those
instances where data transmission is incomplete or delayed rather than a complete Outage as
contemplated in Section 4.5.1.1 above.
5.6 APPLICABILITY
a. If Customer experiences four (4) qualifying violations of the same metric as defined in this
SLA on the same service, Customer shall have the right to terminate the affected service in
lieu of service credits for the fourth or final such qualifying event or any other remedy
agreed to by the Parties which is offered for such events and without liability or obligation
except for unpaid Fees and costs relating to the Services rendered through the date of such
termination and all third party termination, disconnection and cancellation fees relating to
such termination, including, but not limited to, CPE gear or Local Telephone/Access
Company's local access charges, provided Customer has notified Spryware of its intent to
terminate in writing within 30 days after the last such outage.
b. Eligibility for any credits is subject to Customer’s account being held current and having no
outstanding balance due. Customer’s total credit in any contract year shall not exceed 60
days fees for the covered service type. Residual credits may not be carried over to
subsequent years.
c. Customer must have contracted for the specific service covered under each provision to
qualify for any credits against those provisions.
d. EACH CREDIT OR TERMINATION RIGHT SHALL BE THE CUSTOMER’S SOLE AND EXCLUSIVE
REMEDY FOR THE CORRESPONDING SERVICE OUTAGE AND SPRYWARE’S FAILURE TO MEET
THE SERVICE OBJECTIVES. ANY DISPUTES ARISING OUT OF OR RELATING TO THIS SLA MUST
BE BROUGHT WITHIN SIX MONTHS OF THE OCCURRENCE OF ANY SUCH DISPUTE.
e. This SLA applies to new Customers who use the Services in accordance with Spryware’s
policies on or after 6-1-2005. This SLA will be applied to all qualifying incidents occurring on
or after 6-1-2005. Spryware reserves the right to change or modify this SLA and will post
such changes or modifications to a web site designated by Spryware and made available to
Customer, provided that any such change or modification shall not adversely affect
Customer’s then-current SLA.
21. SOP - Master Page 21
6. CUSTOMER SERVICE GUIDE
6.1 OVERVIEW
At SpryWare we understand the urgency and importance of real-time market data. From initial
implementation to the actual use of our technology, you will work closely with our own full-time
professionals who have many years of market data experience and are experts on all of the
SpryWare market data solutions.
Support is divided into the following three functional areas:
Network & Hardware
Database Management
Development and Functionality
6.2 NETWORK MONITORING
SpryWare operates fully managed networks and has secure remote access to all data centers and
servers. An extensive monitoring system is in place that provides statistics and notifications on a
real-time basis. These systems are operational 24/7. Normal system maintenance and updates will
take place outside of regular office hours.
6.3 HOURS OF OPERATION
The SpryWare offices are fully staffed during standard business hours, 8:00 AM CT to 5:00 PM CT on
all market trading business days as defined each year by the New York Stock Exchange (NYSE).
SpryWare provides an additional hour of Customer Support each morning beginning at 7:00AM CT.
SpryWare defines Support Hours as running from 7:00 AM CT to 5:00 PM CT Monday thru Friday on
all market trading days.
System upgrades or other events requiring after-hours support can be accommodated on a case-by-
case basis by SpryWare Support. Instances of after-hours support will need to be arranged in
advance, and are subject to staff availability.
SpryWare staff will also make their best effort to address issues that arise during off-hours. Email is
typically monitored during evening and early morning hours. The SpryWare monitoring system will
email alerts to staff as they are detected. Clients can also email SpryWare at
Support@SpryWare.com to report issues or make inquiries. SpryWare will follow a “Best Efforts”
policy to address system critical issues reported by the monitoring system and client reported issues
emailed to Support prior to the next business day’s market open.
22. SOP - Master Page 22
6.4 EMAIL SUPPORT
Issues, requests, comments and suggestions sent to SpryWare via email should be sent to
Support@SpryWare.com. The SpryWare Support team members are copied on all Support emails.
Emails sent to Support will generate a new case in the SpryWare help desk tool. Each case will be
reviewed and assigned to a qualified team member. The Client will receive an acknowledgement
email from the Support Department telling them a new case has been created. Emails will be
responded to in the timeframes outlined in the Problem Severity, Communication and Response
Definitions section detailed below.
6.5 PHONE SUPPORT
Incoming phone calls to the Chicago Headquarters during the standard Support hours outlined
above are answered by live personal. The Client is instructed to call (312) 922-7779 to be connected
directly to the appropriate Support personnel.
6.6 PROBLEM SEVERITY, COMMUNICATION AND RESPONSE DEFINITIONS
SpryWare defines four levels of problem severity, communication and resolution for client support.
The response times indicated below are not applicable to issues that arise during non-business
hours.
Table 2: Problem Severity, Communication and Response Table
Severity Description Response Time Contact Method
1 Client access is down with no work
around available
30 Minutes Phone
2 Client access is available with service
feature(s) unavailable with no work
around identified
60 Minutes Phone, Email
3 Client access is available with service
feature(s) available using a work
around
3 Business Days Trouble Ticket
4 Client access is available with all
features functioning. Suggestion or
inquiry on usage of service.
5 Business Days Trouble Ticket
6.6.1 Definitions
Response Time: The amount of time SpryWare has to provide the first update to the client
after being notified of a problem.
Contact Method: The method of communication to report and monitor a problem until
resolution.
24. SOP - Master Page 24
8. TOOLSETS
SpryWare relies on a number of tools for day to day productivity. This section provides an overview
of each of these tools. In cases where lengthy documentation is required to detail a specific tool a
brief overview will be provided and more through documentation will be provided in a separate
attachement as noted in that overview.
Toolsets described in this section include:
Active Directory
SaaS Provider Intermedia
o Email – Exchange/Outlook
o SharePoint
o CRM
Salesforce
CVS
Nagios
BGInfo
MISVerify
VPN
8.1 ACTIVE DIRECTORY
SpryWare relies on Microsoft Active Directory and DNS. There are currently three Active Directory
servers, or more commonly known as Infrastructure servers. Each servers runs the following
applications:
Domain Controller
Global Catalog Server
DNS Server
DHCP Server
One of the three servers is located at SpryWare HQ. The other two are Virtual Servers locate at the
Equinix Chicago and NY4 Data centers. The server names are as follows:
SWHQDC01 (SpryWare HQ)
SWILDC02 (Chicago Equinix CH4)
SWNJDC03 (Secaucus Equinix NY4
Additional information on the configuration and use of these resources can be found in the
document: SOP – Active Directory v##.
25. SOP - Master Page 25
8.2 INTERMEDIA
8.2.1 Overview
SpryWare subscribes to the principals of Software as a Service (SaaS). SaaS is a model of software
deployment whereby a provider licenses an application to customers for use as a service on
demand. Intermedia is such a provider offering SpryWare Email services (Exchange & Outlook),
SharePoint and CRM. All three of these Microsoft products run on servers housed at the Intermedia
data centers. No servers or technical expertise in the operation of these environments is required
by SpryWare. Intermedia is solely responsible for the maintenance of their servers. As a user,
SpryWare works with Intermedia to ensure that full advantage is taken of these offerings. The
following subsections detail these three offerings.
8.2.2 Email – Exchange/Outlook
8.2.2.1 Overview
8.2.2.2 Password Policy
8.2.2.3 Process & Procedures
8.2.2.4 Intermedia Technical Support
8.2.2.5 Create New Mailbox
8.2.2.6 Create New Distribution List
8.2.2.7 Storage Management
8.2.2.8 Archive your Mailbox
8.2.2.9 ActiveSync
8.2.2.10 Blackberry Service
8.2.3 SharePoint
8.2.3.1 Overview
The SpryWare SharePoint Home site is located at the following address:
http://spryware.hostpilot.com
As noted in the sections below certain SpryWare customers and business partners will access the
site using an extended URL, example: BNY will require a “/bny” at the end of the URL.
The following sections detail login procedures and the various sections of the SharePoint site.
8.2.3.2 Initial Access
SpryWare offers two levels of access to the SharePoint site; full access for SpryWare employees and
limited access for select customers and business partners. The full access option allows SpryWare
employees access to the Home site as well as access to select customer/business partner sites. The
26. SOP - Master Page 26
limited access option allows a SpryWare customer or business partner access to their specific site
only. These access levels are set by the SpryWare SharePoint administrator.
Credentials for individual SpryWare team members are established first through an entry in
SpryWare’s Active Directory, followed by inclusion in the Intermedia Hostpilot Administration page.
Lastly, from the SharePoint site itself, the user needs to be added to the People and Groups section.
Utilizing an Intermedia feature titled UserPilot, the SpryWare Intermedia Adminstrator will tie the
SpryWare users Active Directory credentials to Intermedia so the user will have a single login and
password to gain access to all Intermedia tools.
For the select SpryWare customers and business partners who have SpryWare provided SharePoint
sites, credentials are established through Intermedia Hostpilot as well as the Users and Groups
section in the individual SharePoint site. These crecentials are established by the SpryWare
Intermedia Administrator.
27. SOP - Master Page 27
The login and password required to gain access to the SharePoint site can be saved so that after
sigining on initially, this information won’t have to be entered again. To do this proceed as follows:
1. Open Internet Explorer
2. Go to the SharePoint Home site. You’ll need to login but this will be the last time.
3. Go to Tools -> Internet Options -> Security -> Trusted Sites -> Click on the Sites button
4. Confirm that the SharePoint URL is listed in the Add this website to the zone: field and
clieck <Add>
5. Click <Close>
6. On the Security tab page, press the Custom level button
7. Scroll down to User Authentication and select Automatic login with current username and
password
Not that these procedures will have to be performed on any computer that is used to gain access to
the SharePoint site; typically a work and home PC.
29. SOP - Master Page 29
8.2.3.13 Phone List
8.2.3.14 Process & Procedures
8.2.3.14.1 Create a new announcement
8.2.3.14.2 Upload a document
8.2.3.14.3 List Updates
8.2.3.14.4 Add/Change/Delete a Calendar Entry
8.2.3.14.5 Server Detail Entry/Modification
8.2.4 CRM
8.2.4.1 Overview
The entire SOP for CRM can be found in the attachment entitled “SOP – CRM User Manual v##”.
This document covers the SpryWare implementation of Microsoft Dynamics CRM Version 4, which
we refer herein to as simply Microsoft CRM or just CRM. The Acronym CRM stands for Customer
Relationship Management. CRM is the tool that will be used to replace Salesforce for Customer
Support ticket processing.
This tool is provided to SpryWare by CRM On Target, which is the leading worldwide provider of
partner-hosted Microsoft Dynamics CRM. TriVenture is the parent company of CRM On Target.
CRM is a highly customizable tool that has been configured to specifically address SpryWare’s
unique Support requirements. Over time this tool will be updated to introduce new features and
functionality.
Initially this tool will be for the processing of Support tickets (Cases). Cases will be created in one of
two ways either automatically via email (Support@SpryWare.com), or manually from detail received
through a customer call or other customer contact.
This document details the installation, training and Support procedures. An overview of Workflow
processing is detailed. Additionally, a section has been included stressing the importance of keeping
accurate and complete contact and account information.
8.3 SALESFORCE
8.3.1 Overview
8.3.1.1 Home
8.3.1.2 Leads
8.3.1.3 Accounts
8.3.1.4 Contacts
8.3.1.5 Opportunities
8.3.1.6 Reports
30. SOP - Master Page 30
8.3.1.7 Dashboards
8.3.1.8 Documents
8.3.1.9 SpryWare Products
8.4 CVS
8.4.1 Overview
8.4.2 Process & Procedures
8.4.2.1 Procedure #1
8.4.2.2 Procedure #2
8.4.2.3 Procedure #3
8.5 NAGIOS
Nagios is a monitoring application for both systems and networks that monitors and reports on:
Hosts – computers, switches, and printers
Services – network resources, host resources (CPU load, Memory Usage, etcetera
The entire SOP for Nagios can be found the attachment entitled “SOP – Nagios Manualv ##”.
Monitored elements are evaluated and assigned one of three values:
OK – Element is within standard working parameters
WARNING – Element nearing the border of acceptable parameters
CRITICAL – Element is Non-Responsive, Not Reachable, Not Executing, or Not within
established acceptable parameters.
Notification is performed in numerous methods:
Email
Web Display
Text Message (requires plugin, additional cost)
Audio Alerts
Phone Calls(requires plugin, additional cost)
Email and Web display are currently configured on the system
31. SOP - Master Page 31
8.6 BGINFO
BGINFO is a utility designed to poll the server for information related to that server’s hardware,
network configuration, operating system and SpryWare version number. This information is
displayed directly on the desktop to allow for easy access to this data. BGINFO collects this
information each time a new user logs into the system thus guaranteeing the most up to date data.
Should the user fail to logoff or remain on the system for an extended period of time BGINFO will
refresh on an hourly basis. The instructions for installing BGINFO are shown below:
Assumptions
The user has logged in as SpryWadm with the /admin switch.
Example: mstsc /v:mis70.spryware.local /admin
Installation Steps:
1. Create the folder: C:SpryWareBGinfo on the server in question.
2. Copy the file BgInfo.zip form the ClientApps directory on the D: drive on Prine to the newly
created BGinfo folder on the server in question.
3. Extract the Zip file into the BGinfo folder.
4. Start -> Run -> cmd (enter)
5. Type sprywarebginfoinfo.cmd <HOSTNAME>
(ex. sprywarebginfoinfo.cmd mis70 )
6. Enter the SpryAdm password when prompted.
7. Close cmd window when finished.
8. Notice the desktop now contains the system information.
9. Log out of the rdp session
10. Log back into the box without using /admin
11. Accept the EULA
12. Notice the desktop now contains the system information.
13. Log out
8.7 MISVERIFY
MISVerify is a utility that checks the status of the Market Information Servers located in the
SpryWare cages at Equinix in Chicago. This utility is scheduled to run via the Task Scheduler utility at
5:00 AM each day.
The MISVerify utility checks for the existence of each server and performs checks against the
following:
Core Tables
o Directory
o Events
o Session
o SIPStats
o Authentication
Specified tables that can include:
32. SOP - Master Page 32
o EquityPrice
o EquityOptionChain
o FuturePrice
o FutureOptionChain
Symbols that exist in the tables:
o IBM
o /ES 09Z
If MISVerify identifies an error condition, an email will be generated and sent to the Operations
Distribution list with the Subject line reading: MISVerify Errors. This email will include detail on the
error condition(s). This email should be reviewed by Operations in the morning and corrective
action taken to address any reported errors. A MISVerify Errors email will not be sent out if no
errors are detected.
The MISVerify utility and associated files reside on the adm1 Server at Equinix in the directory
c:MisVerify. In order to add or remove a server from polling, edit the verifyall.cmd file and make
the appropriate changes.
8.8 VPN
SpryWare provides a number of ways for team members to access SpryWare resources remotely
through VPN connectivity. Establish a VPN connection is frequently the first step to being able to
access remote client servers from outside the SpryWare office.
Refer to the document SOP – VPN Guide v## for further information on the setup and use of
SpryWare VPN resources.
8.9 BUGZILLA
Bugzilla is a Web-based general-purpose bug tracker and testing tool used by the SpryWare
Development and Q/A teams. Refer to the document SOP – Bugzilla v## for further information on
the setup and use of this tool.
33. SOP - Master Page 33
9. DIRECT MARKET SERVER (DMS)
9.1 OVERVIEW
High level stuff. Probably just grab material from Marketing. Talk about the various types of
servers, back end, proxy and eventually MIG.
9.2 TECHNICAL REQUIREMENTS
9.2.1 Hardware
Redundant power
Redundant NICs, market data feeds
9.2.2 Operating System
2008, vs 2003. Standard vs Enterprise. Turn off automatic updates
9.2.3 Network
Detailed in the Installation section… how to configure NICs
Mention Cisco
Two for market data, one for lan and maybe one more for a cross connect
9.3 MIS BUILD PROCEDURES
A MIS can be installed at one of the two SpryWare Hosted Datacenters; Equinix NY4 in Secaucus
New Jersey or Equinix CH4 in Chicago. A MIS can also be installed on a client premises or at a
datacenter of the client’s choosing. Two stand alone documents have been created in order to
document the MIS Build Procedures; one covers the Windows Operating System and one coveres
the Linux based RHEL6 Operating System. The documents are titled:
SOP – Windows MIS Build Procedures v##
SOP – Linux MIS build Procedures v##
The build process starts with documentation, but proper documentation is required throughout.
The SOP also includes a review of the server specification and covers ongoing communications with
the client throughout the process, bringing the Server to a functional state, the physical process of
receiving the server and having it mounted in a cabinet, followed by the installation of power,
network and Keyboard/Video/Mouse (KVM). Also included are the required configuration settings
for the BIOS, steps required to configure the Operating System and setup the network. Lastly, The
SpryWare Server Build section details the steps required to install and configure SpryWare software
on the new server. A Checklist section is provided as a more concise list of steps to be used by a
skilled Operations team member who will be responsible for the actual build process. A Build
Review Checklist is designed to be used by another SpryWare team member (who didn’t do the
original build) as a quality assurance step, to check/verify the work of the person who did the
original build.
34. SOP - Master Page 34
9.4 MIS.XML
This section documents the the three types of MIS.XML files, the Client Side, Data Server Side and
Proxy. The Client Side and Proxy configureation is documented in full detail with all applicable
settings. A brief description of the Data Server Side MIS.XML is also included. A more detailed
review is beyond the scope of this document.
The MIS.XML resides in the SpryWareBin directory and is responsible for the overall configuration
of the client computer or server.
35. SOP - Master Page 35
9.4.1 Client Side
Command Description
<MisConfig> Every MIS.XML must start and end with this.
<Directory> Section for basic client server settings.
<Remote>[True/False]</Remote> This setting should always be set to the default of True.
<Login>Jsmith</Login> Login
<Password>Password</Password> Password
<Domain>Spryware</Domain> Domain
<DataDirectory>C:UsersJsmithData</DataDirectory>
Data directory field only required to redirect the location of the temporary
files. The default is the TEMP directory
</Directory>
<Client> Method of connection to the server.
<RemoteType>[Socket|LBM]</RemoteType> Default is Socket. LBM (Latency Busters Messaging) not yet supported.
</Client>
<Socket>
<TableSocket> host[:port]</TableSocket> Server name or IP and port no. Default port number is 80
<EnableDebugLogging>[Yes|No]</EnableDebugLogging> Enables logging. Yes=On, No=Off
<NoDelay>[On|Off]</NoDelay>
Turns Nagle On/Off. NoDelay ON means Nagle is Disabled, NoDelay OFF
meands Nagle is Enabled
<TCPWindowSize>1048576</TCPWindowSize> Specifically sets TCPWindowSize. The default is left to the O/S.
<BaseTimeout>30</BaseTimeout>
Controls Base Query timeout representing the number of seconds to wait
for a response from the server before reconnecting. Default is 2
</Socket>
<LogManager> Application required for logging
<LogDirectory>C:UsersJsmithLogs</LogDirectory> Directory log files will be written to. Default is C:SpryWareData
</LogManager>
<MisGiver>
<AutoClose>[On|Off]</AutoClose>
Close MisGiver after application utilizing it is stopped. On=Close, Off=Don’t
Close.
<Daemon>[Yes|No]</Daemon> Yes=Launch MisGiver as a Helper as required No=Don’t launch
<UnloadDelay>2</UnloadDelay>
Delay in seconds before MISGiver closes after corresponding application
closes. Default is 5
</MisGiver>
<BufferQueueSize>
<All>500</All> Number of messages. Default is 0 (Unlimited)
</BufferQueueSize>
<PricePlus> Settings specifically for PricePlus example program
<Size>10000</Size> Default table size for local cache. Default is 10000
<CalcBBOSize>[Yes|No]</CalcBBOSize> Control National BBO Calculation. Default is No
<GrowthSize>5000</GrowthSize> Grow Size increments in number of records
<MultiThreaded>[On|Off]</MultiThreaded> Run MultiThread. On=Run, Off=Don’t Run. Default is On
</PricePlus>
<MISRTDServer> Excel Real Time Data Section
<Debug>[On|Off]</Debug> Turn debugging on or off. Default is Off
<TableSocket>host[:port]</TableSocket> Line typically not included. Default to <Socket> section
</MISRTDServer>
</MisConfig> Every MIS.XML must end with this.
36. SOP - Master Page 36
9.4.2 Server Side (Data Server)
The Server Side MIS.XML resides in the SpryWareBin directory and among other things, dictates
which incoming exchange feeds the server will accept. As indicated at the beginning of this section,
comprehensive documentation of this file is beyond the scope of this document due to the length
and complexity of each setting. For a new MIS install, copy a MIS.XML from a similarily configured
MIS and make any required changes specific to the server in question. Consult SpryWare
Operations and/or Development for details associated with specific configuration settings.
37. SOP - Master Page 37
9.4.3 Server Side (Proxy)
The MIS.XML detailed below is from a Proxy server attached to two back end servers. In this
example the back end servers are MIS50 & MIS51.
Command Description
<MisConfig> Every MIS.XML must start and end with this.
<Directory> Section for basic client server settings.
<Remote>[True/False]</Remote> This setting should always be set to the default of True.
<Login>Jsmith</Login> Login
<Password>Password</Password> Password
<Domain>Spryware</Domain> Domain
<DataDirectory>C:UsersJsmithData</DataDirectory>
Data directory field only required to redirect the location of
the temporary files. The default is the TEMP directory
</Directory>
<Client> Method of connection to the server.
<RemoteType>[Socket|LBM]</RemoteType>
Default is Socket. LBM (Latency Busters Messaging) not yet
supported.
</Client>
<Socket>
<TableSocket>mis50:80</TableSocket> Server name and port no. Default port number is 80
<TableSocket>mis51:80</TableSocket> Typically a second (or more) servers are included.
<EnableDebugLogging>true</EnableDebugLogging> Enables logging. Ture=On, False=Off
</Socket>
<MisGiver> Application required for logging
<AutoClose>Off</AutoClose>
Close MisGiver after application utilizing it is stopped.
On=Close, Off=Don’t Close.
<Daemon>yes</Daemon>
Yes=Launch MisGiver as a Helper as required No=Don’t
launch
<ConnectOnStartup>true</ConnectOnStartup> Connect to back end servers on statup.
<SharedInterest>Off</SharedInterest>
<InterestListSize>1000</InterestListSize>
<VerifyBBOInterest>Off</VerifyBBOInterest>
<ForwardOptionOnRoot>Off</ForwardOptionOnRoot>
</MisGiver>
<QuoteServer>
<DataDictionaryType>ZLIB</DataDictionaryType>
<StreamDictionaryType>FAST</StreamDictionaryType>
<MaxQueueCount>200000</MaxQueueCount>
<Priority>High</Priority>
<MaxPacketsPerBuffer>1000</MaxPacketsPerBuffer>
<RejectEntitlements>Off</RejectEntitlements>
</QuoteServer>
<LogManager>
<LogDirectory>c:sprywaredata</LogDirectory>
</LogManager>
39. SOP - Master Page 39
9.5 MIS DECOMMISSION PROCEDURES
Periodically a server will need to be removed from a SpryWare hosted data center. Reasons will
include, but not be limited to:
Client Replacing Older Hardware
Client no longer a SpryWare Customer
The following steps outline the general procedures required to be followed in order to
decommission a server. Each decommission is unique and procedures may vary. No decommission
procedures should be followed without a formal CRM Work Order.
1. Confirm with the client the Decommission schedule
2. Have all Nagios notifications for the server turned off
3. Have server removed from MISVerify script checks
4. Shut Down the Switch Ports
5. Stop SpryWare Services
6. Uninstall SpryWare software
7. Remove SpryWare services
8. Save off the mis.xml to your local machine
9. Delete install directory
10. Delete data directory
11. Delete any Spryware Desktop shortcuts
12. Get rid of random Spryware related files in c:temp
13. Delete the Utils folder
14. Uninstall Nagios then delete the Nagios folder
15. Uninstall NTP then delete NTP folder
16. Remove SNMP Traps & Security
17. Edit the host file and take out all mappings except for ‘local’
18. Remove IP Addresses from Market Data Interfaces, enable any disabled ports
19. Rename SpryAdm user to Administrator
20. Reset the password for Administrator to Password1
21. Reset DRAC Username & Passowrd to root and remove IP config
22. Power Down the Server
23. Remove port assignments from the PDUs
24. Remove KVM from the Dell Remote Access Controller on ADM1
25. SharePoint
a. Change the Status from Active to Decommissioned
b. Update the Journal field stating the reason for the Decommission and any other
pertinent information.
26. Schedule/Coordinate Removal of the Server
a. On-site hand over
b. SmartHands
27. Remove all unused cabling
28. CRM Work Order
a. Updated throughout the decommission steps
b. Close out ticket at the conclusion of the decommission
41. SOP - Master Page 41
9.7 PROCESS & PROCEDURES
9.7.1 SpryWare Upgrade
9.7.2 Data Editing
At some point, either a Spryware customer or Ops member will identify either missing or erroneous
data on a server. The dicussion below will step through the process of editing data on a specific
server and, if need be, populating that change out to all the servers.
9.7.2.1 Procedures
Data is edited following these procedures:
9.7.2.1.1 Verify the data in question.
Typical problems will be with the EquityPrice table and either the open, high, low, or close
values. Check other sources such as Esignal, Yahoo Finance, or Google Finance and come to a
consensus on what the correct value should be.
9.7.2.1.2 Applying Corrections
Open tableviewer on the server in question, go to the specific table, find the symbol and the
turn on ‘Enable Editing’.
42. SOP - Master Page 42
This will allow you to then select the specific field, double click, and manually type in a value.
This will update that value permanently on that server. Be sure to turn of ‘Enable Editing’ when
finished.
9.7.2.1.3 Broadcast Changes
ADM1.spryware.com is the master server that will propogate changes out to all servers. This
requires the creation of a .tbl file that will be processed by the server and pushed to other
servers. There are several ways that this can be done, but below is one way of creating the file.
Using snapshot will pull the data required to produce a file for the whole system. First, the field
headers seen in a tableViewer EquityPrice table need to be added to a file. The following
command will get those:
Snapshot /t:EquityPrice /s:XXXX >filename.csv
Example:
43. SOP - Master Page 43
By opening up the created csv in notepad, you can see those headers. Next, pump the data into that
file using the following command:
Snapshot /t:EquityPrice | findstr “symbol” >>filename.csv
Example:
Next, bring up the data file in Excel and remove all the data that you do not need. In example, we
will only be using the ‘close’ data, so we will remove everything else. Save this new file as
EquityPrice.csv. Finally, open that file in Notepad and then save it as EquityPrice.tbl.
9.7.3 Operating System Patches
9.7.4 Conflation
9.7.5 Arbitration
44. SOP - Master Page 44
9.7.6 Front End/Back End/Proxy/MIG
9.7.7 MIS Redundancy/Fail Over
9.7.8 Server Detail/BGInfo
9.7.9 Backup & Restore Procedures
9.7.10 External RAID Arrays
On select servers withing the SpryWare Equinix Data Centers, CH4 & NY4, SpryWare holds large
amounts of data on external storage RAID arrays. At this time six servers have these RAID arrays
attached to them. The table below lists the server, the RAID array model and capacity:
Server RAID Array Model Capacity
ADM10 (CH4) MD1000 #1
MD1000 #2
26TB
14TB
ADM11 (NY4) MD1000 26TB
MIS74 MD1000 26TB
MIS69 MD1220 3TB
NJMIS21 MD1000 26TB
NJMIS39 MD1220 3TB
9.7.11 Hardware Upgrades
9.7.12 Hardware Failures
45. SOP - Master Page 45
9.8 TROUBLESHOOTING GUIDE
Procedures for checking the health of the system
Tableviewer
MDUMP
Snapshot
MIS.LOG & Tail
Check status of the services
Copytable
Network – PING, Tracert, etc.
Grep
Access via RDP
Access via keyboard
Access to PDUs
46. SOP - Master Page 46
9.8.1 Client Side Logging
Enabling logging on the client system will provide detailed information, specifically how much data is
being consumed by the client and how well or poorly data is being queued, buffered and processed.
In order to enable logging, the Logmanager service will need to be installed and the MIS.XML
configuration file will need to be updated.
9.8.1.1 Installation Procedures
As a prerequisite the SpryWare MIS32 bit SDK must already be loaded on the client computer in
question. For instructions on how to load the SDK, refer to the SpryWare SDK Installation
Procedures.
9.8.1.1.1 Logmanager Service Procedures:
1. From a shell (command prompt, ‘cmd.exe’) CD to the SpryWare installation directory. By
default this is “c:sprywarebin”
2. Execute “logmanager –install” to install the service. An OpenService failed message will
appear but the service should install successfully as shown below:
47. SOP - Master Page 47
9.8.1.1.2 MIS.XML Update Steps:
1. Open the MIS.XML file for editing.
2. In the “<Socket>” section add a tag called EnableDebugLogging and set it to “on”. An
example <Socket> section is shown below:
<Socket>
<TableSocket>mis66.spryware.com:80</TableSocket>
< EnableDebugLogging>yes</EnableDebugLogging>
<Socket>
3. Create a section in the file called Logmanager. This should be at the same level as <Socket>,
<Directory>, etc. That is, it is not embedded as a subsection but contained within the scope
of <MISConfig> (the top layer.) Add a tag to this section called LogDirectory and set it to a
directory path. This is the file system location that the logmanager service will create logs
in. An example is shown below:
<LogManager>
<LogDirectory>c:sprywaredata</LogDirectory>
<LogManager>
Once these configurations are in place, merely starting the “SpryWare Logmanager Service” via the
Services MSC, or at the shell with “net start logmanager”, will allow events to be logged. No
changes need to be made to the application. The EnableDebugLogging value will write an entry to
the MIS.LOG file when an error or failover occurs.
48. SOP - Master Page 48
10. INFRASTRUCTURE
In addition to the Market Information Server (MIS) described above, we have several different types
of other servers utilized in the various SpryWare hosted data centers. An overview of each of these
servers is provided in the following sections.
10.1 ADM
10.2 ESXI
Refer to the document entitled “Standard Operating Procedures – ESXi Overview” for more details.
10.3 NAS
10.4 DELL MD1000
49. SOP - Master Page 49
11. MARKET DATA
11.1 OVERVIEW
The SpryWare MIS is capable of reading a wide variety of domestic and foreign market data feeds.
New feeds are being added or updated all the time. A listing of all currently supported market data
feeds is provided in a sub section shown below.
For all servers residing at one of SpryWare’s hosted data centers, market data is supplied through
SpryWare’s business partner, Essex Radex Company (ERCO). Incoming market data feeds are the
responsibility of the client for all servers that reside outside of one of SpryWare’s hosted data
centers, i.e., client site or non-SpryWare hosted data center.
An Exchange Feed distribution diagram outlining all market data feeds available at SpryWare’s
hosted data centers is available in the Operation section of the SpryWare SharePoint site.
The content and distribution mechanism associated with market data feeds is not static, updates
occur on a regular basis. This may include content changes, hours of operation changes, bandwidth
upgrades, network requirements and various other solutions designed to address expansion
requirements. As such, SpryWare continuously monitors market data change notices that come in
from a variety of sources. Each proposed change outlines the requirements and sets a schedule that
may include test periods and a cut over period. The transition of the new change into Production
may consist of a measured roll out or hot cut at a specific time and date. The handling of these
market data feed changes is detailed in this section.
11.2 SHAREPOINT MARKET DATA CALENDAR
Included in the SpryWare SharePoint Site Home page is a link to a list titled Market Data Calendar.
This list is used to document all upcoming market data feed changes and the required procedures
that need to be followed to ensure a successful roll out. A screen shot of a sample Market Data
Calendar entry is shown below.
SpryWare monitors a number of resources to identify and evaluate changes to market data feeds.
These changes are reviewed by the SpryWare Development team and updated in the SharePoint
Market Data Calendar section. Updates requiring development work are added to the Development
and QA Sections of SharePoint.
50. SOP - Master Page 50
11.3 MARKET DATA CALENDAR SOURCES
A variety of resources are available to notify market data suppliers and recipients of upcoming
changes. This section details the various resources, including how to subscribe to the update
information, how & when updates become available and the procedures SpryWare follows to
monitor, document and act upon the changes.
11.3.1 NASDAQ OMX
News Alerts are made available on the NASDAQ OMX website located at:
http://www.nasdaqtrader.com/HomePage.aspx
Click on the News Alerts tab to view the current headlines.
These news alerts can be emailed by filling out the Enrollment Form supplied by The NASDAQ OMX
Group. This form is located at:
https://www.nasdaqtrader.com/EASP/TraderEASP.aspx?id=NewsSignup
Create an account and select the alerts to be received.
51. SOP - Master Page 51
11.3.2 Clearnet Calendar
11.3.3 Interactive Data Corporation (IDC) Developer Notices
Stand alone SOP
52. SOP - Master Page 52
12. SPRYWARE HOSTED DATA CENTERS
12.1 OVERVIEW
SpryWare allows for flexibility in how The Market Information Server is installed and configured. It
starts with the market data. We can work with existing direct data feeds if a customer is already a
subscriber, or we can provide the data. The same goes for the actual data center location. As
outlined below, we can install in either the customer’s data center or set up a new server in one of
our locations. This section specific to spryware hosted data center…
For customers who are not interested in managing various data lines and networking in a data
center, SpryWare Hosted offers an ideal solution. Take advantage of existing infrastructure and
economies of scale and co-locate your SpryWare MIS Servers with SpryWare at either or both of our
data centers in New York and Chicago. We are located at the very source of the exchange feeds to
eliminate any latency produced by geography.
Each data center leads the industry for physical security, power availability, infrastructure flexibility
and customer support, exceeding the standards set by global Internet brands and leading
enterprises. Each data center is designed to support and protect mission-critical Internet
operations.
12.2 REDUNDANCY
SpryWare integrates redundancy in both the Hosted Data Center and Client premise equipment. All
Market Information Servers are built to a specifications that dictate multiple power supplies,
multiple Network Cards and hard disks.
Incoming Market Data feeds at the Hosted Data Centers are … two vlans multiple nics. This is also
encouraged for CPE devices.
In addition to these server and network specific redundancy options … configured for multiple front
end and back end servers…
12.3 DISASTER RECOVERY
SpryWare has implemented a number of policies to address the issue of disaster recovery. System
monitoring and diagnostics are implemented to identify problems before they reach a critical stage.
For situations where a Severity 1 or 2 issue does occur, these monitoring and diagnostic tools will
notify SpryWare immediately so that steps can be taken to restore service.
12.4 SUPPORT
Qualified technicians are on-site 24 hours a day, 365 days a year at the Chicago Data Center to
perform routine and emergency maintenance/installation procedures. In New York, timely on-site
support is available during each market trading day from 8:00 AM CT to 5:00PM CT. After hours
53. SOP - Master Page 53
support in New York is available on a “Best Efforts” basis as described in Customer Service Guide
section of this document.
12.5 CUSTOMER CARE AREA
The Chicago Data Center offers a variety of support services. To allow efficient and speedy
customer access and to accommodate the demands of lengthy system/software installations,
customer care areas include:
Individual workstations with telephone and Internet connections
Anti-static equipment staging area
Kitchen, relaxation area, video game room, shower facilities
Shared conference rooms
Secure loading docks to facilitate equipment delivery or shipping
12.6 SITE ACCESS
Three types of site access include Direct Access by a SpryWare employee, Smart Hands & Remote
Access. Discuss Equinix Admin access and who can visit. Tony to provide the rules for gaining access
to SIAC.
12.7 DATA CENTER INFRASTRUCTURE
12.7.1 Overview
All elements of each data center – building shell, exterior, floors and roof – meet or surpass local
building codes and standards.
12.7.2 Physical Security
Multi-level physical security features and a rigidly controlled operating environment protect
valuable customer assets and operations. All areas of each data center are monitored and recorded
using CCTV, and all access points are controlled. Additionally:
Each facility is manned by onsite security on a 24x365 basis.
No keys required: In Chicago all doors, including cages, are secured with biometric hand
geometry readers. In New York a security guard is required as an escort directly to the
appropriate cabinet.
Silent alarm and automatic notification of appropriate law enforcement officials protect all
exterior entrances.
To enter a facility, a customer must present a valid picture I.D. Only pictured government-
issued forms of identification will be accepted.
All equipment checked upon arrival.
Shipping and receiving area walled from collocation areas.
12.7.3 Network Operations Center (NOC)
54. SOP - Master Page 54
The environmental components of each data center; Security, Power, HVAC, Fire Protection, are
monitored at the Hosted Data Centers Network Operations Center (NOC). Each NOC is staffed
7/24/365 to ensure seamless operation of their respective facilities.
All Market Information Servers are monitored 24 hours per day, 7 days per week by the SpryWare
Operations department. This includes both servers housed at one of SpryWare’s Hosted Data
Centers and Customer Premise Equipment.
12.7.4 Power Systems & Distributed Redundancy
Highly reliable power is imperative for critical customer operations. The entire electrical system has
built-in redundancy to guarantee continuous operation. The overall system is N+1 redundant,
including each component within the parallel electrical systems.
AC and DC raceways with 2N distribution
AC power delivery via distributed redundant UPS systems
Batteries with at least 7 minutes full load operation (diesel engine generators take roughly 8
seconds to synchronize and assume load); 48 hours worth of generator fuel, contracts with
multiple fuel providers.
Isolation K factor transformers used for 480 volt UPS to 208/120 volt. K factor of K20; 80
degrees Centigrade rise; cobber winding, DC system fuse protection; -48 volt delivery via
fuse panels.
12.7.5 Customer Power
Power systems are designed to meet customers’ diverse needs. Typically each cabinet receives two
20Amp Circuits Primary and two Secondary. AC power systems are capable of delivering both 120v
AC and 208v AC power in a variety of amperage configurations.
12.7.6 Network (Internet & Carrier Detail)
Each of SpryWare’s Hosted Data centers are network neutral utilizing multiple Tier 1 network
providers including…
12.7.7 Environmental Controls
To provide optimum conditions for equipment operation and minimize downtime due to equipment
failure, the HVAC system provides appropriate airflow, temperature and humidity. Redundancy
features provide additional protection for customer operations.
A Power Distribution Unit (PDU) is a highly reliable; multiple outlet power strip designed to deliver
conditioned power to mission-critical server, networking and support equipment. These units are
used throughout SpryWare’s Hosted Data Centers used in conjunction with Uninterruptible Power
Supplies (UPSs).
SpryWare utilizes metered PDUs, which allow for monitoring the total amount of current, in amps,
that flow through the PDU. These multi-outlet devices provide sufficient outlets for the many
55. SOP - Master Page 55
devices that are commonly installed into a rack enclosure. The metered aspect of the PDU helps
ensure that the PDU is not overloaded past the recommended rating of the unit.
Each cabinet at the SpryWare Hosted Data Centers is typically equipped with two to four PDU
devices. For equipment that comes configured with multiple power supplies, two PDUs will be
employed for that particular piece of equipment, thus providing a greater level of redundancy.
Each PDU is accessible via an in-house, or SpryWare VPN enabled, web based interface. The Server
Detail section of the SpryWare SharePoint site lists the PDU number and associated port(s) for each
piece of equipment housed in the data center.
12.7.8 Fire Protection
SpryWare Hosted Data Centers are protected with a dual-alarmed, dualinterlock multi- zoned, dry-
pipe, water-based fire suppression system armed with sensory mechanisms (HSSD) to sample the air
and give alarms prior to pressurization. Production area fire suppression is provided by a
multizoned, pre-action, dry-pipe system. In order for the system to trip, multiple cross-linked events
must occur. These include detection by ceiling mounted smoke-heads and smoke “sniffers” located
throughout the facility. Lastly a sprinkler head must trip in order for the dry- pipe system to activate.
This requires a temperature of 140 degrees F at the head location. Fire suppression is localized at
the event point only.”
12.7.9 Flood Control
The Chicago Data Center offers a number of additional features. The New York facility may offer
similar features, however, document research is still being performed to determine this.
The facility is built above sea-level with no basements. Conduits are tightly sealed, with moisture
barriers on exterior walls. There are dedicated pump rooms and moisture detection sensors are in
place. Drainage/evacuation systems are implemented.
12.7.10 Earthquake Preparedness
The Chicago Data Center offers… Location-specific seismic compliance. Structural systems meet or
exceed seismic design requirements of local building codes for lateral seismic design forces. In
addition, equipment and nonstructural components, including cabinets, are anchored and braced in
accordance with the requirements of the 1997 Uniform Building Code.
