Healthcare Interface Engine Selection

OTB Approach to Healthcare
Interface Engine Selection
The thirteen steps of highly successful HL7 engine
selections
John Traeger
john.traeger@otbsolutions.com

Table of Contents
− Introduction
− The Proven Approach – Preparation
− The 13 Step Process

− Why Your Requirements are Wrong

− The Proven Approach – Collection and
Evaluation
− So Did You Like What They Said?
− Who do you invite to do a “dog and pony
show?”
− What to look for in the demonstration

− Why Your RFI Doesn’t Deliver Information
You Need

− Score each vendor immediately after the
demo

− Don’t Forget to Ask About… Everything

− Sounds Good, Now Let Me Do It

− Why Your RFI Will Be Ignored
− Define gates in the selection process
− Develop scoring system
− Scaling the process for your organization
− How to make sure you send your RFI to all
potential vendors

− They shoot and…Score!

− Now Let’s Talk About Money
− Let’s Talk to the Vendors’ Friends

− The Proven Approach – Making a
Decision
− So What’s All This Really Going to Cost?
− May I Have The Envelope Please?
− Finding the pick of the litter
− Taking it the last mile
− Now the Real Work Begins; Avoid Getting
Lost in the Noise

John Traeger

Last modified:16 September 2013
© 2013 OTB Solutions. All rights reserved. | OTB is a Microsoft Gold Certified Partner

2

John Traeger (john.traeger@otbsolutions.com)


Introduction
− Selecting an interface engine for your healthcare organization is not a trivial
matter.
− impacts the reliability and flexibility of healthcare data delivery within and outside
your organization
− costs are much broader than what you pay the software vendor; deployment,
migration, training, and support hours
− The importance of “fit” cannot be overstated
− internal costs and system effectiveness will vary widely depending on the engine’s
features and the context and culture of your organization

− Why would I bother with a process when I already know what I want?
− You only know what you know; what you don’t know can cause expensive mistakes
− Its an opportunity to clearly define the interface team’s role in your organization
− Learn more about how your team functions which leads to improved effectiveness



Introduction
− Who else would care about using a process like this?
− Anyone who responsible for selecting a new interface engine:
− IT Director

− PM
− Lead Developers
− IT Portfolio Managers

− Executive Team
− The future of healthcare and executive goals are inextricably linked to the movement of
data
− They expect the interface team to be agile enough to meet application changes (new or
updates) seamlessly without impacting business planning or strategy

− When should you start the process and how long is this going to take?
− Start as soon as possible after internal approval to ensure enough time for a
thorough selection process
− For a large organization making a major investment, this is typically a 2-3 month
process
− For small organizations, this may take as little as 1.5 months



Eight reasons HL7 interface engine selections fail
1. Biases
a.

Toward a vendor

b.

Toward a technology

2. Rushed decision making
3. Only a single vendor considered
4. Replacing what is known instead of what could be
a.

Requirements oriented around current state

b.

Requirements that are not looking at the art of the possible

5. Misjudging skill level of development team
6. Misjudging the level of effort required to change interface engines
(replacing an engine while the plane is in flight)
7. Misunderstanding the role of the interface engine in the organization’s
healthcare ecosystem
8. Not identifying or understanding the stakeholders

THE PROVEN APPROACH – PREPARATION



The 13 Step Process
1. Develop requirements
2. Create RFI
3. Identify vendors and submit RFI
4. Score RFI
5. Invite vendors for sales demo
6. Score sales demo
7. Invite vendors for Proof of Concept
8. Score Proof of concept
9. Send out RFP/RFQ
10.Score RFP/RFQ including financial component
11.Select Vendor
12.Negotiate the final details and obtain final signatures
13.Socialize the selection in your organization



The 13 step process:

− Why Your Requirements are Wrong
− Requirements are stated as how instead of what you want accomplish
− your requirements (yes, even technical ones) should describe what the business/interface
team is trying to do, not how they do it.

− Requirements reflect the current state instead of future state
− Thinking outside of the box and designing requirements around the art of the possible will
ensure a more successful selection and implementation.

− Enterprise Context is missing
− Requirements that don’t take into account the current and target enterprise context can
lead to selections that result in a poor fit.

− Requirements are biased
− The team needs to identify and acknowledge preconceived biases to ensure the selection
process is objective and well informed. Examples include:
− The latest and greatest solutions are always best
− Developer needs and wants are the only criteria considered
− Pre-existing vendor relationships
− Cheapest vendor always is best




− Why Your RFI Doesn’t Deliver Information You Need
− Who Cares About This? Understanding the requirements by stakeholder
− Who are the business and technical leaders that your team works with on a regular basis?

− Who are the key players your team has to consult for specific expertise?
− Who do you have to call for permission for large projects?

− Who Are You? Understand the context
− Understanding the context of your organization is important in finding an engine and
technology that is a good fit.
− Interface Environment
− Enterprise Portfolio
− IT Guiding Principles
− Cultural Fit
− Organization’s decision making process

Lesson
Learned:

Clearly understanding the requirements will significantly increase the chance of management and developer buyin at the end of the process

Lesson
Learned:

Clearly understanding the context within which the Interface Engine will be implemented makes the questions
you ask the vendor more relevant and effective




2. Create RFI
− Don’t Forget to Ask About… Everything
− Create your RFI questions with intent
− create an RFI that contains questions that illustrate your intent and reveal you have done
your homework

− Question Types
− Subjective - These often relate to usability.
− Is this tool easy to use?

− Objective – These questions are often Binary yes/No questions
− Does the engine support HL7 3.X?
− Does the engine support web services?

− Quantitative - These questions can be about capacity, performance, scalability, etc.
− What is the maximum number of connections per engine instance?
− What is the maximum daily message volume?

− Qualitative – These questions typically are about reliability, customer service, etc.
− What is the failover experience for end users? For developers and support staff?
− What is the skill level of your on-call support staff?
Lesson
Learned:

Relying too much on subjective questions may lead to a biased selection. A good balance of question types
ensures you look at the offerings from multiple perspectives.




2. Create RFI
Categorize questions to create structure
− Interface team specific:

− Engine Development Environment and
Tools
− Engine Support
− Data Migration/ Conversion from old
system

− Training

− Enterprise context specific:
− Server Configuration
− High Availability - Disaster Recovery
− Software Capacity
− Network
− Database
− Desktop Client and User Devices
− Web Enabled Standards
− Remote Access
− Customer/Vendor Support
− Security/HIPAA
− Business

Keep the door open for the vendors to answer “unknown unknowns”
− Make sure there are open ended questions that encourage the vendors to tell
you what else they can do that your team may not have thought of.
Lesson
Learned:

Be sure your questions get at the skills and staffing requirements for the vendor’s solution. You will need to
know both quantitative and qualitative answers for implementation, ongoing support and upgrades. The
answers to these questions will be important in later evaluation for total lifecycle cost impacts.




2. Create RFI
− Why Your RFI Will Be Ignored
− To get vendors interested your team needs to provide the vendor with:
− Clear business statement
− Describe your business goals and timing

− Demonstrate you understand the scope and scale of an interface engine replacement
− Organization size
− Interface team size and structure
− Number of applications and interfaces

− Average messages per day

− State at what level the initiative is funded and approved
− Executive approval
− Capital funded, Operations expense, etc.

− Describe the selection process
− Purpose of the RFI
− Demo, POC, and RFP/RFQ phases
− Scoring process
− Projected timeline
Lesson
Learned:

The more details you can provide about commitment of funds, organization buy-in and timing, the more interest and
response from qualified vendors you will get. Be clear and up front about your process, timing and what you are
expecting.




2. Create RFI
− Define gates in the selection
process
− Typically there are four scoring
rounds:
1. RFI Score
2. Vendor Demo
3. On-site POC
4. RFP/RFQ Score
− Pricing is NOT mentioned prior to
this step
− Best and final offer made by vendor

Lesson
Learned:

By waiting on pricing until the RFP/RFQ stage, the vendor gets a much better understanding of your business and
can provide more accurate cost estimate




2. Create RFI
− This system has to be
communicated clearly and
applied consistently to all
vendors:
1. Question/Response elements
are grouped by Functionality

FUNCTIONALITY

QUESTION

Factor

Engine Support
22

Is there monitoring in the application? Please describe.

23

Does your system have connection alerting? Please describe.

24

Can your system send emails? Can your system interface with
Active Directory distribution lists?

25

What analytics/metrics can be produced from the system? Is
there a "dashboard" view of this? Please describe.

− Sum of elements in a group =
100
− Weights for individual elements
should be whole numbers

− The weights are usually set by
the team at the start of the
process and not changed after
that point

26

What is the upgrade process?

27

2. Each element within a group
is given a factor weighting
within that group

Weighting

Do upgrades require down time and if so what is the typical
duration?

28

How often are new major versions of your product released?
Minor ones?

29

When was the software first developed and installed? When was
the last major release or upgrade?

30

When is the next major software upgrade planned for this
system? Briefly, what will be new?

31

How are patches delivered?

32

Does the application allow for logging of outbound messages?

33

What are the data retention capabilities and recommendations for
maintaining history on-line?

34

What is the process for resending messages both single and
batches?

35

Does the application use standard Windows or UNIX commands?

36

What system performance monitoring and optimization tools are

2
10

10
10
5
10
5
1
1
1
10
10
10
5
11

included with the application?
TOTAL FOR CATEGORY

Example group of elements with weights

100




2. Create RFI

FUNCTIONALITY

3. Each element of the RFI
response from each vendor is
scored by the team
− Use "Likert Scale" which
includes odd number of choices
such as:
− 0.0

Doesn't meet

− 0.3

Partially meets

− 0.5

Weighting

Vendor

Team

Factor


QUESTION

Response

Raw
Score

Engine Support
22


23


24

Can your system send emails? Can your system interface with

2

Yes

10

Yes

10

Yes

10

No

5

Yes

10

Yes

5

Yes

1

Yes

1

Yes

1

Yes

10

Yes

10

Yes

10

Yes

5

No

11

W/Mods

0.5
0.5
0.5

Active Directory distribution lists?
25

What analytics/metrics can be produced from the system? Is

0.3

there a "dashboard" view of this? Please describe.
26


27

Do upgrades require down time and if so what is the typical

0.5
3.0

duration?
28

How often are new major versions of your product released?

0.5

Minor ones?

Meets requirement

− 0.8
effort

When was the software first developed and installed? When

When is the next major software upgrade planned for this

Exceeds out of the box

0.3

system? Briefly, what will be new?
31


32

− Example group with a vendor’s
raw score by team:

0.5

was the last major release or upgrade?
30

Exceeds with some

− 1.0

29


33

What are the data retention capabilities and recommendations

3.0
0.3
0.5

for maintaining history on-line?

34

What is the process for resending messages both single and

0.5

batches?
35

Does the application use standard Windows or UNIX

0.3

commands?
36

What system performance monitoring and optimization tools
are included with the application?

TOTAL FOR CATEGORY

100

Example group with a vendor’s raw score by team:

0.5




2. Create RFI
4. The factor weighting times the team raw score equals the weighted score for
each element for a vendor.
FUNCTIONALITY

QUESTION

Weighting

Vendor

Factor

Response

Team Raw

Weighted

Score

Score

Engine Support
22


2

Yes

0.5

0.8

23


10

Yes

0.5

5.0

24

Can your system send emails? Can your system interface with Active Directory distribution lists?

10

Yes

0.5

5.0

25

What analytics/metrics can be produced from the system? Is there a "dashboard" view of this? Please

10

No

0.3

3.0

5

Yes

0.5

2.5

10

Yes

3.0

30.0

describe.
26


27

Do upgrades require down time and if so what is the typical duration?

28

How often are new major versions of your product released? Minor ones?

5

Yes

0.5

2.5

29

When was the software first developed and installed? When was the last major release or upgrade?

1

Yes

0.5

0.5

30

When is the next major software upgrade planned for this system? Briefly, what will be new?

1

Yes

0.3

0.3

31


1

Yes

3.0

3.0

32


10

Yes

0.3

3.0

33

What are the data retention capabilities and recommendations for maintaining history on-line?

10

Yes

0.5

5.0

34

What is the process for resending messages both single and batches?

10

Yes

0.5

5.0

35


5

No

0.3

1.5

36

What system performance monitoring and optimization tools are included with the application?

W/Mods

0.5

TOTAL FOR CATEGORY

11
100

5.3
72.3




2. Create RFI
5. Total weighted score for each functionality group is calculated and compared with
the totals for each vendor.

6. Use the same scoring system to re-score the vendors after each round
- The financial component is not scored until the final RFP/RFQ round
Lesson
Learned

Be sure the Factor Weighting within each group are communicated/negotiated with the key stakeholders
(especially leadership) before the first round of vendor scoring happens. They will not have adequate buy-in of
the final selection unless they are very familiar with how and when the weights are decided.




2. Create RFI
− Not every organization has the resources to commit to a two to three month
selection process. Some general guidelines:
− Medium and Large organizations need to use the full process. The risk associated with
an improper fit are high enough to require the full, rigorous approach:
− Small organizations with their own IT resources can use a more streamlined process.
The scale of risk associated with their selection is lower and they can afford a less
rigorous yet well informed approach:
− Small organizations with outsourced IT resources have a unique wrinkle in their process.
They may have to evaluate new contract IT resource to support the technology as well:

Lesson
Learned:

Medium and large organizations that don’t take the time to do the full due diligence are at high risk of making a
selection that results in an improper fit.




2. Create RFI
Organization size and IT model
Medium and Large

Small with own IT

Small with
outsourced IT

Steps

1.

Develop requirements

1

1

1

2.

Create RFI

2

2*

2*

3.

Identify vendors and submit RFI

3

3

3

4.

Score RFI

4

5.

Invite vendors for sales demo

5

4

4

6.

Score sales demo

6

5

5

7.

Invite vendors for Proof of Concept

7

7**

7***

8.

Score Proof of concept

8

9.

Send out RFP/RFQ

9

10. Score RFP/RFQ including financial component

10

8

8

11. Select Vendor

11

6

6****

12. Negotiate the final details and obtain final signatures

12

9

9

13. Socialize the selection in your organization

13

10

10

* (Combine with RFQ)
** (Winner only)
*** (Software and implementation vendor)
**** (includes vendor/contactor for implementation)




3. Identify vendors and submit RFI
− How to make sure you send your RFI to all potential vendors
− Do your homework to ensure you are looking at all the vendors who could
potentially meet your needs.
− web searches
− HL7 web communities
− KLAS reports (http://www.klasresearch.com/segment/19)
− Other comparable healthcare organizations and professional associations.
− Whitepapers or case studies that are released by vendors can be useful though one
should remember the source when reading.

− Once you have generated a list of vendors, send out an initial communication to
identify a sales contact for RFI response.

Lesson
Learned:

Often on the vendor web site there will be a generic sales contact. It may take a few days for them to respond to
the initial communication and give you a sales person to work with.

THE PROVEN APPROACH – COLLECTION AND EVALUATION




4. Score RFI
− So Did You Like What They Said?
− The RFIs are in; how do the vendors look?
− were they timely? Did they answer the questions? Did they bury you in marketing
material? Did they contact you for clarity?
− If there are problems in terms of clarity with your RFI it will be evident at this stage

− Take this opportunity to hone the scoring process, and don’t be afraid to modify it
if there are blocking issues or gaps.
− This is the last opportunity to change the scoring process significantly without creating the
potential for bias.

− Strive for a consistent sense of the rating scale used for individual elements.
− Obtain a consensus on what the difference is between “Meets requirement”, “Exceeds
with some effort”, and “Exceeds out of the box”.
− Wait for all of the responses to come in and then score them at the same time or at least
in the same week.
− In cases of many responses, go back to the first ones you scored and reevaluate to
ensure that you are consistent.
Lesson
Learned:

The sales engineers can be leave a misleading impression of a company and its product. While the timeliness
and quality of the RFI answers should be taken into consideration, don’t rely too heavily on those first
impressions.




− Who do you invite to do a “dog and pony show?”
− The top 3-5 vendors are invited to do a sales demonstration
− This can be a virtual meeting, though in-person tends to be more productive and more
questions get asked by your team.

− What to look for in the slide show
− Features of the product
− A session on a demonstration server to run through some basic scenarios
− Elaboration on the key question elements from the RFI
− New approaches to solving interface engine or business challenges.
− A simplified development process
− Automated tool sets
− Improved documentation model

− Better visibility/sharing of interface status across roles in an organization.
− Etc.

− Snippets of testimonials and case studies for the presentation. Ask for more
detailed information on these at the end of the presentation along with a copy of
the slide deck or other materials reviewed.




− What to look for in the demonstration
− This is the vendor’s chance to show their application in a best case scenario
− Pay close attention and takes notes for later evaluation discussions.
− Provide an ad hoc question during the demo phase to see how each vendor
responds without having the ability to prepare.
− Some key features you should expect to see demonstrated:
− One-to-many interfaces such as ADT
− Many-to-one interface such as Clinical Reporting or Charges
− HL7 versions supported
− Toolsets used to develop interfaces
− Web service strategy
− Look and feel of user interface your operations desk would use to monitor system alerts

− Self-service features for analysts and operations




6. Score sales demo
− Score each vendor immediately after the demo
− Get an accurate response from the scoring team while the information is fresh.
− Reduces the confusion in cases where vendors present on consecutive days (or
the same day).
− Use the same core team to score for every vendor to maintain scoring
consistency:
− Interface Developers
− Interface Analysts
− Server engineering
− Operations support
− Interface Team Manager

Lesson
Learned:

Vendor demonstration interface engine servers will always show best-case scenarios. Don’t be fooled by how
simple they can make it look, but pay attention to the features they show and drill into them during the POC phase.




− Sounds Good, Now Let Me Do It
− Each vendor should be invited to an independent week for POC session
− Your team will perform hands on work using their product.
− Your team will vet at least some of their claims made during the sales demo.
− Watch carefully how your team adapts to the new tools to understand how their
skill set fits with the product

Lesson
Learned

Yes that’s right, three vendors = three separate weeks. Remember, this is a major investment for your team and
organization. Both will have to live with your choice for many years. It is critical you take the time to fully
understand the products offered.




− Setting the stage
− There is a considerable amount of pre-work needed to host successful POC
sessions. Key steps to being ready for the event:
1. Develop interface build scenarios with your development team. These should include
but not be limited to:
− Basic ADT interface
− Complex ADT filtering
− Basic Orders and Results interfaces
− Complex Results formatting interface
− Charges to flat file

2. Obtain a large volume of test messages that are relevant to the build scenarios.
− Ideally these should be de-identified production messages so you can run them through connections
to and from your end use applications’ test instances when validating your build scenarios.
− If de-identified messages are not available, then volume testing will have to be limited to within the
interface engine and not through connecting to the application.

3. Obtain from the vendor their Operating System and server requirements needed to run a
POC with up to 4-5 interfaces and some potentially having some high volume.




− Setting the stage
− There is a considerable amount of pre-work needed to host successful POC
sessions. Key steps to being ready for the event:
4. Obtain a Server (either physical or Virtual)
− Make sure it is accessible by your interface developers.
− Be sure they have Administrator rights to the server so they will be able to install the vendor
software.

5. Verify you can establish connectivity between the POC server and the test instances of
the applications involved in your build scenarios.
6. Book a conference room with data projector for 5 full days just to make sure there is
enough time.
7. Schedule up to 5 days for the demo session with your Developer team.
− This is critical to get a complete set of opinions on the usability of the vendors interface engine
software.

− If your organization requires the use of server engineers to install software, be sure the server
engineer who will be doing the production implementation is available for the first day of each
Vendor POC session.
− Be sure your lead developers are scheduled to attend all five days
− Invite some junior developers to the second day to observe how they respond to the learning curve
of the product.




− Sample POC Agenda
Day
Time
1 Afternoon

2

10:00AM –
4:00PM

3

10:00AM –
4:00PM
10:00AM –
4:00PM

4

5

Lesson
Learned

10:00AM –
Noon

Activity
Vendor Arrives
Introductions
Install the software
Introductions
Build simple scenarios

Build complex scenarios
Run large volumes through test interfaces,
wrap up session with Vendor
Vendor departs
Interface team re-scores the vendor in light
of what was learned during the POC

Attendees
Vendor, PM, Server
Engineer, Lead Developer
Vendor, PM, Lead
Developers, Junior
Developers
Vendor, PM, Lead
Developers
Vendor, PM, Lead
Developers
PM, Lead Developers,
Junior Developers, Server
Engineer

Consider this your time with the application. If your team has a problem on their existing interface engine they
have been unable to resolve, it’s OK to let your developers have a crack at it on the new vendor platforms during
each POC session. However, be careful of rat holes and avoid letting a single technical problem hijack the
process.




8. Score Proof of Concept
− They shoot and…Score!
− This is the most important round of scoring for your interface team.
− The recent hands on the problem solving with the products makes for an informed
assessment
− The scoring must happen at the end of the POC week to maintain consistency
and clarity.

Lesson
Learned

It is critical you schedule your team for enough time to do this critical round of scoring on each vendor so they
will support the final selection.


9. Send out RFP/RFQ
− Most if not all of the feature requirements were discovered during the RFI, demo
and POC phases, the RFP/RFQ includes mostly financial components.
− A detailed list of what you are being
offered
− Number and type of software licenses
− Specific feature bundles (if any)

− Professional Service hours
− Implementation support
− Initial costs
− System purchase

− Recurring costs
− Annual licensing
− Server?
− Connections?
− Processors?
− Annual support
− 24 X 7 or business hours?
− Phone support
− Engineering support
− Response times

− Server?
− Connections?
− Processors?
− Professional fees
John Traeger


31




9. Send out RFP/RFQ
− Most if not all of the feature requirements were discovered during the RFI, demo
and POC phases, the RFP/RFQ includes mostly financial components.

− A detailed list of what you are
being offered

− Recurring costs
− Annual licensing

− Number and type of software
licenses

− Server?

− Specific feature bundles (if any)

− Processors?

− Professional Service hours
− Implementation support

− Initial costs
− System purchase
− Server?
− Connections?
− Processors?

− Professional fees

− Connections?

− Annual support
− 24 X 7 or business hours?
− Phone support

− Engineering support
− Response times




9. Send out RFP/RFQ
− Don’t’ forget to weight the money
− The financial category should have elements that align with the major components
that make up a vendor quote.
− Your team (or perhaps finance) can assign factor weighting to each element, for
example:
Pricing

Category
172
173
174
175
176
177

Seat License
Server License
Consulting Fees
Support Fees
Upgrade costs
Training costs
TOTAL FOR CATEGORY

Weight
0.2000
0.2000
0.2000
0.1000
0.1000
0.2000
1.0000

− When comparing value based on price, be sure you are comparing apples to apples.
− What type of training is the vendor offering?
− Is it onsite or will travel be required?

− What is included in support fees and consulting fees?
Lesson
Learned:

Be careful on the factor weighting given to the financial requirements. If too much weight is given to a single
element you can turn some promising vendors into losers when in fact total cost of ownership and increased
business value to your organization should keep them in consideration for the final selection.




9. Send out RFP/RFQ
− Let’s Talk to the Vendors’ Friends
− References are very important in assessing the fit with your organization.
− Request from the Vendors customers who are the same size/type as your organization (most
important).
− Inclusion of one or two larger and smaller organizations can also provide some useful insights
− Ask for reference customers who have a similar application portfolio mix (HIS, Lab, etc.)

− Call each reference and arranging a time for a phone interview with some of your team.
− This can extend the timeline of the process a bit but references are too important to ignore.

− If time and distance allows, arrange a site visit with some reference customers
− Sit with their developers and see how they use the product.
− Gives a tremendous amount of additional insight into how the vendor’s product behaves in
the real world.

− Be prepared to compromise on finding a reference of the same size or type as your
organization for each vendor that is reasonably close.
− It is worth taking a site visit to a dissimilar organization if it is your only option to seeing the
product running in a production environment.
Lesson
Learned

Don't collect or spend time contacting references until the field is narrowed down to 3 or less vendors as it is time
consuming to schedule the interviews.

THE PROVEN APPROACH – MAKING A DECISION




9. Send out RFP/RFQ
− So What’s All This Really Going to Cost?
− The calculations only start with each vendor’s RFP/RFQ response.
− Hardware purchase and installation
These will vary depending on vendor system requirements and the scope and scale of the interfaces in
your organization. Typical items include:
− Servers (blade, rack mount, or virtual)
− Data Center rack space
− Storage (SAN or on board RAID)

− Network or Server Rack appliances
− Data center or server engineer hours

− Development and Implementation
Labor costs are less precise to calculate than hardware and installation. The costs fall into three buckets:
− Contract Legacy system developers to fill in while your team ramps up on the new platform

− New Platform developers to increase velocity of migration effort
− May be provided by vendor or 3rd party

− Consulting Services from Software Vendor
Lesson
Learned

The differences in labor costs between vendors can be dramatic and can easily change the value propositions
from each. Be sure to take the time to look at availability of labor and size of learning curve for the technology
being used by the vendor. To ignore this can be a costly mistake.




10. Score RFP/RFQ including financial component
− May I Have The Envelope Please?

Vendor 2

Vendor 3

Vendor 4

Vendor 5

Weighted
score

Weighted
score

Weighted
score

Weighted
score

FUNCTIONALITY

Factor
Weighting

Vendor 1

Weighted
score

− The totals for each vendor are rolled up into a summary worksheet that reveals
who is highest and lowest in each of the categories, including financial.

Engine Development Environment and Tools

1.00

0.452

0.417

0.427

0.427

0.427

Engine Support

1.00

0.723

0.731

0.723

0.723

0.723

Data Migration/ Conversion from old system

1.00

0.485

0.485

0.485

0.485

0.485

Training

1.00

0.560

0.560

0.595

0.560

0.560

Server Configuration

1.00

0.470

0.470

0.470

0.680

0.470

High Availability - Disaster Recovery

1.00

0.340

0.340

0.340

0.340

0.440

Software Capacity

1.00

0.190

0.190

0.190

0.190

0.190

Network

1.00

0.630

0.630

0.630

0.630

0.630

Database

1.00

0.470

0.470

0.470

0.470

0.470

Desktop Client and User Devices

1.00

0.470

0.470

0.470

0.470

0.470

Web Enabled Standards

1.00

0.633

0.633

0.633

0.633

0.633

Remote Access

1.00

0.730

0.730

0.730

0.730

0.730

Customer/Vendor Support

1.00

0.363

0.363

0.363

0.363

0.363

Security/HIPAA

1.00

0.500

0.500

0.500

0.500

0.500

Business

1.00

0.517

0.517

0.517

0.517

0.517

Pricing

1.00

0.440

0.440

0.440

0.440

0.440

Hardware Costs

1.00

0.540

0.540

0.580

0.490

0.610

RECOMMENDATION TOTALS

17.00

8.513

8.485

8.563

8.648

8.658

KEY
Lowest
Highest

Lesson
Learned:

Remember to keep vigilant of personal and institutional biases when doing the final scoring.




11. Select Vendor
− Finding the pick of the litter
− For the highest and lowest scorer in each group, write up a couple of sentences
describing why that vendor is great or not.

− The score totals represent the cumulative evaluation of all of the elements and the
sum of the functionality group totals.
− It should be pretty obvious who the winners are and who is out.

− The relative importance of some functionality groups over others is key to pay
attention to.
− For example, a vendor with a higher total score may still not be the right choice if the
business and pricing are most important and that vendor fell behind in those two
categories.

− If after all of the above it remains too close to call between two vendors, fall back
on vendor’s cultural fit with your organization




12. Negotiate the final details and obtain final signatures
− Taking it the last mile
− Once the final selection has been made, it’s time to announce the decision to
stakeholders within your organization.
− Schedule time to review the results in detail with each of the stakeholders who
helped with requirements.
− Be sure to obtain executive approval and leverage this for broader communication
within the organization.
− For each of the major groups impacted get onto the agenda for their next regularly
scheduled team meeting and review both the process used, findings and final
selection.

Lesson
Learned:

Be sure to share the process used to reach the decision and include testimonials from the stakeholders that
participated.

Lesson
Learned:

Final contract negotiations and Legal review always takes longer and is more complex than you think it should be.
Be patient and don’t let it derail the selection in the last mile.




13. Socialize the selection in your organization
− Avoid Getting Lost in the Noise
− At most organizations there are competing priorities for IT resources, even in
calmer times. In our current era of ARRA, HIPAA, business consolidations and
transformations it is harder than ever to retain the commitment from leadership to
provide the resources needed to follow through on implementing a new interface
engine.
Draft a compelling scope document
− You need to drive home the importance of the new interface engine deployment.
Be sure to highlight the dependencies all of the other major IT and business
initiatives have on a reliable, agile interface engine for success.
Keep the project visible and stakeholders informed
− Make sure the project is visible with consistent status updates. Get an early win
on an initial project.

Lesson
Learned:

It is critical to get out of the gate early and deliver an initial success. Both internal customers and the vendor will
be disappointed if there isn’t early results to create momentum.

RFI Requirement/Questions
Engine Development Environment and Tools
Engine Development
Environment and Tools

1
2

Briefly describe the usability of the developer GUI.
Does the application provide for any standard interfaces, libraries, templates, or
routines?

3

Is there an internal HL7 library in the application? How often do customers
receive updates?

4
5
6

What languages are available for scripting?
What other vendors have plug-ins to your application?

How many and what type of user defined fields are available? (memo, free text,
table driven, date, etc.)

7
8

When and how (what UI) is "Hand Coding" performed?

Does the system provide the ability to test message translations, filters, and
routes?

9
10
11

What is the recommended set up for dev./test/production environments?

13
14
15
16
17

How do you promote code from one environment to another?

18
19
20

What is the development environment? Local, shared server or other?

12

John Traeger

Does the application provide interface version control?

What are the troubleshooting capabilities?

21

What happens when automated error recovery fails?

Does the application have any imposed development processes?

Do you provide a simple solution to FTP? Please describe.
Does the application have any batch processing functionality?
Does the application have any file parsing functionality?
What other EDI formats does the system support? Is there a library for them?

What are the de-bugging capabilities?
Describe the automation and scheduling component of the application.
Last modified: 16 September 2013
© 2013 OTB Solutions. All rights reserved.

42

Engine Support

Engine Support
22
23
24


Can your system send emails? Can your system interface with Active Directory
distribution lists?

25

What analytics/metrics can be produced from the system? Is there a "dashboard"

view of this? Please describe.

26
27


28

How often are new major versions of your product released? Minor ones?

29

When was the software first developed and installed? When was the last major

Do upgrades require down time and if so what is the typical duration?

release or upgrade?

30

When is the next major software upgrade planned for this system? Briefly, what will
be new?

31
32
33

What are the data retention capabilities and recommendations for maintaining
history on-line?

34

What is the process for resending messages both single and batches?

35


36

What system performance monitoring and optimization tools are included with the
application?

John Traeger


43

Data Migration/Conversion from old system
Training
Data Migration/
Conversion from old
system
37
38

Do you provide a data conversion service?

39

What format do you require of the data to be converted?

40

Will initial training be performed at our offices or at your facilities? (include costs).

41

How many of your staff will conduct initial training? What are their qualifications and

Can you convert our existing parameters, maps and configurations?

Training

experience?

42

How much time should we budget for training each user? Please list by

different

user roles.

43
44

What is your approach to training users? Describe briefly.
What training materials (e.g. manuals, video tapes, CD-ROM, multi-media, online)
are provided?

45

Do you offer ongoing training classes? What is the typical cost per day or class?

46

Where is your nearest training facility? What resources are available?

47

Is there an organized national user group? Is there a regional user group in our
area? How often and where are meetings?

48

What documentation (i.e. user, operations, technical) is supplied with your system?

49

How often is documentation updated? Will you contractually guarantee
documentation to be current with new software releases?

50
John Traeger

Can users customize on-line help screens and other documentation?


44

High Availability - Disaster Recovery
51

Briefly describe the system architecture (e.g. network, centralized, open systems,
PC based, LAN, etc.).

52

Briefly describe the processor, memory, and disk capacity requirements for the file
servers used in the system architecture for a deployment of your system at a facility
of our size.

53

Are there any Server monitoring tools available such as BMC Patrol?

54

Does your application support running on Virtual Server environments?

55
56

Does your application support drives running on SAN?

57

What safeguards (e.g. fault tolerance, hardware redundancy) are included that

If there is a database server is it required to reside on the application server?

High Availability Disaster Recovery
eliminate unplanned downtime?

58

Describe your disaster recovery - quick restore capabilities and procedures.

59

Does your system provide the ability to configure a high availability environment?
(for example: RAC, clustering, fail over server, HACMP scripts) Please describe and

include architectural diagram with the HA configuration.

John Traeger


45

Software Capacity
Network
Software Capacity
60

What is the maximum number of Developers that can be supported?

61

What is the maximum number of monitor users can be supported?

62
63
64

What is the maximum messages per day?
What is the maximum number of connections?
Does your application have capacity to run the following scenario on a single
instance without queues backing up?
> At or near maximum number of connections on your engine
> At least one interface of these that sends 1.2 million messages/day with peaks of
100 messages/sec

65

Please provide performance metrics from "typical" implementations for small,
medium and large organizations

Network
66

Are there any network requirements integral to the engine (in the data center or on a
rack)?

67

If your solution for our organization requires a multiple server architecture, please
diagram the details of all protocols, ports and flows between servers (For example,
separate database server, configuration server, web server and the connections
between them within the data center).

68

Please specify any other data center or rack based network requirements for the
engine to operate at peak performance and capacity (this refers to between engine
components).

69

Please describe and assess DNS requirements and DNS records that are required
for the system.

John Traeger


46

Database

Database
70 Does the application use a database platform and if so what version?

71 Does your system provide ability to back-up database changes throughout the day with no
user interruption?

72 Describe your database disaster recovery - quick restore capabilities and procedures.
73 What operating system will the database server use?
74 For High Availability; does the DB use Oracle RAC, Standby Database, or SQL Replication?
75 Can the Database be placed on a SAN?
76 Are there database monitoring tools such BMC Patrol available for the Database? If so, are
there DB monitoring scripts for your product to run on the tool available?

77 Does your application support direct links to SQL , ODBC, OCI, OLEDB? Which and are
there limits or restriction?

78 Does the database require a dedicated server or can it be part of a shared server? What is
recommended?

79 Do you offer an API (Application Programming Interface) toolkit that will allow other Windows
applications to connect to your database?

80 Can other Windows programs query your data and obtain access to records and fields?
81 Does your system provide any preventive maintenance tools or utilities to identify database
integrity errors, perform clean-up, re-organize the database, etc.?

82 Does the database include any functions that are designed to ensure integrity and faulttolerance in a network environment? Please describe.

83 Will you provide a complete database dictionary that defines the contents and links for each
record and field, and allow access to the database from other Windows applications?

84 Do you have access to all data fields?
85 If your dB is NOT one that our organization DB team supports then:
Who will provide support?
What is the contact process for requesting support?
When hours is support available?

86 What is the recommended database backup schedule (hourly, nightly, weekly, etc.)?
How will the database be backed up (backup application or system)?

John Traeger

Where will it be backed up (tape, disk, file systems, etc.)?

87 How will the backups be encrypted?


47

Desktop Client and User Devices

Desktop Client and User
Devices
88

What type of client access is used for development? (Browser, Citrix, terminal
emulator, fat client)

89

Most desktops do not use static IP address, instead configured using DHCP. Will
your application have any issues with this configuration?

90

Please provide the developer desktop specifications and requirements. Include OS,
Service Pack level, supported browser types and versions, Processor Speed, Disk
Space, and RAM

91
92

If a client is used on the desktop, can it be pushed using SMS?
Are there any desktop software requirements such as Flash player, .NET, active X,
etc.?

93

Does the application store any patient data in memory or cache on the desktop
when used remotely?

John Traeger


48

Remote Access

Remote Access
112

Do you (the vendor) currently support any other systems in our network?

113

What methods can be used for vendor access? (Land to Land IPSec VPN Tunnel,
SSL VPN, IPSec VPN, Citrix, other)

114

What servers will the vendor need to have access to? Will there be a need to
access any workstation?

115

What are the protocols and UDP/TCP port numbers the vendor needs to access the
application/servers from the remote vendor client devices?

John Traeger


49

Customer/Vendor Support
Customer/Vendor
Support
116

Where is the nearest software support office to our location? How many technical
staff? How many programmers?

117

Is telephone software support available 24 hours per day and 7 days per week?
Are support staff in office at all times or on-call from home?

118

Is your support department fully staffed 8 a.m. to 5 p.m. for customers in all time
zones (e.g. EST to PST)?

119

Do you have an 800 toll free support number? Do you bill telephone charges to
customers?

120

Is telephone support time included in the support fees or do you charge on a per
call or hourly basis?

121

Do you have a guaranteed response time for answering customer problem calls? If
yes, what is it?

122

How and when are known software problems (e.g. bugs, errors) resolved? Do you
publish a list of known problems?

123

What is the SLA (Service Level Agreement) for software problems acknowledged
by your company? What time frame for resolution for each level of severity can we
expect?

124

Does your company assume responsibility for any aspects of software
implementation?

125

Does your company provide custom programming support? If so, at what rate?

126

What is maximum delivery time for custom programming jobs? Will you guarantee
this in the contract?

127
128

Is support available on weekends and all holidays?
What customer/client roles (i.e., managers, supervisors, users, IS/IT, etc.) are
typically involved in the Core and/or User Group for this product?

John Traeger

129

What are the processes and estimated time commitments (i.e., hours per month or
per week) involved for the customer representatives on the Core and/or User September 2013
Last modified:16
Group?


50

Security/HIPAA
Security/HIPAA
130
131

Is the System capable of archiving all data?
Has the system satisfied the criteria for §170.314(d)(2) Auditable events and
tamper-resistance? If not, how does the system track and report on all user activity
including date and time data was accessed?

132

Has the system satisfied the criteria for §170.314(d)(1) Authentication, access,
control, and authorization? If not, how does the system handle authentication,
access, control, and authorization?

133
134
135

What security measures are available for remote access?
Are passwords encrypted?
Has the system satisfied the criteria for §170.314(d)(7) End-user device encryption?
If not, what method of encryption is used for patient data?

136

Has the system satisfied the criteria for §170.314(d)(5) Automatic log-off? If not,
describe the systems auto log off capability.

137

Can the system provide flexible user defined roll based security? Is there any limit to
the number of rolls defined?

138

Has the system satisfied the criteria for §170.314(d)(3) Audit report(s). If not,
describe you tools/reports for HIPAA compliance/privacy requirements?

139

Has the system satisfied the criteria for §170.314(d)(8) Integrity? If not, how does
the system ensure the integrity of the message when required?

140

Does your system allow easy account management? Please describe.

John Traeger


51

Business
Business
141

How many installations of comparable size and complexity (100+ physical interface
connections) do you have?

142

What is the number of successful installs on the current version?

143

What is the name and current release (e.g. version number) of the application
software?

144

Are all software upgrade costs (e.g. custom programming, installation, training)
included in support fees? If not, explain.

145

How are software installation fees billed (e.g. customer milestones, fixed amount,
line item, hourly)?

146

How are monthly support fees billed (e.g. fixed amount, variable, hourly)?

147

Is travel time to our site billable? If so, at that rate? Briefly explain policy.

148
149

What is your software warranty period and what is covered?
Will your company allow your code to be put in escrow for our use if your company
goes out of business?

150
151
152
153
154
155

What percent of your business is Healthcare?
Have you interfaced with [our] HIS system?
Have you interfaced with [our] LIS system?
What sort of training resources do you have?
How many years have you been in business?
To what level will your company guarantee that if the software is kept up to date with
maintenance and service patches it will comply with all current and future federal
and state mandates?

156

Is the system licensed by Enterprise license, max number of ports, specific ports
used, processes or concurrent users? Is there a limit? What is the next level and
associated cost?

John Traeger


52

About OTB
− OTB Solutions Group is a Professional Services Firm founded in 2002, which serves customers in multiple industries in the
Pacific Northwest, across North America and internationally. Our primary business location is in Seattle, Washington. Our
senior team members are all formerly from global consulting firms. Our core team consists of approximately 25 professionals
and is augmented by a worknet of approximately 70 additional resources.
− OTB’s client base spans a diverse range of organizations: from an international non-profit focused on global health issues, to
several major multi-state healthcare networks, to manufacturing companies, enterprise software, and financial services
companies. Our team members have completed significant public sector projects at both the local and state level and have
been recognized as leaders in developing public/private partnerships for enabling information technology projects.
− We pride ourselves in thinking “Outside the Box” to bring our clients innovative business solutions for collaboration and process
automation, enterprise content management, business intelligence and portals. Over the last 6 years, OTB has completed
multiple projects valued over $1,000,000 and many more in the $500,000 to $1,000,000 range. We have never had a “failed
project” and our client retention rate is essentially 100%. The majority of our work involves emerging technologies and state of
the art business systems. OTB Solutions Group is a Microsoft Silver Partner in Portals and Collaboration and Search; and a
SharePoint Deployment Planning Services (SDPS) program member.
− OTB Solutions Group brings an exceptional level of experience as well as world class credentials in project and program
management. Our consultants have successfully managed public sector projects, which were nationally recognized as models
of private/public sector partnership. We understand how to work with public sector agencies and how to create value in this
environment.
− We routinely ask our clients why they have selected us and why they are happy with the results. Several common themes
emerge from these discussions. Generally, the reasons are:
•
•

We take the time to listen, and to understand the problem
We are able to put together the right team with the right skills

•
•

We know how to “tread lightly”, treating everyone with respect
The people introduced at the beginning of the project will be working with you until you achieve your objectives and the project is
complete

•

We have a verifiable track record of bringing projects in on time and on budget

− We know that our long term success depends on the passion we bring to our assignments, the results we achieve for our
clients, and the impact we have on the organizations with which we work. We believe that you will enjoy working with our
people and we are excited about the opportunity to bring our team’s skills and capabilities to assist the KCHA.
John Traeger

© 2013 OTB Solutions. All rights reserved. | OTB is a Microsoft Gold Certified Partner.

53

Healthcare Interface Engine Selection

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Healthcare Interface Engine Selection

Similar to Healthcare Interface Engine Selection (20)

Recently uploaded

Recently uploaded (20)

Healthcare Interface Engine Selection