Lessons Learned from ELN & LIMS Implementations

Lessons Learned From
ELN/LIMS Implementations
04.10.2018
─

Lessons Learned from ELN/LIMS Implementations 1


Introduction 4
Lesson 1 - It’s A Three-Legged Stool 4
People 4
Process 5
Technology 6
Cloud-based Solutions 6
Machine Learning 6
Collaboration Tools 6
IOT in the Lab 7
Lesson 2 - Requirements Are Essential 7
Lesson 3 - Build, Buy or Both 7
Build 7
Buy 8
Both 8
Lesson 4 - Let The Data Drive The Decision 8
Relying On A Friends Opinion 9
Memories of a Previous System 9
Focusing On Part of Your Process 9
Judging A Book By It’s Cover 9
Lesson 5 - Let The Project Drive The Schedule 10
Assessment 10
Requirements Gathering 10
Vendor Assessment 10
Downselection & Demos 10
Implementation 11
Lesson 6 - Prepare Your Vendors 12
Lesson 7 - Know Your Vendors 12
Lesson 8 - Prepare Yourself 13
Roles and Responsibilities 13
Vendor Point Of Contact 13
Solution Evaluator 13
Lesson 9 - Know Thyself 14
Lesson 10 - Cloud or On-Prem 16
Lesson 11 - Enlist the Aid of A Consultancy 16


Frequently Asked Questions 16
What’s the difference between an eLN and a LIMS? 16
Aren’t These Systems All Small-Molecule Oriented? 17
What’s an API and How Does It Affect My Selection Process? 17
What's the difference between IT and informatics? 18
Contact Us 18


Introduction
The process of selecting and implementing an electronic lab notebook (eLN) and laboratory
information management system (LIMS) can be a challenging, time-consuming endeavor. It
is an expensive decision that requires careful analysis of lab operations and processes,
requirements gathering, and understanding of diverse technologies like cloud-based
applications, instrument data parsing & loading, data analysis, machine learning,
small-molecule and biologics registration, batch tracking, request management, and
structure- and sequence-based searching just to name a few.
This white paper draws on lessons learned from multiple implementations in diverse life
science labs.

Lesson 1 - It’s A Three-Legged Stool
It's a commonly held misconception that ELN/LIMS implementations are a wholly
technological solution to a scientific problem. The reality is that every successful
informatics implementation is a three-legged stool consisting of people, process and
technology. Each leg plays an essential role in a successful outcome of the project, and
eliminating any one leg can have disastrous consequences.
People
Regardless of the size of the organisation, most life science companies have some form of
governance structure in place to insure that strategic decision makers have a view of what’s
happening within the company, identify challenges and resources, and set direction.

When selecting an eLN/LIMS system, the governance structures within the company help
set the priorities, drive the agenda for the effort, and that ensure that key decision makers
and internal subject matter experts help advance the selection process. Buy-in at all levels
of the company is essential, and alignment with business goals and priorities ensure that
the teams use the same filters when reviewing vendors and their solutions.


At a minimum, you’ll want the following types of people involved:

● Senior Leadership - these leaders will form the steering committee. Their
responsibility is to identify and recruit key opinion leaders in the organization, set
the tone, vision and agenda for the efforts, identify strategic priorities, and set
budgets.
● Key Opinion Leaders/Subject Matter Experts - Their role is to help explain your
scientific processes, the tools you use, the data you collect, and the analyses you
perform. This information will drive the requirements gathering process. They’ll also
help evaluate each of the solutions. Look for people who are typically early adopters
of new technologies, who have hands-on experience with lab processes and whose
opinions hold sway within the organization.
● Research Informatics/IT - Their role will be to evaluate the solution on its technical
merits, determine how the solution fits with the existing infrastructure, and to
assess the associated costs for each solution.
Process
At a certain level, all drug discovery, diagnostic and medical device companies follow the
same industry-standard, stage-gated approaches approved by their industries. But the
devil is in the details, and key to a successful systems implementation is to have a common
understanding of the process that your company follows to get a product out the door. It’s
this process that you’re looking to support with your new system. You’ll want to understand
the answers to the following questions:

● What are the steps in the process?
● Which steps in the process generate data?
● What instruments and software are involved?
● What types of data files are generated? (Excel, PDF, XML, JSON, proprietary file
types?)
● How big are the data files?
● Are the instrument controllers networked together? Do they automatically deposit
files on a shared drive? How frequently is the drive backed up?
● Who is responsible for that data? For generating it, for verifying it, for recording it?
● When something goes awry, how do you track down the source of the problem? By
and large, variations in experimental data tend to come from two sources: operator


variability and material variability. Are you collecting the information necessary to
track down the source of experimental variation?

One of the consequences of having a rapidly-moving, science-driven company, is that the
velocity of change means that scientific groups are often operating in relative isolation,
unaware of the processes upstream and downstream of them and the impacts on
eachother.
This can result in handoffs between organizations that require additional work. For
example, you might be outsourcing part of your process to a CRO, and the data you get
back must be “munged” in order to get it in the right format for the software application
that you’re using to analyze the results. That data munging process can be time-consuming
and error prone, and wherever possible, we want to try and eliminate these types of
problems by bringing them to the surface and creating organizational awareness, but also
looking to vendors for possible solutions to those problems.
Technology
The technologies that drive your business are constantly changing. And having a broad
view of the players in the landscape can make all the difference in your selection process.
Here are some of the more recent innovations that we’re seeing:
Cloud-based Solutions
Although vendors have always had solutions that could be deployed in your own data
center, over the past 7 years we’ve seen the movement towards the cloud. This makes it
easier and less expensive for startup companies to manage their data and operations
without large capital expenditures in infrastructure. It also means that
geographically-dispersed organizations can work more effectively together. And with an
increasing reliance on partnerships, cloud-based solutions can make it possible for
partners to share data in a secure fashion.


Machine Learning
Machine Learning and Artificial Intelligence have been hot buzzwords over the past few
years, but until recently they’ve been a solution in search of a problem. Lately, that’s
changed, with the emergence of Machine Learning applied to screening data. It allows your
scientists to discover new trends from very diverse data sets. You can examine screening
data for millions of molecules and discover unseen correlations between structural or
sequence, and changes in activity in QSAR (Qualitative Structure Activity Relationship) data
and BSAR (Biological Sequence Activity Relationship) data.
Collaboration Tools
Science is by nature a collaborative endeavour, and vendors are building collaborative
capabilities into their systems. These capabilities come in different forms. Collaborative
image annotations allow researchers to ask for help from colleagues, and direct their
attention to specific regions of interest. Think of it as Google Docs for images.
Event feeds and dashboards help teams keep up-to-date with projects, and communicate
more effectively.
IOT in the Lab
Industry standards organizations like the SiLA (Standards in Lab Automation) have been
working in on standards to allow instruments and robots to communicate with LIMS
software using standard Internet-of-Things (IoT) protocols.
Lesson 2 - Requirements Are Essential
While today’s ELN/LIMS solutions are highly configurable, there is no one-size fits all
solution. Requirements analysis will help insure that your selection process identifies
technologies that can best support your organization. They help you make data-driven
decisions, and minimize bias in the selection process.
By reviewing your process with your teams, requirements come to the surface that might
easily have been overlooked. The data generated by each step is documented, making it
easier to identify software capable of parsing and loading the data.
Priorities for each requirement are identified. This makes it possible to generate weighted
scores when you assess the vendors.


Lesson 3 - Build, Buy or Both
There are a variety of circumstances that can drive the decision to build or buy a system.
Here are a few questions that you can use to help you better understand where you might
fit on the spectrum.
Build
● Do we have the internal resources, processes, experience and technology to build
the application ourselves?
● Can we outsource this effort? Can we do this with the proper oversight necessary to
see this to its conclusion?
● What is the opportunity cost involved in dedicating internal resources to this effort?
If I’ve tied up my informatics department in these software engineering tasks, will it
prevent them from addressing these higher value needs that my scientific groups
have?
● What are the ongoing costs going to be to maintain this system?
● Could we open source the parts of the system that aren’t proprietary? Would that
alleviate some of the maintenance burden on the staff? Is there a community of
practice that this would benefit and that would be able to share some of the costs?
Organizations like the Pistoia Alliance which focus on implementing pre-competitive
technologies and standards might be a way forward.

Buy
● Does the solution address most of our business processes? If not, can multiple
solutions be integrated into a seamless whole?
○ What is the cost of integration?
○ What will the maintenance cost be for each of these integrations? Each time a
vendor updates an application, the integrations between that application and
other applications will have to be re-tested (or revalidated in the case of
validated systems). This the fewer the vendors you select, the lower the
costs.
○ Does the vendor provide validation services?
● What is the true cost of ownership?
○ Yearly Licensing fees. Is it licensed per module? Is it licensed per module/per
seat? Will you end up paying for extra seats for a given module that your
organization never uses?
○ Implementation/configuration costs


○ Integration costs
○ Support costs (both internal and external)
○ On-Prem vs Cloud-based Hosting costs. Does the vendor provide a discount
for cloud-based hosting?
Both
In addition, to each of the previous options, you may find it necessary to buy multiple
solutions, and integrate them together. You may also have internally developed
applications that require integration.
● Do you have experienced programming staff to support this?
● Do the systems you selected have publicly available APIs (Application Programming
Interfaces) that you can write to? (see Frequently Asked Questions for more details).
● How will this affect our upgrade costs if each integration has to be re-tested
whenever the vendor rolls out an upgrade?
Lesson 4 - Let The Data Drive The Decision
A common trap that companies fall into is failing to let the data drive the decision. In other
words, if the decision is not based on your company's requirements, priorities and an
analysis of vendor applications, the result is often islands of disconnected data and
systems, double-entry into different systems, and time spent munging data from one
format into another so that another system can ingest it.
Here are some examples of common scenarios where the data didn’t drive the decision.
Relying On A Friends Opinion
Unless your friend is doing exactly the same things you’re doing, has gone through the
selection process recently and has staff with similar experience and requirements, relying
on their opinion of a system can be a flawed approach to selecting your system. Your
selection should be based on your company’s priorities, budget, and the assessment of
your scientific staff.
Memories of a Previous System
It’s sometimes tempting to say, “let’s use System X. I used it my last company, and it was
pretty easy to use”. There’s no learning curve, and your team can hit the ground running.
But doing so, will cost you the opportunity of surveying the current software market and
evaluating software that may potentially be a better fit, may be cheaper, better-performing
and easier to use.


Focusing On Part of Your Process
It’s often said that the “squeaky wheel gets the grease”, but when making technology
selections, this can have unforeseen consequences. This happens most frequently, when a
critical part of the process is time consuming to do, or a leader is particularly vociferous
about their needs.
But focusing in one area, can create difficulties as you begin to implement additional
solutions further down your pipeline. For example, suppose that the most time consuming
part of your process is designing plasmid maps. You find a solution that your scientists love
and you implement it. The result is a new data island that’s unconnected to any other
system in your lab. You now have to create a way to get that data into other systems, or
have your scientists do double entry.
Judging A Book By It’s Cover
It’s easy to be seduced by a good user interface. You are, after all, going to be spending a
lot of time using the application. You want it to be easy to use. But this can lead you to
select a system that while aesthetically pleasing, doesn’t cover enough of your business
process. The result will be that the company must purchase multiple solutions to cover the
business process, and that each of those solutions must then be integrated together in
order to share data. This will result in recurrent revalidation costs as each integration must
be re-tested each time an application is updated.

Lesson 5 - Let The Project Drive The Schedule
Projects typically consist of two phases: Assessment and Implementation.
Assessment
The goal of the assessment phase is to understand your requirements, and how the
available vendor-based solutions could help you address those requirements.
Requirements Gathering
During the requirements gathering stage, interviews are conducted with process owners.
● What is the critical path that your organization follows to get a product out the
door?
● Who is responsible for each stage of the process?
● What data is generated and what technology is used to generate it?


● Where are the gaps in the process?
● What are the most time-consuming activities in the process?
● What kinds of activities span multiple processes? For example, sample management
& inventory, batch registration, and protocol management.
Team members are interviewed about each stage in the process. Requirements elicited
from these interviews are compiled and circulated to the working group for review and to
senior leadership for approval.
The timeline for executing the requirements gathering stage varies depending on the size
of the organization, the number of sites involved and the complexity of the process, and
can take anywhere from 1-3 weeks.
Vendor Assessment
During the vendor assessment phase, requirements are compared to vendor capabilities to
create a rank ordered list of vendors from which you can then downselect to the primary
vendors that you want to interview. This can take 2-3 weeks depending on the number of
vendors initially selected.
Downselection & Demos
After the downselection has occurred, requirements are sent to the vendors, and scoping
discussions are held with each vendor to communicate your needs, and to setup demo
schedules. At this point you should be looking at no more than 3 vendors.
The working groups, composed of members of your scientific teams, will review each of the
solution demos. It’s best to record the demos so that you can review the solution, or so that
members who weren’t available during the demo period can provide their input.
A master list of follow-up questions is prepared based on the input from each of the teams
and circulated back to the vendors. This may lead to a second round of demos to help
address specific questions.
The assessments of each of the team members is compiled to create an overall solution
assessment and each solution is then given a weighted score. These are assembled into
vendor score cards which can be reviewed by the senior leadership to make the final
selection.
It may be possible to have a proof-of-concept system stood up, that your scientific team
can then evaluate over the course of a week.
Implementation
The implementation phase is a partnership between the vendor and customer. During this
phase the following types of activities will take place:


Vendor
● Stand up the production and test environments.
● Configure the system to support your processes.
● Configure the system to load and parse the high-priority assay data files.
● Configure reports, dashboards, and data analysis tools for the assays you’ll be
running.
● Train staff who will be implementing the first round of assays.
● Test or validate the system. Depending on your regulatory needs, the system will
either be tested with your staff present, or an in-depth validation performed. The
testing may involve performance testing in addition to verifying that the system
meets the functional requirements.
Customer
● Identify and gather the data files that must be used for the initial implementation of
the high priority assays.
● Training
● Provide acceptance test/validation feedback
● Provide configuration feedback.

Lesson 6 - Prepare Your Vendors
All of the solutions on the market today have a high degree of flexibility and configurability,
which means that vendors often need to configure the solution in preparation for the
demo. A demo of “out of the box” functionality can often be disappointing or may skew
your evaluation towards the vendor with the prettiest interface.
Preparing your vendor, allows them to focus the demo to suit the needs of audience and
the organization as a whole. This means that they don’t waste time demonstrating
chemistry features to an audience of biologists or antibody tools to protein engineers.
Communicate your requirements to the vendor. Make sure they understand the audience
and their interests. If you have a fairly diverse audience, then you may get better feedback
from your staff by breaking the demos into smaller more focused sessions. This may make
better use of your staff’s time, and will allow each team to ask more in-depth questions.


Lesson 7 - Know Your Vendors
Knowing your vendors technical capabilities is one thing, knowing their size, experience,
investors and finances are another.
● How stable is the vendor?
● How many employees do they have?
● Are they a growing concern?
● Are they publicly held, or bootstrapped by their own sales?
● Have they had VC investment? If so, when is an exit anticipated. An exit such as a
buyout can result in changes to the company organisation and stability and affect
the pace of releases. If it's a merger with another company, it can mean changes in
pricing, or additional products added to your system with differing user interfaces?
● Who are their customers? How many of your competitors use them?
● Do you have people in your network with personal experience using their software?
● How responsive are they when setting up demos or answering service calls?
Lesson 8 - Prepare Yourself
In practical terms, preparing yourself for the project involves the following types of
activities:
● Communicating expectations.
○ What is the scope of the effort? Is the goal to implement informatics
applications that cover most of the operations, or a select group?
○ Who will be responsible for what? Everyone on the team must understand
what’s expected of them and what impacts the process may have to their
daily activities.
○ What's the time frame for the project? Set aside the appropriate amount
of time, so that people can participate in the effort and provide the feedback
necessary to properly evaluate the solutions.

● Communicate strategic priorities. Are there parts of the organization that need to
be supported first? Are there specific business drivers or milestones that need to be
met within a given time frame? How do those drivers affect the implementation
schedule?


Roles and Responsibilities
Vendor Point Of Contact
Ideally there should be a single point of contact for vendors. This person will be responsible
for setting up vendor demos, and will be the sole person responsible for communicating
back to vendors. This person will ensure that vendors get a single, consistent message from
your company.
Solution Evaluator
Your solution evaluation team will consist of a variety of members of your technical and
scientific staff. Each person should have responsibility for one or more steps within the
overall process. During the solution evaluation process, each person is responsible for the
following:
● Assessing how the solution will support their part of the process.
● Identifying gaps where the solution does not cover a process
● Assessing ease-of-use
It’s important to stress that the evaluator should not provide the vendor with information
about the suitability of the product, as this may compromise future negotiations with the
vendor. All communication with the vendor must go through the Point of Contact.

Lesson 9 - Know Thyself
In ancient Greece the Oracle of Delphi was a mystical seer whose every word could drive
the decisions of kings. One of the two inscriptions on her temple was “Know Thyself”. That
sentiment is no less true now, than it was 2500 years ago, and knowing the strengths and
weaknesses of your organization, is crucial when making application selection decisions.
The key questions that every organization must ask itself when evaluating these solutions
are:
● What kind of an organization do I have?
○ Do I have dedicated informatics personnel to administer the application? Can
I bring in a contractor for this, or am I going to rely on my lab manager to do
this?
○ Who is going to be responsible for configuring each assay that goes into the
system?
○ Who is going to be responsible for access control?


○ Does my organization have its own data center, or should I use this in the
cloud?
○ Do we have the kind of bandwidth necessary to upload large amounts of
imaging data into the cloud?
○ Do I have a multi-site organization in different time zones, which might
require the solution to be deployed in multiple AWS zones?
○ Do I have the informatics organization necessary to build on-top of this
solution? Let’s face it, no solution will address all of our needs, so how do we
fill the gaps? Do we buy additional applications? Do we build custom
software?
○ Do the majority of the employees in my company come from academic labs,
or industry? Often academic labs don’t have the budget for informatics
applications, and therefore employees from academia may lack experience
in making technology selections.
○ Do we need to have purchasing and inventory connected to insure that
reagents can be easily re-ordered?
○ Is our staff used to being mobile? Do they want to roam the lab with a tablet
and enter information as they go, or do they need to do data entry from a
desktop computer?

● What are the critical path processes in my organization which help me get
product out the door? Whether it’s drug discovery, diagnostics, service delivery or
device development (or some combination thereof), knowing the steps involved, the
people who perform those steps, the data they produce, and the current gaps and
time consuming steps in the process is essential to a successful implementation.
Ultimately, the selection process must make clear how the eLN/LIMS will support
the scientific processes within the organization.

● What is your vision for your business? What kinds of capabilities do you want to
have? Do you want to be a cloud-enabled business? Could augmented reality make
your scientific operations more “hands free” and more efficient? How can these
types of technologies position your company as an innovator that attracts the
attention of investors, partners and new employees?
Lesson 10 - Cloud or On-Prem
These days most systems are available either in the cloud, or “on-prem” (meaning in your
company’s data center). Often we see startups with little money to spend on kitting out a
data center, and staffing a large IT group to manage it. They’re looking for a solution that


doesn’t tie them to particular hardware, or a fixed capacity. They want a system that grows
as they do. Or the company has a number of sites, and it doesn’t make sense to put all of
the computing infrastructure in one location. They want a system that can easily be
accessed from multiple sites, in different time zones. A system that is automatically backed
up, and is periodically upgraded.
For other companies, an on-prem solution is a better fit. They already have the computing
infrastructure and staff in place.

Lesson 11 - Enlist the Aid of A Consultancy
As we’ve seen from previous lessons learned, there are a number of pitfalls that can occur
during an eLN/LIMS selection, and an experienced consultant can help you navigate
through them. Look for consultants with the following qualifications:
● Extensive experience in implementing research informatics systems.
● Experience in your industry, therapeutic area and therapeutic classes
● Experience managing the technology selection process from requirements to
selection
● Experience managing the implementation process
● Experience with a wide variety of lab technologies
Bringing in a consultant should provide you with the data you need to make an unbiased
decision about the solution that best fits the needs of your organization.
Frequently Asked Questions
What’s the difference between an eLN and a LIMS?
At it’s core, a LIMS is designed to parse and load experiment data into a database in order
to make those data queryable and analysable. It tracks sample, and reagent information
for you in addition to experimental conditions, and experiment data. With it, you can
answer questions like:
● Is there unexpected variation in my controls over time?
● How do the results obtained by the assay development team, compare with the
results obtained by the automation team for the same assay?
● How do the results of this assay vary over multiple animal cohorts, and multiple
time points?
● What effect does that new kit or reagent have on my assay results?


You can query across experiments to find new trends, and to look for patterns in your data.
The eLN is your legal system of record for scientific data. It’s where verified data is stored,
signed and locked. Each page in the eLN contains the data for an experiment or set of
related experiments for a project.
Aren’t These Systems All Small-Molecule Oriented?
Many solutions started life as tools for managing small-molecule screening data and
registration and grew over the resulting years to encompass more of the workflow.
In recent years, vendors have begun adding modules to their system to support the
biologics discovery process. These modules provide customers with support for Biological
Structure/Sequence Activity Relationship (BSAR) analysis, biologics registration for both
sequence and non-sequence based registration, support for molecular biology and for
workflows for antibody, peptide, and cell-based therapeutics. Most solutions allow the
customer to configure the system to support the registration of new therapeutic classes
like CAR-T therapeutics, microbiome therapeutics, oncolytic viruses, antisense molecules
and more.
Vendors continue to expand the capabilities of their systems and continue to work closely
with customers to provide them with tools that fit their unique needs.
What’s an API and How Does It Affect My Selection Process?
An Application Programming Interface (or API) is a means that applications use for
communicating with one another. For example, suppose you have a request management
system, and a protocol management system. Whenever Jane, your biology assay specialist,
receives a request for a particular assay to be performed, the system should retrieve the
protocol for the assay, and display it on her tablet in the lab. To do this, a programmer
would use the API, to retrieve the protocol from the protocol management system, and
display the results.
As vendors have come to the realization that integrating various systems together is vital to
customers, they have added APIs to their systems. Make sure that your consultant reviews
the API to insure that the types of integrations you want will be supported.
What's the difference between IT and informatics?
IT is typically responsible for hardware, office applications, networking, and email - the
basic infrastructure of a company. The scope of their experience and responsibilities
typically don't extend to managing and developing scientific applications.
Research Informatics is usually staffed by people whose primary responsibilities are
scientific systems. Their responsibilities may also include data analysis. The staff typically


have advanced training in bioinformatics and cheminformatics. They understand the drug
discovery problem domain and the scientific challenges it presents.
Contact Us
Need help getting started? Got more questions? Contact us.
Aspen Biosciences
Info@aspenbiosciences.com

Lessons Learned from ELN & LIMS Implementations

More Related Content

What's hot

Similar to Lessons Learned from ELN & LIMS Implementations

Recently uploaded

Lessons Learned from ELN & LIMS Implementations