This document summarizes a research fellowship project analyzing medical guidelines websites. It introduces the purpose of medical guidelines and common issues with online guidelines like lack of accessibility and usability. The project developed a quality index to evaluate and compare features of different medical guidelines websites. Data was collected from various websites and analyzed to identify trends and opportunities to make guidelines more patient-centered and effective. The goal is to modify guidelines websites to be more accessible, understandable and user-friendly to better educate patients.

1. TRAVELER’S
SUMMER
RESEARCH
FELLOWSHIP
PROGRAM
2014
WEILL
CORNELL
MEDICAL
COLLEGE,
NEW
YORK,
NY
Identifying
Issues
in
Medical
Guidelines
to
Improve
Decision-­‐Making
Sumaiya
N.
Sarawat
Correspondence: Sumaiya N. Sarawat, City University of New York – Brooklyn College, 2900
Bedford Ave., Brooklyn, NY 11210, Email: ss1170@bcmail.brooklyn.cuny.edu. Tel. 347-458-
8385.
Conflicts of Interest: None
Financial Disclosure: None
Word count: 3,349

2. 1
Figures:
Tables:
8
8
7
7
7
7
6
5
4
4
4
4
3
3
3
2
1
0
1
2
3
4
5
6
7
8
9
Total
Score
For
Each
Index
Variables
of
Quality
Table
2:
Common
A?ributes
Among
Medical
Guidelines
44.44%
33.33%
22.22%
Table
3:
CharacterisDcs
of
Sites
Contains
guideline
ar;cles
Contains
symptoms
feature

3. 2
Tables of Contents:
Abstract…………………………………………………………………………………………………….2
Introduction……………………………………………………………..…………………………………3
Methods………………………………………………………………………………….…………………5
Variables…………………………………………………………………………………………………...6
Statistical Analysis………………………………………………………………………………………...7
Results……………………………………………………………………………………………………...7
Analysis…………………………………………………………………………………………………...10
Discussion……………................................................................................................................................10
Conclusion………………………………………………………………………………………………...13
References………………………………………………….......................................................................14
Many
physicians
and
health
care
providers
rely
on
scientific
scholarly
literature
when
it
comes
to
treating
patients
in
addition
to
their
experience
and
knowledge.
Medical
guidelines,
also
known
as
clinical
protocol
guidelines,
clinical
protocols,
clinical
practice
guidelines,
or
clinical
guidelines,
are
documents
whose
purpose
is
to
educate
physicians
and
patients
on
making
the
best
informed
decisions
and
requirements
regarding
diagnoses,
treatments,
recommendations,
and
managements
for
multiple
aspects
of
healthcare.
There
are
various
problems
with
the
online
guidelines
that
deter
patients
from
utilizing
them.
These
problems
include:
the
lack
of
accessibility,
ease
of
site
navigation,
excessive
information,
too
much
medical
jargon,
and
the
lack
of
website
attraction.
Guideline
development
is
improved
when
diverse
stakeholders,
including
patients,
specialists,
and
experts,
share
power
in
the
process,
from
defining
the
scope
of
the
question
to
the
final
stages
of
review.
The
Institute
of
Medicine
(IOM)
and
Guidelines
International
Network
(G-­‐I-­‐N)
have
each
released
best-­‐practice
standards
for
clinical
guidelines
that
place
health
care
consumers
at
the
forefront
of
guideline
development.
The
AAO-­‐
HNSF
has
placed
a
growing
emphasis
on
consumer
engagement
in
its
latest
guideline
development
manual
and
has
developed
clear,
multifaceted
infrastructural
approaches
to
identify
and
utilize
consumers
in
guideline
development
and
guideline
review.
These
article
reviews
those
recommendations
and
emphasizes
that
patient-­‐centered
guideline
development
can
improve
the
quality
and
impact
of
medical
guidelines.
The
hypothesis
for
this
project
states
by
modifying
online
guidelines
to
become
patient-­‐centered,
the
hope
is
to
gravitate
a
larger
patient
populace
to
begin
using
these
websites
to
become
more
willing
to
be
well
informed
about
their
health.
Using
data
collected
from
the
multiple
websites
on
the
internet,
a
Microsoft
Excel
spreadsheet
was
created
to
analyze
the
trends
and
compare
the
different
features
of
medical
guidelines
on
the
web.
The
data
sources
that
were
used
were
numerous
medical
guideline
websites
suggested,
known
prior
due
to
experience,
and
also
found
through
internet
research.
An
index
of
quality
was
developed
to
measure
the
effective
of
each
website
used.
At
the
end
of
each
website
row,
the
numbers
were
auto
summed
into
a
total
score.
An
index
score,
which
is
the
sum
of
all
parts,
were
used
to
rank
the
websites
from
having
the
highest
score
to
the
lowest
score.
The
spreadsheet
was
then
sorted
so
the
websites
were
ordered
in
the
descending
direction
–
the
medical
guideline
with
the
highest
score
was
placed
on
top
of
the
spreadsheet.
It
is
crucial
for
online
guidelines
to
alter
their
websites
to
manifest
websites
that
are
accessible,
comprehendible,
and
user
friendly
in
order
to
perpetuate
the
best
health
care
for
patients.

4. 3
INTRODUCTION:
Medical guidelines are documents and published papers whose purpose is to educate patients and
doctors into making the best-informed decisions and requirements regarding diagnoses,
treatments, and management for multiple aspects of health care.1
Medical guidelines are also
called clinical protocol guidelines, clinical protocols, or clinical guidelines. There are many well-
known medical guideline websites. For instance, there is the National Guideline Clearinghouse
and the American College of Physicians Clinical Practice Guidelines. Alongside guidelines there
are also websites with a function to relay treatments, diagnoses, risk factors, etc. once symptoms
are entered into the site.2
These well-known sites include TheNNT, WebMD, and Mayo Clinic.
All these sites provide recommendations on what a person should be doing with the symptoms
each individual provides. Certain sites have a function specifically geared to provide
recommendations such as appropriate tests, vaccinations, check-ups suggested to patients of a
certain gender or age group. The most common websites to view these recommendations are
HealthFinder and Agency for Healthcare Research and Quality (AHRQ).
Although these guidelines hold purpose their main objective gets lost through the conundrum the
online guideline websites bring. The predicament that surfaces regarding guidelines revolves
around accessibility, navigation, credibility, excessive information and lack of attraction. These
hindrances lead to the lack of knowledge encompassing health. This manifests uneducated
patients, which further impels poor health conditions among this populace. The knowledge
1
Clinical Practice Guidelines We Can Trust - Institute of Medicine. (n.d.). Clinical Practice
Guidelines We Can Trust - Institute of Medicine. Retrieved August 4, 2014, from
http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx
2
Medical Guideline. (n.d.). healthinformatics -. Retrieved August 4, 2014, from
http://healthinformatics.wikispaces.com/Medical+Guideline

5. 4
gained by these guidelines has the ability to prevent certain illnesses from occurring and cures
others rapidly. However, if one does not have the ability to comprehend the material given, how
can they use these guidelines to benefit them? The incompetence of guidelines ignites a breach in
ethical practices. The autonomy of a patient can be compromised if they are not aware and fully
educated about their health and by not providing proper guidelines to patients we ensue non-
malfeasance. Most guidelines are set up for doctors and individuals with a medical background
to gain access to. Therefore, these guidelines are orchestrated in a manner that are on a level that
only individuals with a certain mental capacity would be able to understand. The layouts are
structured as medical school textbooks; they are monotonous, not vibrant and verbose. In terms
of excessive information, options, of which procedure or protocol to follow lay infinite, which
only further confuses the patient. It is essential for not only doctors and health professionals to be
aware of medical procedures but the patients themselves.3
Bestowing patients with knowledge of
their health and preventative medicines or cures as a whole may decrease a number of ailments
from occurring. This would also decrease the number of visits to the hospital or trips to the ER,
which in turn is cost effective. Creating patient centered guidelines can serve as a tool for
patients and allow them to ask for certain checkups that may be overlooked by the physician.
Essentially, a health guideline is an imperative instrument for all individuals to utilize because it
educates one on how to live and lead a healthy lifestyle. Not all individuals have access to this
information but by presenting information in a clear coherent manner the general public can be
well informed about potential health risk, benefits, and procedures that could augment their
3
Standards for Developing Trustworthy Clinical Practice Guidelines - Institute of Medicine.
(2011, March 23). Standards for Developing Trustworthy Clinical Practice Guidelines - Institute
of Medicine. Retrieved August 8, 2014, from
http://www.iom.edu/Activities/Quality/ClinicPracGuide.aspx

6. 5
physical condition. However, the only way for this to be implemented is if the online guidelines
are modified in a way that is patient-centered. The big questions that are yearning for answers
remain (1) Are there online guidelines that are user friendly and if so which ones? (2) Is there a
way to construct an alternative site that could potentially reinvent the way guidelines are
perceived? (3) Will modifying sites help increase patient understanding of guidelines and
encourage them to use these websites for personal health benefits? (4) Will this increase in
knowledge due to sufficient knowledge augment patient’s health as a whole?
METHODS
Design, Data Source, and Sample
Using data collected from the multiple websites on the internet, a Microsoft Excel spreadsheet
was created to analyze the trends and compare the different features of medical guidelines on the
web. The data sources that were used were numerous medical guideline websites suggested,
known prior due to experience, and also found through internet research. These sites include:
TheNNT, WebMD, the Mayo Clinic, Health Finder, AHRQ’s (Agency for Healthcare Research,
and Quality), Electronic Preventive Services Selector (ePSS), National Guideline Clearinghouse,
U.S. Preventive Services Task Force, PubMed, and American College of Physicians. TheNNT,
National Guideline Clearinghouse, American College of Physicians, and the U.S. Preventive
Services Task Force were suggested websites recommended by Dr. Tara F. Bishop, a research
instructor. Data was originally compiled using those four websites by analyzing the benefits and
disadvantages of each site and comparing each other through a pros/cons table on a Microsoft
Word document. After the compilations, an Excel spreadsheet was created using those four sites
in an attempt to see the comparisons against each site and to receive a more quantitative data set

7. 6
as opposed to a qualitative data set. The sites were then ranked by a point system and ordered in
a trajectory.
A sample of medical guideline websites was taken from the vast population of clinical
guidelines on the World Wide Web. The original four sites were recommended as a good place
to start the research and the other sites were gathered as the data progressed. Health Finder
offered an option for people to receive health recommendations depending on their age, gender,
and pregnancy status. AHRQ’s ePSS also has a similar feature but with two added options, one
for being a tobacco user and the other for being sexually active. Both Health Finder and AHRQ’s
ePSS were found along the side of the U.S. Preventive Services Task Force website. This is a
crucial feature for sites to contain since it allows patients to make informed decisions about their
health from recommendations based on their lifestyles. This is why these two sites were included
to be a part of the sample sites. From personal experience, WebMD and the Mayo Clinic were
popular medical sites patients know to refer to when they experience symptoms that might be
explained by certain conditions. An option to list specific symptoms an individual may be
experiencing is a good tool to have on a website so patients can know what to expect before
going to a doctor. This is why these two sites were also included to be part of the sample
websites. PubMed is a classic site students and professionals visit to search research articles. The
free public version of PubMed contains all the abstracts of articles and some full text articles.
This is helpful for people to visit and read the latest scholarly articles in various conditions,
which was why PubMed was added to the sample data websites as well.
Variables

8. 7
The list of the medical guideline websites used was organized on the farthest left hand column on
a Microsoft Excel spreadsheet (“A” column – cells A2:A10). An index of quality was developed
to measure the effective of each website used. The variables of quality were listed through the
first row (B1:R1). The seventeen variables are as follows: organization, navigation, no excessive
information, summary option, tutorials, advanced search, detail, user friendly, medical layout,
disease likelihood, symptoms viewer, review option, color codes, recommendations,
methodology, public access, and effectiveness of symptoms. The variables were defined to have
a yes/no option. In order to rank the medical guideline websites quantitatively, numbers had to
be assigned through the dichotomous scale indicating a ‘1’ to be ‘yes’ and a ‘0’ to be ‘no’. A
score of ‘1’ would indicate that that specific medical guideline has that variable within its
website while a ‘0’ would indicate that that guideline is lacking that variable in its site.
Statistical Analysis
At the end of each website row, the numbers were auto summed into a total score. An index
score, which is the sum of all parts, were used to rank the websites from having the highest score
to the lowest score. The spreadsheet was then sorted so the websites were ordered in a
descending direction – the medical guideline with the highest score was placed on top of the
spreadsheet.
RESULTS
Nine websites were found and the characteristics of the sites are as follows. Four out of the nine
sites include actual scholarly medical guidelines so approximately 44.44% of the websites
contain articles for readers to view. Three out of the nine websites (33.33%) contain a feature
where patients can list or point out their respective symptoms in order to receive suggested

9. 8
diagnoses and treatments. Although all the websites used had areas of recommendations for the
respective searches, only two out of the nine sites (22.22%) offered recommendations
specifically based on one’s age, gender, and lifestyle (Table 3).
TheNNT, WebMD, and AHRQ’s ePSS were tied for the highest, having a score of 11 so these
websites contained most of the variables. TheNNT is the only website included in the samples
that contained color-coding for different features on the site. American College of Physicians
was shown to have the lowest score of 6 which indicates that the specific website does not
contain more than half (64.71%) of the variables used to measure effectiveness of medical
websites. The Mayo Clinic and Health Finder were tied in the rankings with a total score of 10,
and the National Guideline Clearinghouse, the U.S Preventive Services Task Force and PubMed
were tied in the rankings with a score of 8 (Table 1 & 2). It is important to note that although
some websites have the same score, it does not mean that they contain ‘1’s for the same
variables. They can have different options in each respective website, for example one site can
have a specific variable while another site lacks that variable but it can make up the score by
having a different variable.
Table 1:
Table 2: Results Table

10. 9
ANALYSIS:
8
8
7
7
7
7
6
5
4
4
4
4
3
3
3
2
1
0
1
2
3
4
5
6
7
8
9
Total
Score
For
Each
Index
Variables
of
Quality
Table
2:
Common
A?ributes
Among
Medical
Guidelines
44.44%
33.33%
22.22%
Table
3:
CharacterisDcs
of
Sites
Contains
guideline
ar;cles
Contains
symptoms
feature
Contains
recommenda;ons
based
on
lifestyle

11. 10
When analyzing each website and categorizing which website deserved a higher ranking than the
other, key elements were looked upon to arrive to a conclusion. The ones ranked highest were
the most coherent, accessible and easy to navigate. The lowest ranked were those who
demonstrated the opposite characteristics. Finding websites that were average amongst the list
concluded the mean. The information was qualitative. The ranking order followed as so: which
guideline websites stand out in the categories of: organization, navigation, non-excessive,
summary option, tutorials, advanced search, detailed, user friendly, medical layout, disease
likelihood, symptoms viewer, review option, color codes, recommendations, methodology,
public access, effectiveness of treatment. The highest ranked was TheNNT, WebMD, and
AHRQ’s ePSS and the lowest ranked was American College of Physicians.
DISCUSSION:
Starting the opening of the millennium, the World Wide Web became accessible to the general
public. As the Internet was brought into the homes of many individuals, the world became much
easier to navigate and simple questions were answered quickly as the Internet advanced
progressively to include more search engines. Now people the have the ability to look up
anything from the weather to treatments to their various illnesses. In this day and age, many
people do not rush straight to the doctor’s office.4
Instead, many patients can easily type their
symptoms into numerous medical websites and receive close to accurate treatment and diagnosis
results within seconds without having to leave their homes.
4
Standards for Systematic Reviews of Comparative Effectiveness Research. (2011, March 23).
Institute of Medicine of the national academics . Retrieved August 5, 2014, from
http://www.iom.edu/Activities/Quality/SystemReviewCER.aspx

12. 11
An issue arises when the question of whether or not everyone really has access to these
resources. With almost six million New Yorkers dependent on Medicaid, there is a good portion
of the population who barely has any money for food and shelter, much less internet access. A
good portion of the population, such as many immigrants, may be illiterate and unable to
understand the language the guidelines are written in. This can cause a barrier in gaining crucial
information through the web. There is a good chance that these percentages of the population are
unaware of the free online resources available to help them figure out what may be wrong with
their health and whether or not if it is severe enough to make a trip to the doctor’s office.
However when individuals do discover free online medical resources, patients may become
skeptical of how credible the Internet is versus face-to-face interaction between the patient and
the doctor.
The ability to navigate through these websites is another impetus for the lack of affectivity of
online medical guidelines for patients. The patient is already perplexed and desires a simple
solution to their burning question. They don’t want to get onto a website and see clutter. The way
many of these online medical journals are set up mirrors a medical textbook or journal. The
colors and layout are mundane and the language wordy, their purpose is to educate those with a
medical background. However, many patients don’t necessarily have an understanding of
medicine and that may be the sole reason they are searching for a guideline in the first place. All
the medical jargon again daunts patients into not being informed about their health.
Another problem we face is the issue of excessive information. The amount of data the Internet
holds is quite vast. This notion is akin to online medical guidelines. Almost all medical
guidelines provide the patient with an abundance of alternatives, options, and information. This
overload of facts and guidance deters the patient from actually understanding or choosing which

13. 12
guideline is not only most credible but the right fit for them. The deliberation between which
source is the right source drives a wedge in the ability of the patient to trust a certain guideline
over the other. When they are so many different guidelines giving several different protocols
regarding one procedure the patient becomes alarmed and does not know which is the right one.
This uncertainty propels the patient to be misinformed about his or her own health.
There is no clear-cut answer but the best solution may be modifying the way these websites are
being presented to the general public. The websites should display an inviting layout, with colors
that would attract and not frighten a patient. It shouldn’t be verbose and instead dissect all the
medical jargon into layman’s terms. Instead of providing infinite guidelines to one question or
concern, the website should implement the top three. These websites providing guidelines should
help the patients understand their health not further confuse them.
With so many individuals dependent on Medicaid as aforementioned before and several
Americans that are undocumented and living without health insurance it is crucial to educate
these patients about their health. These patients have various reasons that impede their ability to
go seek proper medical care. Going to the doctor’s office for routine check-ups is often not first
on their priority list due to transportation potentially being an issue. Making an appointment with
the doctor might mean missing a day of work and one day’s worth of pay is more important than
getting routine check-ups for some families.
CONCLUSION:
Guidelines are more imperative than many believe. Not only do they serve as a mini check up for
those who do not have the means or face impediments in going to see the doctor but they serve
as an educational tool as well. In order to not breach autonomy it is imperative to make sure that

14. 13
patients are well informed enough to make a medical decision for themselves. By having a sound
knowledge of their health, patients will feel less inept to discuss their medical issues and make
choices that will benefit them. They will question the care of the doctor less and instead work
with the doctor to find an answer. Therefore, it is crucial for online guidelines to alter their
websites to manifest websites that are accessible, comprehendible, and user friendly in order to
perpetuate the best health care for patients.

15. 14
References:
1. Clinical Practice Guidelines We Can Trust - Institute of Medicine. (n.d.). Clinical
Practice Guidelines We Can Trust - Institute of Medicine. Retrieved August 4, 2014,
from http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx
2. Medical Guideline. (n.d.). healthinformatics -. Retrieved August 4, 2014, from
http://healthinformatics.wikispaces.com/Medical+Guideline
3. Standards for Developing Trustworthy Clinical Practice Guidelines - Institute of
Medicine. (2011, March 23). Standards for Developing Trustworthy Clinical Practice
Guidelines - Institute of Medicine. Retrieved August 8, 2014, from
http://www.iom.edu/Activities/Quality/ClinicPracGuide.aspx
4. Standards for Systematic Reviews of Comparative Effectiveness Research. (2011, March
23). Institute of Medicine of the national academics . Retrieved August 5, 2014, from
http://www.iom.edu/Activities/Quality/SystemReviewCER.aspx