Application developments Choices Research into existing Solutions Survey Monkey or similar tools Risk of exposure of patient data Lack of flexibility, not able to brand or function as standalone application Data collection, transfer and storage risks RedCap Lack of flexibility Cannot function as standalone application needs to be hosted and CLASP would have to store the data Limited support from Vanderbilt would require an FTE to manage Web Survey Tool Kit Good option allows surveys to be created using a JSP and Java Servlets Could function as a standalone application No current support of development plans does not function on with newer version of Java JDK or Apace Webserver Choice of Development Platforms To meet customers’ requirements application would have to have ability to function as a hosted web application, mobile compatible website or smart phone or tablet application Possibility exists that each version would have to be developed on separate IDEs, Android SDK, Eclipse, Netbeans, XCODE, etc Development cost would be high Version and change management would be complicate
Carolinas Comfort Scale
Converting Paper to DigitalCarolinas Health Informatics Program Practicum Project John Horan
Carolinas Comfort Scale Presentation OutlineIntroduction of the Carolinas Comfort Scale (CCS)Project Scope and RequirementsResearch and DevelopmentApplication DemonstrationPotential Impact of ApplicationQuestions and General Discussion
What is it?• CCS is survey used to help physicians and patients assess Quality of Life both pre and post hernia repair with mesh• Developed by physicians and researchers from Carolinas Laparoscopic and Advanced Surgery Program (CLASP)• CLASP has become a leader in minimally invasive surgery by focusing on Patient Care, Research, and Education.
How it was developed• Developed by comparing Carolina Comfort Scale results against the SF-36 questionnaire which is considered the gold standard for measuring Quality of Life for patients who have undergone hernia repair surgery• Results of the study published in the Journal of American College of Surgeons http://www.ncbi.nlm.nih.gov/pubmed/18387468• CCS was found to better assess the quality of life and satisfaction of patients who have undergone surgical hernia repair better than the generic SF-36.
Carolinas Comfort Scale CSS UsageThe CCS survey is currentlyused by surgeons in 19 USstates, 39 Countries, andhas been translated intomultiple languages.
Carolinas Comfort ScaleA physician from the United Kingdom willuse the CCS in Nottingham City Hospitalwhich covers 64 general practices andprovides care to 330,000 patients annually.The CCS will be used as a support tool forscreening eligibility for inguinal herniarepair surgery funded by the NottinghamCity Hospital.French physicians will launch a researchtrial using the CCS involving 30 university hospitalteams in France. The study is sponsored by theFrench government.
Carolinas Comfort Scale Current Workflow of Data Collection
Carolinas Comfort Scale How the Survey Works• Patients fill out the paper based survey• Question responses are scored using an algorithm developed by CLASP surgeons and researchers that score five endpoints• Activity, Sensation, Pain, Movements and the Total Score.
Carolinas Comfort Scale SAS Scoring Algorithm SamplePrimary Endpoint #1: Activity Score PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN;Compare mean/median scores for each of the 23 questionnaire items (1a, 1b, 2a, PROC MEANS NOPRINT data= CCS.Responses;2b, 2c, etc) for lightweight vs. heavyweight mesh. This will provide a summary for VAR q2c q3c q4c q5c q6c q7c q8c;all of the 23 questions for each comparative group. OUTPUT OUT=CCS.AvgMOVEMENT MEAN=m1-m7;PROC SORT data= CCS.Responses; by CURRENTMESHTYPE; RUN; BY SYMPTOMATIC;PROC MEANS NOPRINT data= CCS.Responses; WHERE FOLLOWUPTIMEPOINT = ‘POSTOP’;VAR q1a q1b q2a q2b q2c q3a q3b q3c q4a q4b q4c q5a q5b q5c q6a q6b RUN;q6c q7a q7b q7c q8a q8b q8c; Primary Endpoint #5: Total ScoreOUTPUT OUT=CCS.AvgAll MEAN=m1-m23; Compare mean/median scores for QOL scores collapsed acrossby CURRENTMESHTYPE ; pain/sensation/movement & activity for symptomatic vs. non-symptomaticRUN ; patients. This will provide an overall report based on the mean for all 23Primary Endpoint #2: Sensation Score questionnaire items.Broader Endpoint #2: Compare mean/median scores for sensation collapsed across /*Combine result data files from examples #2, #3 and #4. This combination representsactivity (1a, 2a, 3a, 4a, 5a, 6a, 7a, 8a) for symptomatic vs. non-symptomatic patients. all scores collapsed across pain/sensation/movement.*/This will provide a report that specifically focuses in on the sensation of mesh post PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgPAIN FORCE; run;implantation pertaining to a specific activity for each comparative group. PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgMOVEMENT FORCE;PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN; run;PROC MEANS NOPRINT data= CCS.Responses; PROC MEANS NOPRINT data= CCS.avgSENSATION;VAR q1a q2a q3a q4a q5a q6a q7a q8a; VAR m1 m2 m3 m4 m5 m6 m7 m8;OUTPUT OUT=CCS.AvgSensation MEAN=m1-m8; OUTPUT OUT=CCS.QOLall MEAN=m1-m23;BY SYMPTOMATIC; by SYMPTOMATIC; RUN ;RUN; Compare mean/median scores for QOL scores collapsed acrossPrimary Endpoint #3: Pain Score pain/sensation/movement & activity for symptomatic vs. non-symptomaticBroader Endpoint #3: Compare mean/median scores for pain collapsed across patients. This will provide an overall report based on the mean for all 23activity (1b, 2b, 3b, 4b, 5b, 6b, 7b, 8b) for symptomatic vs. non-symptomatic questionnaire items using the Wilcoxon signed-rank test.patients. This will provide a report that details the level of pain that is experienced PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgPAIN FORCE; run;by the patients pertaining to a specific activity either pre or post-operatively for PROC APPEND BASE=ccs.avgSENSATION DATA=ccs.avgMOVEMENT FORCE;each comparative group. run;PROC SORT data= CCS.Responses; by SYMPTOMATIC; RUN; DATA CCCS.meanALL;PROC MEANS NOPRINT data= CCS.Responses; SET CCS.avgSENSATION;VAR q1b q2b q3b q4b q5b q6b q7b q8b; RUN;OUTPUT OUT=CCS.AvgPAIN MEAN=m1-m8; PROC NPAR1WAY WILCOXON data=ccs.meanALL;BY SYMPTOMATIC; 8WHERE FOLLOWUPTIMEPOINT = ‘PREOP’; CLASS SYMPTOMATIC;RUN; 7 VAR m1 m2 m3 m4 m5 m6 m7 m8;Primary Endpoint #4: Movement Score OUTPUT OUT = CCS.wilcoxonresults;Compare mean/median scores for movement collapsed across activity (2c, 3c, 4c, RUN;5c, 6c, 7c, 8c) for symptomatic vs. non-symptomatic patients. This will provide a PROC SORT data=ccs.wilcoxonresults; by SYMPTOMATIC; RUN;report that details movement limitations either pre or post-operatively for each PROC MEANScomparative group.
What CLASP wanted• An electronic version of the CCS survey• Electronic version of survey needs to be easily distributed to current and future users on a variety of platforms• To minimize risk survey needs to function as a standalone application so CLASP does not take on the risk or expense of data collection and data management.• Current users need to have the ability to print PDF versions of the completed survey with results and have the ability to export data for patient records and further analysis.
Additional Considerations• Security of patient information• Data collection and storage• Branding of application to bring recognition to the research and work that CLASP has done to create the Carolina Comfort Scale• Ability to control the use and monitor usage of the application• Ability to update application as future changes are developed
Additional Considerations Continued• Decrease turn around time of data analysis• Make data collection and sharing easier• Create application that would require very little if any additional support• Make application and results generation easy to use and export for current users
Development options• Research into existing tools and solutions • REDCap • Survey Monkey • WebSurvey Toolbox• To meet customer requirements separate versions of the electronic survey would have to be developed for each platform using several different tools • Android: Android SDK or Eclipse • Apple: XCODE • Web: Eclipse, Netbeans, etc.• Development costs and management would be expensive and time consuming if separate platform development was undertaken
Carolinas Comfort ScaleA solution exists to meet allthe needs!• Adobe leader in development of Rich Internet Applications• Using Adobe’s development tools applications can be developed and then distributed to a variety of platforms
Carolinas Comfort Scale Adobe workflow for application development
Carolinas Comfort Scale • Is Flash a good choice for development of this application? • Benefits of Adobe Flash vs. HTML5 • Popularity: estimated that world wide over 90% of internet users have Flash installed on their desktop computers • Many users find it to be easier to vs. use than other development options that exist with HTML5 • Drawbacks of using Flash • Not support on Apple devices, however Adobe provides several tools to convert flash based application for use on IOS
Application Development• A tested and successful design already existed in the paper format of the survey• Screen elements were designed in Photoshop and then imported into Adobe Catalyst. In Catalyst the screen elements were given functionality and page transitions were defined• The project was then imported into Flash builder where additional functionality was added and it was then compiled for testing on the Blackberry Playbook and Adobe Air platforms• The customer was sent an Adobe Air version of the prototype application to evauluate and additional edits to the screen layout were made• The Adobe Air and Playbook tablet prototype was then demonstrated before the Department of Minimally Invasive Surgery at there monthly team meeting
Carolinas Comfort Scale Proposed workflow for application data collection and analysis
Results• Response to the application was very positive• When prompted for feedback on the application the only question was could it have the ability to export the data. I had missed demonstrating the data export screen in my presentation• When scoring the paper survey the algorithm has to take into account that patients may neglect to answer a question or that they may select more than one choice for a question. These issues can be prevented in the application by limiting question responses to one selection and by requiring that users answer each question before the results are generated.• Converting the survey to electronic forms give the customer multiple options for distribution and the ability to manage their intellectual property
Results Continued• Use of the application would allow Physicians to have immediate result that that they could share with their patients• Provides patients with additional information• Streamlines the workflow of data collection and analysis• Provides an example of how other paper based surveys and forms can be converted into applications in a cost effective manner• Next steps develop the full scoring and export functionality into the application