MEDINFO 2013 Panel on Personalized Healthcare and Adherence: Issues and Challenges

7,275 views

Published on

Venue: The 14th World Congress on Medical and Health Informatics will take place in Copenhagen, Denmark.
http://medinfo2013.dk

Moderator: Dr. Marion Ball (IBM Research/JHU); Panelists: Dr. Vimla Patel (NYAM), Dr. Bern Shen (Healthcrowd), Dr. Pei-Yun Sabrina Hsueh (IBM Research)
Organizer: Dr. Pei-Yun Sabrina Hsueh (phsueh@us.ibm.com)

Personalization is key to the delivery of wellness care including preventive measures and disease management regimes, where patients take on increased responsibility for
their own health. While personalized care has already taken a giant leap through genomics, it remains a challenge to understand how individual differences play a role in patient adherence and manage recommended changes accordingly.

Practical methods of creating and evaluating personalized
systems have not been fully established. In particular, the role of data-driven analytics in producing actionable insights for practitioners is unclear, and the use of behavioral data has created additional challenges to the understanding of patient adherence for effective care delivery.

The panel will discuss the challenges that face many countries around personalized care from various perspectives. These range from behavioral aspects such as maintaining good practices, cognitive aspects such as how do individuals make decisions in the lights of good evidence, social aspects such as how to engage patients in sustaining adherence behavior, to technological aspects such as how to evaluate individual applicability of data-driven analytics and personalized technological systems.

The panel is expected to contribute to the global community by presenting lessons learned from
existing pilot designs and a collective list of recommendations for pilot design of personalized services at the conclusion of this panel.

Published in: Health & Medicine
1 Comment
0 Likes
Statistics
Notes
  • the one step ahead vision to connect science of healthcre with art of personal care
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
7,275
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
19
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide
  • I will briefly review the SEMINAL themes that emerged during this conference -- 31 years ago
  • Three instances of weakness PRIOR to cessation of Inderal, however patient hypothesis was that her weakness was due to the cold medication
  • Cessation of Inderal was very salient, and physician selectively focused on that detail. Unfortunately, the diagnosis of anemia was missed.
  • Lay subjects respond to alterations in lifestyle, not to signs and symptoms Physicians act on signs and symptoms to relieve symptoms
  • Expensive Heavyweight Centralized Requires experts to use Data primarily biological… … collected in artificial setting (hospital lab) Vs. Cheaper Lightweight Distributed User-friendly Bio, behavioral, psychosocial data… … collected in situ http://encarta.msn.com/media_461544834/IBM_System_360_Mainframe_Computer.html www.ipunplugged.com/news.asp?mi=5.1&articlekey=47 http://www.eng.ox.ac.uk/samp/diabetes.html http://www.dkimages.com/discover/Home/Health-and-Beauty/Medical-Examinations/Pregnancy-Tests/Pregnancy-Tests-03.html
  • http://www.faceresearch.org/demos/average
  • Adapted from Journal of Communication pages S184-S201, 4 AUG 2006 DOI: 10.1111/j.1460-2466.2006.00289.x http://onlinelibrary.wiley.com/doi/10.1111/j.1460-2466.2006.00289.x/full#f1 Elements of ELM - https://msbfile03.usc.edu/digitalmeasures/priester/intellcont/2009MediaChapterPettyBrinolPriester-1.pdf
  • http://jama.ama-assn.org/cgi/content/abstract/291/10/1238 http://journals.lww.com/lww-Medicalcare/Abstract/2002/09000/Patient_Adherence_and_Medical_Treatment_Outcomes_.9.aspx – met-analysis of 63 studies of mostly medical adherence studies
  • ~54M Americans with disabilities ~7M Americans >15 years old limited in ≥ 1 ADLs Informal caregiving ~$200B/year in US 1.4M receiving home health care
  • http://socialgraphproject.org/blog/2010/12/announcing-the-social-graph-project/
  • http://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2013.pdf
  • JNC7 guideline  2012 ADA EASD Type ii Diabetes guideline Stratified medicine definition: Defines groups of patients that could derive especially big benefits from a certain therapy. E.g., based on certain genetic traits of a tumor, we can now predict very precisely for many types of cancer whether or not the given patient would benefit from chemotherapy. Biobank projects Pan-European biobanking Biomolecular Resource Research Infrastructure Standardization Impedence by the lack of efficient personalization and validation techniques Disruption will involve pushing more medicine into the precision category . ~ Clayton Christensen “The innovator’s Prescription” Companies like Amazon, Netflix, Pandora, Rhapsody, and iTunes offer consumers virtually unlimited choices in real time. What I think mass customization allows is a company that operates in a physical goods environment (i.e. http://www.bluewardrobe.com) the ability to offer the customer anything. Mass customization allows the deployment of a “long tail” strategy without the burden of physical inventory. Companies that can match their product offerings to the true shape of the demand curve (a curve that incorporates both “hits” as well as “niches”) will be able to offset any initial inconvenience that mass customization inherently possesses. Today’s customer-facing technologies are cheaper and more social.  Configurators, which help customers co-design their customized product purchases, are cheaper, better, and more ubiquitous than ever. They can finally be integrated directly into a Facebook site, which will facilitate social sharing and group co-design activities. Tomorrow’s customer-facing technologies will be revolutionary.  Technologies empowering customers to design their own products will become richer and more plentiful. For example, Microsoft’s Xbox Kinect shows the pathway towards the ultra-configurator: a device that can measure the contours of your body and allow you to use gestural inputs to design products.  (I draw pin-stripes on a suit I’m co-designing; I size the steering wheel of a car I’m customizing; etc.) The richer the configuration experience, the more appealing mass customized products will become – and these experiences will indeed be much richer. Platforms are promoting discovery, fulfillment, and scale.  Back-end systems like supply chain software provider Archetype Solutions offer better production-side IT analytics. And explicit platform providers for products, like Ponoko, Zazzle, or Spreadshirt, are popularizing, syndicating, and empowering an ecosystem of partners to devise their own customized products.   80 percent of mass customization is about brand building and for consumer goods mass customization is utilized primarily to increase existing sales of mass produced products.  NIKE ID NIKEiD is a service provided by the sportswear company NIKE allowing customers to customize clothing purchased from Nike. The customer becomes the designer as they change and add a personal look and feel to a selected item. The service can be accessed both online from their homepage and in select physical branches. The service was launched initially in 1999 and could only be accessed through their website. It provided customers the ability to choose from a limited range of different material and colors to develop their own style of tennis shoe. Intuitive medicine is care for conditions loosely diagnosed by symptoms and treated with therapies of unclear efficacy.   intuitive medicine, involves highly trained specialists handling difficult diagnoses and treatment. The second, empirical medicine, deals with the expensive world of chronic care and trial-and-error treatment. Lastly, precision medicine is where the diagnosis and therapy are known. Treatment can be made routine and moved out of the hospital. Precision medicine is the delivery of care for diseases that can be precisely diagnosed and with predictable, evidence-based treatments.  The NAS report calls for "precision medicine," — the use of genomic, epigenomic, exposure, and other data to define individual patterns of disease, potentially leading to better individual treatment. If this sounds like personalized medicine, it is — but more so. With the over-use of 'personalized medicine' in a wide variety of contexts, "precision medicine" conveys a more accurate image of diagnosis that is person-centered and multifaceted. Will subdividing syndromes based on molecular signatures, neuroimaging patterns, inflammatory biomarkers, cognitive style, or history give us subgroups that are more responsive to certain medications or psychosocial treatments? Personalized Healthcare Accelerate the emergence of disruptive innovations in health care by developing and curating a broad, multi-stakeholder approach that addresses all four primary elements of disruption. Provide coordination and orchestration of stakeholder efforts at the scientific/technical level, the commercial/industry ecosystem level, and the societal/social level. Read more: Precision medicine could be the key to better, cheaper care - FierceHealthcare http://www.fiercehealthcare.com/story/precision-medicine-could-be-key-better-cheaper-care/2011-02-28#ixzz26mMPyKB8  As the P4 Medicine predicted, personalization will transform the healthcare industry. It will impact how we live and how businesses operate profoundly Personalized healthcare provides: • It is personalized; it is based on an understanding of how genetic variation drives individual treatment. • It is predictive; it is able to identify what conditions a person might contract in the future and how the person will respond to a given treatment, enabling the development of a tailored health strategy. • It is preventive; it facilitates a proactive approach to health and medicine, which shifts the focus from illness to wellness. • It is participatory; it empowers patients to make informed choices and take responsibility for their own health. Clayton: Hospitals become focused solution shops, practicing intuitive medicine Focused value-adding process hospitals & clinics provide procedures after definitive diagnosis Facilitated networks take dominant role in the care of many chronic diseases
  • Kaiser Permanente – Reduce 5-year CVD risk 2.4 times more than EHR+panel support tool alone (  13% absolute risk reduction)  6,000 myocardial infarctions (MIs) and strokes prevented annually if applied throughout KP (  43% increase over JNC7 guideline for the same cost)  $420M annual net savings if applied throughout KP Fix: Individualized Guideline Primary Diabetes RISK: E.g., DPP Secondary and Tertiary Diabetes RISK: E.g., UKPDS, FIN-D2D Cardiovascular disease RISK: E.g., Individualized guideline for hypertension control pilot in Kaiser & Boston Health Group
  • Source: SA Schroder. We can do better - Improving the Health of the Amarican People. N Engl J Med 2007;357:1221-8. Stampfer MJ, Hu FB, Manson JE, Rimm EB, Willett WC. Primary prevention of coronary heart disease in women through diet and lifestyle. N Engl J Med 2000;343:16-22. Tuomilehto J. et at. N Engl J Med 2001; 344:1343-1350 any cardiovascular disease event 42% reduced risk nonfatal heart attack, stroke, or death from cardiovascular causes 57% reduced risk 1. Finland National Type II Diabetes Prevention Programme (Saaristo et al., 2007) 5  hospital  districts, covering a population of  1.5 million, during the years 2003-2007. Test the hypothesis whether T2D can be prevented or at least delayed in high-risk subjects by life-style modification or by combining  lifestyle intervention and drug treatment 2. CDC's Diabetes Prevention Program, 2002 27 clinical centers around the United States 3,234 study participants were overweight and had pre-diabetes the average follow-up is 2.8 yrs Test whether lifestyle intervention group—those receiving intensive individual counseling and motivational support on effective diet, exercise, and behavior modification -- reduced their risk of developing diabetes
  • are the high-level participation of users and channels of deep user understanding Mass customization is the method of "effectively postponing the task of differentiating a product for a specific customer until the latest possible point in the supply network." (Chase, Jacobs & Aquilano 2006, p. 419). A recent study by a group of consultants at McKinsey* put a figure on the amount that is lost by producing cars to meet a demand that never materialises. Eliminating such losses (and the associated discounts needed to sell off excessive stocks of finished cars) could, according to McKinsey, be worth up to $80 billion a year to the car manufacturers. Nissan Motor has estimated that converting entirely to BTOcould save up to $3,600 per vehicle. Nevertheless, despite some concerted efforts, notably among car makers in Europe, the goal of the “three-day car” (as one research project dubsBTO vehicles) could still be ten years distant. Companies like Amazon, Netflix, Pandora, Rhapsody, and iTunes offer consumers virtually unlimited choices in real time. What I think mass customization allows is a company that operates in a physical goods environment (i.e. http://www.bluewardrobe.com) the ability to offer the customer anything. Mass customization allows the deployment of a “long tail” strategy without the burden of physical inventory. Companies that can match their product offerings to the true shape of the demand curve (a curve that incorporates both “hits” as well as “niches”) will be able to offset any initial inconvenience that mass customization inherently possesses. Today’s customer-facing technologies are cheaper and more social.  Configurators, which help customers co-design their customized product purchases, are cheaper, better, and more ubiquitous than ever. They can finally be integrated directly into a Facebook site, which will facilitate social sharing and group co-design activities. Tomorrow’s customer-facing technologies will be revolutionary.  Technologies empowering customers to design their own products will become richer and more plentiful. For example, Microsoft’s Xbox Kinect shows the pathway towards the ultra-configurator: a device that can measure the contours of your body and allow you to use gestural inputs to design products.  (I draw pin-stripes on a suit I’m co-designing; I size the steering wheel of a car I’m customizing; etc.) The richer the configuration experience, the more appealing mass customized products will become – and these experiences will indeed be much richer. Platforms are promoting discovery, fulfillment, and scale.  Back-end systems like supply chain software provider Archetype Solutions offer better production-side IT analytics. And explicit platform providers for products, like Ponoko, Zazzle, or Spreadshirt, are popularizing, syndicating, and empowering an ecosystem of partners to devise their own customized products.   80 percent of mass customization is about brand building and for consumer goods mass customization is utilized primarily to increase existing sales of mass produced products.  NIKE ID NIKEiD is a service provided by the sportswear company NIKE allowing customers to customize clothing purchased from Nike. The customer becomes the designer as they change and add a personal look and feel to a selected item. The service can be accessed both online from their homepage and in select physical branches. The service was launched initially in 1999 and could only be accessed through their website. It provided customers the ability to choose from a limited range of different material and colors to develop their own style of tennis shoe. Intuitive medicine is care for conditions loosely diagnosed by symptoms and treated with therapies of unclear efficacy.   intuitive medicine, involves highly trained specialists handling difficult diagnoses and treatment. The second, empirical medicine, deals with the expensive world of chronic care and trial-and-error treatment. Lastly, precision medicine is where the diagnosis and therapy are known. Treatment can be made routine and moved out of the hospital. Precision medicine is the delivery of care for diseases that can be precisely diagnosed and with predictable, evidence-based treatments.  The NAS report calls for "precision medicine," — the use of genomic, epigenomic, exposure, and other data to define individual patterns of disease, potentially leading to better individual treatment. If this sounds like personalized medicine, it is — but more so. With the over-use of 'personalized medicine' in a wide variety of contexts, "precision medicine" conveys a more accurate image of diagnosis that is person-centered and multifaceted. Will subdividing syndromes based on molecular signatures, neuroimaging patterns, inflammatory biomarkers, cognitive style, or history give us subgroups that are more responsive to certain medications or psychosocial treatments? Personalized Healthcare Accelerate the emergence of disruptive innovations in health care by developing and curating a broad, multi-stakeholder approach that addresses all four primary elements of disruption. Provide coordination and orchestration of stakeholder efforts at the scientific/technical level, the commercial/industry ecosystem level, and the societal/social level. Read more: Precision medicine could be the key to better, cheaper care - FierceHealthcare http://www.fiercehealthcare.com/story/precision-medicine-could-be-key-better-cheaper-care/2011-02-28#ixzz26mMPyKB8  As the P4 Medicine predicted, personalization will transform the healthcare industry. It will impact how we live and how businesses operate profoundly Personalized healthcare provides: • It is personalized; it is based on an understanding of how genetic variation drives individual treatment. • It is predictive; it is able to identify what conditions a person might contract in the future and how the person will respond to a given treatment, enabling the development of a tailored health strategy. • It is preventive; it facilitates a proactive approach to health and medicine, which shifts the focus from illness to wellness. • It is participatory; it empowers patients to make informed choices and take responsibility for their own health. Clayton: Hospitals become focused solution shops, practicing intuitive medicine Focused value-adding process hospitals & clinics provide procedures after definitive diagnosis Facilitated networks take dominant role in the care of many chronic diseases
  • PWR---> clinical requirements -- user preferences -- personalized plan (activity, nutrition, clinical) - compliance plan  
  • How do we proactively leverage patient data to transform guidelines into actionable insights based on risk and disease progression? How do we generate a specific personalized plan ? How do we monitor effectiveness, adherence risk and adaptation points?
  • A representation of all the interactions of the key stakeholders (Mary, her doc, her case manager, her family..) How do we help? Our proposal: A risk mediator that can trace case history, provide proxy measurements that summarize exception urgency, and sort the case exceptions by urgency Saving costs of handling exceptions and improving customer retention. At planning stage Infer Personalization matrix: Identify the proxy measure of intervention need (e.g., the likelihood of entailing hypoglycemia episode) and the personalization matrix (e.g., exception model in the fitness and dietary intake intervention domain for various user subpopulations) appropriate for the determination of such need Exception importance: tf*idf, correlation (chi), information gain Dynamic data collection: Discover what sources of compliance info (other than the initial vendor) are pertinent, and relay related data to the compliance manager. Measure personal intervention need : (expected risk and incompliance propensity) Monitor wellness context changes: detect significant changes in context (e.g., cardio zoning improvement) At personalization matrix analytics stage: Model the occurrence of exceptions in the intervention domain Keep track of the summative effects on intervention need : the summation of the effects of a set of low frequency variants across a variety of lifestyle interventions, each conferring a moderate but readily detectable increase in relative risk. Infer personalization matrix by aggregating the set of exceptions that are commonly seen in the high-need group but rarely found in the low-need group.
  • How to estimate a target patient’s risk levels and identify personal risk factors based on relevant risk group data? How “personalisable” is a target patient? What are the patient information missing for reliable personalization How do we proactively leverage patient data to transform guidelines into actionable insights based on risk and disease progression?  Local risk factor analysis, Personalizable indicator, Active characterization of patient wellness : How do we generate a specific personalized plan ? How do we monitor effectiveness, adherence risk and adaptation points? What is the proxy measure of ind. diff?
  • How do we proactively leverage patient data to transform guidelines into actionable insights based on risk and disease progression?  Local risk factor analysis, Personalizable indicator, Active characterization of patient wellness : How do we generate a specific personalized plan ? How do we monitor effectiveness, adherence risk and adaptation points? What is the proxy measure of ind. diff?
  • MEDINFO 2013 Panel on Personalized Healthcare and Adherence: Issues and Challenges

    1. 1. AN INTERNATIONALAN INTERNATIONAL HELLOHELLO Brazil - Opa Chinese – nin haoDutch – Hallo, Goededag French – Bonjour German - Guten Tag Hawaiian - AlohaIndonesian -Selamat Japan –Japan – konnichiwakonnichiwa Korean – annyeonghaseyo Norwegian - Goddag Portugese –’OlaPortugese –’Ola Spanish - ¡Hola! Swedish - Hej / HallåSwedish - Hej / Hallå Thailand - sà-wàt-deeThailand - sà-wàt-dee Russian - AlloTurkey - Alo, Efendim Italian – Ciao Israel-ShalomItalian – Ciao Israel-Shalom Africa – Hallo Polish – HALO/SLUCHAM Arabic – As salam ‘alakumArabic – As salam ‘alakum
    2. 2. Personalized Healthcare and Adherence: Issues and challenges 2 Senior Advisor Healthcare Informatics IBM Research,  Professor Emerita, Johns Hopkins University  Member, Institute of Medicine  Member of the Board Of Regents of the National Library of Medicine  Past President, International Medical Informatics Association ( IMIA) Fellow: American College of Medical Informatics (ACMI), Past Board member and Fellow of the Health Information Management and Systems Society( HIMSS), American Health Information Management Association (AHIMA) Medical Library Association (MLA) and the College of Health Information Management Executives (CHIME), American Academy of Nursing (FAAN) Marion J. Ball, Ed.D
    3. 3. Personalized Healthcare and Adherence: Issues and challenges Panel topic: Personalized Healthcare and Adherence: Issues and Challenges Session 984 09. Thursday Aug 22 10:30am-12 Auditorium Panel on Patient-Centered Care - I Panel topic: Personalized Healthcare and Adherence: Issues and Challenges Marion J. Ball
    4. 4. Personalized Healthcare and Adherence: Issues and challenges INTRODUCTION • 10:30-10:40am Introduction Marion Ball • 10:40-11:00am Vimla Patel presentation • 11:00-11:20am Bern Shen presentation • 11:20-11:40am Sabrina Hsueh presentation • 11:40-12:00 Panel discussion/audience Q&A
    5. 5. Personalized Healthcare and Adherence: Issues and challenges Looking back in the rear view mirror to the 1960’s to the early work of Dr. Warner Slack in Wisconsin and to Dr Morris Collen who introduced us to the impotence of empowering the patient as a consumer. ALSO that the computer was an enabling technology that could empower the patient and the consumer. We owe a great debt to these and other wonderful pioneer in our field.
    6. 6. Personalized Healthcare and Adherence: Issues and challenges What Do Consumers Want? • We ask consumers, “What kind of health care do you want?” – They answer in terms of quality, access and cost. • We should ask, “How would you like to interface with the healthcare system?” – They would answer in terms of the kind of services and information they want, as they would for Web-based banking or shopping. JS Parker, Consumer Expectations Demand Client-Focused Technology, in Consumer Informatics, 2005, p 77
    7. 7. Personalized Healthcare and Adherence: Issues and challenges Changing Behaviors to help peopleChanging Behaviors to help people do what they need!do what they need! ““ Transparency”Transparency”
    8. 8. Personalized Healthcare and Adherence: Issues and challenges The Future is here! • Where we need to head is to make IT a transparent enabler to provide the care giver and the consumer / patient Information when, where and how they want it to have an effective and efficient Healthcare system.
    9. 9. Personalized Healthcare and Adherence: Issues and challenges Lessons Learned • The importance of re-education is sorely underestimated. This is true for the healthcare provider as well as the consumer. Thanks to the Department of Health and Aging In Australia, the VA, Rod Kolodner, Barry Weiner, Hans Peterson, and John Tressling.
    10. 10. Personalized Healthcare and Adherence: Issues and challenges Lessons Learned • Need grass roots participation by all levels of health care professionals in developing, implementing and training for the new transformed health care delivery system. Thanks to the Department of Health and Aging In Australia, the VA, Rod Kolodner, Barry Weiner, Hans Peterson, and John Tressling.
    11. 11. Personalized Healthcare and Adherence: Issues and challenges 11 Cultural Change Management •Approximately 80% of critical success factors for clinical systems installs are people and process related, while 20% are technology related* HIMSS 2010 e-Presentation *Journal of American Medical Informatics Association
    12. 12. Personalized Healthcare and Adherence: Issues and challenges Consumer Technology The true benefit of these technologies is not in the quantity of data they provide, but in how they transform data into useful information that can make a difference, and improve value and care.
    13. 13. Personalized Healthcare and Adherence: Issues and challenges Thank You Merci Grazie Gracias Obrigado Danke Japanese English French Russian German Italian Spanish Brazilian PortugueseArabic Traditional Chinese Simplified Chinese Hindi Tamil Thai Korean Hebrew
    14. 14. Personalized Healthcare and Adherence: Issues and challenges Personal Health Care Decision Making by Lay Public Vimla Patel, PhD, DSc, FRSC MedInfo – Copenhagen 22 Aug 2013 Panel Discussion (1):
    15. 15. Personalized Healthcare and Adherence: Issues and challenges Vimla L. Patel, PhD, DSc, FRSC • Senior Research Scientist, The New York Academy of Medicine • Director, Center for Cognitive Studies in Medicine and Public health • Adjunct Professor, Biomedical Informatics. Columbia University, NY • Adjunct Professor, Public Health, Weill Cornell Medical Center, NY • Professor of Biomedical Informatics, Arizona State University • Fellow of the Royal Society of Canada (Academy of Social Sciences) • Fellow, American College of Medical Informatics • Associate Editor, Journal of Biomedical Informative (JBI) • Editorial Boards of Journal of Artificial Intelligence in Medicine (AIM), Advances in Health Science Education (AHSE), Topics in Cognitive Science. • Past Vice-President (Member services), International Medical Informatics Association (IMIA) • Past Vice-Chair, AMIA Scientific Program Committee • Past Editorial Boards: International Journal of Medical Informatics (IJMI), Journal of Medical Decision Making (MDM)
    16. 16. Personalized Healthcare and Adherence: Issues and challenges Models of Reasoning • Causal • Descriptive: Explanatory • Standard of logical consistency between theory and evidence • Episodic • Narrative: Explanatory • Opportunistic • Difficulties in differentiating theory and evidence Clinicians use composite models, which are between the technical and lay models Technical Lay Public
    17. 17. Personalized Healthcare and Adherence: Issues and challenges Empirical Studies: How Lay Reasoning Influences Personal Health Decisions 1. What doctor says and what patient hears and decides about personal health care 2. Why some advice to lay public do not lead to desired outcome 3. What medication instructions are intended to be followed and what lay public practice
    18. 18. Personalized Healthcare and Adherence: Issues and challenges Example 1 What doctor says and what patient does
    19. 19. Personalized Healthcare and Adherence: Issues and challenges Cognition and the Challenge of Communication: Mental Models
    20. 20. Personalized Healthcare and Adherence: Issues and challenges Doctor-Patient Interaction • Eliciting patients’ stories while in the waiting room • Doctor-patient interaction • Stories following D-P interaction • Follow up at home Patel V.L, Arocha JF, Kushniruk A (2002) Patients' and Physicians' Understanding of Health and Biomedical Concepts: Relationship to the Design of EMR Systems. Journal of Biomedical Informatics; 35:8-16.
    21. 21. Personalized Healthcare and Adherence: Issues and challenges Scenario for the “Christmas Problem” The patient was a 72 year old female with past history of heart trouble. Treated with prescription drug Inderal. Previously hospitalized and treated for pneumonia (given antibiotics). She collapsed in a department store while shopping and was taken to hospital but released shortly after being treated. Physician felt that patient’s collapse was due to excessive Inderal. She fell again a week later. EKG indicated that collapse was not due to heart trouble. Patient was asked to stop taking Inderal. One week later, the patient had an angina attack which subsided after she had taken Nitroglycerine. The next day, and approximately three weeks after she was treated for her cold, she visits her physician.
    22. 22. Personalized Healthcare and Adherence: Issues and challenges Prior Conceptualization: Explaining Illness Beautiful Day Go Downtown Buy Xmas Card Patient Falls COND: ORD:TEM GOAL: Wear Heavy Coat Wear Scarf Wear Knit Cap Cond : AND: AND: Patient Falls Patel V.L, Arocha JF, Kushniruk A (2002) EMR: Re–engineering the Organization of Health Information. Journal of Biomedical Informatics 35:8-16
    23. 23. Personalized Healthcare and Adherence: Issues and challenges Christmas Shopping not Completed Fell 3 Weeks Ago Given Cold Medication Event 1 Fluid in Lungs Patient Representation During D-P Interaction: “The Christmas Problem” Collapsed while Shopping Hospitalized Dress Too Warmly Event 2 Event 3 Fell While ShoppingSlipped Ambulance/Hospital Stop Inderal Chest Pain See PhysicianNitroglycerin Event 4 Christmas Shopping not Completed
    24. 24. Personalized Healthcare and Adherence: Issues and challenges Physician’s Problem Representation of the “Christmas Problem” During D-P Interaction Pneumonia Hospital Home 10 Days Collapse Hospital Too Warm? Abrupt Decrease/Cessation in Cardiac Medication (Inderal) Slipped? Recommendation: Go back to Inderal Decrease dosage Side Effects: Weakness Fainting
    25. 25. Personalized Healthcare and Adherence: Issues and challenges Treatment Adherence 27 Stock Broker - Stress Dx: High blood pressure Type A Personality Rec: Relaxation Therapy Stop Smoking 72 66 72 Change Profession No Trouble breathing Water on lungs Obese from surgery NoDx: Weight problem Rec: Eat fruits/vegetables Exercise Obesity not from diet Afraid of exercise - Fears not breathing Pain in stomach Believes has ulcer Dx: Psychosomatic pain due to high stress Rec: See psychiatrist Patient believes she has no Psychological problem No PartialDizzy, Weak - Due to cold medication Dx: Sudden stopping of Inderal Rec: Continue Inderal - decrease dose Back on Inderal Weakness continues Before Interaction Physician Diagnosis & Recommendation After Interaction AdherenceAge
    26. 26. Personalized Healthcare and Adherence: Issues and challenges Signs and SymptomsLay Person Physician Interpretation of Patient Problem by Patient and Physician Wellness, Health Illness Disease Restore Normal Physiology Restore Daily Life Goals: Disrupt Daily Life Disrupted Physiology
    27. 27. Personalized Healthcare and Adherence: Issues and challenges Challenge 1 Understanding What the Patient is Thinking: Listening is Not Enough
    28. 28. Personalized Healthcare and Adherence: Issues and challenges
    29. 29. Personalized Healthcare and Adherence: Issues and challenges Cognition and the Challenge of Communicating across cultures Multiple medical systems Scientific education + cultural belief systems = ?
    30. 30. Personalized Healthcare and Adherence: Issues and challenges Cognition and the Challenge of Communicating across cultures REASONING ABOUT PROTEIN ENERGY MALNUTRITION Pictures of various forms of PEM Verbal description of symptoms Cognitive analysis Sivaramakrishnan, M. & Patel, V.L. (1993) Reasoning about childhood nutritional deficiencies by mothers in rural India: A cognitive analysis. Social Science & Medicine, 37(7), 937-952
    31. 31. Personalized Healthcare and Adherence: Issues and challenges
    32. 32. Personalized Healthcare and Adherence: Issues and challenges food to sit in the stomach no food for arms & legs less food (malnourished) Marasmus Kwashiorkor enlarged stomach tthin arms and legs CAU: CAU: CAU: ASSOC: Mother with No Formal Schooling throwing up of food indigestion Indigestion perceived as primary cause “Common sense” explanation for kwash belly
    33. 33. Personalized Healthcare and Adherence: Issues and challenges Explanation by a Mother with Secondary Level of Schooling Marasmus Jaundice Body and eyes become yellow Large stomach, big head, arms and legs Take to doctor to give injections Take to elders for traditional mantras, one per week for 8 weeks Give strict diet with no oil for one month Give lemon juice in cooked pulses and carrots Liver problem Food not digested Therapy based on local reasoning
    34. 34. Personalized Healthcare and Adherence: Issues and challenges
    35. 35. Personalized Healthcare and Adherence: Issues and challenges
    36. 36. Personalized Healthcare and Adherence: Issues and challenges Cough Syrup • Over-the-counter cough medicine used in the treatment of cough associated with common cold • Correct implementation involves a simple procedure but a complex calculation in determining the proper dosage • Subjects had to execute a complex quantification for a simple procedure Patel,V.L. , Branch T, Arocha JA (2002) Errors in Interpreting Quantities as Procedures: The Case of Pharmaceutical Labels. International Journal of Medical Informatics; 65:193-211.
    37. 37. Personalized Healthcare and Adherence: Issues and challenges Pharmaceutical Instructions for Cough Syrup Each teaspoonful (5ml) contains 15 mg of dextromethorphan hydrobromide U.S.P., in a palatable yellow, lemon flavored syrup. DOSAGE ADULTS: 1 or 2 teaspoonfuls three or four times daily. DOSAGE CHILDREN: 1 mg per kg of body weight daily in 3 or 4 divided doses.
    38. 38. Personalized Healthcare and Adherence: Issues and challenges Calculation of a Single Dose For a 22-Pound Child Calculations for a 22-pound child, where n= 3 : 1ハ teaspoon 5ハmillilitres x 5ハmillilitres 15ハ milligrams x 1ハmilligramハofハ medicine 1ハ kilogram ハofハbodyハweight x 22 poundsx 1ハ kilogram 2.2ハpounds x 1 totalハdaily ハdoseハinハmilligrams = 10 15 teaspoons = 2 3 teaspoon 2 3n teaspoon ntimes daily (where n = 3)= 2 3x3 tteaspoon, 3 times daily = 10ハkilograms 15x3ハtimes ハdaiily = 2 9 teaspoon, 3 times daily
    39. 39. Personalized Healthcare and Adherence: Issues and challenges Results: Cough Syrup – The majority of participants (66.5%) were unable to correctly calculate the appropriate dosage of cough syrup – Even when calculations were correct, they were unable to estimate the actual amount to administer – There were no significant differences based on cultural or educational background (except PhDs)
    40. 40. Personalized Healthcare and Adherence: Issues and challenges Dosage Accuracy for Cough Syrup Accuracy Percentage 0 10 20 30 40 50 60 70 Under dose Correct dose Slight overdose Extreme overdose English Indian Greek
    41. 41. Personalized Healthcare and Adherence: Issues and challenges Think-Aloud Quotation #1 One milligram per kilogram of body weight. So 13, 13 daily, 13 milligrams it would be. One milligram per kilogram? 13 kilograms let's say and one times 13 is 13 daily in three or four divided doses. So it would be four doses, I guess, three milligrams each? How many milligrams in a teaspoon? Oh, gosh, I said three milligrams. Each teaspoonful there's five to a teaspoon. Oh, I don't understand. Well, it wouldn't be a full teaspoon it would be more like half a teaspoon, four times a day.
    42. 42. Personalized Healthcare and Adherence: Issues and challenges Think-Aloud Quotation #2 27 kilos, 27 milligrams? 27 milligrams daily. Each teaspoon is 15 milligrams so you want two teaspoons a day divided by four. Hold it, two teaspoons a day divided by three or four doses. So you are talking either one-half to two-thirds a teaspoon, depending on whether it's three or four times a day.
    43. 43. Personalized Healthcare and Adherence: Issues and challenges Think-Aloud Quotation #3 Each teaspoon contains 15 milligrams. Okay, for the children I have to write the weight of the baby, 10 kilograms. I will give him only two teaspoons for the day. Yeah, I give him only two teaspoons, two times a day, because you see here 15 milligrams is one teaspoon. So, one teaspoon is five ml contains 15 milligrams? Yeah, that's it. I give him only two, two teaspoons.
    44. 44. Personalized Healthcare and Adherence: Issues and challenges
    45. 45. Personalized Healthcare and Adherence: Issues and challenges Summary and Challenges to Personalized Health Care How we think and make decisions create certain challenges for successful acceptance of personalized health care – Nature and use of evidence by lay public is different from designers of systems providing care: Understand the users – Understanding what the patient is thinking: Listening is not enough: Thinking behind behavior (cognition) – Traditional belief systems are powerful and they influence decisions about individual health: Work within the system rather than against them
    46. 46. Personalized Healthcare and Adherence: Issues and challenges Personalized Healthcare & Adherence: the role of technology Bern Shen MD MedInfo – Copenhagen 22 Aug 2013 Panel Discussion (2):
    47. 47. Personalized Healthcare and Adherence: Issues and challenges Bern Shen, MD, MPhil. • Chief Medical Officer, HealthCrowd • Member, Band of Angels • Board of Directors, Clayton Christensen Institute for Disruptive Innovation • Board of Directors, Univ. of Iowa Research Foundation • Board of Advisors, Univ. of Iowa College of Public Health • Adjunct Assoc. Prof., Univ. of Iowa Colleges of Medicine & Business • Adjunct Asst. Prof., UCSF School of Pharmacy • Ex-Board Chair, The Health Trust • Ex-Intel (Chief Healthcare Strategist), Oracle, HP, UCSF, UPMC, Yale • Ex-Health Practice Lead, Institute for the Future 47
    48. 48. Personalized Healthcare and Adherence: Issues and challenges Agenda • Personalization • Adherence • Some lessons learned from real-world projects 48
    49. 49. Personalized Healthcare and Adherence: Issues and challenges Agenda • Personalization • Adherence • Some lessons learned from real-world projects 49
    50. 50. Personalized Healthcare and Adherence: Issues and challenges Healthcare more distributed & differentiated 50 One size fits all (e.g., blockbuster drug, “70kg patient”) Personalized medicine
    51. 51. Personalized Healthcare and Adherence: Issues and challenges Many ways to personalize Policies & interventions Access to quality healthcare Health statusHealth status Behavior Biology Physical environment Social environment Technology BehaviorBehavior TechnologyTechnology 51
    52. 52. Personalized Healthcare and Adherence: Issues and challenges Reception Acceptance Intentions Action • Past behavior • Demographics & culture • Perceived health threat & susceptibility • Perceived benefits & barriers • Stereotypes & stigma • Personality, moods & emotions • Other individual differences • Media or intervention exposure • … • Poverty • Entrenched adversaries • Built environment • Geographic barriers • … Internal External Skills & abilities Environmental constraints Logic model 52 Time frame for impact Weeks Months Years Clinical & financial outcome metrics •Biometrics – e.g., weight, BP, glucose •Visits to clinic, or ED •Hospital admissions •Procedure volumes •Utilization & cost savings •Newly enabled business models Behavior change & subjective indices •Measures of self-efficacy •Measures of disease knowledge •Quality of life, depression •Productivity, absenteeism Process & proxy measures •Engagement & response rates •Number & types of health behaviors •Program completion rates, f/u appointment no-shows •Health screening, HEDIS measures •Streamlined workflow Coaching Patient Outcomes RemindersNew info MotivatorsReinforcers Intelligent text messages
    53. 53. Personalized Healthcare and Adherence: Issues and challenges Agenda • Personalization • Adherence • Some lessons learned from real-world projects 53
    54. 54. Personalized Healthcare and Adherence: Issues and challenges Why behavior matters • Adherence is behavior • High vs. low adherence → ~26% difference in outcomes • Behavior is a major determinant of health 54 000’s of deaths/year Source: : Mokdad, et al. 2004. Actual causes of death in the United States, 2000. JAMA. 2004;291:1238-45.
    55. 55. Personalized Healthcare and Adherence: Issues and challenges Yet behavior is largely “invisible” 55
    56. 56. Personalized Healthcare and Adherence: Issues and challenges 56 Source: Green, L. A., et al. (2001). "The ecology of medical care revisited." N Engl J Med 344(26): 2021-5.
    57. 57. Personalized Healthcare and Adherence: Issues and challenges Behavior not only individual 57
    58. 58. Personalized Healthcare and Adherence: Issues and challenges Agenda • Personalization • Adherence • Some lessons learned from real-world projects 58
    59. 59. Personalized Healthcare and Adherence: Issues and challenges 59 Mobile Personalization +
    60. 60. Personalized Healthcare and Adherence: Issues and challenges 60
    61. 61. Personalized Healthcare and Adherence: Issues and challenges Evolution of personalization 61 Targeting & Optimization Outcomes Analytics Dynamic Content
    62. 62. Personalized Healthcare and Adherence: Issues and challenges
    63. 63. Personalized Healthcare and Adherence: Issues and challenges People are different 63 Individual stopped responding – but still wanted to receive messages Highlights the importance of the right outcome measures ~40% overall response rate
    64. 64. Personalized Healthcare and Adherence: Issues and challenges Timing matters 64 <1 week: 58% response rate >2 weeks: 31% response rate
    65. 65. Personalized Healthcare and Adherence: Issues and challenges 65 Hypothesis: Can a mobile messaging program reduce dropout & increase program adherence among cardiac rehab patients? This analysis was performed using data from our program, supplemented with data from respected sources to estimate cost savings. Analysis: Results: Compared to the control group, patients in the intervention group demonstrated 2x rehab completion, ¼ no-show rate at 3-month follow- up appointment, better exercise tolerance, & lower depression scores Increased rehab completion rate reduces utilization meaningful savings→ Entity: Hospital Population: Commercial + Medicaid; Urban + Rural Average age: 55 n = 100 English level: 6th grade Benefit to hospital: Reduced 30-day readmissions for AMI, CABG, Stent, CHF Cost savings: ~ $1,300 annual savings per member
    66. 66. Personalized Healthcare and Adherence: Issues and challenges 66 sensemaking Source: Institute for the Future "Both builders and users of… systems tend to think of them simply as technical tools or problem-solving aids, assuming them to be value-free. However, …the reasoning embedded in such systems reflects cultural values and disciplinary assumptions, including assumptions about the everyday world of medicine.” - D. Forsythe. 2001. Studying Those Who Study Us: An Anthropologist in the World of Artificial Intelligence. p. 93. 1980 1990 2000 2010
    67. 67. Personalized Healthcare and Adherence: Issues and challenges Agenda • Personalization • Adherence • Some lessons learned from real-world projects 67
    68. 68. Personalized Healthcare and Adherence: Issues and challenges Personal Informatics for Wellness: An Interactive Analytics Framework for Computer-Supported Collaborative Prevention Pei-Yun Sabrina Hsueh, PhD, MIMS MedInfo – Copenhagen 22 Aug 2013 Panel Discussion (3):
    69. 69. Personalized Healthcare and Adherence: Issues and challenges Pei-Yun (Sabrina) Hsueh, PhD Research Data Scientist, Wellness Analytics Lead, Health Informatics Research Group IBM T. J. Watson Research Center  IBM Invention Achievement Awards, Organization Committee of Academy of Technology Conference & Healthcare and Life Science Distinguished Speaker Series  Google European Anita Borg Scholar  Program Committee, ACM HLT, NAACL, EACL, CODATA Chronic Disease Management and Independent Living for the Aged  Invited Session Chair, AHFE, ISREC, IEEE SOLI, CollaborateCom  Board of Director, Chinese Institute of Engineers Greater New York Chapter  National Science Council Merit Award  Book Chapter/Journal Review: Artificial Intelligence, IGI Global Privacy Protection Technologies in Business Organizations, IEEE Intelligent Systems, Transactions on Knowledge and Data Engineering, Statistical Analysis and Data Mining, IEEE Selected Topics in Signal Processing, Journal of Natural Language Engineering PhD in Informatics, University of Edinburgh; Masters in Information Mgmt, University of California, Berkeley; Bachelor in Computer Science, National Taiwan University
    70. 70. Personalized Healthcare and Adherence: Issues and challenges Healthcare becoming Personal 1990 Empirical Medicine Intuitive Medicine Disruption will involve pushing more medicine into the precision category. ~ Clayton Christensen “The innovator’s Prescription” Disease-Centric Guideline Precision Medicine Degree of personalization Degreeofcollaboration (datadimension) Data-Driven Evidence All patients with same diagnosis Patients receiving alternative treatment Patients receiving traditional treatment
    71. 71. Personalized Healthcare and Adherence: Issues and challenges From Genetic Determinants of Health to Personalized Medicine and Prevention • Personalized Medicine • Personalized Prevention
    72. 72. Personalized Healthcare and Adherence: Issues and challenges 72 Introducing Data-Driven Analytics into Personalized Services: Improved Outcome and Reduced CostIndividualized Guideline Improved Clinical Outcomes  ≈ 13% absolute risk reduction Individualized Guideline Reduced Operational Costs  ≈ 6,000 myocardial infarctions (MIs) and strokes prevented annually * $7,000 cost savings per person  ≈ 420M US dollars saved in a US provider alone Source: Eddy, et al. (2011). Individualized Guidelines: The Potential for Increasing Quality and Reducing Costs. Annals of Internal Medicine, vol. 154, no. 9, p.627-634. http://www.annals.org/content/154/9/627.abstract 72
    73. 73. Personalized Healthcare and Adherence: Issues and challenges Delivery (Care flow determinant) Nature (Endogenous determinant) (e.g., genetics predisposition) Nurture (Exogenous determinant) (e,g, environment exposure, behavior pattern, social circumstances) 30% 10% 60% Cardiovascula r disease (73-83%) (NHS, NEJM 2000) Cardiovascula r disease (73-83%) (NHS, NEJM 2000) Type II Diabetes (58-91%) (Finland DPS, NEJM 2001, 2007) (US NHS, 2000; CDC DPP, 2002) (China Da-Qing, 2001) Type II Diabetes (58-91%) (Finland DPS, NEJM 2001, 2007) (US NHS, 2000; CDC DPP, 2002) (China Da-Qing, 2001) Cancer (60-69%) (HALE, JAMA 2004; de lorgeril Arch Intern Med, 1998) Cancer (60-69%) (HALE, JAMA 2004; de lorgeril Arch Intern Med, 1998) Personalized MedicinePersonalized Medicine Personalized CarePersonalized Care Personalized Prevention and Disease Management Personalized Prevention and Disease Management Eye complication (76%), Kidney complication (50%), Nerve complication (60%) (UKPDS, US DCCT) Eye complication (76%), Kidney complication (50%), Nerve complication (60%) (UKPDS, US DCCT) Cardiovascular complication (42-57%) (UKPDS, US EDIC) Cardiovascular complication (42-57%) (UKPDS, US EDIC) Holistic View of Determinants of Health to Personalized Services Huge opportunity space for risk reduction: Progress impeded by the lack of efficient personalization and validation techniquesProgress impeded by the lack of efficient personalization and validation techniques SA Schroder. We can do better - Improving the Health of the Amarican People. NEJM 2007;357:1221-8.
    74. 74. Personalized Healthcare and Adherence: Issues and challenges 1990 Empirical Medicine Intuitive Medicine Personalized Service Patient-Centric Guideline Disease-Centric Guideline Precision Medicine Degree of personalization Degreeofcollaboration (datadimension) Data-Driven Evidence Century of behavior change Healthcare becoming both Personal and Collaborative: Two Concepts to be Introduced… Amazon, Netflix, Pandora, and iTunes v.s. Wellness service providers. What is the middle ground? ….. Thinking in the line of “Mass Customization”… What these companies do not have? ….. Thinking in the line of “Personal Informatics”…
    75. 75. Personalized Healthcare and Adherence: Issues and challenges System Approach to the Personalization Problem Personal wellness data model + Hospital HIS
    76. 76. Personalized Healthcare and Adherence: Issues and challenges 76 Longitudinal record Personal Wellness Record Personal Wellness Record Personal Wellness Record Clinical Requirement Guideline Diabetes CHF Cardiovascular Metabolic Core Service Flow (As-Is Process) 76 76 Monitoring Monitoring Sensor Transactionfeed SmartDeviceApp Compliancefeed Clinical Activity Nutrition Follow-up stage 76
    77. 77. Personalized Healthcare and Adherence: Issues and challenges Value Proposition of Personalized Service Core Service Flow (As-Is, To-Be) As-Is Process As-Is Process Personalized Service Personalized Service Customer/Customer/ PatientPatient To-Be Process: Personalized Services (Consultation, Follow-Up) To-Be Process: Personalized Services (Consultation, Follow-Up) Consultation stage Theme #2 Theme #2 Follow-up stage Theme #3 Theme #3 Theme #1 Theme #1 Value-adding process Adherence
    78. 78. Personalized Healthcare and Adherence: Issues and challenges Adherence service – snapshot
    79. 79. Personalized Healthcare and Adherence: Issues and challenges Personalization: Core Issues Addressed & Remaining Questions Remaining questions in each issueRemaining questions in each issue • Individual difference How to measure and validate? What are the missing information at individual level? • Actionable recommendation • How to translate dynamically changing, multi-faceted adherence factors into a patient-centric view? • How to account for multiple dimensions of wellness decision making? • Adherence risk • How to model incremental response? • How to create differential response to adherence exceptions in absence of individual outcome data? 1 2 3 Customer/Customer/ PatientPatient Adherence Theme #1 Theme #1 Theme #2 Theme #2 Theme #3 Theme #3 Personalization for risk stratification (from population to individual evidence) Personalization for risk stratification (from population to individual evidence) Personalization for in- context recommendation (from disease-centric to patient-centric) Personalization for in- context recommendation (from disease-centric to patient-centric) Personalization for adherence risk mitigation (from status-insensitive to status-sensitive) Personalization for adherence risk mitigation (from status-insensitive to status-sensitive)
    80. 80. Personalized Healthcare and Adherence: Issues and challenges • Individual difference • How to measure and validate? • What are the missing information at individual level? • Actionable recommendation • How to translate multi-faceted disease risks into a patient-centric view? • How to account for multiple criteria of wellness decision making? • Adherence risk • How to model incremental response? • How to create differential response to adherence exceptions in absence of individual outcome data? Personalization: Core Issues Addressed & Analytics Components Remaining questions in each issueRemaining questions in each issue Personalization analytics in responsePersonalization analytics in response 1 2 3 Quantifiable individual difference Actionable recommendation Risk-adverse intervention target Patient-centric outcome- maximizing Patient-centric outcome- maximizing Lift modeling for proactive risk mitigation Lift modeling for proactive risk mitigation
    81. 81. Personalized Healthcare and Adherence: Issues and challenges Summary  Importance of personalization in patient engagement  Clinical touch point identification for mass customization  Incorporating personal informatics tools for user modeling  Innovation opportunities in care delivery and patient engagement models  Interactive Analytics Scheme for Personalization services  PWR + HIS  Analytics Engine: Summarization  Customization  Engagement  Interaction Engine: user modeling  important attribute solicitation/self- assessment  Instant outcome measurement and feedback generation  Ongoing in-market experiments (pilots)  Personalized engagement and customer “stickiness”  Invention: Crowd-sourced DB and dynamic accretion of questions based on patient status estimation  Sustainable value capture?
    82. 82. Personalized Healthcare and Adherence: Issues and challenges Framework to accelerate personalized service design Technologies to enhance wellness services – Guide the identification of customization points in clinical workflow and deployment of the Analytics and IM offerings – Create new tools and infrastructure for client engagements – Explore light-weight approach to connect the components (to prepare for future cloud offerings) New solutions and services – Bring together clients and researchers to understand clinical touch points – Demonstrate how to leverage customization points to engage users and possibly improve health literacy and outcomes Replicable patterns for patient engagement deployment – Create ETL procedures to be repeatedly use in other provider settings – Explore both hosted and internal deployment possibilities Plug-in for other tools – Create a recipe from data collection to summarization to customization to engagement to outcome measurement – Each component can be singled out as a standalone process for other tools
    83. 83. Personalized Healthcare and Adherence: Issues and challenges Thank You Merci Grazie Gracias Obrigado Danke Japanese English French Russian German Italian Spanish Brazilian PortugueseArabic Traditional Chinese Simplified Chinese Hindi Tamil Thai Korean Hebrew

    ×