Pharmabroad a companion chatbot for identifying pharmaceutical products when traveling abroad
•Download as PPTX, PDF•
0 likes•24 views
ENTER2020 eTourism Conference presentation
Recommended
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Pharmabroad a companion chatbot for identifying pharmaceutical products when traveling abroad
- 1. 1 Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad 08/01/2020 – University of Surrey, UK Sammarco Matteo Ruf Boris (Axa), Sammarco Matteo (Axa), Jonathan Aigrain (Axa), Marcin Detyniecki (Axa, Polish Academy of Science)
- 2. 2 A lost in translation story An Italian tourist in France Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad
- 3. 3 A lost in translation story A French tourist in Italy Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad
- 4. 4 More complex situations Different language and alphabet Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad
- 5. 5 Contribution Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Match local pharmaceutical products with travelers’ home country products Pervasive and inexpensive Smartphone ready Simplified interaction (chatbot) Manage stressful situations No need to write product name in different alphabet (OCR) World-wide scalability
- 6. 6 System design Interface/Dialog flow Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Interface/ Dialog flow Core service Matching algorithm
- 7. 7 Medical box image I OCR(I) Host country DB lookup Home country DB lookup Geolocation Host country DB available ? No Yes medicinename ! molecule molecule ! medicinename T(OCR(I)) System design Core service Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Interface/ Dialog flow Core service Matching algorithm
- 8. 8 System design Matching algorithm Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Interface/ Dialog flow Core service Matching algorithm
- 9. 9 Matching algorithm Local drug DB available Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Candidate 1 Candidate 2 Local drug DB (L) OCR Home drug DB (L) Levenshtein distance < 𝜏 Result: molecule Looking for drug name
- 10. 10 Matching algorithm Local drug DB not available Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Home drug DB (L) OCR Molecule translation Levenshtein distance < 𝜏
- 11. 11 Matching results Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Local drug DB available Local drug DB not available OCR mistake OCR mistake Smaller fonts Translation error
- 12. 12 Implementation Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad OCR: OCR.space, Microsoft Azure Vision Service NLP: Microsoft Azure Bot Service Translation: Microsoft Translator Text API Drug database: ChEMBL, AIFA (Agenzia Italiana del Farmaco), FDA (Food and Drug Administration)
- 13. 13 Conclusion Pharmabroad: A Companion Chatbot for Identifying Pharmaceutical Products When Traveling Abroad Black box service Good balance between precision and recall (F1-score = 0.87) with low Levenshtein distance (𝜏 = 3) when the local country drug DB is available Getting the molecule directly is more challenging (F1-score = 0.66) Embed vision and translation services inside the mobile application (no network connection needed)
- 14. 14 Thank you
Editor's Notes
- Hello, I’m glad to present you today Pharmabroad, whici is a system with a chatbot interface to identify pharmaceutical products when traveling abroad.
- This work began from a personal experience: 10 years ago I moved from Italy to France and I packed in my bag some basic medicaments. But when traveling you remain long enough abroad and you run out of personal stock or you forgot to bring something with you or you just didn’t include it in the travel check list, you need to buy some local medical products. In France, eventually, you’ll come across a medicament called « Doliprane » and I had to take some pills of Doliprane too. So I discovered which is simply made by paracetamol as molecule and used maily for fever treatment, or headache or just some pain. It can by bought without medical presciption.
- Reading further the warning label, for the chemical composition, side effects, posology and so on, I discovered that the Doliprane is the equivalent of what, at contrary, a French turist staying in Italy can find as Tachipirina. This is a kind of toy case scenario, as the principal molecule was written clearly on the box, but in many cases it’s not so simple.
- In fact, we can travel in contries where the spoken and written language is different, so that the warning label could be impossible to decrypt. Even the alphabet could be different and it would result impossible for us even to write the name to google it. In addition, when you need some medicines it’s supposed you have some pain or you are sick and it’s a stressful. moment in general.
- This is why we designed Pharmabroad: to match pharmaceutical products from the host country to the home country products. As requirements, we wanted it to be pervasive and cheap: so basically it’s a smartphone application With a simplified interaction: thus we adopted the a chatbot paradigm with a minimal interaction with the user, without even the need to write the medicament name And world-wide scalable.
- More in detail, I divide the system in 3 parts: The HM interface with the dialog flow The core service The matching algo This is the higl level interaction schema… Other intents: I lost my luggage, am I insured for this…
- RESULT: The algorithm can return either the right correspondent drug, or ”nothing” which is a False Negative, or a different drug with is a False Positive.
- Results are achieved using only one image for each pharmaceutical box and figures used in the experiments have a much lower resolution wrt usual smartphones.
- Experiments show that we can get a good balance between precision and recall for medicine name extraction from drug boxes images (F1 score = 0 . 87) keeping a low Levenshtein distance ( τ = 3)