How to get localization and testing for medical devices done right
•Download as PPTX, PDF•
1 like•463 views
This webinar discussed challenges, lessons learned, and solutions for medical device localization and testing. Key challenges included the need for domain expertise, managing sensitive client environments and test data, and meeting FDA and EU language requirements. Lessons highlighted the importance of domain knowledge, effective engagement models, and using the right tools. Solutions presented included the Virtual Radiology Environment tool for automated testing, frameworks for data-driven testing, and industry standards and tools for localization.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to How to get localization and testing for medical devices done right
Similar to How to get localization and testing for medical devices done right (20)
More from Qualitest
More from Qualitest (20)
Recently uploaded
Recently uploaded (20)
How to get localization and testing for medical devices done right
- 1. How to get localization and testing for medical devices done right
- 2. Webinar Presenters Nausheen Sayed VP Knowledge and Innovation US-UK David Sommer Director of Strategic Operations
- 3. Agenda Challenges Lessons Learned Introduction SummarySolutions and Tools Q &A
- 4. Agenda Challenges Lessons Learned Summary Introduction Solutions and Tools Q & A
- 5. About QualiTest Group | World’s 2nd Largest Independent Testing Company | QualiTest recognized for each of the past 3 years as the only visionary pure play QA and Testing Services Company in the “ Magic Quadrant”
- 6. Domain background |The Medical and Healthcare domain can be a vast playground to offer testing services |At QualiTest, we offer a variety of testing services that comply with FDA regulations |Some of our clients including Fujifilm, GE Healthcare, Philips, Carl Zeiss, THINK Surgical and Accuray | For more info visit our website
- 7. About Net- Translators |Global Multilingual Localization Provider |Est: 2001 |Coverage & Solutions: |Over 60 languages |Solution oriented provider of language and technology solutions |Certifications: |ISO 9001:2008, ISO 13485:2003, ISO 17100:2015-05 and CAN/CGSB-131.10-2008 |Major Locations: |United States, Israel, Argentina
- 8. Agenda Challenges Lessons learned Summary Introduction Solutions and Tools Q & A
- 9. Medical Device Testing and Localization |Domain knowledge |FDA rules and regulations |Sensitive Client environment |Test Data–management, compatibility, standards like DICOM |Flexible, light Test tools for every stage: design, development and V&V |Compatibility testing |Localization testing Challenges
- 10. EU Language Requirements for MDD |Requirements to translate safety related materials |Manufacturer ultimately responsible for translation |Medical Device Directive 93/42/EEC article 4.4: |Annex I, point 13 • labels and display, safety instructions and instructions for use |In-Vitro Diagnostic Directive 98/79/EC article 4.4
- 12. Lessons learned Some of our case studies and experience indicate: |Domain knowledge and training - very essential |Effective engagement models |Test Data - very critical, saves time and money if done right |Nativity and fluency in language of engineers for localization testing |Right selection of tools
- 13. ISO 17100:2015 Translation Standard |Translation services - Requirements for translation services |Resources: • Human • Competencies and Qualifications • Technical |Pre-production processes and activities: • Feasibility • Quote • Project and client specific information • Project preparation |Production process: • Project Management • Process
- 14. ISO 17100:2015 Translation Standard |Production process |Project Management |Process: • Translation • Check • Revision • Review • Proofreading • Final verification and delivery |Post-production processes |Feedback |Closing |Confidentiality
- 15. Agenda Challenges Lessons learned Q & ASummarySolutions and Tools Introduction
- 16. Solutions and Tools |What to automate? |Regression tests, memory leaks, memory consumption, input fields testing | How to automate? |Automation tool selection • Revision control and document tracking • Analysis and reporting • Learning curve for tool and maintainability in future • Costs and ROI • Mutli threading |Data driven frameworks |Pseudo code generation
- 17. Test Case Pseudo Code Eggplant Functions 1) DMS Record Test Environment. 2) Launch System Administration Launch System Administration. Func_OpenProcessFromHomeScreen 3) Click the DICOM Hosts Tab From the System Administration, click on the DICOM Hosts tab. Func_ClickImage 4) Select the + icon to create a New DICOM Host Click on the + button to create a new host. Func_ClickImage 5) Enter Host Name Type the new host name. Func_AddNewHost 6) Enter AE Title Type the AE Title Func_AETitle 7) Enter IP Address Type the IP Address Func_IPAddress Test Script 14423 DICOM: Destination Configuration
- 18. Virtual Radiology Environment Tool | Qualitest‘s Virtual Radiology Environment (VRE) is a comprehensive testing solution, aimed to test your Picture Archive Communication Systems (PACS) for scalability and reliability, delivering a comprehensive turnkey solution for synchronized Digital Imaging and Communications in Medicine (DICOM) and Health Level 7 (HL7) Traffic | VRE is a simulation system that provides all of the network patterns and data flow that occur natively in the Radiology domain | VRE effectively reduces time, personnel needs, and a substantial Return on Investment (ROI) without sacrificing accuracy or reproducibility
- 19. Case study on VRE |Client Overview and Business Need: | The client is a diversified technology company with primary divisions focused in the areas of healthcare, consumer lifestyle and lighting | The main issue was managing the memory for a client viewer while updating and processing of DICOM images of a given modality, size and slice count |QualiTest Solution: | We proposed the VRE tool and performed a free-of-charge Proof of Concept, demonstrating the benefits and assuring quality and interoperability while simplifying and dramatically speeding-up all aspects of V&V testing |Key Benefits | VRE reduced testing time by 33% | VRE reduced personnel needs by 90% | The paperwork and coordination for executing such a task were greatly simplified
- 20. Application Strings User Documentation Help Desk Digital Content Process Strategy Web Marketing Content prepare translate QC return extract
- 21. Key Parts of the Process |Content review |Terminology management |Glossary creation |Glossary translation |Training |Translation, editing and proofreading |Post process |DTP |Testing |In-context checks |Final review
- 22. Industry Tools |Computer Assisted Translation tools (CAT) |SDL Trados Studio |SDL Passolo |XTM |Translation Platforms |Net-Proxy |Net-Cloud |Desktop Publishing |Adobe |Articulate |Microsoft |Single Source |DITA
- 23. Agenda Challenges Lessons learned Q & ASummarySolutions and Tools Introduction
- 24. |Are you keeping up with the domain knowledge? |Are you using the right tools to replicate a live environment? |Are you automating efficiently? |How are you engaging with your testing teams? |Are you managing your risks or exposure? Summary
- 25. Agenda Challenges Lessons learned Q & ASummarySolutions and Tools Introduction
- 26. David Sommer – david.sommer@net-translators.com Nausheen Sayed - nsayed@qualitestgroup.com
Editor's Notes
- Nausheen role: manage the US-UK company’s knowledge assets to provide innovations and solutions to our customers, knowledge extraction/insertion, training of employees. David role: David Sommer 13 years @ Net-Translators Huge baseball fan And I am David Sommer the Director of Strategic Operations at Net-Translators, my position today is to help our customers achieve optimal results by acting as a bridge between operations and sales. I have been with Net-Translators for the past 13 years where I have previously been a project manager, and finally I am looking forward to the world Baseball classic due to start in March.
- Read the Agenda During the webinar, I will focus more on medical device testing while David will elaborate on medical device localisationa
- Read the Agenda
- Very brief intro abt qt in the medical domain The Medical and Healthcare domain can be a vast playground to offer testing services - there is radiology and imaging software, pharmaceuticals and their portals, healthcare, health insurance and hospital applications, ICD testing, localization and translation etc. At QualiTest, we offer testing services in all of the above that complies with FDA regulations and help create a wide range of healthcare testing services and medical software verification and validation services including functional, automation and performance. Some of our clients including Fuji Film, GE Healthcare, Phillips, Carl Zeiss, Think Surgical and Accuray, have attested to our services and supported us in publishing case stories. Over to net translators
- Thank you Nausheen, just a brief overview of Net-Translators, we are a global MLV specializing in the technology sectors, with our main vertical being the Medical Device Industry. An expression of our expertise is found in our multiple international quality certifications which include in addition to ISO 9001, also ISO 13485 which represents the requirements for a comprehensive quality management system for the design and manufacture of medical devices. ISO 17100 which provides requirements for the core processes, resources, and other aspects necessary for the delivery of a quality translation service that meets applicable specifications. And The new Canadian national standard that sets out the procedural requirements for delivering translation services that suppliers of translation services to the federal government have to adopt. We have offices in the US, Israel and Argentina, and have a solution oriented approach to providing our customers with the best solution for their language needs.
- Read the Agenda
- Domain knowledge – Engineers testing the medical software have zero to very little knowledge of domain be it radiology, ICD-10 codes, PACS system architecture or DICOM stds, hence a lot of time is lost in education. Cant share the knowledge. Understanding FDA regulations for countries, ISO stds- keeping them in mind while designing tests, high tracebility, detail to documentation vs time is a lot to handle. Compliance when it comes to patient information, clerical data, security constraints across systems/environments. Hence need for light tools for setup/installation on client env Managing test data , their comptabilty across platforms, and to maintain their integrity across platforms Compatibility testing - Medical devices are integrate into hospital networks, transfer data and have CRUD access to records Localization testing – tho not very different than web applications, localization in medical domain becomes more challenging with the above challenges. David from net translators can elabortate further
- To fully understand localization and translation in the Medical Device industry, we need to take a look at the various regulations which govern these requirements. The regulation that probably have the largest impact are probably those mandated by the EU, and this is simply due to the fact that these regulations cover more languages than any other. Device manufacturers that intend to place their products in the European market must carefully comply with all essential requirements in Annex I to Directive 93/42/EEC on medical devices, as well as with the member states' own legislation. The directive provides that member states may require the accompanying information to a medical device to be in their national language or in another community language. All member states require that safety information be provided in their official language to ensure understanding by the final user. The penalties for noncompliance vary from the removal of the product from the market and in some cases, to criminal proceedings. The overriding guidance is that anything which deals with safety of the device must be made available in the users language. The directive's main objective is to harmonize national provisions regarding the safety, health protection, and performance characteristics of medical devices, including the relevant certification and inspection procedures. The directive allows member states to "require the information, which must be made available to the user and the patient to be in their national language or languages or in another Community language, when a device reaches the final user, regardless of whether it is for professional or other use.” Of course each country may also have its own regulations vis-à-vis language requirements, for example The UK Medical Devices Regulations provide that devices placed on the market or put into service must comply with relevant requirements. "the essential requirements specified with regard to information on the packaging and on any label are complied with only if such information is in English (whether or not it is also in another language and whether or not the device is for professional use)." However, instructions for use may be in another EU language, provided that there is "a clear statement in English stating the language in which instructions are given" The Netherlands, provides that "the information, which must be available to the patient, must be in the Dutch language, meaning not necessarily the caregiver. And Sweden provides that all devices that fall under the scope of Swedish law must be labeled in Swedish note that this seems to include patient facing and care-giver facing devices. As for the far east in Japan, medical devices are regulated under Japan's Pharmaceutical Affairs Law. To market a medical product in Japan, a manufacturer must obtain shonin (manufacturing product approval) for high-risk devices ensuring the safety and effectiveness of the product. Medical equipment falls into one of three categories depending on the degree of potential risk—highly controlled (high-risk), controlled (low-risk), and general (very low risk). Typically, all documents submitted for a shonin application, as well as all safety documents, must be in Japanese. In China devices undergo a registration process based on classification. Class III devices, for example, require testing by a Chinese laboratory. All safety information must be in Chinese for high-risk devices. Medical devices must provide instructions for use in Mandarin Chinese translation when importing into China. The translation must be done before the product is allowed to enter the country.
- Read the Agenda
- Due to several challenges, clients have very limited resources available to support knowledge transfer and training of software, hardware and clinical elements of the system. And often engagements require different roles in various technical and domain expertise, from functional testers to system engineers, and each role required a different training plan. Reinventing the wheel every time is an inefficient use of time and knowledge. Hence we learned its important to Train the Trainers ie investment in training and a knowledge base that then transforms into a comprehensive training plan. Due to compliance we cannot share customer data or info but we can provide the basic tools like strong and detailed functional testing skills, background on FDA rules and regulations, education on terminology, list of best practices and criteria for tool selection. Your leads, line managers become your experts and contributors. Gather experiences from around the globe, and then see how the knowledge flows into and out of this knowledge system facilitating quick learning and ramp up/ So when working with a client who is one of the largest international healthcare corporations, on tight schedules with safety-critical and real-time requirements, we learned that an MTS - assuming full responsibility for the project, delivering results based on a pre-defined service-level-agreement (SLA) and measured on agreed key performance indicators (KPIs worked best for us and them – as we like to say leave it to the experts. Also when we talk abt engagement models, we learned that on-shore/ on-site engagement worked very well ie having engineers operate through a local Test Center that was located in close vicinity to the customers premises, as well as on the customer premises per customer need. This enabled better collaboration. For one of our clients Ministry healthcare was updating and migrating its Hospital Network to ICD-10, The International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) is a system used by physicians and other healthcare providers to classify and code all diagnoses, symptoms and procedures recorded in conjunction with hospital care in the United States. A simple and cost effective solution was to create a Golden Data Test set for testing - a set of dual-coded medical records (both ICD-9 and ICD-10). The Golden Data is peer-reviewed and validated by clinical/coding experts. Not only provided accurate revenue impacting testing but also least expensive approach to end-to-end ICD-10 testing. Something to keep in mind for (ICD-11) is due by 2018! 5. For efficient localization testing, fluency and nativity in language being tested for helps, hence designing test teams with atleast one engineer native and fluency in say Russian, French, hindi, Turkish is very helpful specially when it comes to creating test data and edge scenarios – as simple as the longest word that will test the cut off in a form field. 6. Right tooling for every stage of testing is very important. For example when working with Some clients do not prefer heavy installation on client workstation. Hence we researched and recommended a light weight automation tool where only a small agent is needed to be installed – nothing heavy on client side. Plus this tool was able to recognize a lot of 3rd party UI components that other tools were unable to. Plus it worked with iPads as well. so an image based automation tool that was able to integrate with the TMT and jira did the trick than going for open source options
- So knowing the challenges that face MDD manufacturers vis-à-vis language requirements, and here it is important to point out that the underlying reason for these requirements is to ensure patient safety, and in business terms to reduce potential for loss due to liability, we need to see how we can meet these language needs of the customer. A good place to start would be to make sure that any vendor that you engage with be certified at the minimum to ISO 13485, as well as ISO 17100 the translation standard. I want to take a look at why this is the case. Often for customers translation is a black-box, and there is no real way to be certain that the quality is sufficient for their purpose. So like any other industry the proper processes need to be ensured, and the way to achieve this is to look at the proper processes and make certain that your vendor follows these processes. There is no better reference that the ISO 17100 standard, This International Standard specifies requirements for all aspects of the translation process directly affecting the quality and delivery of translation services. It includes provisions for translation service providers (TSPs) concerning the management of core processes, minimum qualification requirements, the availability and management of resources, and other actions necessary for the delivery of a quality translation service. The Standard is a comprehensive manual and addresses all the major aspects of quality. It begins by discussing the basic materials which means the resources involved. The translators and reviewers should have translation competence including Linguistic and textual competence in the source & target languages, research competence, cultural competency, domain competence, obviously extremely important in as far medical devices are concerned and of course technical competency as the industry today is highly technical and while there is no requirement that automated tools be used, the reality of the industry is that these tools are making the work much more efficient. The standard also sets forth translator qualifications such as the requirement to possess a graduate degree. It further discusses requirements for Project Managers as well as technical and technological requirements. The standard takes an holistic approach to the subject of translation, this is due to the fact that so many factors can affect the final outcome and in the pre-production phase includes determining the feasibility of the project, for example is this a rush project, and if so are the risks too great in order to accept it? Is there sufficient budget on the part of the customer to carry out all of the required steps? Do we have enough information about the specific study or project that we are localizing? Are there translation assets in place such as glossaries and terminology databases? The standard also spells out very specifically the required processes/
- This process includes multiple stages of activities and controls such as Translation, Editing, Proofreading checks verifications and QC. And in fact the standard also covers post production issues such as ICR feedback, implementation archiving and data management. Finally,
- Read the Agenda
- We talked about selecting the right automation tool. In addition we would recommend a few more capabilities to keep in mind: Revision control and document tracking Analysis and reporting Learning curve for tool and maintainability in future – this is where we come into picture – we already know the tools, we can set the test env, install, create scripts, train your staff to run and maintain in future Costs – licensing costs Mutli threading else cost of running extra machines for parallel testing Whichever tool you chose its very important that we Always generates a pseudo code document for the tests/scripts
- We talked about selecting the right automation tool. In addition we would recommend a few more capabilities to keep in mind: Revision control and document tracking Analysis and reporting Learning curve for tool and maintainability in future – this is where we come into picture – we already know the tools, we can set the test env, install, create scripts, train your staff to run and maintain in future Costs – licensing costs Mutli threading else cost of running extra machines for parallel testing Whichever tool you chose its very important that we Always generates a pseudo code document for the tests/scripts
- Usage: Testing (e.g. automatic tests, manual tests, workflow tests and simulation) Development (e.g. prototyping, reverse engineering, bug fixing) Support (e.g. assistance in problem analysis) Training (e.g. application training, field service training on network traffic)
- Unfortunately I wont be able to demo the tool today but I thought a case study will help explain it better: Client Overview The client is a diversified technology company with primary divisions focused in the areas of healthcare, consumer lifestyle and lighting. It is one of the largest electronics companies in the world and employs over 100,000 people across more than 60 countries. Business Needs To resolve a customer-facing problem in managing memory for a client viewer, the client test needed data to be loaded into a vendor’s Picture Archive Communication System (PACS) using actual DICOM images of a given modality, size and slice count. The main issues included the updating and processing of the images. The coordination and running of software across each file on disk and updating the header takes days to correctly update one terabyte of DICOM images for multiple studies and patients, along with correctly edited HL7 for each. The datasets had to enter the PACS using C-STORE and consolidated by the system with required orders. QualiTest’s Solution QualiTest proposed using their Virtual Radiology Environment (VRE), a simulation system that combines a comprehensive multi-modality/multi-vendor radiology dataset to assure quality and interoperability while simplifying and dramatically speeding-up all aspects of V&V testing. QualiTest proved their VRE system’s capabilities to the client by performing a free-of-charge Proof of Concept, demonstrating the benefits that the client could contrive from QualiTest’s tools and services. VRE accomplished the issues of updating and processing images automatically by using a proprietary rules engine internally to update all of the header data without loading the full image data into memory. This saved system resources and time on the VRE server while realistically simulating a live Radiology Department. After the data was converted into the proprietary format, the issue was fully reproducible. The real-world execution of the VRE System allowed this vendor to reproduce the issue in-house without delays and provide a patch to the impacted customer faster than previously possible. Key Benefits VRE reduced testing time by 33% VRE reduced personnel needs by 90% The paperwork and coordination for executing such a task were greatly simplified A substantial Return on Investment (ROI) was realized, without sacrificing accuracy or reproducibility
- Net translators solutions – process strategy So having looked at the language requirements for Medical devices and the ISO 17100 standard, I want to take a moment to “translate (for lack of a better word) this into real life application. Essentially the strategy consists of three concentric rings with the outermost ring dealing with the content layer, by this I mean the actual content for translation. Today more than ever, we see a widening array of varied content that needs to be translated, it used to be that all we need concern ourselves with was the UI and the accompanying documentation, today however there are more and more types of material requiring translation and the ability to interface with these various sources in an efficient manner via API’s, cloudbased management platforms, platform specific plugins and so on helps contribute to an effective localization strategy. The second circle is the basic outline of what happens to the content, and this includes the identification and extraction of the content, the preparation of the content, translation and qc and finally the implementation of the translations into their proper place. The final circle is what happens inside at the vendor. TRANSITION Looking at our internal processes you may note that much of what we do is what is laid out in ISO 17100, the first stage as we noted is the analysis stage, this allows us to scope the project from a business standpoint and takes into account things such as DTP effort, Testing Effort, Word count, leveraged word count and terminology requirements. I will note that even prior to this stage that insofar as software localization is concerned that ensuring that your application supports localization is critical, one of the ways to test for this is to undertake a pseudo localization which applies a series of extended characters to the resource files and then may be visually checked to see if there are any issues which arise, in addition to character support, we also enable the expansion of the text as well as indicating start and finish lines which will help identify whether or not there is enough room to account for text expansion, non-extracted strings, cosmetic issues and so on. During the kick-off phase, the content is prepared and packaged for the linguists, and today much of this process is automated which streamlines the project. Also here we check the project requirements and goals and update the glossary if needed. The next phase is handled by the linguists and cover TEP and automated checks for consistency and adherence to glossary as well as number integrity which is crucial when speaking about medical devices. The post translation tasks vary according to the type of content, it could be DTP, or preparation of the content for implementation. And naturally the QA/QC phase also depends upon the type of content, for instance if it is web, then an in-context review would be necessary, if it is UI, then UI testing, if it is documentation then final review of the layout in context. Finally when this is completed we deliver the content to the customers in-country reviewer for their feedback.
- Before I go into the tool set that we use and recommend, I do want to emphasize that for the most part the success of a localization project hinges on implementing the correct process, and I do want to take a moment to point out some key activities which will help ensure a quality localization that will resonate with your users. Ensuring that your content is localization ready entails a more disciplined approach to authoring and change management. Terminology management which would include glossary creation, and this glossary would include any proprietary as well as professional terms. We take a concept based approach to terminology management as a term can be more than a single word but a concept for example “emergency room”. This glossary would then be translated by the translator, and then approved on behalf of the customer by their own resources. One of the keys is also to enable the translator to be trained up in order to understand the uniqueness of your product. Of course we mentioned earlier that the remainder of the process
- While Net-Translators is for the most part tool agnostic and for good reason, we do of course need and use a wide range of tools in our production as well as platforms. So we tend to use well known tools with portability and compatibility with international industry standards such as TMX and TBX (Translation Memory exchange, termbase exchange). Our main tools are SDL Trados Passolo and XTM cloud. These tools are intended for a wide variety of content, some is specialized, for example Passolo is a visual localization tool. Our most exciting offerings today Net-Proxy our solution for website localization, the idea behind which is that we can localize your website with little to no effort on your part. This solution is intended to help speed to market as well as for companies that have limited resources dedicated to internationalizing their website. Net-Cloud is our customer facing portal solution which enables on-line submission of translation projects as well as tracking and reporting. Finally of course, we use industry standard tools for DTP, since we need to localize whatever formats our customers give us.
- Read the Agenda
- So just to sum it all up, we ask you to introspect
- Read the Agenda