AI-influenced workflows in RT
•
0 likes•12 views
Cognitive Ergonomics in Complex Systems: AI-influenced workflows
Recommended
Recommended
More Related Content
Similar to AI-influenced workflows in RT
Similar to AI-influenced workflows in RT (20)
More from RojinMoghadam
More from RojinMoghadam (20)
Recently uploaded
Recently uploaded (20)
AI-influenced workflows in RT
- 1. AI-Infused Contouring Workflow in RT Cognitive Ergonomics Assignment 3: AI-infused workflows Alexandra Maria Bartas - 5165369 Rojin Ghorbani Moghadam - 5088135 Jeroen Lieveloo - 4567382 Parastou Raeis Ghanavati - 5242568
- 4. Analysis of current AI-infused workflow (overlap)
- 5. Analysis of current AI-infused workflow (overlap) Level of uncertainty in tasks
- 6. Analysis of current AI-infused workflow (possibility)
- 7. Research
- 8. Research: “The influence of automation on tumor contouring” Aselmaa, A., van Herk, M., Song, Y. et al. The influence of automation on tumor contouring. Cogn Tech Work 19, 795–808 (2017). https://doi.org/10.1007/s10111-017- 0436-0
- 9. Research
- 11. Workflow Ideas
- 12. Workflow Ideas
- 13. Workflow Ideas
- 14. Workflow Ideas
- 15. Workflow Ideas
- 16. Context and Goal
- 17. Performance Metrics Primary: - Time spend on tasks - Amount of clinically significant inaccuracies. Secondary: - Cognitive workload - Total Journey duration
- 20. General How much the patient deviates from the average human in the region of interest. The quality of the CT scan Prosthetics (dental fillings). Placement of the tumor. Doctor Specific Doctors experience. Doctors training Doctors personal bias Doctors specialty AI specific Size of the Training Dataset Quality of the Training Dataset Context Factors
- 21. Proposal for new AI-infused workflow
- 23. Refined Envisioned Workflow Human-AI interaction
- 24. Uncertainty indicator Translates the layer uncertainty of the contours into a scalar value (percentage). AI aided manipulations By indicating where the doctor does not agree with the AI contouring the AI can reevaluate the choices and propose changes. VR environment Use VR to view the model as a whole and make changes directly to the model instead of on a per- slice basis. User Interface
- 25. Uncertainty indicator Translates the layer uncertainty of the contours into a scalar value (percentage). AI aided manipulations By indicating where the doctor does not agree with the AI contouring the AI can reevaluate the choices and propose changes. VR environment Use VR to view the model as a whole and make changes directly to the model instead of on a per- slice basis. User Interface
- 26. Uncertainty indicator The contouring of the individual parts can be reviewed by the RTT first if the uncertainty is too high.
- 27. The use of colour is used to indicate different tissue. Green is nerve, violet is gland, and orange is bone. Refined Uncertainty indicator
- 28. By clicking on the uncertainty collum the user can order the organs on their uncertainty in ascending or descending order. Refined Uncertainty indicator
- 29. There is a mistake in the contours that needs to be corrected. AI Assisted Contouring
- 30. This symbol indicates on which layers the high uncertainty occurs AI Assisted Contouring
- 31. The RTT starts to adjust the contours. AI Assisted Contouring
- 32. Based on the changes the RTT made, RYCA can re-evaluate the contours. AI Assisted Contouring
- 33. Apply proposed contours. AI Assisted Contouring
- 34. Thank You
Editor's Notes
- We first studied the current manual workflow.
- Then, compared it to the AI-infused workflow, that the RTT is removed from the flow and the whole process takes less time.
- We wanted to see if the current workflows can give us some insight to start with, therefore by overlapping the two workflows (with and without AI), we could compare some important information.
- We figured out that there are some tasks that AI can do precisely with a low uncertainty level. However, RTT might be too slow and uncertain in those tasks. On the other hand, there are some tasks that RTT does with higher certainty level than AI. So at least one of them (AI or RTT) is good and efficient in those tasks. The most problematic tasks are those that neither AI not RTT can do it with high certainty.
- Based on the previous slide, our first attempt to make an overlapped workflow that is both fast and consistent, was to implement RTT as a peer reviewer in tasks that AI is weak at. If RTT was also uncertain, RO comes to triple check the result. Therefore, we would reduce the pressure on RO.
- Figure 1 Contouring workflow with RTT (2A) and AI (2B) Compared contouring with RTT and AI for 3 different patient cases
- Figure 2 (based on Figure 1) AI seems to be more time efficient (C-1 is case 1, C-2 is case 2 and C-3 is case 3, All is all of them together) https://link.springer.com/article/10.1007/s10111-017-0436-0 Other research:Tumor delineation: The weakest link in the search for accuracy in radiotherapy It can be subjective and observer-dependent. Some researchers have identified the lack of continuous education and training as a cause of the variability in tumor delineation It is also recommended that radiation oncologists should collaborate with other specialties, such as radiologists. https://www.jmp.org.in/article.asp?issn=0971-6203;year=2008;volume=33;issue=4;spage=136;epage=140;aulast=Njeh Other research: applications and limitations of machine learning in radiotherapy high degree of interobserver variability Atlas method is the most commonly used, however atlas-based methods are highly sensitive to the atlas-selection strategy,48 as well as the robustness of the—often time-consuming—registration itself https://www.birpublications.org/doi/epub/10.1259/bjr.20190001
- We brainstormed all the possibilities with the aim of reducing contouring time while enhancing accuracy. The following slides demonstrate 5 idea proposals. P.S. Could the RTT learn from their mistakes as well as the AI? Yes! We implemented this comment in slide 23.
- Why is there no feedback to the RTT so they can learn too, like the AI?
- We needed more information to evaluate and choose between those ideas. Hence, we studied the context and the goal of this process. We divided the goals in two parts, Primary and Secondary goals that you can see in this slide.
- Activity Theory is a theoretical framework for the analysis and understanding of human interaction through their use of tools and artefacts
- Based on activity theory, the Motives, Goals, and Subconcious are described.
- And lastly, the contextual factors are described.
- Q: Is giving the RTT the entire organ for context the only way? A: we think this is the best way to be more accurate. However, they know the organ they are working on has a high uncertainty level.
- Based on the reflections during presentation, we liked the idea that RTT can also learn their mistakes from RO refinements, so we implemented it in the workflow.
- Uncertainty indicator: the uncertainty percentage is used to forward the organ for reviewing to either the RTT first and then the RO if it is highly uncertain. Or to the RO directly if the uncertainty is low.
- We chose to continue with the Uncertainty indicator and the AI aided manipulations but not the VR environment. The idea behind the VR environment was to cut down the time spend on misc tasks such as between slide navigation. However, to say that this new interaction would be faster while maintaining the same precision would be a huge assumption so we chose to not use it.
- This is just a selected part of the UI for presentation relevance. This part of the UI is used to navigate between organs, patients and different scans. Q: Why do you use uncertainty rather than certainty? A: We chose to indicate uncertainty because that indicates how much initial outlines is left to be done, and how much more the RTT’s opinion supersedes that of RYCA. Moreover, we aim to reduce uncertainty level more and more every time that RYCA learns. Q: The RTT still has to scroll through all the slices in order to find the uncertainty within the organ. A: We added an indicator (as seen in slide 30) that shows on which slices the high uncertainty occurs. Q: Why are all the colours green? A: this is how we got the data and assumed the colours should be used to indicate what kind of material the organ is (for instance bone, tumor, …) So we did not find it proper to change colors for uncertainty level when we don’t know the mental background of the color indications for RTT and RO. We also implemented this in slide 27-28
- This is just a selected part of the UI for presentation relevance visible on the same screen as the previous part of the UI. The UI for navigating the slices is kept mostly the same. The two added functions are the Uncertainty indicator of the per slice basis as shown in the next slide.
- This indicator symbol is added to aid the RTT/ RO in finding where the higher uncertainties are, in order to correct them and decrease the time it takes to find them (misc/between slice navigation).
- We chose to use a pop up to get the attention of the user. If it would happen in the corner of the screen the user would not notice or continually check if the AI proposes new contours, thereby increasing mental load.The placement of the Popup should be in the middle because that area is mostly not occupied with critical information (such as the center of CT scans or the 3D model). The popup should be a bit smaller so it does not block sight of the CT scans, but we chose to make it a little bit bigger for this presentation.
- After the AI changed the model, it is now more certain that this is correct due to the input of the RTT/RO so it will remove the markers.