- 1. Lecture 4: Informatics skills (Part 2) – searching and making decisions Dr. Martin Chapman Principles of Health Informatics (7MPE1000). https://martinchapman.co.uk/teaching
- 2. Lecture structure 1. How can communication go wrong? (Communicating) 2. How can we improve communication? (Structuring and Questioning) 3. How can we find the data we need? (Searching) 4. What do we do with data once we have it? (Making decisions) 1. and 2. were covered in Lecture 3 (Informatics Skills Part 1)
- 3. Learning outcomes 1. To again understand how other principles from informatics can guide the way we work. Informatics; the study of information: representation, processing, and how it is communicated. It all comes back to interventions… 2. To again understand how principles from informatics can influence clinical practice.
- 4. Principles from Informatics we will touch upon 1. Search strategies 2. Logical inference (we started looking at this in Lecture 2) 3. Bayes’ theorem 4. Decision trees
- 5. Searching How can we find the data we need?
- 6. Recall: Data channels Sinks 1 + 2 X, Y X=1, Y=2 Data model (language) Labelled data Knowledge base Input Output 3 Z = X + Y First-hand (verbal) Second-hand (verbal) Written (data or literature) Device
- 7. Searching the data we receive In Lecture 3, we looked at strategies that allow us to improve communication. By following these strategies, we can increase the chance that we receive the data that we need. But how do we accurately find the specific data we need from within a large body of data once we receive it? We need to consider how to search through data.
- 8. Search strategies We could adopt a simple approach to searching through the data we receive, such as quickly scanning through everything (search mode) or focussing in on a section where we think the information we need might be (reading mode). But really we need something more formal than this. We need a dedicated search strategy.
- 9. Search space Our search space is the place in which we will find the information we need, in this case the data we have received. Before we can define a search strategy that dictates how we move through this search space, we need to structure or map through the space. A common way to do this is to structure the space as a tree, with nodes and edges. Nodes at the end of the tree are called leaf nodes, and the node at the top of the tree is our root node, and is where our search starts.
- 10. Search space Root node Leaf node e.g. Patient Encounter e.g. Individual diagnosis
- 11. Breadth-first search and Depth-first search strategies 1 2 3 4 5 6 7 8 9 10 11 1 2 9 10 11 3 6 7 8 4 5 Rule of thumb: Visit each child in order, and queue each child to have its children visited next Rule of thumb: Always proceed to the next depth if you can, if you cannot, backtrack. Target item Target item How do we now search this space?
- 12. Analytic search: Heuristic and Model-based strategies 1 2 3 4 5 1 2 Target item Target item We might have some heuristic, or rule of thumb, to guide our search, such as ‘the item is at depth 3’. Similarly, we might have an exact model (like a map) of where to find the item. How do we now search this space?
- 13. Shameless plug…
- 14. There’s more to search: (1) Other search spaces We may not just find ourselves searching the data we receive, but also searching, at a higher level, through different information sources themselves, such as other people or literature resources. We would do all of this before we actually search the data receive.
- 15. There’s more to search: (2) Broader process
- 16. Making decisions What do we do with data once we have it?
- 17. Making decisions So far, we’ve looked at how we can effectively receive data (Lecture 3), and how we can find what we need within that data (Searching). Often, the data we receive tells us that there is a problem and, as such, we need to make a decision as to how to rectify that problem. In the remainder of this lecture, we will consider: 1. How to determine what data tells us 2. How to make a decision accordingly
- 18. Determining what data tells us Everything we’ve looked at so far has been about collecting data The things we will discuss are connected as a part of a wider decision-making process
- 19. Determining what data tells us We can determine what a piece of data tells us by drawing conclusions from it. We can draw conclusions from data using logical inference or rules of probability. Using logic inference is already familiar to us…
- 20. Recall: Inference procedure Knowledge, from our model: If a plane does not have pontoons, then it will sink. We refer to this type of model as an inference procedure. It is supported by three other (sub-) models: (1) A data model (2) A knowledge base (3) An ontology We can break down the actual content of our model… Cause and effect
- 21. Recall (again): An information system Sinks 1 + 2 X, Y X=1, Y=2 Data model (language) Labelled data Knowledge base Input Output 3 Z = X + Y We later formalised the process on the previous slide as an information system…
- 22. Logical inference: Deduction Sinks 1 + 2 X, Y X=1, Y=2 Data model (language) Labelled data Knowledge base Input Output ? Z = X + Y The kind of inference we’ve seen so far is known as deduction. We use our knowledge base to determine which output results from our input.
- 23. Logical inference: Abduction Sinks ? X, Y X=1, Y=2 Data model (language) Labelled data Knowledge base Input Output 3 Z = X + Y Abduction works in the opposite way, seeking to determine the range of possible inputs that might have led to this output. For example, here we could have X=1;Y=2 or X=2;Y=1.
- 24. Logical inference: Induction Sinks 1 + 2 X, Y X=1, Y=2 Data model (language) Labelled data Knowledge base Input Output 3 f(X,Y)? = Z? Finally, induction tries to determine what connects inputs and outputs together. This is often a difficult and error-prone process, as it may appear that a rule connects two outputs, when it does not. What if, for example, we (incorrectly) decided our knowledge base just returned ‘3’ to every input. It is correct in this case, but would not generalise.
- 25. Logic inference in clinical practice Deduction: The patient has pneumonia (input), so the patient will develop a fever (output). Abduction: The patient has a fever (output) so could have pneumonia OR COVID-19 (possible inputs). Induction: The patient has pneumonia (input) and then develops a fever (output). We can (carefully) conclude that pneumonia causes fever. This is simplified and does not necessarily constitute actual medical practice.
- 26. Rules of probability: Bayes’ theorem While logical inference aims to give us exact conclusions, this might, in reality, not always be possible. Instead, a piece of data might tell us one of a number of different things. In this case, either implicitly or explicitly, a probability is assigned to each outcome, depending on how likely it is to occur or be true. But we need to be careful how we assign these probabilities, given our existing beliefs. Note: The following example is adapted from the (excellent) video by 3Blue1Brown, ‘Bayes theorem, the geometry of changing beliefs’.
- 27. Exercise: A question You are told that a person named Alex has the following personality traits: ‘Quiet’, ‘Reserved’, ‘Good at maths’ Do you think it’s more likely that Alex: (A)Works in retail (e.g. a shop assistant) or (B) Is a computer programmer? This is our data
- 28. Exercise: A question A similar study was carried out by two academics who determined that (when transposed for this example) the majority of people would say that Alex is most likely to be a computer programmer. 75% chance programmer
- 29. Common judgements vs. rationality While understandable, saying that these traits mean that Alex is a programmer is actually irrational (i.e. goes against standard laws of probability). This is not necessarily due to stereotypes (although they likely play some part – retail workers can be quiet, reserved and good at maths too!), but instead due to, in the researchers’ opinions, background information, or an estimate thereof, not being included when making this determination.
- 30. Background information in decision making Relevant background information when answering our retail vs. programmer question is the number of individuals in these roles in the population. From UK census data, we can see that there are over three times as many people working in retail as programmers. Industry People Retail trade 2,844,470 Computer programming 790,258
- 31. Bayes’ theorem: The ‘prior’ We can again visualise this, and it actually tells us a very different story from the 75% chance we thought of earlier. Probabilities drawn from existing data in this way are known as the prior. 25% chance programmer 1 3 75% chance retail For every 4 people, we can expect one to be a programmer.
- 32. Bayes’ theorem: The ‘likelihood’ We can now consider again the likelihood that, given the data we have about Alex’s traits, they are a programmer. This time though, we must do so in the context of there being a greater number of retail workers. We’ve extrapolated, but the proportions are the same. 4 12 25% 75%
- 33. Bayes’ theorem: The ‘likelihood’ To do this, we now consider our percentage (the chance of someone with Alex’s traits being a programmer) as a proportion of the total programmers that would be in this representative sample, and do the same for the retail workers: 75% 25% 4 12 Note: these percentages don’t necessarily have to add up to 100%. 25% 75%
- 34. Bayes’ theorem: Final calculation With these percentages, 3 programmers fit the description we have, and 3 retail workers. To determine our new discounted probability score, therefore, of Alex being a programmer, we must relate the number of programmers fitting the description to the total number of people fitting the description: = 50% 3 4 12 3 6 3
- 35. Bayes’ theorem: Core idea This shows us how the strength of our new data has been discounted by our existing knowledge. In other words, our new data has not been taken as it is, but has instead updated what we originally thought to be true, from the data (our prior beliefs). 25% chance programmer (Prior) 50% chance programmer (Bayes’) 75% chance programmer (Likelihood)
- 36. Bayes’ theorem: Formula We can write this process out as a formula: (75% × 25%) ((75% × 25%)+(25% × 75%)) (Programmer likelihood × Programmer prior) ((Programmer likelihood × Programmer prior) + (Retail likelihood × Retail prior)) or = 50%
- 37. Aside: Bayes’ in Artificial Intelligence (AI) A robot may update its beliefs about the world after receiving new data, using Bayes’ theorem.
- 39. Prelude: Combining probabilities Bayes’ theorem (and other methods of inference) tell us (accurately) the chance of certain outcomes occurring, e.g. Alex being a programmer. However, more complex outcomes are often based upon multiple things being true, and as such we need to take into account multiple probabilities. We can do this (somewhat unintuitively at this point, given the fact that no decisions are shown) using something called a decision tree: Before we think about actually making decisions…
- 40. ‘Decision trees’ We want to know the chance that Alex could fill a Python programmer role: Suitable for the job: (33% × 80%) + (67% × 1%) = 27% Not suitable for the job: (33% × 20%) + (67% × 99%) = 73% Alex turns up Alex is a programmer Alex is not a programmer Alex knows Python Alex does not know Python Alex knows Python Alex does not know Python Suitable for the job Not suitable for the job Suitable for the job Not suitable for the job 67% 33% 80% 20% 99% 1% We multiply the probabilities along each branch. Chance node We could again apply Bayes to get our Python %s Outcome This is quite like our search space, except now we’re representing a decision space.
- 41. Decision trees Let’s now add in a decision node and consider whether to hire Alex. With just our percentages, there isn’t much to go on here. Fortunately though, when making a decision preferences are also a factor; which outcomes do we prefer. Hire Alex Don’t hire Alex Let’s assume we don’t have the opportunity to interview Alex, we just hire them!
- 42. Decision trees with utility In this context, we express preferences with something called utility. This is a number between 0 and 1, showing how much we prefer a given outcome, after a particular decision has been made. Hire Alex Don’t hire Alex Utility
- 43. Decision trees with utility We may decide that missing a programmer by mistakenly not hiring them is worse (in our minds) than actually hiring a good programmer (imagine the missed potential!). We can reflect this as follows: Hire Alex Don’t hire Alex Utility 0.8 0.8 0.5 0 0.5 0 0 0 …but there is a worse outcome for us if we miss out on Alex’s skills. We’re pretty happy if we hire Alex and they are a programmer…
- 44. Decision trees with utility Working: Hire Alex: (33% × 80% × 0.8) + (33% × 20% × 0) + (67% × 1% × 0.8) + (67% × 99% × 0) = 0.2166 Don’t hire Alex: (33% × 80% × 0.5) + (33% × 20% × 0) + (67% × 1% × 0.5) + (67% × 99% × 0) = 0.1354 Hire Alex Don’t hire Alex Utility 0.8 0.8 0.5 0 0.5 0 0 0 Now we include the utility in our calculation. Now we include the utility in our calculation. Highest utility
- 45. Where do utility values come from? You may not agree with the utility values used previously, and that’s ok! It’s all about preference. Broadly, utility values can be drawn from: 1. Rating scales (as we have seen). 2. Standard gamble (the more likely a person is to choose a low probability option, the higher utility they ascribe to that outcome). 3. Time trade-off (compare time periods with an unknown utility to those with a known utility, when they are considered equivalent, and adjust accordingly).
- 46. A note on rationality and utility Calculations of utility here seem to give us a fairly straightforward way of determining how to make a decision. However, the concept of utility is based upon assumptions of rationality; that humans will always act in a manner that maximises their utility. This, unfortunately, is not always the case. The best thing for each prisoner to do is to lie (-1, -1), but the most attractive option is to confess despite a worse outcome (-8, -8), due to the risk of the other confessing (-10).
- 47. Summary To again understand how other principles from informatics can guide the way we work. • Proper search strategies are needed to locate the specific data we need when we receive a large amount of it. • When we receive new data, we might be able to determine its exact implications using logical inference. • If we receive new data and we are unsure of its consequences, we can determine the probability of different outcomes, making sure to consider the wider context, using Bayes’ theorem. • Once we know what a piece of data tells us, or may tell us, we can decide between alternatives using a decision tree.
- 48. It all comes back to interventions… To again understand how principles from informatics can influence clinical practice. • Much like poor communication, not being able to efficiently search the information received can cause issues with the delivery of interventions. • The more efficiently a clinician can search data, the more efficient intervention delivery will be. • Not accurately inferring the likelihood of a piece of data indicating that a patient has a certain condition can lead to the incorrect delivery of interventions. • As such, leveraging techniques from informatics in order to gain an accurate view on probabilities is important.
- 49. References and Images Enrico Coiera. Guide to Health Informatics (3rd ed.). CRC Press, 2015. https://www.riomed.com/electronic-patient-records-impact-on-healthcare-industry/ https://www.lego.com/en-gb/service/buildinginstructions/3178 https://uxwing.com/computer-internet-woman-icon/ https://www.flaticon.com/free- icon/cashier_4469878?term=shop+assistant&page=1&position=7&origin=search&related_id=4469878 https://www.electronicsforu.com/news/robots-at-duty-to-prevent-spread-of-covid-19-infection