Recommended
Recommended
More Related Content
Similar to Role of theory in data analytics
Similar to Role of theory in data analytics (20)
Recently uploaded
Recently uploaded (20)
Role of theory in data analytics
- 1. The Role of Theory in Data Analytics G. Scott Alamanach Mikalauskis
- 4. • (First slide): We once checked the Global Terrorism Database for modes of terror attacks (bombing, assassination, vehicle ramming, etc.) relative to their distance from Tampa, and found a band circling the globe in which certain attack modes had never occurred. Riyadh happened to be in that band, as well as some other interesting cities. Most likely, this is just statistical happenstance, but who knows, perhaps there's some deep reason for it. It's an example of the kinds of patterns that are hidden in the data, waiting to be found. In this case, the deep causes- - if any-- are not nearly as interesting to us as the number of terror bombings that we forecast for Saudi Arabia next month.
- 5. This band includes: • Angola • South Sudan • Riyadh • Bahrain • Kuwait • Herat, Dushanbe, Tashkent • All of Kyrgyzstan • Beijing • Tokyo
- 6. • (Second slide): I predict for Saudi Arabia that over the following month, either six or seven bombings will occur. I am 64.33% certain of this, meaning that a prediction of this sort is wrong exactly 35.67% of the time. There is a 99.42% chance that between five and eight bombings will occur. Clients should budget for eight bombings in Saudi Arabia during the month indicated. Do we care about the reasons behind that peculiar band of attack modes around the world? No, not nearly as much as we care about a forecast of eight bombings. Theory wouldn’t be of much value to us here.
- 8. • (Third slide): This map shows some of the general area of that band. These red arcs are very approximate. It includes some interesting territory. We have no theory for why attack modes should have this peculiar geographic pattern, but if our goal is to travel safely, then we don’t really need one.
- 10. • (Fourth slide): Theory, however, is powerful, and can help us discover secrets that predictive analytics, for all its strength, leave invisible. For example, consider the retina. The physiology of our photoreceptors is such that when the lights go dark, the photoreceptors start admitting higher concentrations of calcium ions. This in turn triggers sensitivity changes that help our eyes adapt to the dark. The process reverses when the light returns. Using lasers and some fluorescent dyes, neuroscientists were able to measure the diffusion rates for these calcium ions.
- 12. • (Fifth slide): Using numerical regression techniques, they found the diffusion to be best described by the sum of two exponential time constants, like so: • Rods: [Ca++](t) = 0.5e(-t/260ms) + 0.5e(- t/2200ms) • Red-sensitive cones: [Ca++](t) = 0.37037e(- t/43ms) + 0.62963e(-t/640ms) • Blue-sensitive cones: [Ca++](t) = 0.5e(- t/140ms) + 0.5e(-t/1400ms) • Predictive analytics would stop here, and be very happy; knowing those time constants allows us to predict the behavior of calcium ion very precisely, and inquiry need go no further. We can market our products to ion channels, or whatever we're going to do.
- 13. • Rods: • 𝐶𝑎++ 𝑡 = 0.5𝑒 −𝑡 260𝑚𝑠 + 0.5𝑒 −𝑡 2200𝑚𝑠 • Red-sensitive cones: • 𝐶𝑎++ 𝑡 = 0.37037𝑒 −𝑡 43𝑚𝑠 + 0.62963𝑒 −𝑡 640𝑚𝑠 • Blue-sensitive cones: • 𝐶𝑎++ 𝑡 = 0.5𝑒 −𝑡 140𝑚𝑠 + 0.5𝑒 −𝑡 1400𝑚𝑠
- 14. • (Sixth slide): But hold the phone: theory, particularly, the laws of thermodynamics and the unyielding logic of geometry, tell us that if it were just a matter of ion channels, then there would be one time constant only. If there are two time constants, then there must be two ways by which calcium ions exit the cell. There are two ways, whether we like it or not; one of them is the ion channels, which we know about. The second is something of a mystery. Thanks to analytics and some theory, we now know that that mystery is out there, and we can go hunting for it.
- 16. • (Seventh slide): Let’s take a second look at forecasting. We can model the effects on transportation networks when some incident disturbs normal operations. For example, if a bad actor blows up an oil tanker inside the Malacca Strait, then nearby Sunda Strait– the nearest available alternative– will get overloaded. This will push some traffic into other alternates; effects will be felt as far away as the rail lines in Siberia.
- 17. Rail
- 18. • (Eighth slide): Now, given the right mathematical models and data, we can measure what those effects will be. If we have (I'm making this up) a chip manufacturer in Manchuria who ships products by rail to Europe, it's possible to forecast the extra time and cost of shipping if something happens to far-off Malacca Strait, even though this manufacturer never himself uses Malacca Strait.
- 19. Highway
- 20. • (Ninth slide): Now, to be able to make a forecast like that requires some kind of theory about how parallel transportation routes interact. Given enough data, one could use predictive analytics and identify a law that describes how they interact, though we would gain no insights into why. The why is theory, and sometimes theory is really nice to have.
- 21. SHIP 800 Ω HIGHWAY 2400 Ω RAIL 1600 Ω 1 𝑻𝑶𝑻𝑨𝑳 = 1 𝑺𝑯𝑰𝑷 + 1 𝑹𝑨𝑰𝑳 + 1 𝑯𝑰𝑮𝑯𝑾𝑨𝒀 GDP 1.2M V FOREIGNTRADE2700A
- 22. • (Tenth slide): Whatever the mode of travel, the movement of physical material is described by second-order differential equations. The movement of current through an electric circuit is a second-order differential equation too, and the parallels line up exactly. Then, econometrics gives us models of foreign trade, and we can augment those econometric models through the judicious application of physical theory. What results is the ability to determine the relative value of parallel transportation modes; parallel modes behave like parallel resistors in an electric circuit. With that model in place, we can then introduce a disturbance anywhere– a blockage of the Malacca Strait, for example– and see how the effects will ripple throughout the entire system.
- 23. QUESTIONS
- 24. • (Eleventh slide): So my point is, analytics is great; analytics with theory is dramatically greater. Theory is not always available to us, but when it is, we're smart to avail ourselves of it. I can now take your questions…
Editor's Notes
- We once checked the Global Terrorism Database for modes of terror attacks (bombing, assassination, vehicle ramming, etc.) relative to their distance from Tampa, and found a band circling the globe in which certain attack modes had never occurred. Riyadh happened to be in that band, as well as some other interesting cities. Most likely, this is just statistical happenstance, but who knows, perhaps there's some deep reason for it. It's an example of the kinds of patterns that are hidden in the data, waiting to be found. In this case, the deep causes-- if any-- are not nearly as interesting to us as the number of terror bombings that we forecast for Saudi Arabia next month.
- I predict for Saudi Arabia that over the following month, either six or seven bombings will occur. I am 64.33% certain of this, meaning that a prediction of this sort is wrong exactly 35.67% of the time. There is a 99.42% chance that between five and eight bombings will occur. Clients should budget for eight bombings in Saudi Arabia during the month indicated. Do we care about the reasons behind that peculiar band of attack modes around the world? No, not nearly as much as we care about a forecast of eight bombings. Theory wouldn’t be of much value to us here.
- Theory, however, is powerful, and can help us discover secrets that predictive analytics, for all its strength, leave invisible. For example, consider the retina. The physiology of our photoreceptors is such that when the lights go dark, the photoreceptors start admitting higher concentrations of calcium ions. This in turn triggers sensitivity changes that help our eyes adapt to the dark. The process reverses when the light returns. Using lasers and some fluorescent dyes, neuroscientists were able to measure the diffusion rates for these calcium ions.
- Using numerical regression techniques, they found the diffusion to be best described by the sum of two exponential time constants, like so: Rods: [Ca++](t) = 0.5e(-t/260ms) + 0.5e(-t/2200ms) Red-sensitive cones: [Ca++](t) = 0.37037e(-t/43ms) + 0.62963e(-t/640ms) Blue-sensitive cones: [Ca++](t) = 0.5e(-t/140ms) + 0.5e(-t/1400ms) Predictive analytics would stop here, and be very happy; knowing those time constants allows us to predict the behavior of calcium ion very precisely, and inquiry need go no further. We can market our products to ion channels, or whatever we're going to do.
- But hold the phone: theory, particularly, the laws of thermodynamics and the unyielding logic of geometry, tell us that if it were just a matter of ion channels, then there would be one time constant only. If there are two time constants, then there must be two ways by which calcium ions exit the cell. There are two ways, whether we like it or not; one of them is the ion channels, which we know about. The second is something of a mystery. Thanks to analytics and some theory, we now know that that mystery is out there, and we can go hunting for it.
- Let’s take a second look at forecasting. We can model the effects on transportation networks when some incident disturbs normal operations. For example, if a bad actor blows up an oil tanker inside the Malacca Strait, then nearby Sunda Strait– the nearest available alternative– will get overloaded. This will push some traffic into other alternates; effects will be felt as far away as the rail lines in Siberia.
- Now, given the right mathematical models and data, we can measure what those effects will be. If we have (I'm making this up) a chip manufacturer in Manchuria who ships products by rail to Europe, it's possible to forecast the extra time and cost of shipping if something happens to far-off Malacca Strait, even though this manufacturer never himself uses Malacca Strait.
- Now, to be able to make a forecast like that requires some kind of theory about how parallel transportation routes interact. Given enough data, one could use predictive analytics and identify a law that describes how they interact, though we would gain no insights into why. The why is theory, and sometimes theory is really nice to have.
- Whatever the mode of travel, the movement of physical material is described by second-order differential equations. The movement of current through an electric circuit is a second-order differential equation too, and the parallels line up exactly. Then, econometrics gives us models of foreign trade, and we can augment those econometric models through the judicious application of physical theory. What results is the ability to determine the relative value of parallel transportation modes; parallel modes behave like parallel resistors in an electric circuit. With that model in place, we can then introduce a disturbance anywhere– a blockage of the Malacca Strait, for example– and see how the effects will ripple throughout the entire system.