2018 Princeton Fintech & Quant Conference: AI, Machine Learning & Deep Learning Risk Management & Controls: Beyond Deep Learning and Generative Adversarial Networks: Model Risk Management in AI, Machine Learning & Deep Learning

2018 Princeton Fintech & Quant Conference: AI, Machine Learning & Deep Learning Risk Management & Controls: Beyond Deep Learning and Generative Adversarial Networks: Model Risk Management in AI, Machine Learning & Deep Learning
[1] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: 2018 Princeton Fintech & Quant Conference Princeton University, April 21, 2018 Princeton Presentations in AI-ML Risk Management & Control Systems 2016 Princeton Quant Trading Conference, Princeton University How to Navigate ‘Uncertainty’... When ‘Models’ Are ‘Wrong’... and ‘Knowledge’... ‘Imperfect’! Knight Reconsidered Again: Risk, Uncertainty, & Profit beyond ZIRP & NIRP 2015 Princeton Quant Trading Conference, Princeton University Future of Finance Beyond 'Flash Boys': Risk Modeling for Managing Uncertainty in an Increasingly Non-Deterministic Cyber World: Knight Reconsidered: Risk, Uncertainty, and Profit for the Cyber Era Yogi Dr. Yogesh Malhotra Post-Doctoral R&D in AI, Machine Learning & Deep Learning Marquis Who's Who in the World® 1999-, Marquis Who's Who in America® 2002-, Marquis Who's Who in Finance & Industry® 2001-, Marquis Who's Who in Science & Engineering® 2006- www.yogeshmalhotra.com (646) 770-7993 dr.yogesh.malhotra@gmail.com Global Risk Management Network, LLC 757 Warren Road, Cornell Business & Technology Park, Ithaca, NY 14852-4892 http://www.linkedin.com/in/yogeshmalhotra www.FutureOfFinance.org
[2] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: The European Parliament Think Tank's Research Policy document 'Should we fear artificial intelligence?' reflects the ongoing mainstream debate between the Utopian and Dystopian aspects of AI and Machine Learning. "Powerful AIs can in principle be given nearly any goal, which is a source of both risk and opportunity. There are myriad possible malicious uses of AI and many ways in which it might be used in a harmful manner unintentionally, such as with algorithmic bias. Perhaps most fundamentally, the control problem will have to be addressed – that is, we will need to learn how to ensure that AI systems achieve the goals we want them to without causing harm during their learning process, misinterpreting what is desired of them, or resisting human control." Third in the series of the Princeton Presentations on AI and Machine Learning Risk Management & Control Systems, the current presentation develops fundamental guidance on the design, development, and implementation of AI, Machine Learning, and Deep Learning Models and Methods. The 2018 Princeton presentation will focus on "the control problem" which is a critical prerequisite for AI systems to have positive impacts by further developing upon my prior two presentations that pioneered Cyber-Finance-Trust™ Model Risk Management & Model Risk Arbitrage™ practices at prior Princeton Quant Trading Conferences. Starting with the first technical report on the Bitcoin Blockchain Cryptographic Proof of Work; spanning latest developments in AI, Machine, Learning, Deep Learning, and, Generative Adversarial Networks; and, hedge fund algorithmic trading, the presentation generates interesting insights about the most critical role of risk management controls. Such role of risk management controls is most critical in not only getting the best out of AI, but also ensuring that the worst fears about the AI do not really come true. Abstract
[3] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include:  SHOULD WE FEAR ARTIFICIAL INTELLIGENCE CURRENT GLOBAL CONTEXT & BACKGROUND  AI, MACHINE LEARNING, DEEP LEARNING, GANs: 1 SENSE MAKING vs. INFORMATION PROCESSING  AI, MACHINE LEARNING, DEEP LEARNING, GANs: 2 SENSE MAKING vs. INFORMATION PROCESSING  AI-ML-DL BIASES, INTERPRETIBILITY, EXPLAINABILITY WITH GREAT POWER COMES GREAT RESPONSIBILITY  AI-ML-DL and VaR: WHY ALL ‘MODELS’ ARE WRONG “THIS IS NOT INTELLIGENCE”, NOT ‘COMMON SENSE’  RISK MODELING TO UNCERTAINTY MANAGEMENT WHEN ‘BEST PRACTICES’ BECOME ‘WORST PRACTICES’  AI-ML-DL-GANs: KNOWING ‘MATH’ AND ITS ‘REAL’ LIMITS RELY UPON INTUITION TO GO BEYOND LIMITS OF ‘MATH’ OUTLINE OF PRESENTATION Need Copy of Presentation? Contact via LinkedIn: http://www.linkedin.com/in/yogeshmalhotra
[4] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Adaptability-Generalizability Past Prediction vs. Future Anticipation KMS & Risk Management Controls Self-Adaptive Complex Systems AI-ML Knowledge Management Systems Sense Making Past vs. Future ‘Historical Data’ Known vs. Unknown
[5] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include:
[7] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "Recently, such probabilistic, statistical, and numerical methods related concerns are in globally popular press related to cybersecurity controls and compliance. Earlier, similar probabilistic, statistical, and numerical methods related concerns were in the global popular press in the context of the global financial crisis. Future questions focused on the underlying assumptions and logic may focus on related implications for compliance, controls, valuation, risk management, etc. Likewise, recent developments about mathematical entropy measures shedding new light on apparently greater vulnerability of prior encryption mechanisms may offer additional insights for compliance and control experts. For instance, given related mathematical, statistical and numerical frameworks, analysis may also focus on potential implications for pricing, valuation and risk models. The important point is that many such fundamental assumptions and logic underlying widely used probabilistic, statistical, and numerical methods may not as readily meet the eye." Interpretability, Explainability, and, Model Risk are Related Issues Hence, they need to be addressed together for AI and Machine Learning Future of Bitcoin & Statistical Probabilistic Quantitative Methods: Global Financial Regulation (Interview: Hong Kong Institute of CPAs) http://yogeshmalhotra.com/Future_of_Bitcoin.html Bitcoin Protocol: Model of ‘Cryptographic Proof’ Based Global Crypto-Currency & Electronic Payments System http://yogeshmalhotra.com/BitcoinProtocol.html January 20, 2014 December 04, 2013 GDPR
[8] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Harvard Business Review: If Your Data Is Bad, Your Machine Learning Tools Are Useless In addition to Data, the challenges of accurate AI-ML Models and Methods are equally, if not even more so, critical given that they are hidden from the users' eyes (WWW: Society of Actuaries in Ireland: Cybersecurity & Cyber-Finance Risk Management - Yogesh Malhotra, PhD) https://lnkd.in/eDb897h "[T]he approaches to mitigate operating risk associated with the use of models need to evolve to reflect recent trends in the Finance Industry. In particular there are a number of new areas where it is not possible for the "human eye" to necessarily detect material flaws: in the case of models operating over very small time scales in high frequency algorithmic trading, or for portfolio risk measurement models where outputs lack interpretability due to highdimensionality and complex interactions in inputs, the periodic inspection of predicted versus realized outcomes is unlikely to be an effective risk mitigate." https://lnkd.in/eV79T6C
[9] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.europarl.europa.eu /thinktank/en/document.html? reference=EPRS_IDA(2018)6 14547 http://www.europarl.europa.eu/ RegData/etudes/IDAN/2018/61 4547/EPRS_IDA(2018)614547 _EN.pdf Adaptability-Generalizability Past Prediction vs. Future Anticipation KMS & Risk Management Controls Self-Adaptive Complex Systems AI-ML Knowledge Management Systems Creativity, Imagination, Innovation, Intuition, Insight Known vs. Unknown Routine, Structured, Procedural Non-routine, Unstructured, Non-procedural With Great Power Comes Great Responsibility
[10] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI-ML Risk Management & Controls Most Critical Lesser Concern about the Next ‘AI Winter’ Greater Concern about the ‘Nuclear Winter’*
[11] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI-ML Risk Management & Controls Most Critical Interpretability vs. Sense Making Past vs. Future
[12] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI-ML Risk Management & Controls Most Critical
[13] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI-ML Risk Management & Controls Most Critical Adaptability-Generalizability SACS 4 AI Types Human Driving in Most Unpredictable Environments Past vs. Future
[14] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www.linkedin. com/feed/update/urn: li:activity:639162502 6721890304 *M5: What is being Human?: Qualities such as "freedom of will, intentionality, self- consciousness, moral agency and a sense of personal identity." http://www.robotics -openletter.eu/
[15] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www.linkedin.com/feed/update/ urn:li:activity:6391798889275547648
[16] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: 1998 First Quant MIS- IT PhD on KMS & Risk Management Controls Cybernetic & Control Systems http://www.aacsb.edu//media/aacsb/publications/ research-reports/impact-of-research.ashx?la=en * 20-Year R&D Adaptability- Generalizability SACS Past Prediction vs. Future Anticipation
[17] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: My LinkedIn Page accessible also from my Home Page: https://www.linkedin.c om/pulse/dear-ceo-ai- machine-learning- advice-top-industry- leading-malhotra/ MASTER REFERENCE FOR MOST TERMS & CONCEPTS http://www.kmnetwork.com/RealTime.pdf Adaptability- Generalizability SACS KMS & Risk Management Controls Sense Making Past vs. Future ‘Historical Data’ Malhotra, Y., Integrating Knowledge Management Technologies in Organizational Business Processes: Getting Real Time Enterprises to Deliver Real Business Performance, Journal of Knowledge Management, Vol. 9, Issue 1, April 2005, 7-28. Past Prediction vs. Future Anticipation Known vs. Unknown 20-Year R&D KMS-Controls Risk Mgmt. Strategies Technologies People Processes
[18] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.yogeshmalhotra.com/ publications.html http://www.brint.org/expertsystems.pdf Malhotra, Y., Expert Systems for Knowledge Management: Crossing the Chasm between Information Processing and Sense Making, Expert Systems with Applications: An International Journal, 20(1), 7-16, 2001. https://www.linkedin.com/in/ yogeshmalhotra/ Adaptability-Generalizability Past Prediction vs. Future Anticipation KMS & Risk Management Controls Self-Adaptive Complex Systems AI-ML Knowledge Management Systems Sense Making Past vs. Future ‘Historical Data’
[20] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: MIT Technology Review: The GANfather: The man who’s given machines the gift of imagination MIT AI-Strategy Executive Guide (continued) https://lnkd.in/eknKzm5 Malhotra, Yogesh, "Knowledge Management in Inquiring Organizations" (1997). AMCIS 1997 Proceedings. 181. https://lnkd.in/eKR3p8s https://lnkd.in/eGbhayW "Hegelian inquiry systems are based on a synthesis of multiple completely antithetical representations that are characterized by intense conflict because of the contrary underlying assumptions. Knowledge management systems based upon the Hegelian inquiry systems, would facilitate multiple and contradictory interpretations of the focal information. This process would ensure that the focal information is subjected to continual re- examination and modification given the changing reality. Continuously challenging the current 'company way,' such systems are expected to prevent the core capabilities of yesterday from becoming core rigidities of tomorrow." https://lnkd.in/eQNXzkN
[21] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Example of Latest on GANs: At Least 20-Years Behind! Not in MATH, But in INTUITION... See: Derman on Models & Intuition Key Problems of AI-ML Models: Socio-Psychology & Learning Constructs - Correct AI-ML REPRESENTATION? - Valid & Reliable MEASURES? - Valid & Reliable RELATIONSHIPS? Recipe for the Next AI-ML Crisis “Baked” in underlying METHODs And MODELs And assumed as a GIVEN Concern Less about the ‘Next AI Winter’ but More about the ‘Next AI Nuclear Holocaust’ If Risk Management Controls are Non-existent or Bypassed
[22] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: • Malhotra, Y., Galletta, D.F., and, Kirsch, L.J. How Endogenous Motivations Influence User Intentions: Beyond the Dichotomy of Extrinsic and Intrinsic User Motivations, Journal of Management Information Systems, Summer 2008, Vol. 25, No. 1, 267-299. • Malhotra, Y. and Galletta, D.F., A Multidimensional Commitment Model of Volitional Systems Adoption and Usage Behavior, Journal of Management Information Systems, Summer 2005, Vol. 22, No. 1; 117-151. • Malhotra, Y., and, Kirsch, L.J., Personal Construct Analysis of Self-Control in IS Adoption: Empirical Evidence from Comparative Case Studies of IS Users & IS Champions. Proceedings of the First INFORMS Conference on Information Systems and Technology, 105-114, Washington, DC, May, 1996. • Malhotra, Y., Expert Systems for Knowledge Management: Crossing the Chasm between Information Processing and Sense Making, Expert Systems with Applications: An International Journal, 20(1), 7-16, 2001. (Holland Communication - 1995) Example of Latest on GANs: At Least 20-Years Behind! Not in MATH, But in INTUITION... See: Derman on Models & Intuition Example of Latest in Generative Adversarial Networks – 20 Years earlier Research Applied by NASA, Big Banks, and, Top Intelligence Agencies Artificial Curiosity, Intrinsic Motivation, Information Seeking Behavior, Reward Function http://www.yogeshmalhotra.com/ publications.html Sense Making Past vs. Future ‘Historical Data’ KMS & Risk Management Controls
[23] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Stanislav Petrov was on duty in a secret command centre outside Moscow on 26 September 1983 when a radar screen showed that five Minuteman intercontinental ballistic missiles had been launched by the US towards the Soviet Union. Red Army protocol would have been to order a retaliatory strike, but Petrov – then a 44- year-old lieutenant colonel – ignored the warning, relying on a “gut instinct” that told him it was a false alert. It later emerged that the false alarm was the result of a satellite mistaking the reflection of the sun’s rays off the tops of clouds for a missile launch. “We are wiser than the computers,” Petrov said in a 2010 interview with the German magazine Der Spiegel. “We created them.” “false alarm” ‘fake news’
[24] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: LET US DO A THOUGHT EXPERIMENT DOTs: WHAT IS ITS “MEANING”?FEATURE MATH vs. INTUITION
[25] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: LET US DO A THOUGHT EXPERIMENT LINEs: WHAT IS ITS “MEANING”?FEATURE VECTOR MATH vs. INTUITION
[26] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: LET US DO A THOUGHT EXPERIMENT PLANEs: WHAT IS ITS “MEANING”?FEATURE MAP Interpretability vs. Sense Making Past vs. Future MATH vs. INTUITION Known vs. Unknown
[27] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: CUBEs: WHAT IS ITS “MEANING”? STACKED FEATURE MAP The Building Blocks of Interpretability Interpretability techniques are normally studied in isolation. We explore the powerful interfaces that arise when you combine them and the rich structure of this combinatorial space. MATH vs. INTUITION
[28] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Labrador retriever and tiger cat Several floppy ear detectors seem to be important when distinguishing dogs, whereas pointy ears are used to classify "tiger cat". https://distill.pub/2018/building-blocks/ The Building Blocks of Interpretability Interpretability techniques are normally studied in isolation. We explore the powerful interfaces that arise when you combine them and the rich structure of this combinatorial space. MATH vs. INTUITION
[29] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://devblogs.nvidia.com/deep- learning-nutshell-core-concepts/ Deep Learning in a Nutshell consolidation
[30] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www.theverge.com/20 18/4/11/17224984/artificial- intelligence-idxdr-fda-eye- disease-diabetic-rethinopathy
[31] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www.fda.gov/N ewsEvents/Newsroom/ PressAnnouncements/ ucm604357.htm
[33] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www.wsj.com/articles /the-key-to-smarter-ai-copy- the-brain-1523369923
[34] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why ‘Humans-in-the Loop’ are Even More Critical for Interpretability https://ssrn.com/abstract=2940467 Socio-Technical Systems Malhotra, Yogesh, Advancing Cognitive Analytics Using Quantum Computing for Next Generation Encryption (Presentation Slides) (March 24, 2017). Available at SSRN: https://ssrn.com/a bstract=2940467
[35] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why Interpretability is complicated Why ‘Humans-in-the Loop’ are Even More Critical for Interpretability Malhotra, Y., Bringing the Adopter Back Into the Adoption Process: A Personal Construction Framework of Information Technology Adoption. Journal of High Technology Management Research, 10(1), 1999, 79-104. Socio-Technical Systems
[36] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why Interpretability is complicated Why ‘Humans-in-the Loop’ are Even More Critical for Interpretability Socio-Technical Systems
[37] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Socio-Technical Systems Sense Making Past vs. Future ‘Historical Data’ Adaptability-Generalizability Self-Adaptive Complex Systems AI-ML -KMS Known vs. Unknown LET US DO A THOUGHT EXPERIMENT
[40] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: What Caused the Failure of the Socio-Technical System? 3 Key Systems Failed Perfect Weather Conditions and Perfect Road Conditions in AZ What Would Happen in the “Typical” “Zero-Visibility” Winter Weather in Central NY? When 65 MPH I-90 “Thruway” Traffic Drives ‘Normally’ in Day at 10 MPH for Safety Or When All Traffic is Off the 65 MPH I-90 “Thruway” as it’s Frozen. Socio-Technical Systems Adaptability- Generalizability Self-Adaptive Complex Systems AI-ML -KMS Sense Making Past vs. Future ‘Historical Data’ Known vs. Unknown
[41] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "When you do physics you're playing against God; in finance [just like all other sociotechnical systems], you're playing against God's creatures.“ - Emanuel Derman [Generalized Model Risk Management: Bayesian vs. VaR: https://lnkd.in/eGr9eCi ] "While robot cars are being created to follow traffic rules, interactions with humans continue to present hurdles. Pedestrians, in particular, can confuse systems because they are "unpredictable"." “The computer vision systems are incredibly brittle in these cars. There’s a strong, high probability that the computer vision system failed to detect the person.” Tempe Police confirmed in a press conference that the Uber vehicle was traveling at around 40mph (with no signs yet that it was slowing down) when it struck the pedestrian.
[42] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: MATH vs. INTUITION "Not everything that counts can be counted, and not everything that can be counted counts." "As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality." "If you give a pilot an altimeter that is sometimes defective he will crash the plane. Give him nothing and he will look out the window. Technology is only safe if it is flawless.” NNT
[44] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: SPACE + CYBERSPACE ‘OFFENSIVE’ ‘DEFENSIVE’ Analysis of full-motion video data from tactical aerial drone platforms such as the ScanEagle and medium- altitude platforms such as the MQ-1C Gray Eagle and the MQ-9 Reaper. Project Maven: First operational use of deep learning AI technologies in the defense intelligence enterprise. Malhotra, Yogesh, Cognitive Computing for Anticipatory Risk Analytics in Intelligence, Surveillance, & Reconnaissance (ISR) (January 28, 2018). Available at SSRN: https://ssrn.com /abstract=3111837 MATH vs. INTUITION https://thebulletin.org/project-maven-brings-ai-fight-against-isis11374 Algorithmic Warfare Cross-Functional Team
[45] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: “Maven is designed to be that pilot project, that pathfinder, that spark that kindles the flame front of artificial intelligence across the rest of the Department.” https://thebulletin.org/project-maven-brings-ai-fight-against-isis11374 With Great Power Comes Great Responsibility MODELS RISKS ISR SIGNALS Data in Transit Data in Use Malhotra, Yogesh, Cognitive Computing for Anticipatory Risk Analytics in Intelligence, Surveillance, & Reconnaissance (ISR) (January 28, 2018). Available at SSRN: https://ssrn.com/ab stract=3111837 MATH vs. INTUITION
[46] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include:  Maven’s success is clear proof that AI-ML-DL is ready to revolutionize many national security missions even if DoD is not yet ready for the organizational, ethical, and strategic implications of that revolution.  Having met sky-high expectations of the DoD, it’s likely  to spawn 100 copycat ‘Mavens’ in ISR.  “I don't think honestly there is any aspect of Department that is not ripe for introducing some type of AI and machine learning into it.”  Agile Manifesto + Quant Models Manifesto + CyberISR “Convolutional Neural Networks are doomed” – Geofferey Hinton Malhotra, Yogesh, Cognitive Computing for Anticipatory Risk Analytics in Intelligence, Surveillance, & Reconnaissance (ISR) (January 28, 2018). Available at SSRN: https://ssrn.com/abstract=3111837 With Great Power Comes Great Responsibility SPACE + CYBERSPACE ‘OFFENSIVE’ ‘DEFENSIVE’
[47] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://thebulletin.org /daniel-ellsberg- dismantling- doomsday- machine11539 Lesser Concern about the Next ‘AI Winter’... Greater Concern about the ‘Nuclear Winter’*
[48] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.dailymail.co. uk/sciencetech/article- 5603367/AI-studies- CCTV-predict-crime- happens-rolled- India.html
[49] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: “I think the way we’re doing computer vision is just wrong,” he says. “It works better than anything else at present but that doesn’t mean it’s right.” Dynamic Routing Between Capsules https://arxiv.org/abs/1710.09829 Matrix capsules with EM routing https://openreview.net/forum?id=HJWLfGWRb&noteId=HJWLfGWRb “I think the way we’re doing computer vision is just wrong.”
[50] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: MATH vs. INTUITION “Imagine a face. What are the components? We have the face oval, two eyes, a nose and a mouth. For a CNN, a mere presence of these objects can be a very strong indicator to consider that there is a face in the image. Orientational and relative spatial relationships between these components are not very important to a CNN.” = https://www.cs.toronto.edu/~hinton/csc2535/notes/lec6b.pdf https://medium.com/ai%C2%B3-theory-practice-business/understanding-hintons-capsule-networks-part-i-intuition-b4b559d1159b Internal data representation of a convolutional neural network does not take into account important spatial hierarchies between simple and complex objects. "As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality." “Certainly the statement 2 x (1/2) = 1 is arithmetically correct. But do two half-sheets of paper make one whole sheet and do two half-shoes make one whole shoe?” – Morris Kline
[51] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www2.deloitte.com/insights/us/en/deloitte-review/issue-20/augmented-intelligence-human- computer-collaboration.html MATH vs. INTUITION What’s HARD? What’s EASY? Computationally? Intuitively? Computationally: Routine, Structured, Procedural Intuitively: Non-routine, Unstructured, Non-procedural "Though machines are, in speed, accuracy, and endurance, superior to the human brain, one should not infer, as many popular writers are now suggesting, that machines will ultimately replace brains. Machines do not think. They perform the calculations which they are directed to perform by people who have the brains to know what calculations are wanted.” - Morris Kline
[52] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://www2.deloitte.com/insights/us/en/deloitte-review/issue-20/augmented-intelligence-human-computer- collaboration.html
[56] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "Patrick Winston, a professor of AI and computer science at MIT, says it would be more helpful to describe the developments of the past few years as having occurred in “computational statistics” rather than in AI. One of the leading researchers in the field, Yann LeCun, Facebook’s director of AI, said at a Future of Work conference at MIT in November that machines are far from having “the essence of intelligence.” That includes the ability to understand the physical world well enough to make predictions about basic aspects of it—to observe one thing and then use background knowledge to figure out what other things must also be true. Another way of saying this is that machines don’t have common sense." "The computer that wins at Go is analyzing data for patterns. It has no idea it’s playing Go as opposed to golf, or what would happen if more than half of a Go board was pushed beyond the edge of a table... " AI has No ‘Common Sense’... No Sense for ‘Sense Making’... No Sense of ‘Meaning’...
[57] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: “I personally think the problem of intelligence is the greatest problem in science. AlphaGo is one of the two main successes of AI, and the other is the autonomous-car story. Very soon they’ll be quite autonomous. Is this getting us closer to human intelligence? " Tomaso Poggio, a professor at the McGovern Institute for Brain Research at MIT said these programs are no closer to real human intelligence than before. "These systems are pretty dumb." He says no one knows how to make a broader general intelligence, like what humans have, and you can’t do it by “gluing together” existing programs that play games or categorize images. A self-driving Go player would bring us no closer to a "general" AI, or one that can think for itself and solve many kinds of novel problems. “We have not yet solved AI by far. This is not intelligence," says Poggio. He thinks the next AI breakthroughs are going to come from neuroscience, something he works on as head of a 10-yr, $50 million program called the Center for Brains, Minds, and Machines, which is exploring how the brain creates human visual awareness. This is not intelligence
[58] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "Insofar as certainty of knowledge is concerned, mathematics serves as an ideal, an ideal toward we shall strive, even though it may be one that we shall never attain. Certainty may be no more than a phantom constantly pursued and interminably elusive.“ – Morris Kline https://www.linkedin.com/pulse/designing-smart- minds-using-tools-utopian-view-ai-yogesh-/ http://www.linkedin.com/in/yogeshmalhotra Fischer Black and the Revolutionary Idea of Finance Hedge Funds Trading and Risk Management On Fischer Black: Intuition is a Merging of the Understander with the Understood – Emanuel Derman A Man for All Markets – Ed Thorp "Future strategic advantage and competitive performance will not derive from simply adoption and use of new information and communication technologies. Rather, they will be determined by smart minds using smart technologies, with greater emphasis being on smart minds.
[59] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Starting from original article on AI & ML inspired by Genetic Algorithms pioneer Dr. John Holland that outlined these key distinctions 20 years ago: https://lnkd.in/eDE-W3z... To recent observations in: Making AI & Deep Learning Work Better: Designing 'Smart Minds' Using 'Smart Tools': https://lnkd.in/gcp_yHe . Conclusions in this week's MIT-Strategy discussions on AlphaZero, AlphaGoZero, and, AlphaGo: "From a Strategic and Psychological perspective, the 'games' humans are capable of imagining and playing are at a different level as compared to machines, only, if we can recognize so, as discussed in Module 5 with reference to [my] articles such as on AI & Machine Learning Strategy and Psychological Games." Response to: "we're always outdated..." To never be outdated, always "Know Forward" instead of "Knowing Backward"... use Real Intelligence... How: "Obsolete what you know before others obsolete it and profit by creating the challenges and opportunities others haven't even thought about." - Inc. Magazine Interview, Inc. Technology special issue #3, 1999. https://lnkd.in/dhrXpwq BEYOND THE MASTER ALGORITHM
[60] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: More on Partnering 'Smart Minds' with 'Smart Tools': Making AI & Deep Learning Work Better: Designing 'Smart Minds' Using 'Smart Tools': https://lnkd.in/gcp_yHe . MIT Sloan Management Review: "Companies are succeeding with AI by partnering smart machines with smart people who are learning how to take advantage of what those machines can do. In short, AI implementation success depends on your ability to hire and develop problem- solvers, equip them with data (and potentially AI), and then empower them to actually solve problems. Note that addressing skill requirements this way may well require major changes to your existing hiring and development practices. Companies that view smart machines purely as a cost-cutting opportunity are likely to insert them in all the wrong places and all the wrong ways. These companies will automate existing processes rather than imagine new ones. They will cut jobs rather than upgrade roles. These are the companies who will find that implementing AI is little more than a reprise of the ERP nightmare." https://lnkd.in/dBHEYXh
[61] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI: Model Risk Management to counter Spurious ML "Patterns": MIT AI-Strategy Executive Guide (continued) https://lnkd.in/eknKzm5 "All models are wrong, some models are useful." Problem: FT: Spurious correlations are kryptonite of Wall St’s AI rush https://lnkd.in/ednTEiS "Machine learning is a valuable tool to analyse vast data sets. But it really is just data mining to find patterns. Sometimes a signal might make money for a few days or weeks, and when it disappears or even leads to losses it can be hard to be certain whether it was arbitraged away by other traders, or if it was spurious from the start. Although data mining is often used simply to mean looking for patterns in huge data sets, for quants the term typically has negative connotations, implying a selective hunt for data points to support a specific thesis. It is frequently used interchangeably with the more technical expression “overfitting”, building a faulty model on a bedrock of shaky data." Model Risk Management: Model Risk Management Paper (JP Morgan) (follow up to MIT Sloan Management Review Paper) https://lnkd.in/eGr9eCi Model Risk Management Presentation (Princeton) https://lnkd.in/eyP9Npd Model Risk Arbitrage™ Presentation (Princeton) https://lnkd.in/dJ-Gnxx https://lnkd.in/ednTEiS
[62] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Yogesh Malhotra, PhD, 2016www.yogeshmalhotra.com Malhotra, Yogesh, Cybersecurity & Cyber-Finance Risk Management: Strategies, Tactics, Operations, &, Intelligence: CROs-CSOs Keynote: Enterprise Risk Management to Model Risk Management: Understanding Vulnerabilities, Threats, & Risk Mitigation (September 15, 2015). Available at SSRN: https://ssrn.com/abstract=2693886. All Models are Wrong... Some Models are Useful. Why Intuition is most critical for System Performance
[63] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Yogesh Malhotra, PhD, 2016www.yogeshmalhotra.com Malhotra, Yogesh, Cybersecurity & Cyber-Finance Risk Management: Strategies, Tactics, Operations, &, Intelligence: Enterprise Risk Management to Model Risk Management: Understanding Vulnerabilities, Threats, & Risk Mitigation (Presentation Slides) (September 15, 2015). Available at SSRN: https://ssrn.com/abstract=2693886. Why it is most critical to remember that Model is Not the Reality
[64] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Yogesh Malhotra, PhD, 2016www.yogeshmalhotra.com • Embrace subjectivity • Acknowledge uncertainty • Integrate objective & subjective info Why ‘Common Sense’ is most critical to know how wrong a Model can be
[66] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Yogesh Malhotra, PhD, 2016www.yogeshmalhotra.com Model use entails model risk (Derman, 1996; Morini, 2011) because a statistical model is used for risk estimation. The problem of model risk for any risk model such as VaR results from the fact that risk cannot be measured, but must be estimated using a statistical model (Boucher et al., 2014; Danielsson et al., 2014) . Using a range of different plausible models which can be robustly discriminated between, the variance between corresponding range of estimates is a succinct measure of model risk (Danielsson et al., 2014). We apply this notion of multi-model comparison of estimates and extend it to multi-methods comparison to manage model risk advancing estimation of cyber risk related loss beyond the limitations of VaR discussed earlier. Why it is most critical to manage model risk using Model Risk Management
[67] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "The world has too much texture than [quants] can squeeze into the framework they're used to. I see a huge incidence of pure speculative gambling on the part of these folks who are hired on the strength of their knowledge of quantitative methods." "You're worse off relying on misleading information than on not having any information at all. If you give a pilot an altimeter that is sometimes defective he will crash the plane. Give him nothing and he will look out the window. Technology is only safe if it is flawless." "To me, VaR is charlatanism because it tries to estimate something that is not scientifically possible to estimate, namely the risks of rare events. It gives people misleading precision that could lead to the build up of positions by hedgers. It lulls people to sleep." http://www.yogeshmalhotra.com/risk.html "The only Constant used to be Change... Even it is not Constant anymore...."
[68] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.actuaries.org/ ASTIN/Colloquia/Helsink i/Presentations/Embrechts .pdf https://www.wired.com/2009/02/wp-quant/ At the heart of it all was Li's formula. When you talk to market participants, they use words like beautiful, simple, and, most commonly, tractable... Li's approach made no allowance for unpredictability: It assumed that correlation was a constant rather than something mercurial... “They didn't know, or didn't ask. One reason was that the outputs came from "black box" computer models and were hard to subject to a commonsense smell test. Another was that the quants, who should have been more aware of the copula's weaknesses, weren't the ones making the big asset-allocation decisions. Their managers, who made the actual calls, lacked the math skills to understand what the models were doing or how they worked.” “The most dangerous part is when people believe everything coming out of it.” - Li
[69] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: XXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXX
[70] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.yogeshmalhotra.com/risk.html
[72] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://ssrn.com/abstract=2538401
[75] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: National Association of Insurance Commissioners Expert Paper Most Models are Wrong. Some Models are Useful. - Derman https://ssrn.com/abstract=3081492
[77] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: GUIDING THE BIG-4 CONSULTING BEST PRACTICES ABOUT ‘BEST PRACTICES’
[79] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: “Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes.”
[80] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: “Images on the Right are slightly distorted versions of the images on the Left.” “The difference between Left and Right set of images is imperceptible to the human eye.” “However, where human eye sees the SAME OBJECT on the Right, the Convolutional Neural Network sees an OSTRICH for all the three images.”
[81] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: What Caused the Failure of the Socio-Technical System? 3 Key Systems Failed Perfect Weather Conditions and Perfect Road Conditions in AZ What Would Happen in the “Typical” “Zero-Visibility” Winter Weather in Central NY? When 65 MPH I-90 “Thruway” Traffic Drives ‘Normally’ in Day at 10 MPH for Safety Or When All Traffic is Off the 65 MPH I-90 “Thruway” as it’s Frozen. Socio-Technical Systems Adaptability- Generalizability Self-Adaptive Complex Systems AI-ML -KMS Sense Making Past vs. Future ‘Historical Data’ Known vs. Unknown
[82] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: MATH vs. INTUITION "Not everything that counts can be counted, and not everything that can be counted counts." "As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality." https://alexiajm.github.io/GANs/
[83] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://developers.googleblog.co m/2018/04/text-embedding- models-contain-bias.html https://papers.nips.cc/paper/6228- man-is-to-computer-programmer- as-woman-is-to-homemaker- debiasing-word-embeddings.pdf
[84] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://datavizblog.com/2013/07/27/why-you-should-never-trust-a-data-visualization/ “The mathematician really creates models of reality. Each model has a limited applicability. Moreover, one must distinguish between the mathematical model and the physical world or between mathematical theories and physical reality.” Morris Kline “That one can draw pictures to represent what one is thinking about in geometry has its drawbacks. One is prone to confuse the abstract concept with the picture and to accept unconsciously properties of the pictures.”
[85] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: It’s hard to explain to people who haven’t worked with machine learning, but we’re still back in the dark ages when it comes to tracking changes and rebuilding models from scratch. It’s so bad it sometimes feels like stepping back in time to when we coded without source control... This is an optimistic scenario with a conscientious researcher, but you can already see how hard it would be for somebody else to come in and reproduce all of these steps and come out with the same result. Every one of these bullet points is an opportunity to inconsistencies to creep in. To make things even more confusing, ML frameworks trade off exact numeric determinism for performance, so if by a miracle somebody did manage to copy the steps exactly, there would still be tiny differences in the end results! In many real-world cases, the researcher won’t have made notes or remember exactly what she did, so even she won’t be able to reproduce the model. Even if she can, the frameworks the model code depend on can change over time, sometimes radically, so she’d need to also snapshot the whole system she was using to ensure that things work. https://petewarden.com/2018/03/19 /the-machine-learning- reproducibility-crisis/
[86] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: https://petewarden.c om/2013/07/18/why -you-should-never- trust-a-data- scientist/
[87] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Arguably the world's greatest mathematician, he worked out a solution to one of the seven great unsolved mathematical problems, the Poincaré conjecture, in 2002. It was a magnificent achievement. Honours, cash, offers of world lecture tours and lucrative teaching posts were hurled at the Russian theorist. But Perelman turned down the lot, including the Fields medal, the mathematical world's equivalent of a Nobel prize, and a million dollars in prize money that the Clay Institute wanted to give him for his work. Since then, he has announced he has given up the study of mathematics altogether and has cut off communications with all journalists and nearly all his friends. https://www.theguardian.com/books/2011/ma r/27/perfect-rigour-grigori-perelman-review
[88] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.pravdareport.com/science/tech/28-04- 2011/117727-Grigori_Perelman-0/ According to the newspaper, both Russian and foreign special services are showing interest in Perelman's discoveries. The scientist has learned some super-knowledge which helps realize creation. Special services need to know whether Perelman and his knowledge may pose a threat to humanity. With his knowledge he can fold the Universe into a spot and then unfold it again. Will mankind survive after this fantastic process? Do we need to control the Universe at all? http://www.claymath.org/library/proceedings /cmip19.pdf
[89] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Reflecting on Math, Theory vs. [Socio-]Physical Reality - Morris Kline “One of the first difficulties in applying statistics is to decide the meaning of the concepts involved.” “In the search for a method of proof, as in finding what to prove, the mathematician must use audacious imagination, insight, and creative ability. His mind must see possible lines of attack where others would not.” “When creating a mathematical proof, the mind does not see the cold, ordered arguments which one reads in texts, but rather it perceives an idea or a scheme which when properly formulated constitutes deductive proof. The formal proof, so to speak, merely sanctions the conquest already made by the intuition.”
[90] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Reflecting on Math, Theory vs. Physical Reality "In my lifetime, I have never bought any stock, much less their derivatives. I deposit money only in an ordinary bank account, and I have rarely even had a fixed deposit account.” Kiyosi Itô, Founder of Itô Calculus aka Stochastic Calculus of Quantitative Finance “There is nothing so practical as a good theory.” - Kurt Lewin “There is nothing so practical as good practice of theory.” - Yogesh Malhotra - (A Personal Constructivist Corollary) "Is then mathematics a collection of diamonds hidden in the depths of the universe and gradually unearthed one by one or is it a collection of synthetic stones manufactured by man but nevertheless so brilliant that it bedazzles those mathematicians who are already partially blinded by pride in their own creations? Several considerations incline us to the latter point of view.“ - Morris Kline "One should question the extent to which mathematics really represents the physical world. It treats those physical concepts which can be represented by numbers or geometrical figures. But physical objects possess other properties was well. We do not usually think of human beings as chunks of matter moving in space and time.“ - Morris Kline "All scientific work depends upon measurement. However, all measurements are approximate.“ - Morris Kline
[91] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Reflecting on Math, Theory vs. Physical Reality - Morris Kline "One finds among the supreme mathematicians men, such as Newton, Lagrange, and Laplace, who even cared little or nothing for mathematics proper, but felt compelled to take up mathematical problems in order to solve physical problems." "If herds of cattle behaved like volumes of gases or like raindrops, then the arithmetic would not apply, and it is only through experience that we learn how they do behave. Hence, we have no guarantee that arithmetic per se represents truths about the physical world." "The mathematician really creates models of reality. Each model has a limited applicability. Moreover, one must distinguish between the mathematical model and the physical world or between mathematical theories and physical reality." "Human nature is a more complicated structure than a mass sliding down an inclined plane or a bob vibrating on a spring." "Suppose, next, that one raindrop is added to another raindrop. Do we now have two raindrops? If one cloud is joined to another cloud do we now have two clouds? One may protest that in these examples the merged objects have lost their identity, and that the addition process of arithmetic does not contemplate such loss. And precisely for this reason, arithmetic in the normal sense no longer applies."
[93] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why Interpretability is complicated Why ‘Humans-in-the Loop’ are Even More Critical for Interpretability Malhotra, Y., Bringing the Adopter Back Into the Adoption Process: A Personal Construction Framework of Information Technology Adoption. Journal of High Technology Management Research, 10(1), 1999, 79-104. Socio-Technical Systems
[94] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why Interpretability is complicated Why ‘Humans-in-the Loop’ are Even More Critical for Interpretability Socio-Technical Systems
[95] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Why Machine Learning Doesn’t ‘Make Sense’ Or Sense ‘Meaning’ Malhotra, Y., Bringing the Adopter Back Into the Adoption Process: A Personal Construction Framework of Information Technology Adoption. Journal of High Technology Management Research, 10(1), 1999, 79-104.
[96] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Cognition ActionAffect Socio-Psychology and Neuroscience of ‘Making Sense’ and Sensing ‘Meaning’ Damásio presents the "somatic marker hypothesis", a proposed mechanism by which emotions guide (or bias) behavior and decision-making, and positing that rationality requires emotional input. He argues that René Descartes' "error" was the dualist separation of mind and body, rationality and emotion. https://en.wikipedia.org/wiki/Descartes%27_Error “Damasio’s essential insight is that feelings are “mental experiences of body states,” which arise as the brain interprets emotions, themselves physical states arising from the body’s responses to external stimuli. (The order of such events is: I am threatened, experience fear, and feel horror.) He has suggested that consciousness, whether the primitive “core consciousness” of animals or the “extended” self- conception of humans, requiring autobiographical memory, emerges from emotions and feelings.” https://www.technologyreview.com/s/528151 /the-importance-of-feelings/ “Thinking, feeling, and deciding are the most intimately human of all things, and yet we understand them hardly at all.” https://www.technologyreview.com/s/528221 /peering-inside-the-workings-of-the-brain/
[97] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Cognition ActionAffect How Humans “Make Sense” Where every “aspiring” ‘Data Scientist’ starts by rote Function Form “In the search for a method of proof, as in finding what to prove, the mathematician must use audacious imagination, insight, and creative ability. His mind must see possible lines of attack where others would not.” Morris Kline
[99] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: "Recently, such probabilistic, statistical, and numerical methods related concerns are in globally popular press related to cybersecurity controls and compliance. Earlier, similar probabilistic, statistical, and numerical methods related concerns were in the global popular press in the context of the global financial crisis. Future questions focused on the underlying assumptions and logic may focus on related implications for compliance, controls, valuation, risk management, etc. Likewise, recent developments about mathematical entropy measures shedding new light on apparently greater vulnerability of prior encryption mechanisms may offer additional insights for compliance and control experts. For instance, given related mathematical, statistical and numerical frameworks, analysis may also focus on potential implications for pricing, valuation and risk models. The important point is that many such fundamental assumptions and logic underlying widely used probabilistic, statistical, and numerical methods may not as readily meet the eye." Interpretability, Explainability, and, Model Risk are Related Issues Hence, they need to be addressed together for AI and Machine Learning Future of Bitcoin & Statistical Probabilistic Quantitative Methods: Global Financial Regulation (Interview: Hong Kong Institute of CPAs) http://yogeshmalhotra.com/Future_of_Bitcoin.html Bitcoin Protocol: Model of ‘Cryptographic Proof’ Based Global Crypto-Currency & Electronic Payments System http://yogeshmalhotra.com/BitcoinProtocol.html January 20, 2014 December 04, 2013 GDPR
[100] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: AI-ML Risk Management & Controls Most Critical Lesser Concern about the Next ‘AI Winter’ Greater Concern about the ‘Nuclear Winter’*
[101] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: http://www.europarl.europa.eu /thinktank/en/document.html? reference=EPRS_IDA(2018)6 14547 http://www.europarl.europa.eu/ RegData/etudes/IDAN/2018/61 4547/EPRS_IDA(2018)614547 _EN.pdf Adaptability-Generalizability Past Prediction vs. Future Anticipation KMS & Risk Management Controls Self-Adaptive Complex Systems AI-ML Knowledge Management Systems Creativity, Imagination, Innovation, Intuition, Insight Known vs. Unknown Routine, Structured, Procedural Non-routine, Unstructured, Non-procedural With Great Power Comes Great Responsibility
[102] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: Adaptability-Generalizability Past Prediction vs. Future Anticipation KMS & Risk Management Controls Self-Adaptive Complex Systems AI-ML Knowledge Management Systems Sense Making Past vs. Future ‘Historical Data’ Known vs. Unknown
[103] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: The European Parliament Think Tank's Research Policy document 'Should we fear artificial intelligence?' reflects the ongoing mainstream debate between the Utopian and Dystopian aspects of AI and Machine Learning. "Powerful AIs can in principle be given nearly any goal, which is a source of both risk and opportunity. There are myriad possible malicious uses of AI and many ways in which it might be used in a harmful manner unintentionally, such as with algorithmic bias. Perhaps most fundamentally, the control problem will have to be addressed – that is, we will need to learn how to ensure that AI systems achieve the goals we want them to without causing harm during their learning process, misinterpreting what is desired of them, or resisting human control." Third in the series of the Princeton Presentations on AI and Machine Learning Risk Management & Control Systems, the current presentation develops fundamental guidance on the design, development, and implementation of AI, Machine Learning, and Deep Learning Models and Methods. The 2018 Princeton presentation will focus on "the control problem" which is a critical prerequisite for AI systems to have positive impacts by further developing upon my prior two presentations that pioneered Cyber-Finance-Trust™ Model Risk Management & Model Risk Arbitrage™ practices at prior Princeton Quant Trading Conferences. Starting with the first technical report on the Bitcoin Blockchain Cryptographic Proof of Work; spanning latest developments in AI, Machine, Learning, Deep Learning, and, Generative Adversarial Networks; and, hedge fund algorithmic trading, the presentation generates interesting insights about the most critical role of risk management controls. Such role of risk management controls is most critical in not only getting the best out of AI, but also ensuring that the worst fears about the AI do not really come true. Abstract
[104] Model Risk Management in AI, Machine Learning & Deep Learning AI, Machine Learning & Deep Learning Risk Management & Controls Beyond Deep Learning and Generative Adversarial Networks... Copyright, Dr. Yogesh Malhotra, 2018www.yogeshmalhotra.com Princeton Fintech and Quant Conference @ , April 21, 2018 Conference sponsors include: 2018 Princeton Fintech & Quant Conference Princeton University, April 21, 2018 Princeton Presentations in AI-ML Risk Management & Control Systems 2016 Princeton Quant Trading Conference, Princeton University How to Navigate ‘Uncertainty’... When ‘Models’ Are ‘Wrong’... and ‘Knowledge’... ‘Imperfect’! Knight Reconsidered Again: Risk, Uncertainty, & Profit beyond ZIRP & NIRP 2015 Princeton Quant Trading Conference, Princeton University Future of Finance Beyond 'Flash Boys': Risk Modeling for Managing Uncertainty in an Increasingly Non-Deterministic Cyber World: Knight Reconsidered: Risk, Uncertainty, and Profit for the Cyber Era Yogi Dr. Yogesh Malhotra Post-Doctoral R&D in AI, Machine Learning & Deep Learning Marquis Who's Who in the World® 1999-, Marquis Who's Who in America® 2002-, Marquis Who's Who in Finance & Industry® 2001-, Marquis Who's Who in Science & Engineering® 2006- www.yogeshmalhotra.com (646) 770-7993 dr.yogesh.malhotra@gmail.com Global Risk Management Network, LLC 757 Warren Road, Cornell Business & Technology Park, Ithaca, NY 14852-4892 http://www.linkedin.com/in/yogeshmalhotra www.FutureOfFinance.org

Check these out next

2018 Princeton Fintech & Quant Conference: AI, Machine Learning & Deep Learning Risk Management & Controls: Beyond Deep Learning and Generative Adversarial Networks: Model Risk Management in AI, Machine Learning & Deep Learning

Recommended

More Related Content

Slideshows for you(18)

Similar to 2018 Princeton Fintech & Quant Conference: AI, Machine Learning & Deep Learning Risk Management & Controls: Beyond Deep Learning and Generative Adversarial Networks: Model Risk Management in AI, Machine Learning & Deep Learning(20)

Recently uploaded(20)

2018 Princeton Fintech & Quant Conference: AI, Machine Learning & Deep Learning Risk Management & Controls: Beyond Deep Learning and Generative Adversarial Networks: Model Risk Management in AI, Machine Learning & Deep Learning

Editor's Notes