Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

分子のグラフ表現と機械学習

169 views

Published on

2018年9月18日
第79回応用物理学会秋季学術講演会 特別シンポジウム
「インフォマティクスへの招待」~機械学習・インフォマティクスは応用物理をどう変えるか?~
https://meeting.jsap.or.jp/jsapm/wp-content/uploads/2018/06/SP1-1.pdf

Published in: Science
  • Be the first to comment

  • Be the first to like this

分子のグラフ表現と機械学習

  1. 1. !2 
 
 

  2. 2. !2 
 
 

  3. 3. !3
  4. 4. !4 All models are wrong, but some are useful (George Box)
  5. 5. !4 All models are wrong, but some are useful (George Box) David Hand Theory-driven models can be wrong But data-driven models cannot be wrong
  6. 6. !4 All models are wrong, but some are useful (George Box) David Hand Theory-driven models can be wrong But data-driven models cannot be wrong or right
  7. 7. !4 All models are wrong, but some are useful (George Box) David Hand Theory-driven models can be wrong But data-driven models cannot be wrong or right Data-driven are not trying to describe an underlying reality. so they could be poor or useless, but not wrong But are merely intended to be useful
  8. 8.
  9. 9. !6 REVIEW Inverse molecular design using machine learning: Generative models for matter engineering Benjamin Sanchez-Lengeling1 and Alán Aspuru-Guzik2,3,4 * The discovery of new materials can bring enormous societal and technological progress. In this context, exploring completely the large space of potential materials is computationally intractable. Here, we review methods for achieving inverse design, which aims to discover tailored materials from the starting point of a particular desired functionality. Recent advances from the rapidly growing field of artificial intelligence, mostly from the subfield of machine learning, have resulted in a fertile exchange of ideas, where approaches to inverse molecular design are being proposed and employed at a rapid pace. Among these, deep generative models have been applied to numerous classes of materials: rational design of prospective drugs, synthetic routes to organic compounds, and optimization of photovoltaics and redox flow batteries, as well as a variety of other solid-state materials. M any of the challenges of the 21st century (1), from personalized health care to energy production and storage, share a common theme: materials are part of the solution (2). In some cases, the solu- tions to these challenges are fundamentally limited by the physics and chemistry of a ma- terial, such as the relationship of a materials bandgap to the thermodynamic limits for the generation of solar energy (3). Several important materials discoveries arose by chance or through a process of trial and error. For example, vulcanized rubber was prepared in the 19th century from random mixtures of com- pounds, based on the observation that heating with additives such as sulfur improved the rubber’s durability. At the molecular level, in- dividual polymer chains cross-linked, forming bridges that enhanced the macroscopic mechan- ical properties (4). Other notable examples in this vein include Teflon, anesthesia, Vaseline, Perkin’s mauve, and penicillin. Furthermore, these materials come from common chemical compounds found in nature. Potential drugs either were prepared by synthesis in a chem- ical laboratory or were isolated from plants, soil bacteria, or fungus. For example, up until 2014, 49% of small-molecule cancer drugs were natural products or their derivatives (5). In the future, disruptive advances in the dis- covery of matter could instead come from unex- plored regions of the set of all possible molecular and solid-state compounds, known as chemical space (6, 7). One of the largest collections of molecules, the chemical space project (8), has mapped 166.4 billion molecules that contain at most 17 heavy atoms. For pharmacologically rele- vant small molecules, the number of structures is estimated to be on the order of 1060 (9). Adding consideration of the hierarchy of scale from sub- nanometer to microscopic and mesoscopic fur- ther complicates exploration of chemical space in its entirety (10). Therefore, any global strategy for covering this space might seem impossible. Simulation offers one way of probing this space without experimentation. The physics and chemistry of these molecules are governed by quantum mechanics, which can be solved via the Schrödinger equation to arrive at their ex- act properties. In practice, approximations are used to lower computational time at the cost of accuracy. Although theory enjoys enormous progress, now routinely modeling molecules, clusters, and perfect as well as defect-laden periodic solids, the size of chemical space is still overwhelming, and smart navigation is required. For this purpose, machine learning (ML), deep learning (DL), and artificial intelligence (AI) have a potential role to play because their computational strategies automatically improve through experience (11). In the context of materials, ML techniques are often used for property prediction, seeking to learn a function that maps a molecular material to the property of choice. Deep generative models are a special class of DL methods that seek to model the underlying probability distribution of both structure and property and relate them in a nonlinear way. By exploiting patterns in massive datasets, these models can distill average and salient features that characterize molecules (12, 13). Inverse design is a component of a more complex materials discovery process. The time scale for deployment of new technologies, from discovery in a laboratory to a commercial pro- duct, historically, is 15 to 20 years (14). The pro- cess (Fig. 1) conventionally involves the following steps: (i) generate a new or improved material concept and simulate its potential suitability; (ii) synthesize the material; (iii) incorporate the ma- terial into a device or system; and (iv) characterize and measure the desired properties. This cycle generates feedback to repeat, improve, and re- fine future cycles of discovery. Each step can take up to several years. In the era of matter engineering, scientists seek to accelerate these cycles, reducing the FRONTIERS IN COMPUTATION Sanchez-Lengeling et al., Science 361, 360–365 (2018) 27 July 2018 1 of 6 1 Department of Chemistry and Chemical Biology, Harvard University 12 Oxford Street, Cambridge, MA 02138, USA. 2 Department of Chemistry and Department of Computer Science, University of Toronto, Toronto Ontario, M5S 3H6, Canada. 3 Vector Institute for Artificial Intelligence, Toronto, Ontario M5S 1M1, Canada. 4 Canadian Institute for Advanced Research (CIFAR) Senior Fellow Toronto, Ontario M5S 1M1, Canada. *Corresponding author. Email: aspuru@utoronto.ca Fig. 1. Schematic comparison of material discovery paradigms. The current paradigm is outlined at left and exemplified in the center with organic redox flow batteries (92). A closed-loop paradigm is outlined at right. Closing the loop requires incorporating inverse design, smart software (93), AI/ML, embedded systems, and robotics (87) into an integrated ecosystem. IMAGE:ADAPTEDBYK.HOLOSKI onJuly26,2018http://science.sciencemag.org/Downloadedfrom REVIEW https://doi.org/10.1038/s41586-018-0337-2 Machine learning for molecular and materials science Keith T. Butler1 , Daniel W. Davies2 , Hugh Cartwright3 , Olexandr Isayev4 * & Aron Walsh5,6 * Here we summarize recent progress in machine learning for the chemical sciences. We outline machine-learning techniques that are suitable for addressing research questions in this domain, as well as future directions for the field. We envisage a future in which the design, synthesis, characterization and application of molecules and materials is accelerated by artificial intelligence. T he Schrödinger equation provides a powerful structure– property relationship for molecules and materials. For a given spatial arrangement of chemical elements, the distribution of electrons and a wide range of physical responses can be described. The development of quantum mechanics provided a rigorous theoretical foundationforthechemicalbond.In1929,PaulDiracfamouslyproclaimed that the underlying physical laws for the whole of chemistry are “completely known”1 . John Pople, realizing the importance of rapidly developing computer technologies, created a program—Gaussian 70—that could perform ab initio calculations: predicting the behaviour, for molecules of modest size, purely from the fundamental laws of physics2 . In the 1960s, the Quantum Chemistry Program Exchange brought quantum chemistry to the masses in the form of useful practical tools3 . Suddenly, experi- mentalists with little or no theoretical training could perform quantum calculations too. Using modern algorithms and supercomputers, systems containing thousands of interacting ions and electrons can now be described using approximations to the physical laws that govern the world on the atomic scale4–6 . The field of computational chemistry has become increasingly pre- dictive in the twenty-first century, with activity in applications as wide ranging as catalyst development for greenhouse gas conversion, materials discovery for energy harvesting and storage, and computer-assisted drug design7 . The modern chemical-simulation toolkit allows the properties of a compound to be anticipated (with reasonable accuracy) before it has been made in the laboratory. High-throughput computational screening has become routine, giving scientists the ability to calculate the properties of thousands of compounds as part of a single study. In particular, den- sity functional theory (DFT)8,9 , now a mature technique for calculating the structure and behaviour of solids10 , has enabled the development of extensive databases that cover the calculated properties of known and hypothetical systems, including organic and inorganic crystals, single molecules and metal alloys11–13 . The emergence of contemporary artificial-intelligence methods has the potential to substantially alter and enhance the role of computers in science and engineering. The combination of big data and artificial intel- ligence has been referred to as both the “fourth paradigm of science”14 and the “fourth industrial revolution”15 , and the number of applications in the chemical domain is growing at an astounding rate. A subfield of artificial intelligence that has evolved rapidly in recent years is machine learning. At the heart of machine-learning applications lie statistical algo- rithms whose performance, much like that of a researcher, improves with training. There is a growing infrastructure of machine-learning tools for generating, testing and refining scientific models. Such techniques are suitable for addressing complex problems that involve massive combi- natorial spaces or nonlinear processes, which conventional procedures either cannot solve or can tackle only at great computational cost. As the machinery for artificial intelligence and machine learning matures, important advances are being made not only by those in main- stream artificial-intelligence research, but also by experts in other fields (domain experts) who adopt these approaches for their own purposes. As we detail in Box 1, the resources and tools that facilitate the application of machine-learning techniques mean that the barrier to entry is lower than ever. In the rest of this Review, we discuss progress in the application of machine learning to address challenges in molecular and materials research. We review the basics of machine-learning approaches, iden- tify areas in which existing methods have the potential to accelerate research and consider the developments that are required to enable more wide-ranging impacts. Nuts and bolts of machine learning With machine learning, given enough data and a rule-discovery algo- rithm, a computer has the ability to determine all known physical laws (and potentially those that are currently unknown) without human input. In traditional computational approaches, the computer is little more than a calculator, employing a hard-coded algorithm provided by a human expert. By contrast, machine-learning approaches learn the rules that underlie a dataset by assessing a portion of that data and building a model to make predictions. We consider the basic steps involved in the construction of a model, as illustrated in Fig. 1; this constitutes a blueprint of the generic workflow that is required for the successful application of machine learning in a materials-discovery process. Data collection Machine learning comprises models that learn from existing (train- ing) data. Data may require initial preprocessing, during which miss- ing or spurious elements are identified and handled. For example, the Inorganic Crystal Structure Database (ICSD) currently contains more than 190,000 entries, which have been checked for technical mistakes but are still subject to human and measurement errors. Identifying and removing such errors is essential to avoid machine-learning algorithms being misled. There is a growing public concern about the lack of reproducibility and error propagation of experimental data 1 ISIS Facility, Rutherford Appleton Laboratory, Harwell Campus, Harwell, UK. 2 Department of Chemistry, University of Bath, Bath, UK. 3 Department of Chemistry, Oxford University, Oxford, UK. 4 Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. 5 Department of Materials Science and Engineering, Yonsei University, Seoul, South Korea. 6 Department of Materials, Imperial College London, London, UK. *e-mail: olexandr@olexandrisayev.com; a.walsh@imperial.ac.uk 2 6 J U L Y 2 0 1 8 | V O L 5 5 9 | N A T U R E | 5 4 7 © 2018 Springer Nature Limited. All rights reserved. DNA to be sequences into distinct pieces, parcel out the detailed work of sequencing, and then reassemble these independent ef- forts at the end. It is not quite so simple in the world of genome semantics. Despite the differences between genome se- quencing and genetic network discovery, there are clear parallels that are illustrated in Table 1. In genome sequencing, a physical map is useful to provide scaffolding for assembling the fin- ished sequence. In the case of a genetic regula- tory network, a graphical model can play the same role. A graphical model can represent a high-level view of interconnectivity and help isolate modules that can be studied indepen- dently. Like contigs in a genomic sequencing project, low-level functional models can ex- plore the detailed behavior of a module of genes in a manner that is consistent with the higher level graphical model of the system. With stan- dardized nomenclature and compatible model- ing techniques, independent functional models can be assembled into a complete model of the cell under study. To enable this process, there will need to be standardized forms for model representa- tion. At present, there are many different modeling technologies in use, and although models can be easily placed into a database, they are not useful out of the context of their specific modeling package. The need for a standardized way of communicating compu- tational descriptions of biological systems ex- tends to the literature. Entire conferences have been established to explore ways of mining the biology literature to extract se- mantic information in computational form. Going forward, as a community we need to come to consensus on how to represent what we know about biology in computa- tional form as well as in words. The key to postgenomic biology will be the computa- tional assembly of our collective knowl- edge into a cohesive picture of cellular and organism function. With such a comprehen- sive model, we will be able to explore new types of conservation between organisms and make great strides toward new thera- peutics that function on well-characterized pathways. References 1. S. K. Kim et al., Science 293 , 2087 (2001). 2. A. Hartemink et al., paper presented at the Pacific Symposium on Biocomputing 2000, Oahu, Hawaii, 4 to 9 January 2000. 3. D. Pe’er et al., paper presented at the 9th Conference on Intelligent Systems in Molecular Biology (ISMB), Copenhagen, Denmark, 21 to 25 July 2001. 4. H. McAdams, A. Arkin, Proc. Natl. Acad. Sci. U.S.A. 94 , 814 ( 1997 ). 5. A. J. Hartemink, thesis, Massachusetts Institute of Technology, Cambridge (2001). V I E W P O I N T Machine Learning for Science: State of the Art and Future Prospects Eric Mjolsness* and Dennis DeCoste Recent advances in machine learning methods, along with successful applications across a wide variety of fields such as planetary science and bioinformatics, promise powerful new tools for practicing scientists. This viewpoint highlights some useful characteristics of modern machine learn- ing methods and their relevance to scientific applications. We conclude with some speculations on near-term progress and promising directions. Machine learning (ML) (1) is the study of computer algorithms capable of learning to im- prove their performance of a task on the basis of their own previous experience. The field is closely related to pattern recognition and statis- tical inference. As an engineering field, ML has become steadily more mathematical and more successful in applications over the past 20 years. Learning approaches such as data clus- tering, neural network classifiers, and nonlinear regression have found surprisingly wide appli- cation in the practice of engineering, business, and science. A generalized version of the stan- dard Hidden Markov Models of ML practice have been used for ab initio prediction of gene structures in genomic DNA (2). The predictions correlate surprisingly well with subsequent gene expression analysis (3). Postgenomic bi- ology prominently features large-scale gene ex- pression data analyzed by clustering methods (4), a standard topic in unsupervised learning. Many other examples can be given of learning and pattern recognition applications in science. Where will this trend lead? We believe it will lead to appropriate, partial automation of every element of scientific method, from hypothesis generation to model construction to decisive experimentation. Thus, ML has the potential to amplify every aspect of a working scientist’s progress to understanding. It will also, for better or worse, endow intelligent computer systems with some of the general analytic power of scientific thinking. Machine Learning at Every Stage of the Scientific Process Each scientific field has its own version of the scientific process. But the cycle of observing, creating hypotheses, testing by decisive exper- iment or observation, and iteratively building up comprehensive testable models or theories is shared across disciplines. For each stage of this abstracted scientific process, there are relevant developments in ML, statistical inference, and pattern recognition that will lead to semiauto- matic support tools of unknown but potentially broad applicability. Increasingly, the early elements of scientific method—observation and hypothesis genera- tion—face high data volumes, high data acqui- sition rates, or requirements for objective anal- ysis that cannot be handled by human percep- tion alone. This has been the situation in exper- imental particle physics for decades. There automatic pattern recognition for significant events is well developed, including Hough transforms, which are foundational in pattern recognition. A recent example is event analysis for Cherenkov detectors (8) used in neutrino oscillation experiments. Microscope imagery in cell biology, pathology, petrology, and other fields has led to image-processing specialties. So has remote sensing from Earth-observing satellites, such as the newly operational Terra spacecraft with its ASTER (a multispectral thermal radiometer), MISR (multiangle imag- ing spectral radiometer), MODIS (imaging Machine Learning Systems Group, Jet Propulsion Lab- oratory/California Institute of Technology, Pasadena, CA, 91109, USA. *To whom correspondence should be addressed. E- mail: mjolsness@jpl.nasa.gov Table 1. Parallels between genome sequencing and genetic network discovery. Genome sequencing Genome semantics Physical maps Graphical model Contigs Low-level functional models Contig reassembly Module assembly Finished genome sequence Comprehensive model www.sciencemag.org SCIENCE VOL 293 14 SEPTEMBER 2001 2051 C O M P U T E R S A N D S C I E N C E onAugust29,2018http://science.sciencemag.org/Downloadedfrom Nature, 559
 pp. 547–555 (2018) Science, 293 pp. 2051-2055 (2001) Science, 361 pp. 360-365 (2018) Science is changing, the tools of science are changing. And that requires different approaches. (Erich Bloch, 1925-2016)
  10. 10. !7 
 
 CH3 N H3C H NS N O CH3 N OH x ˆyˆy = f✓(x)
  11. 11. CH3 N H3C H NS N O CH3 N OH !8 n f✓ ✓ x ˆyˆy = f✓(x) (x1, y1), (x2, y2), . . . , (xn, yn) ˆyi = f✓(xi)min ✓ nX i=1 error(yi, ˆyi) yiˆyi {f✓(x) : ✓ 2 ⇥}<latexit sha1_base64="Emyg88Fl66FZsIM8Adx6yyqIHAE=">AAACuHichVFNT9RQFD0URUSFETcmbhonGNhMbsEEMiuCG5cwH0BCydDWNzMv9Cvtm4nQ9A/wB4hxpYkLwg9wyYIN/gAW8xOIS0zcuOC20wSVALdp37nn3XN73rt26MpYEQ1GtNEHD8cejT+eePL02eRU6fn0ehz0Ikc0ncANok3bioUrfdFUUrliM4yE5dmu2LB332X7G30RxTLwG2ovFNue1fFlWzqWYqpVWjCTdisxbS8xVVcoK01ns+RjOlfV/2J1U/q62ciwmbZKZapQHvpNYBSgjCJWg9J3mPiAAA568CDgQzF2YSHmZwsGCCFz20iYixjJfF8gxQRre1wluMJidpe/Hc62CtbnPOsZ52qH/+LyG7FSxwyd0xFd0hkd0wX9ubVXkvfIvOzxag+1ImxNHbys/75X5fGq0L1W3elZoY2l3Ktk72HOZKdwhvr+/uFlvVqbSd7QV/rJ/r/QgE75BH7/l/NtTdQ+3+HHZi98Yzwg4/9x3ATr8xWDKsba2/LySjGqcbzCa8zyPBaxjPdYRZP7f8IJzvBDq2o7WkeTw1JtpNC8wD+hRVdv06U+</latexit><latexit sha1_base64="Emyg88Fl66FZsIM8Adx6yyqIHAE=">AAACuHichVFNT9RQFD0URUSFETcmbhonGNhMbsEEMiuCG5cwH0BCydDWNzMv9Cvtm4nQ9A/wB4hxpYkLwg9wyYIN/gAW8xOIS0zcuOC20wSVALdp37nn3XN73rt26MpYEQ1GtNEHD8cejT+eePL02eRU6fn0ehz0Ikc0ncANok3bioUrfdFUUrliM4yE5dmu2LB332X7G30RxTLwG2ovFNue1fFlWzqWYqpVWjCTdisxbS8xVVcoK01ns+RjOlfV/2J1U/q62ciwmbZKZapQHvpNYBSgjCJWg9J3mPiAAA568CDgQzF2YSHmZwsGCCFz20iYixjJfF8gxQRre1wluMJidpe/Hc62CtbnPOsZ52qH/+LyG7FSxwyd0xFd0hkd0wX9ubVXkvfIvOzxag+1ImxNHbys/75X5fGq0L1W3elZoY2l3Ktk72HOZKdwhvr+/uFlvVqbSd7QV/rJ/r/QgE75BH7/l/NtTdQ+3+HHZi98Yzwg4/9x3ATr8xWDKsba2/LySjGqcbzCa8zyPBaxjPdYRZP7f8IJzvBDq2o7WkeTw1JtpNC8wD+hRVdv06U+</latexit><latexit sha1_base64="Emyg88Fl66FZsIM8Adx6yyqIHAE=">AAACuHichVFNT9RQFD0URUSFETcmbhonGNhMbsEEMiuCG5cwH0BCydDWNzMv9Cvtm4nQ9A/wB4hxpYkLwg9wyYIN/gAW8xOIS0zcuOC20wSVALdp37nn3XN73rt26MpYEQ1GtNEHD8cejT+eePL02eRU6fn0ehz0Ikc0ncANok3bioUrfdFUUrliM4yE5dmu2LB332X7G30RxTLwG2ovFNue1fFlWzqWYqpVWjCTdisxbS8xVVcoK01ns+RjOlfV/2J1U/q62ciwmbZKZapQHvpNYBSgjCJWg9J3mPiAAA568CDgQzF2YSHmZwsGCCFz20iYixjJfF8gxQRre1wluMJidpe/Hc62CtbnPOsZ52qH/+LyG7FSxwyd0xFd0hkd0wX9ubVXkvfIvOzxag+1ImxNHbys/75X5fGq0L1W3elZoY2l3Ktk72HOZKdwhvr+/uFlvVqbSd7QV/rJ/r/QgE75BH7/l/NtTdQ+3+HHZi98Yzwg4/9x3ATr8xWDKsba2/LySjGqcbzCa8zyPBaxjPdYRZP7f8IJzvBDq2o7WkeTw1JtpNC8wD+hRVdv06U+</latexit><latexit sha1_base64="Emyg88Fl66FZsIM8Adx6yyqIHAE=">AAACuHichVFNT9RQFD0URUSFETcmbhonGNhMbsEEMiuCG5cwH0BCydDWNzMv9Cvtm4nQ9A/wB4hxpYkLwg9wyYIN/gAW8xOIS0zcuOC20wSVALdp37nn3XN73rt26MpYEQ1GtNEHD8cejT+eePL02eRU6fn0ehz0Ikc0ncANok3bioUrfdFUUrliM4yE5dmu2LB332X7G30RxTLwG2ovFNue1fFlWzqWYqpVWjCTdisxbS8xVVcoK01ns+RjOlfV/2J1U/q62ciwmbZKZapQHvpNYBSgjCJWg9J3mPiAAA568CDgQzF2YSHmZwsGCCFz20iYixjJfF8gxQRre1wluMJidpe/Hc62CtbnPOsZ52qH/+LyG7FSxwyd0xFd0hkd0wX9ubVXkvfIvOzxag+1ImxNHbys/75X5fGq0L1W3elZoY2l3Ktk72HOZKdwhvr+/uFlvVqbSd7QV/rJ/r/QgE75BH7/l/NtTdQ+3+HHZi98Yzwg4/9x3ATr8xWDKsba2/LySjGqcbzCa8zyPBaxjPdYRZP7f8IJzvBDq2o7WkeTw1JtpNC8wD+hRVdv06U+</latexit>
  12. 12. !9 
 
 

  13. 13. !10 CH3 N H3C H NS N O CH3 N OH x ˆyˆy = f✓(x) @<TRIPOS>MOLECULE ***** 13 13 0 0 0 SMALL GASTEIGER @<TRIPOS>ATOM 1 C -2.5458 -9.4750 0.0000 C.2 1 UNL1 0.3080 2 C -3.3708 -9.4750 0.0000 C.2 1 UNL1 0.2529 3 C -2.2875 -8.6917 0.0000 C.2 1 UNL1 0.3838 4 C -3.6208 -8.6917 0.0000 C.3 1 UNL1 0.2067 5 O -2.9583 -8.2042 0.0000 O.3 1 UNL1 -0.4441 6 C -4.3583 -8.3125 0.0000 C.3 1 UNL1 0.2245 7 O -1.5000 -8.4375 0.0000 O.2 1 UNL1 -0.2412 8 O -2.0583 -10.1417 0.0000 O.2 1 UNL1 -0.2764 9 O -3.8500 -10.1417 0.0000 O.2 1 UNL1 -0.2843 10 O -5.0500 -8.7542 0.0000 O.3 1 UNL1 -0.2164 11 O -3.6958 -7.0417 0.0000 O.3 1 UNL1 -0.2174 12 C -4.3958 -7.4875 0.0000 C.3 1 UNL1 0.2185 13 H -4.2083 -9.2667 0.0000 H 1 UNL1 0.0853 @<TRIPOS>BOND 1 2 1 2 2 3 1 1 3 4 2 1 4 5 3 1 5 6 4 1 6 7 3 2 7 8 1 1 8 9 2 1 9 6 10 1 10 11 12 1 11 12 6 1 12 4 13 1 13 5 4 1 OC[C@H](O)[C@H]1OC(=O)C(=C1O)O InChI=1S/C6H8O6/ c7-1-2(8)5-3(9)4(10)6(11)12-5/ h2,5,7-10H,1H2/t2-,5+/m0/s1 CIWBSHSKHKDKBQ-JLAZNSOCSA-N 1 2 3 4 5 6 7 8 9 10 11 12 13
  14. 14. !11 O N N NH NHN N N CH3 CH3 1 2 3 284 Atom + Bond SYBYL MOL2 Pharmacophore 分⼦グラフ表現 … 
 

  15. 15. !12 分⼦グラフ •頂点に原⼦不変量 •辺や頂点対に原⼦対不変量 記述⼦ (3次元や表⾯形状の幾何的量など) (表⾯や堆積など⽴体依存量など) (原⼦の種類,電⼦配置,電荷など) (原⼦間の距離,結合の種類など) DFT B3LYP/6-31G(2df,p) Structure in the ground state 15 properties in the ground state
  16. 16. !13
  17. 17. !14 

  18. 18. !15 Edward Pyro Prob Torch BayesFlow LightGBM (Microsoft) (RuleQuest)
 See5/C5.0 & Cubist (Salford Systems)
 CART® MARS® TreeNet®
 Random Forests® CatBoost (Yandex) TFBoost (Google) TenscentBoost (Tenscent) Sherwood decision forests
  19. 19. !16 SVM, LogReg, GPR, RF, etc. 
 … … 

  20. 20. !17
  21. 21. !18 • Descriptors • Descriptors3D • GraphDescriptors • Fingerprints • ChemicalFeatures • ChemicalForceFields rdkit.Chem rdkit.ML.Descriptors
  22. 22. !19 
 

  23. 23. !20 x =  x1 x2 y x1 x2 y 1 1 w0 ji w00 i wkj 1 bk b0 j b00 i
  24. 24. !21 y 1 w3<latexit sha1_base64="f61CmxDBjyiOdqe46d2urO1LcF8=">AAAChnichVE9S8NQFD3Gr/pddRFcikVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8CatiIN6w8u777x77jvvHd21TD8ganZJ3T29ff2xgcGh4ZHRsfj4xKbvVD1DFA3HcrxtXfOFZdqiGJiBJbZdT2gV3RJb+tFKuL9VE55vOvZGcOKKvYpWts0D09AChgrHJaUUT5KcUhUloyZIVrLpjEKcLC4oWUonUjJFkUQn8k78AbvYhwMDVVQgYCPg3IIGn78dpEBwGdtDnTGPMzPaFzjDIHOrXCW4QmP0iP9lXu10UJvXYU8/Yht8isXDY2YCs/RMd9SiJ7qnF/r4tVc96hFqOeFZb3OFWxo7nyq8/8uq8Bzg8Jv1p+YAB8hGWk3W7kZIeAujza+dNlqFpfXZ+hzd0Cvrv6YmPfIN7Nqbcbsm1q/+0KOzFn4xNujLhcTvyeaCnGIf19LJ3HLHqhimMYN59kNFDqvIo8j9y7jAJRpSTJKljKS2S6WuDmcSP0LKfQLoGZDc</latexit><latexit sha1_base64="f61CmxDBjyiOdqe46d2urO1LcF8=">AAAChnichVE9S8NQFD3Gr/pddRFcikVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8CatiIN6w8u777x77jvvHd21TD8ganZJ3T29ff2xgcGh4ZHRsfj4xKbvVD1DFA3HcrxtXfOFZdqiGJiBJbZdT2gV3RJb+tFKuL9VE55vOvZGcOKKvYpWts0D09AChgrHJaUUT5KcUhUloyZIVrLpjEKcLC4oWUonUjJFkUQn8k78AbvYhwMDVVQgYCPg3IIGn78dpEBwGdtDnTGPMzPaFzjDIHOrXCW4QmP0iP9lXu10UJvXYU8/Yht8isXDY2YCs/RMd9SiJ7qnF/r4tVc96hFqOeFZb3OFWxo7nyq8/8uq8Bzg8Jv1p+YAB8hGWk3W7kZIeAujza+dNlqFpfXZ+hzd0Cvrv6YmPfIN7Nqbcbsm1q/+0KOzFn4xNujLhcTvyeaCnGIf19LJ3HLHqhimMYN59kNFDqvIo8j9y7jAJRpSTJKljKS2S6WuDmcSP0LKfQLoGZDc</latexit><latexit sha1_base64="f61CmxDBjyiOdqe46d2urO1LcF8=">AAAChnichVE9S8NQFD3Gr/pddRFcikVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8CatiIN6w8u777x77jvvHd21TD8ganZJ3T29ff2xgcGh4ZHRsfj4xKbvVD1DFA3HcrxtXfOFZdqiGJiBJbZdT2gV3RJb+tFKuL9VE55vOvZGcOKKvYpWts0D09AChgrHJaUUT5KcUhUloyZIVrLpjEKcLC4oWUonUjJFkUQn8k78AbvYhwMDVVQgYCPg3IIGn78dpEBwGdtDnTGPMzPaFzjDIHOrXCW4QmP0iP9lXu10UJvXYU8/Yht8isXDY2YCs/RMd9SiJ7qnF/r4tVc96hFqOeFZb3OFWxo7nyq8/8uq8Bzg8Jv1p+YAB8hGWk3W7kZIeAujza+dNlqFpfXZ+hzd0Cvrv6YmPfIN7Nqbcbsm1q/+0KOzFn4xNujLhcTvyeaCnGIf19LJ3HLHqhimMYN59kNFDqvIo8j9y7jAJRpSTJKljKS2S6WuDmcSP0LKfQLoGZDc</latexit><latexit sha1_base64="f61CmxDBjyiOdqe46d2urO1LcF8=">AAAChnichVE9S8NQFD3Gr/pddRFcikVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8CatiIN6w8u777x77jvvHd21TD8ganZJ3T29ff2xgcGh4ZHRsfj4xKbvVD1DFA3HcrxtXfOFZdqiGJiBJbZdT2gV3RJb+tFKuL9VE55vOvZGcOKKvYpWts0D09AChgrHJaUUT5KcUhUloyZIVrLpjEKcLC4oWUonUjJFkUQn8k78AbvYhwMDVVQgYCPg3IIGn78dpEBwGdtDnTGPMzPaFzjDIHOrXCW4QmP0iP9lXu10UJvXYU8/Yht8isXDY2YCs/RMd9SiJ7qnF/r4tVc96hFqOeFZb3OFWxo7nyq8/8uq8Bzg8Jv1p+YAB8hGWk3W7kZIeAujza+dNlqFpfXZ+hzd0Cvrv6YmPfIN7Nqbcbsm1q/+0KOzFn4xNujLhcTvyeaCnGIf19LJ3HLHqhimMYN59kNFDqvIo8j9y7jAJRpSTJKljKS2S6WuDmcSP0LKfQLoGZDc</latexit> w1<latexit sha1_base64="ooGd9XNWxcj5k2mqweAC3uUeHVM=">AAAChnichVE9S8NQFD2NX/W76iK4FIviFG5Ma4tT0cWxtVYFlZLEZw2mSUjSihb/gOBqBycFB/EH+ANc/AMO/QniWMHFwdu0Ig7qDS/vvvPuue+8d3TXMv2AqBmRenr7+geig0PDI6Nj47GJyU3fqXqGKBqO5XjbuuYLy7RFMTADS2y7ntAquiW29KPV9v5WTXi+6dgbwYkr9ipa2TYPTEMLGCocl5RSLEGyklbVVDpOsppJplTiZGlRzVAyrsgURgLdyDmxB+xiHw4MVFGBgI2AcwsafP52oIDgMraHOmMeZ2a4L3CGIeZWuUpwhcboEf/LvNrpojav2z39kG3wKRYPj5lxzNEz3VGLnuieXujj1171sEdbywnPeocr3NL4+XTh/V9WhecAh9+sPzUHOEAm1GqydjdE2rcwOvzaaaNVWF6fq8/TDb2y/mtq0iPfwK69Gbd5sX71hx6dtfCLsUFfLsR/TzYXZYV9zCcT2ZWuVVHMYBYL7EcaWawhhyL3L+MCl2hIUUmWUlK6UypFupwp/Agp+wnj2ZDa</latexit><latexit sha1_base64="ooGd9XNWxcj5k2mqweAC3uUeHVM=">AAAChnichVE9S8NQFD2NX/W76iK4FIviFG5Ma4tT0cWxtVYFlZLEZw2mSUjSihb/gOBqBycFB/EH+ANc/AMO/QniWMHFwdu0Ig7qDS/vvvPuue+8d3TXMv2AqBmRenr7+geig0PDI6Nj47GJyU3fqXqGKBqO5XjbuuYLy7RFMTADS2y7ntAquiW29KPV9v5WTXi+6dgbwYkr9ipa2TYPTEMLGCocl5RSLEGyklbVVDpOsppJplTiZGlRzVAyrsgURgLdyDmxB+xiHw4MVFGBgI2AcwsafP52oIDgMraHOmMeZ2a4L3CGIeZWuUpwhcboEf/LvNrpojav2z39kG3wKRYPj5lxzNEz3VGLnuieXujj1171sEdbywnPeocr3NL4+XTh/V9WhecAh9+sPzUHOEAm1GqydjdE2rcwOvzaaaNVWF6fq8/TDb2y/mtq0iPfwK69Gbd5sX71hx6dtfCLsUFfLsR/TzYXZYV9zCcT2ZWuVVHMYBYL7EcaWawhhyL3L+MCl2hIUUmWUlK6UypFupwp/Agp+wnj2ZDa</latexit><latexit sha1_base64="ooGd9XNWxcj5k2mqweAC3uUeHVM=">AAAChnichVE9S8NQFD2NX/W76iK4FIviFG5Ma4tT0cWxtVYFlZLEZw2mSUjSihb/gOBqBycFB/EH+ANc/AMO/QniWMHFwdu0Ig7qDS/vvvPuue+8d3TXMv2AqBmRenr7+geig0PDI6Nj47GJyU3fqXqGKBqO5XjbuuYLy7RFMTADS2y7ntAquiW29KPV9v5WTXi+6dgbwYkr9ipa2TYPTEMLGCocl5RSLEGyklbVVDpOsppJplTiZGlRzVAyrsgURgLdyDmxB+xiHw4MVFGBgI2AcwsafP52oIDgMraHOmMeZ2a4L3CGIeZWuUpwhcboEf/LvNrpojav2z39kG3wKRYPj5lxzNEz3VGLnuieXujj1171sEdbywnPeocr3NL4+XTh/V9WhecAh9+sPzUHOEAm1GqydjdE2rcwOvzaaaNVWF6fq8/TDb2y/mtq0iPfwK69Gbd5sX71hx6dtfCLsUFfLsR/TzYXZYV9zCcT2ZWuVVHMYBYL7EcaWawhhyL3L+MCl2hIUUmWUlK6UypFupwp/Agp+wnj2ZDa</latexit><latexit sha1_base64="ooGd9XNWxcj5k2mqweAC3uUeHVM=">AAAChnichVE9S8NQFD2NX/W76iK4FIviFG5Ma4tT0cWxtVYFlZLEZw2mSUjSihb/gOBqBycFB/EH+ANc/AMO/QniWMHFwdu0Ig7qDS/vvvPuue+8d3TXMv2AqBmRenr7+geig0PDI6Nj47GJyU3fqXqGKBqO5XjbuuYLy7RFMTADS2y7ntAquiW29KPV9v5WTXi+6dgbwYkr9ipa2TYPTEMLGCocl5RSLEGyklbVVDpOsppJplTiZGlRzVAyrsgURgLdyDmxB+xiHw4MVFGBgI2AcwsafP52oIDgMraHOmMeZ2a4L3CGIeZWuUpwhcboEf/LvNrpojav2z39kG3wKRYPj5lxzNEz3VGLnuieXujj1171sEdbywnPeocr3NL4+XTh/V9WhecAh9+sPzUHOEAm1GqydjdE2rcwOvzaaaNVWF6fq8/TDb2y/mtq0iPfwK69Gbd5sX71hx6dtfCLsUFfLsR/TzYXZYV9zCcT2ZWuVVHMYBYL7EcaWawhhyL3L+MCl2hIUUmWUlK6UypFupwp/Agp+wnj2ZDa</latexit> b1<latexit sha1_base64="grn1ZBaXoJImbjZpbmJ4Zi61F48=">AAAChnichVE9S8NQFD3Gr/rZqovgUiyKU7hpWlucii6Oam0VVEoSnzU0TUKSFmrxDwiudnBScBB/gD/AxT/g4E8QxwouDt6mFXFQb3h59513z33nvaO7lukHRM99Uv/A4NBwZGR0bHxiMhqbmi76Ts0zRMFwLMfb1TVfWKYtCoEZWGLX9YRW1S2xo1fWOvs7deH5pmNvBw1XHFS1sm0emYYWMJTXS0opliBZyahqOhMnWc2m0ipxspxUs5SKKzKFkUAvNpzYPfZxCAcGaqhCwEbAuQUNPn97UEBwGTtAkzGPMzPcFzjFKHNrXCW4QmO0wv8yr/Z6qM3rTk8/ZBt8isXDY2YcC/REt9SmR7qjF/r4tVcz7NHR0uBZ73KFW4qezebf/2VVeQ5w/M36U3OAI2RDrSZrd0Okcwujy6+ftNr5la2F5iJd0yvrv6JneuAb2PU342ZTbF3+oUdnLfxibNCXC/Hfk2JSVtjHzVQit9qzKoI5zGOJ/cggh3VsoMD9yzjHBVpSRJKltJTplkp9Pc4MfoSU+wS3D5DF</latexit><latexit sha1_base64="grn1ZBaXoJImbjZpbmJ4Zi61F48=">AAAChnichVE9S8NQFD3Gr/rZqovgUiyKU7hpWlucii6Oam0VVEoSnzU0TUKSFmrxDwiudnBScBB/gD/AxT/g4E8QxwouDt6mFXFQb3h59513z33nvaO7lukHRM99Uv/A4NBwZGR0bHxiMhqbmi76Ts0zRMFwLMfb1TVfWKYtCoEZWGLX9YRW1S2xo1fWOvs7deH5pmNvBw1XHFS1sm0emYYWMJTXS0opliBZyahqOhMnWc2m0ipxspxUs5SKKzKFkUAvNpzYPfZxCAcGaqhCwEbAuQUNPn97UEBwGTtAkzGPMzPcFzjFKHNrXCW4QmO0wv8yr/Z6qM3rTk8/ZBt8isXDY2YcC/REt9SmR7qjF/r4tVcz7NHR0uBZ73KFW4qezebf/2VVeQ5w/M36U3OAI2RDrSZrd0Okcwujy6+ftNr5la2F5iJd0yvrv6JneuAb2PU342ZTbF3+oUdnLfxibNCXC/Hfk2JSVtjHzVQit9qzKoI5zGOJ/cggh3VsoMD9yzjHBVpSRJKltJTplkp9Pc4MfoSU+wS3D5DF</latexit><latexit sha1_base64="grn1ZBaXoJImbjZpbmJ4Zi61F48=">AAAChnichVE9S8NQFD3Gr/rZqovgUiyKU7hpWlucii6Oam0VVEoSnzU0TUKSFmrxDwiudnBScBB/gD/AxT/g4E8QxwouDt6mFXFQb3h59513z33nvaO7lukHRM99Uv/A4NBwZGR0bHxiMhqbmi76Ts0zRMFwLMfb1TVfWKYtCoEZWGLX9YRW1S2xo1fWOvs7deH5pmNvBw1XHFS1sm0emYYWMJTXS0opliBZyahqOhMnWc2m0ipxspxUs5SKKzKFkUAvNpzYPfZxCAcGaqhCwEbAuQUNPn97UEBwGTtAkzGPMzPcFzjFKHNrXCW4QmO0wv8yr/Z6qM3rTk8/ZBt8isXDY2YcC/REt9SmR7qjF/r4tVcz7NHR0uBZ73KFW4qezebf/2VVeQ5w/M36U3OAI2RDrSZrd0Okcwujy6+ftNr5la2F5iJd0yvrv6JneuAb2PU342ZTbF3+oUdnLfxibNCXC/Hfk2JSVtjHzVQit9qzKoI5zGOJ/cggh3VsoMD9yzjHBVpSRJKltJTplkp9Pc4MfoSU+wS3D5DF</latexit><latexit sha1_base64="grn1ZBaXoJImbjZpbmJ4Zi61F48=">AAAChnichVE9S8NQFD3Gr/rZqovgUiyKU7hpWlucii6Oam0VVEoSnzU0TUKSFmrxDwiudnBScBB/gD/AxT/g4E8QxwouDt6mFXFQb3h59513z33nvaO7lukHRM99Uv/A4NBwZGR0bHxiMhqbmi76Ts0zRMFwLMfb1TVfWKYtCoEZWGLX9YRW1S2xo1fWOvs7deH5pmNvBw1XHFS1sm0emYYWMJTXS0opliBZyahqOhMnWc2m0ipxspxUs5SKKzKFkUAvNpzYPfZxCAcGaqhCwEbAuQUNPn97UEBwGTtAkzGPMzPcFzjFKHNrXCW4QmO0wv8yr/Z6qM3rTk8/ZBt8isXDY2YcC/REt9SmR7qjF/r4tVcz7NHR0uBZ73KFW4qezebf/2VVeQ5w/M36U3OAI2RDrSZrd0Okcwujy6+ftNr5la2F5iJd0yvrv6JneuAb2PU342ZTbF3+oUdnLfxibNCXC/Hfk2JSVtjHzVQit9qzKoI5zGOJ/cggh3VsoMD9yzjHBVpSRJKltJTplkp9Pc4MfoSU+wS3D5DF</latexit> b2<latexit sha1_base64="MFl5ubUl7Inr9VexK9vFPrBeDjQ=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoIkKihLT1iQ2lbdpCgsQ/YOIqg5MmDsYf4A9w8Q84+BOMoyYuDl4Kxjigt3l99513z33nvaM5puH5RM8D0uDQ8MhoaGx8YnJqOhyZmd3z7Lqri7xum7Zb1FRPmIYl8r7hm6LouEKtaaYoaNWNzn6hIVzPsK1dv+mIUk2tWMaRoas+QzmtnChHYiTH04qSSkdJVjLJlEKcrCaUDCWjcZmCiKEXW3bkHgc4hA0dddQgYMHn3IQKj799xEFwGCuhxZjLmRHsC5xinLl1rhJcoTJa5X+FV/s91OJ1p6cXsHU+xeThMjOKJXqiW3qjR7qjF/rs26sV9OhoafKsdbnCKYfP5nMf/7JqPPs4/mH9qdnHETKBVoO1OwHSuYXe5TdO2m+5tZ2l1jJd0yvrv6JneuAbWI13/WZb7Fz+oUdjLfxibNC3C9H+yV5CjrOP28lYdr1nVQgLWMQK+5FGFpvYQp77V3COC7SlkCRLKSndLZUGepw5/Aop+wW5L5DG</latexit><latexit sha1_base64="MFl5ubUl7Inr9VexK9vFPrBeDjQ=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoIkKihLT1iQ2lbdpCgsQ/YOIqg5MmDsYf4A9w8Q84+BOMoyYuDl4Kxjigt3l99513z33nvaM5puH5RM8D0uDQ8MhoaGx8YnJqOhyZmd3z7Lqri7xum7Zb1FRPmIYl8r7hm6LouEKtaaYoaNWNzn6hIVzPsK1dv+mIUk2tWMaRoas+QzmtnChHYiTH04qSSkdJVjLJlEKcrCaUDCWjcZmCiKEXW3bkHgc4hA0dddQgYMHn3IQKj799xEFwGCuhxZjLmRHsC5xinLl1rhJcoTJa5X+FV/s91OJ1p6cXsHU+xeThMjOKJXqiW3qjR7qjF/rs26sV9OhoafKsdbnCKYfP5nMf/7JqPPs4/mH9qdnHETKBVoO1OwHSuYXe5TdO2m+5tZ2l1jJd0yvrv6JneuAbWI13/WZb7Fz+oUdjLfxibNC3C9H+yV5CjrOP28lYdr1nVQgLWMQK+5FGFpvYQp77V3COC7SlkCRLKSndLZUGepw5/Aop+wW5L5DG</latexit><latexit sha1_base64="MFl5ubUl7Inr9VexK9vFPrBeDjQ=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoIkKihLT1iQ2lbdpCgsQ/YOIqg5MmDsYf4A9w8Q84+BOMoyYuDl4Kxjigt3l99513z33nvaM5puH5RM8D0uDQ8MhoaGx8YnJqOhyZmd3z7Lqri7xum7Zb1FRPmIYl8r7hm6LouEKtaaYoaNWNzn6hIVzPsK1dv+mIUk2tWMaRoas+QzmtnChHYiTH04qSSkdJVjLJlEKcrCaUDCWjcZmCiKEXW3bkHgc4hA0dddQgYMHn3IQKj799xEFwGCuhxZjLmRHsC5xinLl1rhJcoTJa5X+FV/s91OJ1p6cXsHU+xeThMjOKJXqiW3qjR7qjF/rs26sV9OhoafKsdbnCKYfP5nMf/7JqPPs4/mH9qdnHETKBVoO1OwHSuYXe5TdO2m+5tZ2l1jJd0yvrv6JneuAbWI13/WZb7Fz+oUdjLfxibNC3C9H+yV5CjrOP28lYdr1nVQgLWMQK+5FGFpvYQp77V3COC7SlkCRLKSndLZUGepw5/Aop+wW5L5DG</latexit><latexit sha1_base64="MFl5ubUl7Inr9VexK9vFPrBeDjQ=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoIkKihLT1iQ2lbdpCgsQ/YOIqg5MmDsYf4A9w8Q84+BOMoyYuDl4Kxjigt3l99513z33nvaM5puH5RM8D0uDQ8MhoaGx8YnJqOhyZmd3z7Lqri7xum7Zb1FRPmIYl8r7hm6LouEKtaaYoaNWNzn6hIVzPsK1dv+mIUk2tWMaRoas+QzmtnChHYiTH04qSSkdJVjLJlEKcrCaUDCWjcZmCiKEXW3bkHgc4hA0dddQgYMHn3IQKj799xEFwGCuhxZjLmRHsC5xinLl1rhJcoTJa5X+FV/s91OJ1p6cXsHU+xeThMjOKJXqiW3qjR7qjF/rs26sV9OhoafKsdbnCKYfP5nMf/7JqPPs4/mH9qdnHETKBVoO1OwHSuYXe5TdO2m+5tZ2l1jJd0yvrv6JneuAbWI13/WZb7Fz+oUdjLfxibNC3C9H+yV5CjrOP28lYdr1nVQgLWMQK+5FGFpvYQp77V3COC7SlkCRLKSndLZUGepw5/Aop+wW5L5DG</latexit> w2<latexit sha1_base64="KnpGkPeMEcSDto2IGA0dJDFYgro=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoImCihrT1iY2lbdqCUeIfMHHVwUkTB+MP8Ae4+Acc+AnGERMXBy8FYxzA27y++867577z3tEc0/B8okaf1D8wODQcGhkdG5+YDEempgueXXV1kddt03Z3NNUTpmGJvG/4pthxXKFWNFMUteO11n6xJlzPsK1t/9QR+xW1bBmHhq76DOVOSolSJEZyPK0oqXSUZCWTTCnEyXJCyVAyGpcpiBg6sWFHnrCHA9jQUUUFAhZ8zk2o8PjbRRwEh7F91BlzOTOCfYFzjDK3ylWCK1RGj/lf5tVuB7V43erpBWydTzF5uMyMYoFe6YGa9EKP9EZfXXvVgx4tLac8a22ucErhi9nc57+sCs8+jn5ZPTX7OEQm0GqwdidAWrfQ2/za2XUzt7K1UF+kO3pn/bfUoGe+gVX70O83xdZNDz0aa+EXY4N+XIh2TwoJOc4+biZj2dWOVSHMYR5L7EcaWaxjA3nuX8YlrnAthSRZSknpdqnU1+HM4E9I2W/l+ZDb</latexit><latexit sha1_base64="KnpGkPeMEcSDto2IGA0dJDFYgro=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoImCihrT1iY2lbdqCUeIfMHHVwUkTB+MP8Ae4+Acc+AnGERMXBy8FYxzA27y++867577z3tEc0/B8okaf1D8wODQcGhkdG5+YDEempgueXXV1kddt03Z3NNUTpmGJvG/4pthxXKFWNFMUteO11n6xJlzPsK1t/9QR+xW1bBmHhq76DOVOSolSJEZyPK0oqXSUZCWTTCnEyXJCyVAyGpcpiBg6sWFHnrCHA9jQUUUFAhZ8zk2o8PjbRRwEh7F91BlzOTOCfYFzjDK3ylWCK1RGj/lf5tVuB7V43erpBWydTzF5uMyMYoFe6YGa9EKP9EZfXXvVgx4tLac8a22ucErhi9nc57+sCs8+jn5ZPTX7OEQm0GqwdidAWrfQ2/za2XUzt7K1UF+kO3pn/bfUoGe+gVX70O83xdZNDz0aa+EXY4N+XIh2TwoJOc4+biZj2dWOVSHMYR5L7EcaWaxjA3nuX8YlrnAthSRZSknpdqnU1+HM4E9I2W/l+ZDb</latexit><latexit sha1_base64="KnpGkPeMEcSDto2IGA0dJDFYgro=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoImCihrT1iY2lbdqCUeIfMHHVwUkTB+MP8Ae4+Acc+AnGERMXBy8FYxzA27y++867577z3tEc0/B8okaf1D8wODQcGhkdG5+YDEempgueXXV1kddt03Z3NNUTpmGJvG/4pthxXKFWNFMUteO11n6xJlzPsK1t/9QR+xW1bBmHhq76DOVOSolSJEZyPK0oqXSUZCWTTCnEyXJCyVAyGpcpiBg6sWFHnrCHA9jQUUUFAhZ8zk2o8PjbRRwEh7F91BlzOTOCfYFzjDK3ylWCK1RGj/lf5tVuB7V43erpBWydTzF5uMyMYoFe6YGa9EKP9EZfXXvVgx4tLac8a22ucErhi9nc57+sCs8+jn5ZPTX7OEQm0GqwdidAWrfQ2/za2XUzt7K1UF+kO3pn/bfUoGe+gVX70O83xdZNDz0aa+EXY4N+XIh2TwoJOc4+biZj2dWOVSHMYR5L7EcaWaxjA3nuX8YlrnAthSRZSknpdqnU1+HM4E9I2W/l+ZDb</latexit><latexit sha1_base64="KnpGkPeMEcSDto2IGA0dJDFYgro=">AAAChnichVE9T8JQFD3WL/wEdTFxIRKNU3OhIMSJ6OKoImCihrT1iY2lbdqCUeIfMHHVwUkTB+MP8Ae4+Acc+AnGERMXBy8FYxzA27y++867577z3tEc0/B8okaf1D8wODQcGhkdG5+YDEempgueXXV1kddt03Z3NNUTpmGJvG/4pthxXKFWNFMUteO11n6xJlzPsK1t/9QR+xW1bBmHhq76DOVOSolSJEZyPK0oqXSUZCWTTCnEyXJCyVAyGpcpiBg6sWFHnrCHA9jQUUUFAhZ8zk2o8PjbRRwEh7F91BlzOTOCfYFzjDK3ylWCK1RGj/lf5tVuB7V43erpBWydTzF5uMyMYoFe6YGa9EKP9EZfXXvVgx4tLac8a22ucErhi9nc57+sCs8+jn5ZPTX7OEQm0GqwdidAWrfQ2/za2XUzt7K1UF+kO3pn/bfUoGe+gVX70O83xdZNDz0aa+EXY4N+XIh2TwoJOc4+biZj2dWOVSHMYR5L7EcaWaxjA3nuX8YlrnAthSRZSknpdqnU1+HM4E9I2W/l+ZDb</latexit> w4<latexit sha1_base64="rK3TajMk0HyhnVhAbPot5Dyr0NQ=">AAAChnichVE9S8NQFD3Gr/pddRFcgkVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8DatiIN6w8u777x77jvvHcOzrSAkanRJ3T29ff2xgcGh4ZHRsfj4xGbgVnxTFEzXdv1tQw+EbTmiEFqhLbY9X+hlwxZbxtFKa3+rKvzAcp2N8MQTe2W95FgHlqmHDOWPi6liPEFKUlPVtCaTomZSaZU4WVxQM5SSkwpFkUAncm78AbvYhwsTFZQh4CDk3IaOgL8dJEHwGNtDjTGfMyvaFzjDIHMrXCW4Qmf0iP8lXu10UIfXrZ5BxDb5FJuHz0wZs/RMd9SkJ7qnF/r4tVct6tHScsKz0eYKrzh2PpV//5dV5jnE4TfrT80hDpCJtFqs3YuQ1i3MNr96Wm/ml9Zna3N0Q6+s/5oa9Mg3cKpv5u2aWL/6Q4/BWvjF2KAvF+Tfk80FJck+rqUS2eWOVTFMYwbz7IeGLFaRQ4H7l3CBS9SlmKRIaUlrl0pdHc4kfoSU/QTqOZDd</latexit><latexit sha1_base64="rK3TajMk0HyhnVhAbPot5Dyr0NQ=">AAAChnichVE9S8NQFD3Gr/pddRFcgkVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8DatiIN6w8u777x77jvvHcOzrSAkanRJ3T29ff2xgcGh4ZHRsfj4xGbgVnxTFEzXdv1tQw+EbTmiEFqhLbY9X+hlwxZbxtFKa3+rKvzAcp2N8MQTe2W95FgHlqmHDOWPi6liPEFKUlPVtCaTomZSaZU4WVxQM5SSkwpFkUAncm78AbvYhwsTFZQh4CDk3IaOgL8dJEHwGNtDjTGfMyvaFzjDIHMrXCW4Qmf0iP8lXu10UIfXrZ5BxDb5FJuHz0wZs/RMd9SkJ7qnF/r4tVct6tHScsKz0eYKrzh2PpV//5dV5jnE4TfrT80hDpCJtFqs3YuQ1i3MNr96Wm/ml9Zna3N0Q6+s/5oa9Mg3cKpv5u2aWL/6Q4/BWvjF2KAvF+Tfk80FJck+rqUS2eWOVTFMYwbz7IeGLFaRQ4H7l3CBS9SlmKRIaUlrl0pdHc4kfoSU/QTqOZDd</latexit><latexit sha1_base64="rK3TajMk0HyhnVhAbPot5Dyr0NQ=">AAAChnichVE9S8NQFD3Gr/pddRFcgkVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8DatiIN6w8u777x77jvvHcOzrSAkanRJ3T29ff2xgcGh4ZHRsfj4xGbgVnxTFEzXdv1tQw+EbTmiEFqhLbY9X+hlwxZbxtFKa3+rKvzAcp2N8MQTe2W95FgHlqmHDOWPi6liPEFKUlPVtCaTomZSaZU4WVxQM5SSkwpFkUAncm78AbvYhwsTFZQh4CDk3IaOgL8dJEHwGNtDjTGfMyvaFzjDIHMrXCW4Qmf0iP8lXu10UIfXrZ5BxDb5FJuHz0wZs/RMd9SkJ7qnF/r4tVct6tHScsKz0eYKrzh2PpV//5dV5jnE4TfrT80hDpCJtFqs3YuQ1i3MNr96Wm/ml9Zna3N0Q6+s/5oa9Mg3cKpv5u2aWL/6Q4/BWvjF2KAvF+Tfk80FJck+rqUS2eWOVTFMYwbz7IeGLFaRQ4H7l3CBS9SlmKRIaUlrl0pdHc4kfoSU/QTqOZDd</latexit><latexit sha1_base64="rK3TajMk0HyhnVhAbPot5Dyr0NQ=">AAAChnichVE9S8NQFD3Gr/pddRFcgkVxCremNcWp6OJYrVVBpSTxWYNpEpK0osU/ILjawUnBQfwB/gAX/4BDf4I4VnBx8DatiIN6w8u777x77jvvHcOzrSAkanRJ3T29ff2xgcGh4ZHRsfj4xGbgVnxTFEzXdv1tQw+EbTmiEFqhLbY9X+hlwxZbxtFKa3+rKvzAcp2N8MQTe2W95FgHlqmHDOWPi6liPEFKUlPVtCaTomZSaZU4WVxQM5SSkwpFkUAncm78AbvYhwsTFZQh4CDk3IaOgL8dJEHwGNtDjTGfMyvaFzjDIHMrXCW4Qmf0iP8lXu10UIfXrZ5BxDb5FJuHz0wZs/RMd9SkJ7qnF/r4tVct6tHScsKz0eYKrzh2PpV//5dV5jnE4TfrT80hDpCJtFqs3YuQ1i3MNr96Wm/ml9Zna3N0Q6+s/5oa9Mg3cKpv5u2aWL/6Q4/BWvjF2KAvF+Tfk80FJck+rqUS2eWOVTFMYwbz7IeGLFaRQ4H7l3CBS9SlmKRIaUlrl0pdHc4kfoSU/QTqOZDd</latexit> x<latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit> x<latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit><latexit sha1_base64="W7ixVpw5NdsUFOv7TzhL+JCgRfw=">AAAChHichVFNLwNRFD0GVZ8tNhIb0RALae74CLGQho1li5YEaWbGa006X5mZNqrxB9gSCysSC/ED/AAbf8DCTxDLSmws3E4nEUTdyZt33nn33Hfeu6pj6J5P9NwmtXd0Rrqi3T29ff0DsfjgUM6zy64msppt2O62qnjC0C2R9XXfENuOKxRTNcSWWlpt7G9VhOvptrXpVx2xZypFSy/omuIzlTnMxxOUpCDGfgM5BAmEkbbj99jFPmxoKMOEgAWfsQEFHn87kEFwmNtDjTmXkR7sCxyjh7VlzhKcoTBb4n+RVzsha/G6UdML1BqfYvBwWTmGCXqiW6rTI93RC338WasW1Gh4qfKsNrXCycdORjbe/1WZPPs4+FK19OyjgMXAq87enYBp3EJr6itHF/WNpfWJ2iRd0yv7v6JneuAbWJU37SYj1i9b+FHZC78YN0j+2Y7fIDeTlCkpZ+YSqZWwVVGMYhxT3I8FpLCGNLJcX+AUZziXItK0NCvNN1OltlAzjG8hLX8C01WP7w==</latexit> y y = w3 · max(w1x b1, 0)<latexit sha1_base64="byCiVtPVF/c5ukBwX0pAigYb2i4=">AAAConicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIVtqWxxvHJKfkl8TkJlZolMcbKlQo6CokxRvqGGjGCygb6BmAgQImwxDKUGaAgoB8ge0MMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNxuOeJKBbgCEGjCBD9OjAZIQZ6Rka6BkGmig7OEGjioNBmkGJQQMYH+YMDgweDAEMoUDzmxlWMmxi2MykwuTFFMgUDFHKxAjVI8yAAphiAFPZmdI=</latexit><latexit sha1_base64="byCiVtPVF/c5ukBwX0pAigYb2i4=">AAAConicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIVtqWxxvHJKfkl8TkJlZolMcbKlQo6CokxRvqGGjGCygb6BmAgQImwxDKUGaAgoB8ge0MMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNxuOeJKBbgCEGjCBD9OjAZIQZ6Rka6BkGmig7OEGjioNBmkGJQQMYH+YMDgweDAEMoUDzmxlWMmxi2MykwuTFFMgUDFHKxAjVI8yAAphiAFPZmdI=</latexit><latexit sha1_base64="byCiVtPVF/c5ukBwX0pAigYb2i4=">AAAConicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIVtqWxxvHJKfkl8TkJlZolMcbKlQo6CokxRvqGGjGCygb6BmAgQImwxDKUGaAgoB8ge0MMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNxuOeJKBbgCEGjCBD9OjAZIQZ6Rka6BkGmig7OEGjioNBmkGJQQMYH+YMDgweDAEMoUDzmxlWMmxi2MykwuTFFMgUDFHKxAjVI8yAAphiAFPZmdI=</latexit><latexit sha1_base64="byCiVtPVF/c5ukBwX0pAigYb2i4=">AAAConicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIVtqWxxvHJKfkl8TkJlZolMcbKlQo6CokxRvqGGjGCygb6BmAgQImwxDKUGaAgoB8ge0MMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNxuOeJKBbgCEGjCBD9OjAZIQZ6Rka6BkGmig7OEGjioNBmkGJQQMYH+YMDgweDAEMoUDzmxlWMmxi2MykwuTFFMgUDFHKxAjVI8yAAphiAFPZmdI=</latexit> +w4 · max(w2x b2, 0)<latexit sha1_base64="b/vy4QCaFarrU78/CVIFLGCIO4M=">AAACoXichVFNLyRRFD1qxkz7Gm1mIzGLig4hTOdWR0JmJWzYtaaRdEulqjxU1FeqXnejY2PpD1hYkUiI1czGD7DxByz8BJmlSWws3K6uRBDcyqt33nn33Hfeu2bg2JEkumlRPn1u/fI11dbe0dn1rTvd830x8iuhJYqW7/jhsmlEwrE9UZS2dMRyEArDNR2xZG5ON/aXqiKMbN9bkNuBWHGNdc9esy1DMqWnf47U9LGyterLsmtsDdX0nLql/lJNPTdKw3o6Q1mKQ30NtARkkETeT1+gjFX4sFCBCwEPkrEDAxF/JWggBMytoM5cyMiO9wV20c7aCmcJzjCY3eT/Oq9KCevxulEzitUWn+LwCFmpYoCu6Yzu6IrO6ZYe3qxVj2s0vGzzbDa1ItC793vn7z9UuTxLbDyp3vUssYaJ2KvN3oOYadzCauqrOwd3878LA/VBOqZ/7P+IbuiSb+BV/1snc6Jw+I4fk73wi3GDtJfteA0Wc1mNstrcWGZyKmlVCn3oxxD3YxyTmEEeRa6/h1P8wV8lo8wqeaXQTFVaEs0PPAul9Aj96plA</latexit><latexit sha1_base64="b/vy4QCaFarrU78/CVIFLGCIO4M=">AAACoXichVFNLyRRFD1qxkz7Gm1mIzGLig4hTOdWR0JmJWzYtaaRdEulqjxU1FeqXnejY2PpD1hYkUiI1czGD7DxByz8BJmlSWws3K6uRBDcyqt33nn33Hfeu2bg2JEkumlRPn1u/fI11dbe0dn1rTvd830x8iuhJYqW7/jhsmlEwrE9UZS2dMRyEArDNR2xZG5ON/aXqiKMbN9bkNuBWHGNdc9esy1DMqWnf47U9LGyterLsmtsDdX0nLql/lJNPTdKw3o6Q1mKQ30NtARkkETeT1+gjFX4sFCBCwEPkrEDAxF/JWggBMytoM5cyMiO9wV20c7aCmcJzjCY3eT/Oq9KCevxulEzitUWn+LwCFmpYoCu6Yzu6IrO6ZYe3qxVj2s0vGzzbDa1ItC793vn7z9UuTxLbDyp3vUssYaJ2KvN3oOYadzCauqrOwd3878LA/VBOqZ/7P+IbuiSb+BV/1snc6Jw+I4fk73wi3GDtJfteA0Wc1mNstrcWGZyKmlVCn3oxxD3YxyTmEEeRa6/h1P8wV8lo8wqeaXQTFVaEs0PPAul9Aj96plA</latexit><latexit sha1_base64="b/vy4QCaFarrU78/CVIFLGCIO4M=">AAACoXichVFNLyRRFD1qxkz7Gm1mIzGLig4hTOdWR0JmJWzYtaaRdEulqjxU1FeqXnejY2PpD1hYkUiI1czGD7DxByz8BJmlSWws3K6uRBDcyqt33nn33Hfeu2bg2JEkumlRPn1u/fI11dbe0dn1rTvd830x8iuhJYqW7/jhsmlEwrE9UZS2dMRyEArDNR2xZG5ON/aXqiKMbN9bkNuBWHGNdc9esy1DMqWnf47U9LGyterLsmtsDdX0nLql/lJNPTdKw3o6Q1mKQ30NtARkkETeT1+gjFX4sFCBCwEPkrEDAxF/JWggBMytoM5cyMiO9wV20c7aCmcJzjCY3eT/Oq9KCevxulEzitUWn+LwCFmpYoCu6Yzu6IrO6ZYe3qxVj2s0vGzzbDa1ItC793vn7z9UuTxLbDyp3vUssYaJ2KvN3oOYadzCauqrOwd3878LA/VBOqZ/7P+IbuiSb+BV/1snc6Jw+I4fk73wi3GDtJfteA0Wc1mNstrcWGZyKmlVCn3oxxD3YxyTmEEeRa6/h1P8wV8lo8wqeaXQTFVaEs0PPAul9Aj96plA</latexit><latexit sha1_base64="b/vy4QCaFarrU78/CVIFLGCIO4M=">AAACoXichVFNLyRRFD1qxkz7Gm1mIzGLig4hTOdWR0JmJWzYtaaRdEulqjxU1FeqXnejY2PpD1hYkUiI1czGD7DxByz8BJmlSWws3K6uRBDcyqt33nn33Hfeu2bg2JEkumlRPn1u/fI11dbe0dn1rTvd830x8iuhJYqW7/jhsmlEwrE9UZS2dMRyEArDNR2xZG5ON/aXqiKMbN9bkNuBWHGNdc9esy1DMqWnf47U9LGyterLsmtsDdX0nLql/lJNPTdKw3o6Q1mKQ30NtARkkETeT1+gjFX4sFCBCwEPkrEDAxF/JWggBMytoM5cyMiO9wV20c7aCmcJzjCY3eT/Oq9KCevxulEzitUWn+LwCFmpYoCu6Yzu6IrO6ZYe3qxVj2s0vGzzbDa1ItC793vn7z9UuTxLbDyp3vUssYaJ2KvN3oOYadzCauqrOwd3878LA/VBOqZ/7P+IbuiSb+BV/1snc6Jw+I4fk73wi3GDtJfteA0Wc1mNstrcWGZyKmlVCn3oxxD3YxyTmEEeRa6/h1P8wV8lo8wqeaXQTFVaEs0PPAul9Aj96plA</latexit>
  25. 25. !22 c = a + b d = b + 1 e = c ⇤ d add mult add 1a = 2 b = 1 c = 3 d = 2 e = 6 @e @c = 2 @c @a = 1 @c @b = 1 @d @b = 1 @e @d = 3 add mult add 1a<latexit sha1_base64="w1H+UvjBINhKz9i1Oa4hc7xKuHc=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXttKZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGidY/Y</latexit><latexit sha1_base64="w1H+UvjBINhKz9i1Oa4hc7xKuHc=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXttKZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGidY/Y</latexit><latexit sha1_base64="w1H+UvjBINhKz9i1Oa4hc7xKuHc=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXttKZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGidY/Y</latexit><latexit sha1_base64="w1H+UvjBINhKz9i1Oa4hc7xKuHc=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXttKZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGidY/Y</latexit> b<latexit sha1_base64="RVQwNCydlFDYCWbrBUurD8/CjzA=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXt9KZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGklY/Z</latexit><latexit sha1_base64="RVQwNCydlFDYCWbrBUurD8/CjzA=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXt9KZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGklY/Z</latexit><latexit sha1_base64="RVQwNCydlFDYCWbrBUurD8/CjzA=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXt9KZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGklY/Z</latexit><latexit sha1_base64="RVQwNCydlFDYCWbrBUurD8/CjzA=">AAAChHichVHLLgRBFD3TXmO8BhuJjZgQC5nc9gixkAkbSzNjjASR7lbo6Fe6ayZh4gfYEgsrEgvxAT7Axg9YzCeIJYmNhds9nQiC26muU6fuuXWqru5ZZiCJ6gmlqbmltS3Znuro7OruSff2rQZuxTdEyXAt11/TtUBYpiNK0pSWWPN8odm6Jcr6/mK4X64KPzBdZ0UeeGLT1nYdc8c0NMlUXt9KZyhLUQz9BGoMMohj2U3fYQPbcGGgAhsCDiRjCxoC/tahguAxt4kacz4jM9oXOEKKtRXOEpyhMbvP/11ercesw+uwZhCpDT7F4uGzcggj9Eg39EIPdEtP9P5rrVpUI/RywLPe0Apvq+d4oPj2r8rmWWLvU/WnZ4kdzEZeTfbuRUx4C6Ohrx6evxTnCiO1UbqiZ/Z/SXW65xs41VfjOi8KF3/40dkLvxg3SP3ejp9gdSKrUlbNT2VyC3GrkhjEMMa4HzPIYQnLKHF9gROc4kxpVcaVSWW6kaokYk0/voQy/wGklY/Z</latexit> c<latexit sha1_base64="AMfIQwy5NM6Hmn9KqC42nSTSUKg=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbA7jrq4L3Y3gRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1V3T8AOi5yaluaW1rb2js6u7p7cvkewfKPhOyRMyLxzT8TZ1zZemYct8YASm3HQ9qVm6KTf0w+Vwf6MsPd9w7PWg4sodS9u3jT1DaAFTOVFMpilDUaQagRqDNOLIOsl7bGMXDgRKsCBhI2BsQoPP3xZUEFzmdlBlzmNkRPsSx+hibYmzJGdozB7yf59XWzFr8zqs6UdqwaeYPDxWpjBKT3RLNXqkO3qhj19rVaMaoZcKz3pdK91i4mRo7f1flcVzgIMv1Z+eA+xhLvJqsHc3YsJbiLq+fHRRW5tfHa2O0TW9sv8reqYHvoFdfhM3Obl6+Ycfnb3wi3GD1J/taASFyYxKGTU3nV5cilvVgWGMYJz7MYtFrCCLPNeXOMUZzpU2ZUKZUmbqqUpTrBnEt1AWPgGmtY/a</latexit><latexit sha1_base64="AMfIQwy5NM6Hmn9KqC42nSTSUKg=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbA7jrq4L3Y3gRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1V3T8AOi5yaluaW1rb2js6u7p7cvkewfKPhOyRMyLxzT8TZ1zZemYct8YASm3HQ9qVm6KTf0w+Vwf6MsPd9w7PWg4sodS9u3jT1DaAFTOVFMpilDUaQagRqDNOLIOsl7bGMXDgRKsCBhI2BsQoPP3xZUEFzmdlBlzmNkRPsSx+hibYmzJGdozB7yf59XWzFr8zqs6UdqwaeYPDxWpjBKT3RLNXqkO3qhj19rVaMaoZcKz3pdK91i4mRo7f1flcVzgIMv1Z+eA+xhLvJqsHc3YsJbiLq+fHRRW5tfHa2O0TW9sv8reqYHvoFdfhM3Obl6+Ycfnb3wi3GD1J/taASFyYxKGTU3nV5cilvVgWGMYJz7MYtFrCCLPNeXOMUZzpU2ZUKZUmbqqUpTrBnEt1AWPgGmtY/a</latexit><latexit sha1_base64="AMfIQwy5NM6Hmn9KqC42nSTSUKg=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbA7jrq4L3Y3gRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1V3T8AOi5yaluaW1rb2js6u7p7cvkewfKPhOyRMyLxzT8TZ1zZemYct8YASm3HQ9qVm6KTf0w+Vwf6MsPd9w7PWg4sodS9u3jT1DaAFTOVFMpilDUaQagRqDNOLIOsl7bGMXDgRKsCBhI2BsQoPP3xZUEFzmdlBlzmNkRPsSx+hibYmzJGdozB7yf59XWzFr8zqs6UdqwaeYPDxWpjBKT3RLNXqkO3qhj19rVaMaoZcKz3pdK91i4mRo7f1flcVzgIMv1Z+eA+xhLvJqsHc3YsJbiLq+fHRRW5tfHa2O0TW9sv8reqYHvoFdfhM3Obl6+Ycfnb3wi3GD1J/taASFyYxKGTU3nV5cilvVgWGMYJz7MYtFrCCLPNeXOMUZzpU2ZUKZUmbqqUpTrBnEt1AWPgGmtY/a</latexit><latexit sha1_base64="AMfIQwy5NM6Hmn9KqC42nSTSUKg=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbA7jrq4L3Y3gRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1V3T8AOi5yaluaW1rb2js6u7p7cvkewfKPhOyRMyLxzT8TZ1zZemYct8YASm3HQ9qVm6KTf0w+Vwf6MsPd9w7PWg4sodS9u3jT1DaAFTOVFMpilDUaQagRqDNOLIOsl7bGMXDgRKsCBhI2BsQoPP3xZUEFzmdlBlzmNkRPsSx+hibYmzJGdozB7yf59XWzFr8zqs6UdqwaeYPDxWpjBKT3RLNXqkO3qhj19rVaMaoZcKz3pdK91i4mRo7f1flcVzgIMv1Z+eA+xhLvJqsHc3YsJbiLq+fHRRW5tfHa2O0TW9sv8reqYHvoFdfhM3Obl6+Ycfnb3wi3GD1J/taASFyYxKGTU3nV5cilvVgWGMYJz7MYtFrCCLPNeXOMUZzpU2ZUKZUmbqqUpTrBnEt1AWPgGmtY/a</latexit> d<latexit sha1_base64="5Xp2n3/VxdDBB/UDy5G/H9reqxw=">AAAChHichVHLSsNAFD2NWmt9Vd0Iboql4kLKjQ8UF1J049K2Vgu1SBKnNTRNQpIWtPgDulVcuFJwIX6AH+DGH3DRTxCXCm5ceJsGREW9YTJnztxz58xc1TZ01yNqhaSu7p5wb6Qv2j8wODQcGxndcq26o4m8ZhmWU1AVVxi6KfKe7hmiYDtCqamG2Fara+397YZwXN0yN70DW5RqSsXUy7qmeExl9nZjCUqRH/GfQA5AAkFsWLE77GAPFjTUUYOACY+xAQUuf0XIINjMldBkzmGk+/sCR4iyts5ZgjMUZqv8r/CqGLAmr9s1XV+t8SkGD4eVcSTpkW7ohR7olp7o/ddaTb9G28sBz2pHK+zd4ePx3Nu/qhrPHvY/VX969lDGku9VZ++2z7RvoXX0jcPzl9xyNtmcoit6Zv+X1KJ7voHZeNWuMyJ78Ycflb3wi3GD5O/t+Am2ZlMypeTMfCK9GrQqgglMYpr7sYg01rGBPNcXOMEpzqSwNCPNSQudVCkUaMbwJaSVD6jVj9s=</latexit><latexit sha1_base64="5Xp2n3/VxdDBB/UDy5G/H9reqxw=">AAAChHichVHLSsNAFD2NWmt9Vd0Iboql4kLKjQ8UF1J049K2Vgu1SBKnNTRNQpIWtPgDulVcuFJwIX6AH+DGH3DRTxCXCm5ceJsGREW9YTJnztxz58xc1TZ01yNqhaSu7p5wb6Qv2j8wODQcGxndcq26o4m8ZhmWU1AVVxi6KfKe7hmiYDtCqamG2Fara+397YZwXN0yN70DW5RqSsXUy7qmeExl9nZjCUqRH/GfQA5AAkFsWLE77GAPFjTUUYOACY+xAQUuf0XIINjMldBkzmGk+/sCR4iyts5ZgjMUZqv8r/CqGLAmr9s1XV+t8SkGD4eVcSTpkW7ohR7olp7o/ddaTb9G28sBz2pHK+zd4ePx3Nu/qhrPHvY/VX969lDGku9VZ++2z7RvoXX0jcPzl9xyNtmcoit6Zv+X1KJ7voHZeNWuMyJ78Ycflb3wi3GD5O/t+Am2ZlMypeTMfCK9GrQqgglMYpr7sYg01rGBPNcXOMEpzqSwNCPNSQudVCkUaMbwJaSVD6jVj9s=</latexit><latexit sha1_base64="5Xp2n3/VxdDBB/UDy5G/H9reqxw=">AAAChHichVHLSsNAFD2NWmt9Vd0Iboql4kLKjQ8UF1J049K2Vgu1SBKnNTRNQpIWtPgDulVcuFJwIX6AH+DGH3DRTxCXCm5ceJsGREW9YTJnztxz58xc1TZ01yNqhaSu7p5wb6Qv2j8wODQcGxndcq26o4m8ZhmWU1AVVxi6KfKe7hmiYDtCqamG2Fara+397YZwXN0yN70DW5RqSsXUy7qmeExl9nZjCUqRH/GfQA5AAkFsWLE77GAPFjTUUYOACY+xAQUuf0XIINjMldBkzmGk+/sCR4iyts5ZgjMUZqv8r/CqGLAmr9s1XV+t8SkGD4eVcSTpkW7ohR7olp7o/ddaTb9G28sBz2pHK+zd4ePx3Nu/qhrPHvY/VX969lDGku9VZ++2z7RvoXX0jcPzl9xyNtmcoit6Zv+X1KJ7voHZeNWuMyJ78Ycflb3wi3GD5O/t+Am2ZlMypeTMfCK9GrQqgglMYpr7sYg01rGBPNcXOMEpzqSwNCPNSQudVCkUaMbwJaSVD6jVj9s=</latexit><latexit sha1_base64="5Xp2n3/VxdDBB/UDy5G/H9reqxw=">AAAChHichVHLSsNAFD2NWmt9Vd0Iboql4kLKjQ8UF1J049K2Vgu1SBKnNTRNQpIWtPgDulVcuFJwIX6AH+DGH3DRTxCXCm5ceJsGREW9YTJnztxz58xc1TZ01yNqhaSu7p5wb6Qv2j8wODQcGxndcq26o4m8ZhmWU1AVVxi6KfKe7hmiYDtCqamG2Fara+397YZwXN0yN70DW5RqSsXUy7qmeExl9nZjCUqRH/GfQA5AAkFsWLE77GAPFjTUUYOACY+xAQUuf0XIINjMldBkzmGk+/sCR4iyts5ZgjMUZqv8r/CqGLAmr9s1XV+t8SkGD4eVcSTpkW7ohR7olp7o/ddaTb9G28sBz2pHK+zd4ePx3Nu/qhrPHvY/VX969lDGku9VZ++2z7RvoXX0jcPzl9xyNtmcoit6Zv+X1KJ7voHZeNWuMyJ78Ycflb3wi3GD5O/t+Am2ZlMypeTMfCK9GrQqgglMYpr7sYg01rGBPNcXOMEpzqSwNCPNSQudVCkUaMbwJaSVD6jVj9s=</latexit> e<latexit sha1_base64="N6RtWhRfVnc1QfeCeGTBb+uclkI=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbC7jrq4L3YngRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1T3LDCTRc5PS3NLa1t7R2dXd09uXSPYPFAK35Bsib7iW62/qWiAs0xF5aUpLbHq+0GzdEhv64XK4v1EWfmC6zrqseGLH1vYdc880NMlUThSTacpQFKlGoMYgjTiybvIe29iFCwMl2BBwIBlb0BDwtwUVBI+5HVSZ8xmZ0b7AMbpYW+IswRkas4f83+fVVsw6vA5rBpHa4FMsHj4rUxilJ7qlGj3SHb3Qx6+1qlGN0EuFZ72uFV4xcTK09v6vyuZZ4uBL9adniT3MRV5N9u5FTHgLo64vH13U1uZXR6tjdE2v7P+KnumBb+CU34ybnFi9/MOPzl74xbhB6s92NILCZEaljJqbTi8uxa3qwDBGMM79mMUiVpBFnusLnOIM50qbMqFMKTP1VKUp1gziWygLn6r1j9w=</latexit><latexit sha1_base64="N6RtWhRfVnc1QfeCeGTBb+uclkI=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbC7jrq4L3YngRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1T3LDCTRc5PS3NLa1t7R2dXd09uXSPYPFAK35Bsib7iW62/qWiAs0xF5aUpLbHq+0GzdEhv64XK4v1EWfmC6zrqseGLH1vYdc880NMlUThSTacpQFKlGoMYgjTiybvIe29iFCwMl2BBwIBlb0BDwtwUVBI+5HVSZ8xmZ0b7AMbpYW+IswRkas4f83+fVVsw6vA5rBpHa4FMsHj4rUxilJ7qlGj3SHb3Qx6+1qlGN0EuFZ72uFV4xcTK09v6vyuZZ4uBL9adniT3MRV5N9u5FTHgLo64vH13U1uZXR6tjdE2v7P+KnumBb+CU34ybnFi9/MOPzl74xbhB6s92NILCZEaljJqbTi8uxa3qwDBGMM79mMUiVpBFnusLnOIM50qbMqFMKTP1VKUp1gziWygLn6r1j9w=</latexit><latexit sha1_base64="N6RtWhRfVnc1QfeCeGTBb+uclkI=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbC7jrq4L3YngRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1T3LDCTRc5PS3NLa1t7R2dXd09uXSPYPFAK35Bsib7iW62/qWiAs0xF5aUpLbHq+0GzdEhv64XK4v1EWfmC6zrqseGLH1vYdc880NMlUThSTacpQFKlGoMYgjTiybvIe29iFCwMl2BBwIBlb0BDwtwUVBI+5HVSZ8xmZ0b7AMbpYW+IswRkas4f83+fVVsw6vA5rBpHa4FMsHj4rUxilJ7qlGj3SHb3Qx6+1qlGN0EuFZ72uFV4xcTK09v6vyuZZ4uBL9adniT3MRV5N9u5FTHgLo64vH13U1uZXR6tjdE2v7P+KnumBb+CU34ybnFi9/MOPzl74xbhB6s92NILCZEaljJqbTi8uxa3qwDBGMM79mMUiVpBFnusLnOIM50qbMqFMKTP1VKUp1gziWygLn6r1j9w=</latexit><latexit sha1_base64="N6RtWhRfVnc1QfeCeGTBb+uclkI=">AAAChHichVG7SgNBFD2u72eiNoJNMCgWEu76QLEQ0cYyURMFlbC7jrq4L3YngRj8AW0VCysFC/ED/AAbf8DCTxDLCDYW3t0siAb1LrNz5sw9d87M1T3LDCTRc5PS3NLa1t7R2dXd09uXSPYPFAK35Bsib7iW62/qWiAs0xF5aUpLbHq+0GzdEhv64XK4v1EWfmC6zrqseGLH1vYdc880NMlUThSTacpQFKlGoMYgjTiybvIe29iFCwMl2BBwIBlb0BDwtwUVBI+5HVSZ8xmZ0b7AMbpYW+IswRkas4f83+fVVsw6vA5rBpHa4FMsHj4rUxilJ7qlGj3SHb3Qx6+1qlGN0EuFZ72uFV4xcTK09v6vyuZZ4uBL9adniT3MRV5N9u5FTHgLo64vH13U1uZXR6tjdE2v7P+KnumBb+CU34ybnFi9/MOPzl74xbhB6s92NILCZEaljJqbTi8uxa3qwDBGMM79mMUiVpBFnusLnOIM50qbMqFMKTP1VKUp1gziWygLn6r1j9w=</latexit> @c @a = 1 <latexit sha1_base64="s9akH8AUhIT8vU4k7OSqBXUKGFw=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hEXXXredejZHeUrCvgucFOSQRiHIXqKGjwgg0UITCj4MYw8CMX9VOCCEzO2iw1zESCf7Cl1MsbbFWYozBLOH/N/nVTVlfV4PasaJWvIpHo+IlTYW6IrOqU8/6Rv9oj/31uokNQZejnh2h1oV1mdOX27d/FfV5NngYKR60LNBA6uJV83ew4QZ3EIO9e3jT/2ttdJC5w19pd/s/wv16AffwG9fy7OiKn1+wI/LXvjFuEHOv+24C7aX8g7lneJybmMzbVUG83iNRe7HCjbwDgWUuf4JvuMCl9ai9d4qW5VhqjWWal7gr7D2bgHF4p0U</latexit><latexit sha1_base64="s9akH8AUhIT8vU4k7OSqBXUKGFw=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hEXXXredejZHeUrCvgucFOSQRiHIXqKGjwgg0UITCj4MYw8CMX9VOCCEzO2iw1zESCf7Cl1MsbbFWYozBLOH/N/nVTVlfV4PasaJWvIpHo+IlTYW6IrOqU8/6Rv9oj/31uokNQZejnh2h1oV1mdOX27d/FfV5NngYKR60LNBA6uJV83ew4QZ3EIO9e3jT/2ttdJC5w19pd/s/wv16AffwG9fy7OiKn1+wI/LXvjFuEHOv+24C7aX8g7lneJybmMzbVUG83iNRe7HCjbwDgWUuf4JvuMCl9ai9d4qW5VhqjWWal7gr7D2bgHF4p0U</latexit><latexit sha1_base64="s9akH8AUhIT8vU4k7OSqBXUKGFw=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hEXXXredejZHeUrCvgucFOSQRiHIXqKGjwgg0UITCj4MYw8CMX9VOCCEzO2iw1zESCf7Cl1MsbbFWYozBLOH/N/nVTVlfV4PasaJWvIpHo+IlTYW6IrOqU8/6Rv9oj/31uokNQZejnh2h1oV1mdOX27d/FfV5NngYKR60LNBA6uJV83ew4QZ3EIO9e3jT/2ttdJC5w19pd/s/wv16AffwG9fy7OiKn1+wI/LXvjFuEHOv+24C7aX8g7lneJybmMzbVUG83iNRe7HCjbwDgWUuf4JvuMCl9ai9d4qW5VhqjWWal7gr7D2bgHF4p0U</latexit><latexit sha1_base64="s9akH8AUhIT8vU4k7OSqBXUKGFw=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hEXXXredejZHeUrCvgucFOSQRiHIXqKGjwgg0UITCj4MYw8CMX9VOCCEzO2iw1zESCf7Cl1MsbbFWYozBLOH/N/nVTVlfV4PasaJWvIpHo+IlTYW6IrOqU8/6Rv9oj/31uokNQZejnh2h1oV1mdOX27d/FfV5NngYKR60LNBA6uJV83ew4QZ3EIO9e3jT/2ttdJC5w19pd/s/wv16AffwG9fy7OiKn1+wI/LXvjFuEHOv+24C7aX8g7lneJybmMzbVUG83iNRe7HCjbwDgWUuf4JvuMCl9ai9d4qW5VhqjWWal7gr7D2bgHF4p0U</latexit> @c @b = 1 <latexit sha1_base64="NW+oZxACFGd1rI9krunRkFUgSbY=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hN2uvW479WyO8pSEfRc4KcghjUKQvUQNHxFAooUmFHwYxh4EYv6qcEAImdtFh7mIkU72FbqYYm2LsxRnCGYP+b/Pq2rK+rwe1IwTteRTPB4RK20s0BWdU59+0jf6RX/urdVJagy8HPHsDrUqrM+cvty6+a+qybPBwUj1oGeDBlYTr5q9hwkzuIUc6tvHn/pba6WFzhv6Sr/Z/xfq0Q++gd++lmdFVfr8gB+XvfCLcYOcf9txF2wv5R3KO8Xl3MZm2qoM5vEai9yPFWzgHQooc/0TfMcFLq1F671VtirDVGss1bzAX2Ht3QLIB50V</latexit><latexit sha1_base64="NW+oZxACFGd1rI9krunRkFUgSbY=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hN2uvW479WyO8pSEfRc4KcghjUKQvUQNHxFAooUmFHwYxh4EYv6qcEAImdtFh7mIkU72FbqYYm2LsxRnCGYP+b/Pq2rK+rwe1IwTteRTPB4RK20s0BWdU59+0jf6RX/urdVJagy8HPHsDrUqrM+cvty6+a+qybPBwUj1oGeDBlYTr5q9hwkzuIUc6tvHn/pba6WFzhv6Sr/Z/xfq0Q++gd++lmdFVfr8gB+XvfCLcYOcf9txF2wv5R3KO8Xl3MZm2qoM5vEai9yPFWzgHQooc/0TfMcFLq1F671VtirDVGss1bzAX2Ht3QLIB50V</latexit><latexit sha1_base64="NW+oZxACFGd1rI9krunRkFUgSbY=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hN2uvW479WyO8pSEfRc4KcghjUKQvUQNHxFAooUmFHwYxh4EYv6qcEAImdtFh7mIkU72FbqYYm2LsxRnCGYP+b/Pq2rK+rwe1IwTteRTPB4RK20s0BWdU59+0jf6RX/urdVJagy8HPHsDrUqrM+cvty6+a+qybPBwUj1oGeDBlYTr5q9hwkzuIUc6tvHn/pba6WFzhv6Sr/Z/xfq0Q++gd++lmdFVfr8gB+XvfCLcYOcf9txF2wv5R3KO8Xl3MZm2qoM5vEai9yPFWzgHQooc/0TfMcFLq1F671VtirDVGss1bzAX2Ht3QLIB50V</latexit><latexit sha1_base64="NW+oZxACFGd1rI9krunRkFUgSbY=">AAACpnichVFBSxtBFP5cbU21rdFeCl62DRZP4a0IiiCIvfRUktgYIZE4O53o4GZ32Z0ENOQs+Ad66KkFD1J61R/QS/+Ah/yE0mMELz34slkIrWjfMjvffPO+N9/Mc0NPx4aoN2aNTzx6PJl5MjX99Nnzmezs3HYctCKpyjLwgmjHFbHytK/KRhtP7YSREk3XUxX38O1gv9JWUawD/4M5CtVuU+z7uqGlMEzVs69qjUjITi0UkdHCs2V3hN2uvW479WyO8pSEfRc4KcghjUKQvUQNHxFAooUmFHwYxh4EYv6qcEAImdtFh7mIkU72FbqYYm2LsxRnCGYP+b/Pq2rK+rwe1IwTteRTPB4RK20s0BWdU59+0jf6RX/urdVJagy8HPHsDrUqrM+cvty6+a+qybPBwUj1oGeDBlYTr5q9hwkzuIUc6tvHn/pba6WFzhv6Sr/Z/xfq0Q++gd++lmdFVfr8gB+XvfCLcYOcf9txF2wv5R3KO8Xl3MZm2qoM5vEai9yPFWzgHQooc/0TfMcFLq1F671VtirDVGss1bzAX2Ht3QLIB50V</latexit> @d @b = 1 <latexit sha1_base64="dQwBzum9INSxQbu/yBUD4VE4fi8=">AAACpnichVFBSxtBFP6yak2trVEvgpdtg8VTeFsERRCCXjyJRmMEI+nsOrGDm91ldxKIIWfBP+DBUws9iHg1P8CLf8BDfkLpMQUvHvqyWZBWat8yO998874338yzA1dFmqibMoaGR16Npl+PvRl/+24iMzm1G/n10JFFx3f9cM8WkXSVJ4taaVfuBaEUNduVJft4rb9fasgwUr63o5uBPKiJI09VlSM0U5XM+3I1FE6rHIhQK+Gah+0nbLfNFdOqZLKUozjM58BKQBZJbPqZDso4hA8HddQg4UEzdiEQ8bcPC4SAuQO0mAsZqXhfoo0x1tY5S3KGYPaY/0e82k9Yj9f9mlGsdvgUl0fIShNzdE+X1KM7uqIf9PjPWq24Rt9Lk2d7oJVBZeJsZvvhv6oazxpfnlQvetaoYin2qth7EDP9WzgDfePkvLe9XJhrfaRv9JP9f6Uu3fINvMYv5/uWLFy84MdmL/xi3CDr73Y8B7ufchblrK2FbH41aVUas/iAee7HIvJYxyaKXP8U17hBx5g3NoyiURqkGqlEM40/wvj8G8o4nRY=</latexit><latexit sha1_base64="dQwBzum9INSxQbu/yBUD4VE4fi8=">AAACpnichVFBSxtBFP6yak2trVEvgpdtg8VTeFsERRCCXjyJRmMEI+nsOrGDm91ldxKIIWfBP+DBUws9iHg1P8CLf8BDfkLpMQUvHvqyWZBWat8yO998874338yzA1dFmqibMoaGR16Npl+PvRl/+24iMzm1G/n10JFFx3f9cM8WkXSVJ4taaVfuBaEUNduVJft4rb9fasgwUr63o5uBPKiJI09VlSM0U5XM+3I1FE6rHIhQK+Gah+0nbLfNFdOqZLKUozjM58BKQBZJbPqZDso4hA8HddQg4UEzdiEQ8bcPC4SAuQO0mAsZqXhfoo0x1tY5S3KGYPaY/0e82k9Yj9f9mlGsdvgUl0fIShNzdE+X1KM7uqIf9PjPWq24Rt9Lk2d7oJVBZeJsZvvhv6oazxpfnlQvetaoYin2qth7EDP9WzgDfePkvLe9XJhrfaRv9JP9f6Uu3fINvMYv5/uWLFy84MdmL/xi3CDr73Y8B7ufchblrK2FbH41aVUas/iAee7HIvJYxyaKXP8U17hBx5g3NoyiURqkGqlEM40/wvj8G8o4nRY=</latexit><latexit sha1_base64="dQwBzum9INSxQbu/yBUD4VE4fi8=">AAACpnichVFBSxtBFP6yak2trVEvgpdtg8VTeFsERRCCXjyJRmMEI+nsOrGDm91ldxKIIWfBP+DBUws9iHg1P8CLf8BDfkLpMQUvHvqyWZBWat8yO998874338yzA1dFmqibMoaGR16Npl+PvRl/+24iMzm1G/n10JFFx3f9cM8WkXSVJ4taaVfuBaEUNduVJft4rb9fasgwUr63o5uBPKiJI09VlSM0U5XM+3I1FE6rHIhQK+Gah+0nbLfNFdOqZLKUozjM58BKQBZJbPqZDso4hA8HddQg4UEzdiEQ8bcPC4SAuQO0mAsZqXhfoo0x1tY5S3KGYPaY/0e82k9Yj9f9mlGsdvgUl0fIShNzdE+X1KM7uqIf9PjPWq24Rt9Lk2d7oJVBZeJsZvvhv6oazxpfnlQvetaoYin2qth7EDP9WzgDfePkvLe9XJhrfaRv9JP9f6Uu3fINvMYv5/uWLFy84MdmL/xi3CDr73Y8B7ufchblrK2FbH41aVUas/iAee7HIvJYxyaKXP8U17hBx5g3NoyiURqkGqlEM40/wvj8G8o4nRY=</latexit><latexit sha1_base64="dQwBzum9INSxQbu/yBUD4VE4fi8=">AAACpnichVFBSxtBFP6yak2trVEvgpdtg8VTeFsERRCCXjyJRmMEI+nsOrGDm91ldxKIIWfBP+DBUws9iHg1P8CLf8BDfkLpMQUvHvqyWZBWat8yO998874338yzA1dFmqibMoaGR16Npl+PvRl/+24iMzm1G/n10JFFx3f9cM8WkXSVJ4taaVfuBaEUNduVJft4rb9fasgwUr63o5uBPKiJI09VlSM0U5XM+3I1FE6rHIhQK+Gah+0nbLfNFdOqZLKUozjM58BKQBZJbPqZDso4hA8HddQg4UEzdiEQ8bcPC4SAuQO0mAsZqXhfoo0x1tY5S3KGYPaY/0e82k9Yj9f9mlGsdvgUl0fIShNzdE+X1KM7uqIf9PjPWq24Rt9Lk2d7oJVBZeJsZvvhv6oazxpfnlQvetaoYin2qth7EDP9WzgDfePkvLe9XJhrfaRv9JP9f6Uu3fINvMYv5/uWLFy84MdmL/xi3CDr73Y8B7ufchblrK2FbH41aVUas/iAee7HIvJYxyaKXP8U17hBx5g3NoyiURqkGqlEM40/wvj8G8o4nRY=</latexit> @e @c = d <latexit sha1_base64="X8uof/xJTZpU/i/WuwN8FgOiU/4=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Mm1CrYKKfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOv2dSw==</latexit><latexit sha1_base64="X8uof/xJTZpU/i/WuwN8FgOiU/4=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Mm1CrYKKfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOv2dSw==</latexit><latexit sha1_base64="X8uof/xJTZpU/i/WuwN8FgOiU/4=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Mm1CrYKKfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOv2dSw==</latexit><latexit sha1_base64="X8uof/xJTZpU/i/WuwN8FgOiU/4=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Mm1CrYKKfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOv2dSw==</latexit> @e @d = c <latexit sha1_base64="umdjrpvWGoioaNdrJ/W/yMK0a18=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Cm1CrYKyfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOwKdSw==</latexit><latexit sha1_base64="umdjrpvWGoioaNdrJ/W/yMK0a18=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Cm1CrYKyfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOwKdSw==</latexit><latexit sha1_base64="umdjrpvWGoioaNdrJ/W/yMK0a18=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Cm1CrYKyfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOwKdSw==</latexit><latexit sha1_base64="umdjrpvWGoioaNdrJ/W/yMK0a18=">AAACpnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoxqQVJSZXxxQkFpVkJuYopNYi2Cm1CrYKyfECygZ6BmCggMkwhDKUGaAgIF9gO0MMQwpDPkMyQylDLkMqQx5DCZCdw5DIUAyE0QyGDAYMBUCxWIZqoFgRkJUJlk9lqGXgAuotBapKBapIBIpmA8l0IC8aKpoH5IPMLAbrTgbakgPERUCdCgyqBlcNVhp8NjhhsNrgpcEfnGZVg80AuaUSSCdB9KYWxPN3SQR/J6grF0iXMGQgdOF1cwlDGoMF2K2ZQLcXgEVAvkiG6C+rmv452CpItVrNYJHBa6D7FxrcNDgM9EFe2ZfkpYGpQbPxuCcJ6BZgiAEjyBA9OjAZYUZ6hgZ6hoEmyg5O0KjiYJBmUGLQAMaHOYMDgwdDAEMo0Px2hrUM2xi2M2kw+TGFMoVDlDIxQvUIM6AApgQAOwKdSw==</latexit> add mult add 1 @e @c = 2 @c @a = 1 @c @b = 1 @d @b = 1 @e @d = 3 @e @e = 1 @e @c = 2 @e @d = 3 @e @b = 5 @e @a = 2 @e @b = @e @c @c @b + @e @d @d @b
  26. 26. !23 min ✓ L(✓) L(✓) = nX i=1 error(yi, f✓(xi)) ✓t+1 ✓t ⌘ · r✓L(✓t) r✓L(✓t) = 2 6 4 @L(✓)/@✓1 |✓=✓t @L(✓)/@✓2 |✓=✓t ... 3 7 5
  27. 27. !24 

  28. 28. !25
  29. 29. !26 x(t) h(t) y(t) x(1) x(2) y(2)y(1) h(0) h(1) h(2) x(t) y(t) h(t) 
 h(t) y(t) x(t) h(t 1) x(t) ⇥ + ⇥ tanh(·) (·)tanh(·)(·)(·) ⇥ y(t) x(t) (·)(·) tanh(·) 1 · ⇥ +⇥ ⇥ y(t) x(t) {x(t)} 7! {y(t)}
  30. 30. !27 
 w1 i j w1 w2 w3 w4 i j
  31. 31. !28 https://arxiv.org/abs/1806.01261
  32. 32. !29 
 
 

  33. 33. !30 Donor 1 0 1 0 0 Acceptor 0 1 1 0 0 Aromatic 0 0 0 1 0 Halogen 0 0 0 0 0 Basic 0 0 1 0 0 Acidic 0 0 0 0 1 Implicit Hydrogens Explicit HydrogensStructural Formula atomic number 8 8 7 6 6 6 total degree 2 1 3 3 3 3 #Hs 1 0 2 0 0 1 formal charge 0 0 0 0 0 0 isotope 0 0 0 0 0 0 in Ring? 0 0 0 1 0 1
  34. 34. !31 • the number of immediate neighbors who are “heavy” (non-hydrogen) atoms • the valence minus the number of hydrogens • the atomic number • the atomic mass • the atomic charge • the number of attached hydrogens • whether the atom is contained in at least one ring • hydrogen-bond acceptor or not? • hydrogen-bond donor or not? • negatively ionizable or not? • positively ionizable or not? • aromatic or not? • halogen or not? Rogers+, Extended-Connectivity Fingerprints. J. Chem. Inf. Model., 2010, 50 (5), pp 742–754 Faber+, Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error. J. Chem. Theory Comput., 2017, 13 (11), pp 5255–5264
  35. 35. !32 • Graph Convolution (Duvenaud+ 2015) • Weave (Kearnes+ 2016) • Graph Neural Networks (Scarselli+ 2009) • Gated Graph Neural Networks (Li+ 2016) • Neural Message Passing (Gilmer+ 2017) • Non-Local Neural Network (Wang+ 2018) • Graph Networks (Battaglia+ 2018)
  36. 36. !33 

  37. 37. !34 v xv<latexit sha1_base64="xMPgqSyD/mawPfrhGPiiW2hkCAw=">AAACi3ichVHNSgJRFD5Of2aZVpugjSRGKzlTQSEtpAha+pM/oCIz09UG54+ZUTLxBVq2aWGbghbRA/QAbXqBFj5CtDRo06Iz40CUZGe4c7/73fOd+917REORLRux7+MmJqemZ/yzgbn54EIovLiUt/SmKbGcpCu6WRQFiymyxnK2bCusaJhMUEWFFcTGgbNfaDHTknXt2G4brKIKdU2uyZJgE1Usi2rnrFttVcNRjKMbkVHAeyAKXqT08COU4QR0kKAJKjDQwCasgAAWfSXgAcEgrgId4kxCsrvPoAsB0jYpi1GGQGyD/nValTxWo7VT03LVEp2i0DBJGYEYvuA9DvAZH/AVP/+s1XFrOF7aNItDLTOqoYuV7Me/KpVmG06/VWM921CDXderTN4Nl3FuIQ31rfOrQTaRiXXW8RbfyP8N9vGJbqC13qW7NMv0xvgRyQu9GDWI/92OUZDfjPMY59Pb0eS+1yo/rMIabFA/diAJR5CCnNuHS+jBNRfktrgEtzdM5XyeZhl+BHf4BTUVky0=</latexit><latexit sha1_base64="xMPgqSyD/mawPfrhGPiiW2hkCAw=">AAACi3ichVHNSgJRFD5Of2aZVpugjSRGKzlTQSEtpAha+pM/oCIz09UG54+ZUTLxBVq2aWGbghbRA/QAbXqBFj5CtDRo06Iz40CUZGe4c7/73fOd+917REORLRux7+MmJqemZ/yzgbn54EIovLiUt/SmKbGcpCu6WRQFiymyxnK2bCusaJhMUEWFFcTGgbNfaDHTknXt2G4brKIKdU2uyZJgE1Usi2rnrFttVcNRjKMbkVHAeyAKXqT08COU4QR0kKAJKjDQwCasgAAWfSXgAcEgrgId4kxCsrvPoAsB0jYpi1GGQGyD/nValTxWo7VT03LVEp2i0DBJGYEYvuA9DvAZH/AVP/+s1XFrOF7aNItDLTOqoYuV7Me/KpVmG06/VWM921CDXderTN4Nl3FuIQ31rfOrQTaRiXXW8RbfyP8N9vGJbqC13qW7NMv0xvgRyQu9GDWI/92OUZDfjPMY59Pb0eS+1yo/rMIabFA/diAJR5CCnNuHS+jBNRfktrgEtzdM5XyeZhl+BHf4BTUVky0=</latexit><latexit sha1_base64="xMPgqSyD/mawPfrhGPiiW2hkCAw=">AAACi3ichVHNSgJRFD5Of2aZVpugjSRGKzlTQSEtpAha+pM/oCIz09UG54+ZUTLxBVq2aWGbghbRA/QAbXqBFj5CtDRo06Iz40CUZGe4c7/73fOd+917REORLRux7+MmJqemZ/yzgbn54EIovLiUt/SmKbGcpCu6WRQFiymyxnK2bCusaJhMUEWFFcTGgbNfaDHTknXt2G4brKIKdU2uyZJgE1Usi2rnrFttVcNRjKMbkVHAeyAKXqT08COU4QR0kKAJKjDQwCasgAAWfSXgAcEgrgId4kxCsrvPoAsB0jYpi1GGQGyD/nValTxWo7VT03LVEp2i0DBJGYEYvuA9DvAZH/AVP/+s1XFrOF7aNItDLTOqoYuV7Me/KpVmG06/VWM921CDXderTN4Nl3FuIQ31rfOrQTaRiXXW8RbfyP8N9vGJbqC13qW7NMv0xvgRyQu9GDWI/92OUZDfjPMY59Pb0eS+1yo/rMIabFA/diAJR5CCnNuHS+jBNRfktrgEtzdM5XyeZhl+BHf4BTUVky0=</latexit><latexit sha1_base64="xMPgqSyD/mawPfrhGPiiW2hkCAw=">AAACi3ichVHNSgJRFD5Of2aZVpugjSRGKzlTQSEtpAha+pM/oCIz09UG54+ZUTLxBVq2aWGbghbRA/QAbXqBFj5CtDRo06Iz40CUZGe4c7/73fOd+917REORLRux7+MmJqemZ/yzgbn54EIovLiUt/SmKbGcpCu6WRQFiymyxnK2bCusaJhMUEWFFcTGgbNfaDHTknXt2G4brKIKdU2uyZJgE1Usi2rnrFttVcNRjKMbkVHAeyAKXqT08COU4QR0kKAJKjDQwCasgAAWfSXgAcEgrgId4kxCsrvPoAsB0jYpi1GGQGyD/nValTxWo7VT03LVEp2i0DBJGYEYvuA9DvAZH/AVP/+s1XFrOF7aNItDLTOqoYuV7Me/KpVmG06/VWM921CDXderTN4Nl3FuIQ31rfOrQTaRiXXW8RbfyP8N9vGJbqC13qW7NMv0xvgRyQu9GDWI/92OUZDfjPMY59Pb0eS+1yo/rMIabFA/diAJR5CCnNuHS+jBNRfktrgEtzdM5XyeZhl+BHf4BTUVky0=</latexit> hG<latexit sha1_base64="peoO2C4CZtQgbVCeJaYQ32lHrFU=">AAACl3ichVHLLgRBFD3aa7wHG2IjJsRqclskxMaEBEuDQWJk0t3KTEe/0l0zQWd+wA9YSCQkgvgAH2DjByx8gliS2Fi409OJILid6jp16p5bp+rqnmUGkuixQWlsam5pTbS1d3R2dfcke/vWA7fsGyJnuJbrb+paICzTETlpSktser7QbN0SG/refG1/oyL8wHSdNXngiW1bKzrmrmlokqlCsi+v22GpWsjbmiwZmhUuVgvJFKUpiuGfQI1BCnEsu8lb5LEDFwbKsCHgQDK2oCHgbwsqCB5z2wiZ8xmZ0b5AFe2sLXOW4AyN2T3+F3m1FbMOr2s1g0ht8CkWD5+VwxilB7qmF7qnG3qi919rhVGNmpcDnvW6VniFnqOB1bd/VTbPEqVP1Z+eJXYxHXk12bsXMbVbGHV95fD4ZXVmZTQco3N6Zv9n9Eh3fAOn8mpcZMXKyR9+dPbCL8YNUr+34ydYn0irlFazk6nMXNyqBIYwgnHuxxQyWMIyclx/H6e4xJUyqMwqC8pSPVVpiDX9+BJK9gPHYJex</latexit><latexit sha1_base64="peoO2C4CZtQgbVCeJaYQ32lHrFU=">AAACl3ichVHLLgRBFD3aa7wHG2IjJsRqclskxMaEBEuDQWJk0t3KTEe/0l0zQWd+wA9YSCQkgvgAH2DjByx8gliS2Fi409OJILid6jp16p5bp+rqnmUGkuixQWlsam5pTbS1d3R2dfcke/vWA7fsGyJnuJbrb+paICzTETlpSktser7QbN0SG/refG1/oyL8wHSdNXngiW1bKzrmrmlokqlCsi+v22GpWsjbmiwZmhUuVgvJFKUpiuGfQI1BCnEsu8lb5LEDFwbKsCHgQDK2oCHgbwsqCB5z2wiZ8xmZ0b5AFe2sLXOW4AyN2T3+F3m1FbMOr2s1g0ht8CkWD5+VwxilB7qmF7qnG3qi919rhVGNmpcDnvW6VniFnqOB1bd/VTbPEqVP1Z+eJXYxHXk12bsXMbVbGHV95fD4ZXVmZTQco3N6Zv9n9Eh3fAOn8mpcZMXKyR9+dPbCL8YNUr+34ydYn0irlFazk6nMXNyqBIYwgnHuxxQyWMIyclx/H6e4xJUyqMwqC8pSPVVpiDX9+BJK9gPHYJex</latexit><latexit sha1_base64="peoO2C4CZtQgbVCeJaYQ32lHrFU=">AAACl3ichVHLLgRBFD3aa7wHG2IjJsRqclskxMaEBEuDQWJk0t3KTEe/0l0zQWd+wA9YSCQkgvgAH2DjByx8gliS2Fi409OJILid6jp16p5bp+rqnmUGkuixQWlsam5pTbS1d3R2dfcke/vWA7fsGyJnuJbrb+paICzTETlpSktser7QbN0SG/refG1/oyL8wHSdNXngiW1bKzrmrmlokqlCsi+v22GpWsjbmiwZmhUuVgvJFKUpiuGfQI1BCnEsu8lb5LEDFwbKsCHgQDK2oCHgbwsqCB5z2wiZ8xmZ0b5AFe2sLXOW4AyN2T3+F3m1FbMOr2s1g0ht8CkWD5+VwxilB7qmF7qnG3qi919rhVGNmpcDnvW6VniFnqOB1bd/VTbPEqVP1Z+eJXYxHXk12bsXMbVbGHV95fD4ZXVmZTQco3N6Zv9n9Eh3fAOn8mpcZMXKyR9+dPbCL8YNUr+34ydYn0irlFazk6nMXNyqBIYwgnHuxxQyWMIyclx/H6e4xJUyqMwqC8pSPVVpiDX9+BJK9gPHYJex</latexit><latexit sha1_base64="peoO2C4CZtQgbVCeJaYQ32lHrFU=">AAACl3ichVHLLgRBFD3aa7wHG2IjJsRqclskxMaEBEuDQWJk0t3KTEe/0l0zQWd+wA9YSCQkgvgAH2DjByx8gliS2Fi409OJILid6jp16p5bp+rqnmUGkuixQWlsam5pTbS1d3R2dfcke/vWA7fsGyJnuJbrb+paICzTETlpSktser7QbN0SG/refG1/oyL8wHSdNXngiW1bKzrmrmlokqlCsi+v22GpWsjbmiwZmhUuVgvJFKUpiuGfQI1BCnEsu8lb5LEDFwbKsCHgQDK2oCHgbwsqCB5z2wiZ8xmZ0b5AFe2sLXOW4AyN2T3+F3m1FbMOr2s1g0ht8CkWD5+VwxilB7qmF7qnG3qi919rhVGNmpcDnvW6VniFnqOB1bd/VTbPEqVP1Z+eJXYxHXk12bsXMbVbGHV95fD4ZXVmZTQco3N6Zv9n9Eh3fAOn8mpcZMXKyR9+dPbCL8YNUr+34ydYn0irlFazk6nMXNyqBIYwgnHuxxQyWMIyclx/H6e4xJUyqMwqC8pSPVVpiDX9+BJK9gPHYJex</latexit> v h(1) v =  xv 0<latexit sha1_base64="t0YIAtusNim9537WktegaHytwZo=">AAACxXichVFNS9xQFD1GW+3Y1lE3QjdDB4vdDDelYCkIgy506UdHBWOHvPgcH+aL5E1Qw+DeP+DClQWF0h/QH+DGhVsL/oTiUsFNF73JhJZWam9I3nnn3nNz3rsidFWsia56jN6+R4/7B56UBp8+ez5UHh5ZjoN25MiGE7hBtCrsWLrKlw2ttCtXw0jannDlitieyfIriYxiFfgf9G4o1z275atN5diaqWa5bgkv3eo0k4/phPm6U5mqWEK2lJ8Kz9aR2umUsoIdLrCsHBIz0t/4lW+Wq1SjPCr3gVmAKoqYD8pfYWEDARy04UHCh2bswkbMzxpMEELm1pEyFzFSeV6igxJr21wlucJmdpu/Ld6tFazP+6xnnKsd/ovLb8TKCsbpkj7TDZ3TF/pOP/7ZK817ZF52eRVdrQybQwdjS3f/VXm8amz9Vj3oWWMT73Kvir2HOZOdwunqk73Dm6X3i+PpK/pE1+z/mK7ojE/gJ7fOyYJcPHrAj2AvfGM8IPPvcdwHy29qJtXMhbfV+nQxqgG8wEtM8DwmUccc5tHg/qe4wCW+GbOGZ2gj6ZYaPYVmFH+Esf8T3nyqPg==</latexit><latexit sha1_base64="t0YIAtusNim9537WktegaHytwZo=">AAACxXichVFNS9xQFD1GW+3Y1lE3QjdDB4vdDDelYCkIgy506UdHBWOHvPgcH+aL5E1Qw+DeP+DClQWF0h/QH+DGhVsL/oTiUsFNF73JhJZWam9I3nnn3nNz3rsidFWsia56jN6+R4/7B56UBp8+ez5UHh5ZjoN25MiGE7hBtCrsWLrKlw2ttCtXw0jannDlitieyfIriYxiFfgf9G4o1z275atN5diaqWa5bgkv3eo0k4/phPm6U5mqWEK2lJ8Kz9aR2umUsoIdLrCsHBIz0t/4lW+Wq1SjPCr3gVmAKoqYD8pfYWEDARy04UHCh2bswkbMzxpMEELm1pEyFzFSeV6igxJr21wlucJmdpu/Ld6tFazP+6xnnKsd/ovLb8TKCsbpkj7TDZ3TF/pOP/7ZK817ZF52eRVdrQybQwdjS3f/VXm8amz9Vj3oWWMT73Kvir2HOZOdwunqk73Dm6X3i+PpK/pE1+z/mK7ojE/gJ7fOyYJcPHrAj2AvfGM8IPPvcdwHy29qJtXMhbfV+nQxqgG8wEtM8DwmUccc5tHg/qe4wCW+GbOGZ2gj6ZYaPYVmFH+Esf8T3nyqPg==</latexit><latexit sha1_base64="t0YIAtusNim9537WktegaHytwZo=">AAACxXichVFNS9xQFD1GW+3Y1lE3QjdDB4vdDDelYCkIgy506UdHBWOHvPgcH+aL5E1Qw+DeP+DClQWF0h/QH+DGhVsL/oTiUsFNF73JhJZWam9I3nnn3nNz3rsidFWsia56jN6+R4/7B56UBp8+ez5UHh5ZjoN25MiGE7hBtCrsWLrKlw2ttCtXw0jannDlitieyfIriYxiFfgf9G4o1z275atN5diaqWa5bgkv3eo0k4/phPm6U5mqWEK2lJ8Kz9aR2umUsoIdLrCsHBIz0t/4lW+Wq1SjPCr3gVmAKoqYD8pfYWEDARy04UHCh2bswkbMzxpMEELm1pEyFzFSeV6igxJr21wlucJmdpu/Ld6tFazP+6xnnKsd/ovLb8TKCsbpkj7TDZ3TF/pOP/7ZK817ZF52eRVdrQybQwdjS3f/VXm8amz9Vj3oWWMT73Kvir2HOZOdwunqk73Dm6X3i+PpK/pE1+z/mK7ojE/gJ7fOyYJcPHrAj2AvfGM8IPPvcdwHy29qJtXMhbfV+nQxqgG8wEtM8DwmUccc5tHg/qe4wCW+GbOGZ2gj6ZYaPYVmFH+Esf8T3nyqPg==</latexit><latexit sha1_base64="t0YIAtusNim9537WktegaHytwZo=">AAACxXichVFNS9xQFD1GW+3Y1lE3QjdDB4vdDDelYCkIgy506UdHBWOHvPgcH+aL5E1Qw+DeP+DClQWF0h/QH+DGhVsL/oTiUsFNF73JhJZWam9I3nnn3nNz3rsidFWsia56jN6+R4/7B56UBp8+ez5UHh5ZjoN25MiGE7hBtCrsWLrKlw2ttCtXw0jannDlitieyfIriYxiFfgf9G4o1z275atN5diaqWa5bgkv3eo0k4/phPm6U5mqWEK2lJ8Kz9aR2umUsoIdLrCsHBIz0t/4lW+Wq1SjPCr3gVmAKoqYD8pfYWEDARy04UHCh2bswkbMzxpMEELm1pEyFzFSeV6igxJr21wlucJmdpu/Ld6tFazP+6xnnKsd/ovLb8TKCsbpkj7TDZ3TF/pOP/7ZK817ZF52eRVdrQybQwdjS3f/VXm8amz9Vj3oWWMT73Kvir2HOZOdwunqk73Dm6X3i+PpK/pE1+z/mK7ojE/gJ7fOyYJcPHrAj2AvfGM8IPPvcdwHy29qJtXMhbfV+nQxqgG8wEtM8DwmUccc5tHg/qe4wCW+GbOGZ2gj6ZYaPYVmFH+Esf8T3nyqPg==</latexit> h(t 1) v a(t) v h(t) v yv zv  h (T ) v xv tanh X v2V (yv) tanh(zv) !hG =<latexit sha1_base64="ARMGXVwsPnaLefmPlcb2NUBMhVE=">AAACmXichVHLSsNAFD3GV33Xx0JwUyyKq3IjgiIIPhaKK7VWBSsliVMbzItkWqihP+APuHCjggv1A/wAN/6ACz9BXCq4ceFtGhAV9YbJnDlzz50zc3XPMgNJ9NikNLe0trUnOjq7unt6+5L9A1uBW/YNkTNcy/V3dC0QlumInDSlJXY8X2i2bolt/XCpvr9dEX5gus6mrHpiz9YOHLNoGppkqpAcyut2WKoV8rYmS4Zmhcu11FwhmaYMRZH6CdQYpBHHmpu8RR77cGGgDBsCDiRjCxoC/nahguAxt4eQOZ+RGe0L1NDJ2jJnCc7QmD3k/wGvdmPW4XW9ZhCpDT7F4uGzMoUxeqAreqF7uqEnev+1VhjVqHup8qw3tMIr9B0PZ9/+Vdk8S5Q+VX96lihiJvJqsncvYuq3MBr6ytHJS3Z2Yywcpwt6Zv/n9Eh3fAOn8mpcrouN0z/86OyFX4wbpH5vx0+wNZlRKaOuT6XnF+NWJTCCUUxwP6YxjxWsIcf1j3CGK1wrI8qCsqKsNlKVplgziC+hZD8A53GYIg==</latexit><latexit sha1_base64="ARMGXVwsPnaLefmPlcb2NUBMhVE=">AAACmXichVHLSsNAFD3GV33Xx0JwUyyKq3IjgiIIPhaKK7VWBSsliVMbzItkWqihP+APuHCjggv1A/wAN/6ACz9BXCq4ceFtGhAV9YbJnDlzz50zc3XPMgNJ9NikNLe0trUnOjq7unt6+5L9A1uBW/YNkTNcy/V3dC0QlumInDSlJXY8X2i2bolt/XCpvr9dEX5gus6mrHpiz9YOHLNoGppkqpAcyut2WKoV8rYmS4Zmhcu11FwhmaYMRZH6CdQYpBHHmpu8RR77cGGgDBsCDiRjCxoC/nahguAxt4eQOZ+RGe0L1NDJ2jJnCc7QmD3k/wGvdmPW4XW9ZhCpDT7F4uGzMoUxeqAreqF7uqEnev+1VhjVqHup8qw3tMIr9B0PZ9/+Vdk8S5Q+VX96lihiJvJqsncvYuq3MBr6ytHJS3Z2Yywcpwt6Zv/n9Eh3fAOn8mpcrouN0z/86OyFX4wbpH5vx0+wNZlRKaOuT6XnF+NWJTCCUUxwP6YxjxWsIcf1j3CGK1wrI8qCsqKsNlKVplgziC+hZD8A53GYIg==</latexit><latexit sha1_base64="ARMGXVwsPnaLefmPlcb2NUBMhVE=">AAACmXichVHLSsNAFD3GV33Xx0JwUyyKq3IjgiIIPhaKK7VWBSsliVMbzItkWqihP+APuHCjggv1A/wAN/6ACz9BXCq4ceFtGhAV9YbJnDlzz50zc3XPMgNJ9NikNLe0trUnOjq7unt6+5L9A1uBW/YNkTNcy/V3dC0QlumInDSlJXY8X2i2bolt/XCpvr9dEX5gus6mrHpiz9YOHLNoGppkqpAcyut2WKoV8rYmS4Zmhcu11FwhmaYMRZH6CdQYpBHHmpu8RR77cGGgDBsCDiRjCxoC/nahguAxt4eQOZ+RGe0L1NDJ2jJnCc7QmD3k/wGvdmPW4XW9ZhCpDT7F4uGzMoUxeqAreqF7uqEnev+1VhjVqHup8qw3tMIr9B0PZ9/+Vdk8S5Q+VX96lihiJvJqsncvYuq3MBr6ytHJS3Z2Yywcpwt6Zv/n9Eh3fAOn8mpcrouN0z/86OyFX4wbpH5vx0+wNZlRKaOuT6XnF+NWJTCCUUxwP6YxjxWsIcf1j3CGK1wrI8qCsqKsNlKVplgziC+hZD8A53GYIg==</latexit><latexit sha1_base64="ARMGXVwsPnaLefmPlcb2NUBMhVE=">AAACmXichVHLSsNAFD3GV33Xx0JwUyyKq3IjgiIIPhaKK7VWBSsliVMbzItkWqihP+APuHCjggv1A/wAN/6ACz9BXCq4ceFtGhAV9YbJnDlzz50zc3XPMgNJ9NikNLe0trUnOjq7unt6+5L9A1uBW/YNkTNcy/V3dC0QlumInDSlJXY8X2i2bolt/XCpvr9dEX5gus6mrHpiz9YOHLNoGppkqpAcyut2WKoV8rYmS4Zmhcu11FwhmaYMRZH6CdQYpBHHmpu8RR77cGGgDBsCDiRjCxoC/nahguAxt4eQOZ+RGe0L1NDJ2jJnCc7QmD3k/wGvdmPW4XW9ZhCpDT7F4uGzMoUxeqAreqF7uqEnev+1VhjVqHup8qw3tMIr9B0PZ9/+Vdk8S5Q+VX96lihiJvJqsncvYuq3MBr6ytHJS3Z2Yywcpwt6Zv/n9Eh3fAOn8mpcrouN0z/86OyFX4wbpH5vx0+wNZlRKaOuT6XnF+NWJTCCUUxwP6YxjxWsIcf1j3CGK1wrI8qCsqKsNlKVplgziC+hZD8A53GYIg==</latexit> 

  38. 38. !35 M M vv hv N(v) m U U R 
 hvhv
  39. 39. !36 ↑ “part position paper, part review, and part unification” (DeepMind) 11 Jun 2018 arXiv:1806.01261 [cs.LG]
  40. 40. !37 We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. •Combinatorial optimization 
 (Bello+ 2016; Nowak+ 2017; Dai+ 2017) •Boolean satisfiability 
 (Selsam+ 2018) •Program representation and verification (Allamanis+ 2018; Li+ 2016) •Modeling cellular automata and Turing machines (Johnson, 2017) •Performing inference in graphical models (Yoon+ 2018)
  41. 41. !38 
 
 


×