Hearn - Optimization and Discrete Mathematics - Spring Review 2012

605 views
403 views

Published on

Dr. Hearn give an overview of his program - Optimization and Discrete Mathematics at the AFOSR Spring Review 2012.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
605
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hearn - Optimization and Discrete Mathematics - Spring Review 2012

  1. 1. Optimization and Discrete Mathematics 6 Mar 2012 Don Hearn Program Manager AFOSR/RSL Integrity  Service  Excellence Air Force Research Laboratory15 February 2012 DISTRIBUTION A: Approved for public release; distribution is unlimited.. 1
  2. 2. Optimization and Discrete MathematicsPM: Don HearnBRIEF DESCRIPTION OF PORTFOLIO:Development of optimization and discrete mathematicsfor solving large, complex problems in support of futureAir Force science and engineering, command and controlsystems, logistics, and battlefield managementLIST SUB-AREAS IN PORTFOLIO:• Analysis based optimization• Continuous and Discrete Search methods• Dynamic, Stochastic and Simulation Optimization• Combinatorial Optimization DISTRIBUTION A: Approved for public release; distribution is unlimited.. 2
  3. 3. Outline• Portfolio Overview• Optimization Research• Program Goals and Strategies• AF Future Applications Identified• Program Trends• Transitions• Scientific Opportunities and Transformation Opportunities; Relation of Other Agency Funding• Examples from the portfolio• Summary DISTRIBUTION A: Approved for public release; distribution is unlimited.. 3
  4. 4. Optimization Research• Starts with an important hard problem• Optimization models are proposed• Theoretical properties – existence and uniqueness of solutions, computational complexity, approximations, duality, …• Solution methods – convergence theory, error analysis, bounds, …• If successful, generalize! DISTRIBUTION A: Approved for public release; distribution is unlimited.. 4
  5. 5. Optimization ModelsMaximize (or Minimize) f(x)Subject to g(x) ≤ 0 h(x) = 0where x is an n-dimensional vector and g, h are vector functions.Data can be deterministic or stochastic and time-dependent.The simplest (in terms of functional form) is the Linear Program:Minimize 3x + 5y + z + 6u + ……..Subject to 3 x + y + 2z + …….. <= 152 2x + 7y + 4z +u ….. >= 130 12x + 33y + 2u ….. = 217 etc DISTRIBUTION A: Approved for public release; distribution is unlimited.. 5
  6. 6. Textbook Nonlinear Example• Local optima are a problem: DISTRIBUTION A: Approved for public release; distribution is unlimited.. 6
  7. 7. Program Goals & Strategies• Goals – • Cutting edge optimization research • Of future use to the Air Force• Strategies – • Stress rigorous models and algorithms , especially for environments with rapidly evolving and uncertain data • Engage more recognized optimization researchers, including more young PIs • Foster collaborations – • Optimizers, engineers and scientists • Between PIs and AF labs • With other AFOSR programs • With NSF, ONR, ARO DISTRIBUTION A: Approved for public release; distribution is unlimited.. 7
  8. 8. Future AF Applications Identified• Engineering design, including peta-scale methods• Risk Management and Defend-Attack-Defend models• Real-time logistics• Optimal learning/machine learning• Optimal photonic material design (meta-materials)• Satellite and target tracking• Embedded optimization for control• Network data mining• …….. (to be continued) DISTRIBUTION A: Approved for public release; distribution is unlimited.. 8
  9. 9. Scientific Challenges and Potential ImpactsSome Scientific Challenges • Non-differentiable Optimization • Nonlinear Optimization for Integer and Combinatorial Models • Simulation Optimization • Optimization for analysis of Biological and Social Networks • Real Time Logistics • Risk and Uncertainty in Engineering DesignPotential Transformational Impacts • Meta-material design • Engineering optimization • AI, Machine Learning, Reinforcement Learning • Air Force Logistics DISTRIBUTION A: Approved for public release; distribution is unlimited.. 9
  10. 10. Program TrendsAnalysis based optimization – Emphasis on theoretical results to exploit problem structure and establish convergence properties supporting the development of algorithmsSearch based optimization – New methods to address very large continuous or discrete problems, but with an emphasis on the mathematical underpinnings, provable bounds, etcDynamic, stochastic and simulation optimization– New algorithms that address data dynamics and uncertainty and include optimization of simulation parametersCombinatorial optimization – New algorithms that address fundamental problems in networks and graphs such as identifying substructures DISTRIBUTION A: Approved for public release; distribution is unlimited.. 10
  11. 11. Recent Transition: Model Adaptive Search for detoxification process design Dennis (Rice), Audet (École Polytechnique de Montréal), Abramson (Boeing)• The MADS algorithm, based on (Clarke) generalized derivatives, reduced the solution times by 37% on this problem in which expensive simulations were need to evaluate the objective function.• Series of publications on the theory (SIAM Optimization + others)• Numerous transitions DISTRIBUTION A: Approved for public release; distribution is unlimited.. 11
  12. 12. New Meta Algorithms for Engineering Design Using Surrogate Functions Dennis (Rice), Audet (École Polytechnique de Montréal), Abramson (Boeing) Helicopter rotor tilt design at Boeing Gross Weight (lbs) Cabin Length (ft) Original airfoil 14 92000 21 ’ Cabin Length 13psfRotor Disk Loading, 90750 12 89500 Infeasible 36.5 Region 0.25 Overlap to Diameter ratio 0.25 Overlap to Diameter ratio 0.25 Overlap to Diameter ratio 11 88250 87710 39.8 Optimized airfoil, 89% noise reduction 10 43.1 9 59.4 56.1 52.9 49.6 46.4 8 0.10 0.15 0.20 0.25 0.30 Rotor Overlap to Diameter Ratio Optimization of angle and graft size for Snow water equivalent estimation bypass surgery Current design Proposed y-graft design Highly fragmented domain Feasible domain represented by the dark pixels. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 12
  13. 13. Defend-Attack-Defend Optimization Models NPS (Joint work with ONR), Brown, Wood, et al.“Operate-Attack-Defend” San FranciscoBridges – 3 objectives: 3Defend: critical arcs with resources 2 1 available 4 5Attack: impedes traffic flows 7 6Operate: after the attack 9 10 11Formulation: nonlinear D-A-D model 8 12 14 (extended Stackelberg game) 13 15 17Methods: new nonlinear 16 20 19 18 decomposition algorithms 23 22 21Model provides: 24 29 25• 26 Relative importance of 28 27 components 30 31 32• 34 Improves on std risk analysis 33 35 36 37• Cost assessments 38 39 40 DISTRIBUTION A: Approved for public release; distribution is unlimited.. 13
  14. 14. Recent TransitionRESILIENCE REPORT:ELECTRIC POWERINFRASTRUCTURE SUPPORTINGMISSION ASSURANCE ATVANDENBERG AIR FORCE BASE Javier Salmerón David L. Alderson Gerald G. Brown NPS Center for Infrastructure Defense December 2011 DISTRIBUTION A: Approved for public release; distribution is unlimited.. 14
  15. 15. Other Organizations That Fund Related Work• NSF Several projects jointly funded in the 4 subareas• ONR • 3 projects (5 PIs) jointly funded – in D- A-D modeling, military logistics and stochastic optimization• DOE • No joint grants. One PI has additional funding in energy management• ARO • No joint project funding. One PI is on Army MURIDISTRIBUTION A: Approved for public release; distribution is unlimited.. 15
  16. 16. International Collaborations• Ecole Polytechnique de Montreal • Claude Audet (with US researchers at Rice, Boeing, AFIT)• University of Waterloo • S. Vavasis & H. Wolkowicz• SOARD • Claudia Sagastizabal, RJ Brazil (with R. Mifflin, Wash. St.)(plus many unsponsored collaborations) DISTRIBUTION A: Approved for public release; distribution is unlimited.. 16
  17. 17. Examples from the Portfolio DISTRIBUTION A: Approved for public release; distribution is unlimited.. 17
  18. 18. Discrete Search - Theory and Methods– Jacobson, Howe and Whitley • Study of landscapes in combinatorial problems • Applications in radar scheduling, routing, general NP Hard problems such as TSP, Air Fleet assignment, homeland security, … DISTRIBUTION A: Approved for public release; distribution is unlimited.. 18
  19. 19. Landscape Analysis and Algorithm Development for Plateau Plagued Search Spaces Colorado State University, Howe & Whitley Origin - tough scheduling problems Led to new research on landscapes orbital and TSPs track path sign Traveling Salesman Problem al (e.g., supply chain, logistics, circuit designs, vehicle routing)• Generalized Partition Crossover (GPX): technique for combining two locally optimal solutions to construct better locally optimal solution in O(n) time• Results: Analytical and empirical analysis shows GPX can dominate state of the art methodsImplications: Can dramatically change how TSP instances are solved DISTRIBUTION A: Approved for public release; distribution is unlimited.. 19
  20. 20. Optimal Dynamic Asset Allocation Problems: Addressing the Impact of Information Uncertainty & Quality University of Illinois, Sheldon JacobsonGoal: Create a general theoretical framework for addressing optimal dynamic assetallocation problems in the presence of information uncertainty and limited information quality. Technical Approaches: • Sequential Stochastic Assignment • Surrogate Model Formulation and Analysis • Dynamic Programming Algorithm Design Applications Assignment Problems Screening and Protection Scheduling Problems Payoff to Stakeholders: • Theory: New fundamental understanding of how data deficiencies and uncertainties impact sequential stochastic assignment problem solutions. •Long-term payoff: Ability to solve problems that more closely20 mimic real- world systems driven DISTRIBUTION A: Approved for public release; distribution is unlimited.. by real-time data. 20
  21. 21. Analysis-Based Optimization Theory and Methods– Mifflin and Sagastizabal • New theory leading to rapid convergence in non- differentiable optimization– Freund and Peraire • SDP for meta-material design– Srikant • Wireless Network Operation DISTRIBUTION A: Approved for public release; distribution is unlimited.. 21
  22. 22. Breakthrough: Super-linear convergence for certain non-smooth problems Bob Mifflin (Washington State) and Claudia Sagastizabal (CEPEL, Rio de Janeiro) jointly supported with NSF and SOARD DISTRIBUTION A: Approved for public release; distribution is unlimited.. 22
  23. 23. Band-gap optimization for wave propagation in periodic media MIT, Freund, Parrilo & Peraire Design of materials that give very precise control of light or sound waves New in this work is the use of Semi-Definite Optimization for band structure calculation and optimization in phononic crystalsApplications includedesign of elasticwaveguides, beamsplitters, acoustic lasers,perfect acoustic mirrors,vibration protectiondevices DISTRIBUTION A: Approved for public release; distribution is unlimited.. 23
  24. 24. Optimization of photonic band gaps MIT, Freund & Peraire Goal: Design of photonic crystals with multiple and combined band gaps Approach: Solution Strategies • Pixel-based topology optimization • Large-scale non-convex eigenvalue optimization problem • Relaxation and linearization Methods: Numerical optimization tools • Semi-definite programming • Subspace approximation method • Discrete system via Finite Element discretezation and Adaptive mesh refinement Results: Square and triangular lattices • Very large absolute band gaps • Multiple complete band gaps • Much larger than existing results. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 24
  25. 25. Dynamic and Stochastic Methods• Stochastic Optimization – – Marcus and Fu • Simulation-based optimization• Dynamic Optimization – Powell, Sen, Magnanti and Levi • Fleet scheduling, Refueling, ADP, Learning methods, Logistics DISTRIBUTION A: Approved for public release; distribution is unlimited.. 25
  26. 26. The research challenge• Solve   max E  C  St , X ( St )   t   t  given a system model (transition function) St 1  S  St , xt ,Wt 1 ()  M – Goal: a method that reliably finds near-optimal policies for the broadest class of problems, with an absolute minimum of tunable parameters. DISTRIBUTION A: Approved for public release; distribution is unlimited.. Slide 26 26
  27. 27. Stochastic programming Stochastic search Model predictive control Optimal control Q  learning Reinforcement learning On-policy learning Off-policy learning Markov decision processes Simulation optimization Policy searchSlide 27 DISTRIBUTION A: Approved for public release; distribution is unlimited.. 27
  28. 28. The Knowledge Gradient for Optimal Learning In Expensive Information Collection Problems Princeton University, Warren B. PowellChallenge• Finding the best material, technology, operating policy or design when field or laboratory experiments are expensive.Breakthrough• The knowledge gradient for correlated beliefs (KGCB) is a powerful method for guiding the efficient collection of information. Proving to be robust across broad problem classes. A knowledge gradient surface Technical challenge • Optimal information collection policies are computationally intractable. • Efficient algorithm for capturing correlated beliefs, making it possible to solve problems with thousands of choices on small budgets. Impact • Used to find new drugs, improve manufacturing processes, and optimize policies for stochastic simulators. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 28
  29. 29. Ongoing (high-risk) research: Simulation-Based Methodologies for Estimation and Control UMD (with NSF), S. Marcus & M. Fu Convergence of Distribution to Optimum in Simulation-Based OptimizationACCOMPLISHMENTS• Simulation-based optimizationalgorithms with probable convergence &efficient performance• Evolutionary and sampling-basedmethods for Markov Decision processes DISTRIBUTION A: Approved for public release; distribution is unlimited.. 29
  30. 30. An Optimization Framework for Air Force Logistics of the Future MIT, Magnanti & Levi• Air Force Logistics – – 20 new initiatives to revolutionize supply chains and logistics• DOD logistics + inventory costs – $130 + $80 billion• Magnanti and Levi (MIT) - fast real- time algorithms for – Depot to base supply and joint replenishment• Future war space example - sensor asset allocation• Workshop in 2010 on identification of specific AF models for the future DISTRIBUTION A: Approved for public release; distribution is unlimited.. 30
  31. 31. ASYNCHRONOUS DISTRIBUTED OPTIMIZATION: COOPERATION WITH MINIMAL COMMUNICATION Cassandras, Boston U.OPTIMAL COVERAGE PROBLEM:Deploy vehicles/sensors to maximize “event” detection probability – unknown event locations – event sources may be mobileTheoretical results suggest that asynchronous communication is best. R(x) (Hz/m2) 50 40 30 20 ? 10 ? ? ???? ? 0 ? 10 10 8 5 6 4 2 0 0 Perceived event density over given region (mission space) DISTRIBUTION A: Approved for public release; distribution is unlimited.. 31
  32. 32. SYNCHRONOUS v ASYNCHRONOUS OPTIMIZATION PERFORMANCESYNCHRONOUS v ASYNCHRONOUS: SYNCHRONOUS v ASYNCHRONOUS: No. of communication events Achieving optimality for a problem with nonsmooth gradients in a problem with nonsmooth gradients DISTRIBUTION A: Approved for public release; distribution is unlimited.. 32
  33. 33. Combinatorial OptimizationButenko (Texas A&M), Vavasis (U. Waterloo),Prokopyev (U. Pittsburgh), Krokhmal (U. Iowa)Underlying structures are often graphs; analysis isincreasingly continuous non-linear optimization DISTRIBUTION A: Approved for public release; distribution is unlimited.. 33
  34. 34. Cohesive subgroups in graphs/networks• Data set generated by a certain complex system – Represent using graphs or networks: Nodes = agents, Links = interactions• Interested in detecting cohesive (tightly knit) groups of nodes representing clusters of the systems components• Earliest mathematical models of cohesive subgroups were based on the graph-theoretic concept of a clique• Other subgroups include k-clubs DISTRIBUTION A: Approved for public release; distribution is unlimited.. 34
  35. 35. Applications• Acquaintance networks - criminal network analysis• Wire transfer database networks - detecting money laundering• Call networks - organized crime detection• Protein interaction networks - predicting protein complexes• Gene co-expression networks - detecting network motifs• Internet graphs - information search and retrieval DISTRIBUTION A: Approved for public release; distribution is unlimited.. 35
  36. 36. Finding hidden structures with convex optimization U. Waterloo, S. Vavasis & H. WolkowiczThe following random graph contains a subset of 5 nodes allmutually interconnected:Recent data mining discoveries fromWaterloo: •Finding hidden cliques and bi-cliques in large graphs (see example) •Finding features in images from an image database •Clustering data in the presence of significant background noise using convex optimization 5-clique found via convex optimization DISTRIBUTION A: Approved for public release; distribution is unlimited.. 36
  37. 37. Another complex networkProtein-protein interaction map of H. Pylori DISTRIBUTION A: Approved for public release; distribution is unlimited.. 37
  38. 38. Find substructures via fractional optimization – S. Butenko, Texas A&M H. Pylori – largest 2-club Yeast –largest 2-club Yeast – largest 3-club DISTRIBUTION A: Approved for public release; distribution is unlimited.. 38
  39. 39. Polyomino Tiling U. Pittsburgh, O. Prokopyev• Application: • Time delay control of phased array systems• Jointly supported with Dr. Nachman of AFOSR/RSE, motivated by work at the Sensors Directorate: • Irregular Polyomino-Shaped Subarrays for Space-Based Active Arrays (R.J. Mailloux, S.G. Santarelli, T.M. Roberts, D. Luu) DISTRIBUTION A: Approved for public release; distribution is unlimited.. 39
  40. 40. Motivation• Introducing time delay into phased array systems causes significant quantization side lobes which severely degrade obtained pattern.• Mailloux et al. [2006] have shown empirically that an array of `irregularly` packed polyomino-shaped subarrays has peak side lobes suppressed more than 14db when compared to an array with rectangular subarrays. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 40
  41. 41. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 41
  42. 42. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 42
  43. 43. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 43
  44. 44. Initial Approach• The aim is to mathematically model and solve the problem of designing and arranging polyomino shaped subarrays using optimization.• From information theory it is know that entropy is a measure of disorder or irregularity.• How to `measure’ irregularity of a given tiling of the board? Start from the concept of `center of gravity’. Every polyomino is thought as a solid and its center of gravity is found.• Then, measure irregularity of the tiling by `irregularity of the centers of gravity’ on the board.• So the model maximizes information entropy. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 44
  45. 45. 16x16 Board Tiled with Tetrominoes Maximize entropy. Minimize entropy. 50 x 50 in 1 hr (Observe how centers of gravity are aligned: order = less entropy)4 Tetrominoes - 18 rotations and reflections DISTRIBUTION A: Approved for public release; distribution is unlimited.. 45
  46. 46. Summary• Goals – • Cutting edge optimization research • Of future use to the Air Force• The primary strategy is to identify future AF applications that will require important developments in optimization theory and new algorithms. Questions? DISTRIBUTION A: Approved for public release; distribution is unlimited.. 46
  47. 47. Ongoing research: Search Algorithms for Scheduling Under Uncertainty Colorado State University, Howe & Whitley orbital path track signalApproach/Technical Challenges• Disconnect between theory and practice in search.• Current theory needs to be extended to account for complex structures of real world applications.• Uncertainty and dynamism are not well captured by current theory.Accomplishments Algorithms and complexity proof for a scheduling Proofs about plateaus in Elementary Landscapes Predictive models of plateau size for some classes of landscapes based on percolation theory Recent results on next slide… DISTRIBUTION A: Approved for public release; distribution is unlimited.. 47
  48. 48. Distributed Scheduling UIUC, R. Srikant• Wireless Networks: Links may not be able to transmit simultaneously due to interference.• Collision: Two interfering users cannot transmit at the same time Main Result: The first, fully distributed• Medium Access Control algorithm which maximizes network (MAC) Protocol: throughput and explicitly takes Determines which links interference into account. should be scheduled to transmit at each time Theoretical Contribution: An optimality instant. proof that blends results from convex• (Relates to RSL Complex optimization, stochastic networks and Networks portfolio) mixing times of Markov chains. DISTRIBUTION A: Approved for public release; distribution is unlimited.. 48

×