Azerbaijan2.2.3 sofi and text scanning


Published on

A look at data mining for use in futures and foresight research.

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • 03/03/11
  • 03/03/11 Optional
  • 03/03/11
  • 03/03/11 Works nicely to initiate a live VP demo
  • 03/03/11 See the separate set of slides on Tech Mining case examples Our ceramics engine case is appended here too.
  • Loser only in the sense of no return on prior investment Ceramic vapor deposition = the thin films Sandia developed; spun off to a company. Purpose was for electronics mfg – components without etching. People in electronics used different terminology – able to explore very diff discipline with diff applications. (And Bob knew nothing about these) Interest from early analyses – showed publication not up too much, but complexity of the research well up. Beyond standard bibliometrics – counting – “dead area”. Kinds of organizations and richness of R&D – implying transition to real applications advancing. Bridging resources, able to show also patent activity and business lit interest rising. ~1987 talk about this R&D, but settled down to some key areas. Running ceramics to failure. By early 90s depth of R&D up – e.g., Mitsubishi patenting ceramic truck engine.
  • Azerbaijan2.2.3 sofi and text scanning

    1. 1. State of the Future Index and Text Mining Second Semester Class 3 The Azerbaijan State Economic University March 4, 2011 Ted Gordon, Senior Fellow and Jerome Glenn, Executive Director, The Millennium Project Alan Porter, Director of R&D, Search Technology, Inc. & Co-Director, Technology Policy & Assessment Center Georgia Tech
    2. 2. Assignments From Last Session <ul><li>Create a Futures Wheel on the technology you selected </li></ul><ul><li>Read Chapter on State of the Future Index </li></ul><ul><li>Read Text Scanning </li></ul><ul><li>Visit (collective intelligence example) </li></ul><ul><li>SOFI Assignment </li></ul><ul><li>Peace Scenarios Assignment: What are the conditions for Peace and what are previous agreements </li></ul>
    3. 3. Today’s Topics <ul><li>The concept of SOFI </li></ul><ul><ul><li>The role of Trend Impact Analysis </li></ul></ul><ul><ul><li>The role of Real Time Delphi </li></ul></ul><ul><li>Text Mining </li></ul>
    4. 4. The Class Project (Session 1) <ul><li>Produce an actual (not just a demonstration) State of the Future Index for Azerbaijan. This requires: </li></ul><ul><ul><li>Understanding data availability </li></ul></ul><ul><ul><li>Choosing variables for the SOFI and collecting data </li></ul></ul><ul><ul><li>Identifying future events that could change the course of the variables </li></ul></ul><ul><ul><li>Forecasting the variables </li></ul></ul><ul><ul><li>Computing the SOFI </li></ul></ul><ul><ul><li>Finding policy implications </li></ul></ul>
    5. 5. State of the Future Index <ul><li>A synthesis of variables to help answer the question “Is the outlook for the future improving?” </li></ul><ul><li>A tool for </li></ul><ul><ul><li>Policy analysis </li></ul></ul><ul><ul><li>Improving discussion about the future </li></ul></ul><ul><ul><li>Education </li></ul></ul><ul><ul><li>National comparisons </li></ul></ul>
    6. 6. The Concept <ul><li>Developed by the Millennium Project in 2002 </li></ul><ul><li>A dynamic method for studying whether the future seems to be improving or not and for testing the effects of policy on the future outlook </li></ul><ul><li>Combines 20-30 variables important to the world or the country </li></ul>
    7. 7. Examples of Indexes <ul><li>Human Development Index </li></ul><ul><li>Dow Jones Industrial Average </li></ul><ul><li>Index of Leading Indicators </li></ul>
    8. 8. The State of the Future Index (SOFI) <ul><li>Constructed of weighted variables such as life expectancy and employment. </li></ul><ul><li>Retrospective 20 years; prospective 10 years </li></ul><ul><li>Judgments about variables, weights, expectations, and future developments </li></ul><ul><li>For the world index, experts were recommended by the Millennium Project’s Nodes. </li></ul>
    9. 9. Indexes: Caveats <ul><li>Indexes may lead to oversimplification and loss of detail </li></ul><ul><li>An index may hide cultural differences </li></ul><ul><li>Data behind the index must be preserved and transparent </li></ul><ul><li>Apparent precision may be mistaken for accuracy </li></ul>
    10. 10. Three Types of SOFI <ul><li>Global (comprised of global variables) </li></ul><ul><li>National comparison (comprised of a standard set of variables, national data) </li></ul><ul><li>National focus (comprised of variables of nation’s choosing, national data) </li></ul>
    11. 11. The Global SOFI Variables R&D expenditures  Corruption  Improved water sources  Percent unemployment  School Enrollment, secondary  Renewable Energy   Energy consumption per GDP  Internet Users  Carbon Dioxide emissions People killed or injured in terrorist attacks GDP per capita  Physicians per 1000 population  Literacy rate Population growth rate  Forest Lands  People voting in elections  Life expectancy at birth  Food availability  Infant mortality  Number of refugees Homicides, intentional  Debt Service (% of GNI) Seats held by women in parliaments  Prevalence of HIV  People Living in Extreme Poverty Global Temperature Anomalies Nuclear Proliferation Number of infectious diseases Number of armed conflicts   People in Countries that are Free
    12. 12. Global SOFI Example of SOFI (State of the Future Index) variables: Infant mortality Food availability GNP per capita Access to fresh water CO2 emissions Literacy Wars AIDS deaths Terrorist attacks Debt ratio Unemployment Calories per capita Health care Forest lands Rich poor gap …
    13. 13. .
    14. 14. Trend Impact Analysis <ul><li>Used to assess consequences of future developments on the course of the extrapolations of the variables </li></ul><ul><li>Developments expressed in probabilistic terms </li></ul><ul><li>Impacts in terms of percentage change in the end point of a variable </li></ul><ul><li>Monte Carlo solution </li></ul>
    15. 15. Examples of Developments <ul><li>A nuclear accident such as Three Mile Island (causes many nuclear nations to de-nuclearize). (10%) </li></ul><ul><li>A very good, fast $150 laptop computer becomes available everywhere. (65%) </li></ul><ul><li>Advent of a “teachers without borders” movement (50,000 new teachers in the field) (30%) </li></ul><ul><li>A pandemic of the scale of HIV/AIDS (30%) </li></ul><ul><li>At least 10 countries introduce effective policies designed to increase birth rates to avoid population implosion (75%) </li></ul><ul><li>Automation and robotics increase productivity 25% in enough countries to make “jobless&quot; economic growth (50%) </li></ul><ul><li>Availability of a cheap effective anti-aging therapy (35%) </li></ul><ul><li>Bad weather (storms, hurricanes, floods) cause wide spread crop failures in at least one year (25%) </li></ul>
    16. 16. Example of TIA Results
    17. 18. SOFI in IFs <ul><li>The IFs model </li></ul><ul><ul><li>Is a multi equation model that links economic, demographic, and social measures </li></ul></ul><ul><ul><li>Covers 182 countries </li></ul></ul><ul><ul><li>Uses both systems dynamic relationships (flows and stocks) and econometric relationships </li></ul></ul><ul><ul><li>Includes “cohort-component systems for population; markets for production, exchange, and consumption; and social accounting matrices for financial flows” </li></ul></ul><ul><ul><li>SOFI computations have been integrated into the model </li></ul></ul>
    18. 19. Dashboard Experiments: Getting Better
    19. 20. Dashboard Experiments: Getting Worse
    20. 21. Global SOFI Example of SOFI (State of the Future Index) variables: Infant mortality Food availability GNP per capita Access to fresh water CO2 emissions Literacy Wars AIDS deaths Terrorist attacks Debt ratio Unemployment Calories per capita Health care Forest lands Rich poor gap …
    21. 22. Collecting Judgments for SOFI <ul><li>Judgments include: </li></ul><ul><ul><li>What variables? </li></ul></ul><ul><ul><li>What weights? </li></ul></ul><ul><ul><li>What best and worst expectations? </li></ul></ul><ul><ul><li>What developments can affect the variables? </li></ul></ul><ul><ul><li>What are their probabilities and impacts? </li></ul></ul><ul><li>Techniques </li></ul><ul><ul><li>Delphi </li></ul></ul><ul><ul><li>Real Time Delphi </li></ul></ul><ul><ul><li>Others </li></ul></ul>
    22. 23. RTD Features <ul><li>The choice of invited participants is careful and deliberate </li></ul><ul><li>The questionnaire is presented on line </li></ul><ul><li>There are no sequential questionnaires as in conventional Delphi </li></ul><ul><li>Group averages and reasons for answers of prior participants are shown </li></ul><ul><li>When the respondent comes back to the study in a minute or a day, the original input form with their answers is presented </li></ul><ul><li>By then others may have contributed judgments, the averages and reasons may have changed </li></ul>
    23. 24. An Azerbaijan Student RTD <ul><li>Collects judgments about variables in SOFI </li></ul><ul><li>URL: </li></ul><ul><li>Sign on </li></ul><ul><li>Use code “azer” (no quotes, lower case) </li></ul><ul><li>You may answer the questions </li></ul>
    24. 25. Developments Variables (ordered by importance): Prob Import Ready 28. Massive financial crisis triggers a world depression as large as in the 1930's 42.39 7.56 7.14 53. New technology displaces carbon fuels as cheapest energy source 30.70 7.33 6.43 45. Gulf Cooperation Council moves countries in the region toward EU-like arrangements agreements  46.43 7.22 7.04 26. Extremist political religious groups change the current direction of the region  47.25 7.21 7.42 48. Aging population doubles government social costs  65.42 7.16 6.44 44. Water scarcity problems are essentially solved (for example, through low cost desalination)  27.69 7.08 6.77 37. Extremists detonate nuclear devices, dirty bombs, or other weapons of mass destruction  24.97 7.00 5.92 39. Stability achieved in Iraq  45.38 7.00 7.60 32. Renewable energy sources, like wind and solar, provide 50% of the world's power  26.30 6.84 6.70 29. Most glaciers melt twice as fast as in the decade 2000 - 2009 51.19 6.67 6.21 41. OPEC's ability to control oil production dramatically dissipates 41.66 6.65 7.29 40. Cyber warfare is more difficult to detect and triples in damages from 2010 levels.  63.62 6.54 6.45 51. Economic growth spike in other global regions limits ability to attract talent 73.46 6.44 6.79 42. Middle East oil producing countries successfully diversify beyond energy production (50% of income from sources other than oil)  34.57 6.32 6.17
    25. 26. Resources <ul><li>Real Time Delphi on line </li></ul><ul><ul><li> code “azer” </li></ul></ul><ul><li>Millennium Project reports </li></ul><ul><li><www.stateof> </li></ul><ul><li>Tutorials for Real Time Delphi and SOFI < > </li></ul><ul><li>International Futures (IFs) </li></ul><ul><ul><li> </li></ul></ul>
    26. 27. Why Tech Mining? <ul><li>Welcome to the age of too much information. </li></ul><ul><li>We need to treat: text as data to gain intelligence. </li></ul><ul><li>Mine “ST&I” [Science, Technology & Innovation] information resources to answer technology management questions = Tech Mining. </li></ul><ul><li>Enable Open Innovation </li></ul>© 2009 Search Technology, Inc.
    27. 28. Tech Mining leverages text and data mining to analyze information. Since tech mining allows us to use computers to “read” the information, we can digest far more information than we could traditionally absorb. What does Tech Mining give us?
    28. 29. <ul><li>Text and data mining techniques are good at addressing: </li></ul><ul><ul><ul><li>WHO? </li></ul></ul></ul><ul><ul><ul><li>WHAT? </li></ul></ul></ul><ul><ul><ul><li>WHEN? </li></ul></ul></ul><ul><ul><ul><li>WHERE? </li></ul></ul></ul><ul><li>Additional questions usually require more human insight: </li></ul><ul><ul><ul><li>HOW? </li></ul></ul></ul><ul><ul><ul><li>WHY? </li></ul></ul></ul>Types of Questions
    29. 30. <ul><li>Technical Information </li></ul><ul><li>Science, Technology & Innovation (“ST&I”) Databases (e.g., Web of Science; CSCD, Thomson Innovation) </li></ul><ul><li>Internet Sources (e.g., Googling) </li></ul><ul><li>Technical Expertise </li></ul><ul><li>Contextual Information </li></ul><ul><li>Business, competition, customer, policy, popular content Databases (e.g., Thomson One) </li></ul><ul><li>Internet Sources (e.g., blogs, website profiling) </li></ul><ul><li>Business Expertise </li></ul>Six information types
    30. 31. <ul><li>VantagePoint Import Filters and Tools </li></ul>On-line Data Sources Custom Data Cambridge Scientific Abstracts Factiva Patbase Comma/tab delimited tables Delphion ISI Web Of Knowledge Questel-Orbit Microsoft Excel and Access Dialog Lexis Nexis SilverPlatter SmartCharts EBSCOHost Micropatent STN XML Ei Engineering Village Ovid Thomson Innovation Databases Record/Field Tools Aerospace Focust Pascal Combine duplicate records Art Abstracts Food Sci & Tech Patent Citation Index Remove duplicate records Biobase Foodline Market PCT Create “frankenrecords” Biological Abstracts Foodline Science PCTPAT (merge records from Biological Sciences Forege Phin dissimilar sources) Biosis Frosti Pira Classify records Biotechno FSTA Pluspat Merge fields Business & Industry Gale PROMT PROMT Clean up fields CAPlus (AnaVist export) GeoRef PsycINFO Apply thesauri Cassis Global Reporter PubMed CBNB IFIPAT Rapra Claims IFIUDB Recent Refs Computer & Info Systems INPADOC Reference Manager Corrosion INSPEC Science Citation Index Current Contents IPA SciSearch Derwent Biotech Abstracts ISD Scopus Derwent Innovations Index ITRD Tech Research Derwent World Patent Index JAPIO ToxFile Ei Compendex JICST Transport EMBase Kosmet USApps EnCompass Literature LGST USPat EnCompass Patents MATBUS Waternet Energy Medline WaterResAbs EnergySciTech METADEX Web of Science Engineering Materials Abstr Mgmt and Org Studies WeldaSearch Envr Sci & Pollution Mgmt Micropatent Materials Wisdomain ERIC Mobility EuroPat NSF Awards FamPat NTIS
    31. 32. How to do Tech Mining: 8-steps <ul><li>Spell out the questions and how to answer them </li></ul><ul><li>Get suitable data </li></ul><ul><li>Search (iterate) </li></ul><ul><li>Import into text mining software (e.g., VantagePoint) </li></ul><ul><ul><li> </li></ul></ul><ul><li>Clean the data </li></ul><ul><li>Analyze & interpret </li></ul><ul><li>Represent the information well – communicate! </li></ul><ul><li>Standardize and semi-automate where possible </li></ul>Doing the Tech Mining Process
    32. 33. Sample Applications <ul><li>Developments for use in TIA, a step in SOFI </li></ul><ul><li>R&D Portfolio Management </li></ul><ul><li>Research Evaluation </li></ul><ul><li>Research Profiling </li></ul><ul><li>Tracking R&D over time </li></ul><ul><li>Research Network Analyses </li></ul><ul><li>Monitoring Research Knowledge Flows </li></ul><ul><li>Geo-mapping </li></ul><ul><li>ST&I Indicators </li></ul>
    33. 34. Ceramics in Engines <ul><li>Overcoming Management Resistance </li></ul><ul><li>Jumping Domains </li></ul><ul><li>“ Discovering” new technology </li></ul>Case Examples
    34. 35. A Success Story: Ceramics Save Tax Dollars <ul><li>TARDEC (Tank-Automotive Research, Development & Engineering Center) </li></ul><ul><li>Task in 1996: Reassess a “loser technology” – ceramics for automotive engine applications </li></ul><ul><li>R&D Profile: Amount of activity declines </li></ul><ul><li>But -- Uncovers clues of significant maturation </li></ul>Case Examples
    35. 36. Ceramics Engine Publications (85-96) Technology Maturity & Keyword Diffusion Case Examples
    36. 40. Cognitive Sci. Agri Sci Biomed Sci Chemistry Physics Engineering Env Sci & Tech Mtls Sci Infectious Diseases Psychology Social Studies Clinical Med Computer Sci. Business & MGT Geosciences Ecol Sci Econ. Polit. & Geography Health & Social Issues Azerbaijan Research, 2005-09 on Global Map of Science, SCI-SSCI 2007
    37. 41. Research Profile: Azerbaijan 2005-09 by Disciplines (top 5) Case Examples Macro-Discipline Author Affiliations Key Terms Authors Year   Top 3 Top 5 Top 3 2008-09 Chemistry[475] Natl Acad Sci Azerbaijan [119] Baku State Univ [95] Azerbaijan Acad Sci [48] synthesis [72] thermodynamic properties [27] Density [24] Water [23] methanol [21] Abdulagatov, I M [25] Magerramov, A M [19] Chyragov, F M [18] 48% of 475 Materials Sci[382] Azerbaijan Acad Sci [95] Baku State Univ [66] Azerbaijan Natl Acad Sci [64] effect [29] TlInS2 [19] Incommensurate phase [17] CRYSTALS [17] SINGLE-CRYSTALS [14] Suleymanov, R A [16] Altindal, S [14] Tagiev, O B [13] Mammadov, T S [13] 51% of 382 Engineering[333] Natl Acad Sci Azerbaijan [83] Baku State Univ [74] Azerbaijan Acad Sci [38] methanol [14] Initial stresses [11] sufficient conditions [10] thermodynamic properties [10] approximation [10] boundedness [10] Akbarov, S D [22] Guliyev, V S [16] Khanmamedov, A K [9] Abdulagatov, I M [9] Nasibov, S M [9] 50% of 333 Physics[231] Azerbaijan Acad Sci [58] Baku State Univ [47] Azerbaijan Natl Acad Sci [35] MODEL [22] PHYSICS [12] SCATTERING [10] VARIABILITY [10] SYSTEMS [9] Shahverdiev, E M [13] Shore, K A [13] Aliev, T M [12] Sultansoy, S [12] 51% of 231 Biomed Sci[105] Baku State Univ [27] Azerbaijan Med Univ [9] Azerbaijan Acad Sci [7] EFFICIENCY [10] sturgeons [8] diencephalon [7] CYTOARCHITECTONIC ANALYSIS [7] Azerbaijan [7] EXPRESSION [7] organization [7] Zeynalov, R [9] Musayev, I [9] Rustamov, E K [8] Dadasheva, N [8] 39% of 105
    38. 42. Research Profile: Baku State University – 5 Researchers Case Examples Authors Subject Category Key Terms Authors Year   Top 5 Items Top 5 Items Top 5 Items 2008-09 Magerramov, A M[19] Chemistry, Applied [8] Chemistry, Organic [7] Chemistry, Physical [3] Materials Science, Multidisciplinary [2] Optics [2] reaction [9] synthesis [9] DERIVATIVES [4] reactions [3] formation [3] Magerramov, A M [19] Allakhverdiev, M A [9] Mamedov, I G [5] Bairamov, M R [5] Farzaliev, V M [4] Rzaeva, I A [4] 68% of 19 Chyragov, F M[18] Chemistry, Analytical [14] Chemistry, Inorganic & Nuclear [4] complexation [15] photometric determination [11] stability constants [8] Chyragov, F M [18] Gadzhieva, S R [12] Makhmudov, K T [6] Alieva, R A [5] Guseinov, F E [4] 17% of 18 Allakhverdiev, M A[14] Chemistry, Applied [8] Chemistry, Organic [5] Chemistry, Physical [3] Energy & Fuels [2] Engineering, Chemical [2] Engineering, Petroleum [2] synthesis [8] reaction [6] antioxidant activity [3] cumene [3] Allakhverdiev, M A [14] Magerramov, A M [9] Farzaliev, V M [6] Rzaeva, I A [6] Guseinova, A T [4] 79% of 14 Gadzhieva, S R[14] Chemistry, Analytical [9] Chemistry, Inorganic & Nuclear [5] complexation [11] photometric determination [8] stability constants [6] Gadzhieva, S R [14] Chyragov, F M [12] Makhmudov, K T [6] Guseinov, F E [4] Pashaev, F G [3] 29% of 14 Babanly, M B[12] Materials Science, Multidisciplinary [8] Electrochemistry [2] Chemistry, Inorganic & Nuclear [2] X-ray diffraction [7] thermodynamic properties [6] differential thermal analysis [6] standard entropies [5] Babanly, M B [12] Babanly, N B [2] Sadygov, F M [2] Shykhyev, Y M [2] Imamalieva, S Z [2] Yusibov, Y A [2] 100% of 12
    39. 43. Co-authoring among top Baku State Researchers Case Examples
    40. 44. Assignments <ul><ul><li>Lecture 4, March 11: Delphi, Cross Impact and Trend Impact Analysis; read appropriate chapters in FRMv3 </li></ul></ul><ul><ul><li>Go to Real Time Delphi on line </li></ul></ul><ul><ul><ul><li> ; Code: azer </li></ul></ul></ul><ul><ul><ul><li>Register to participate. </li></ul></ul></ul><ul><ul><ul><li>Provide answers to given questions </li></ul></ul></ul><ul><ul><ul><li>Generate list of variables and developments to consider </li></ul></ul></ul><ul><ul><li>Peace Scenarios Assignment: Based on the conditions for Peace and previous agreements, generate several “causal paths” by which peace, in retrospect, may have been reached. </li></ul></ul>