#FAOdata                                                                    #FAOdata    http://data.fao.org Why, What and ...
#FAOdata                                                        #FAOdata Food and Agriculture Organization of the United N...
#FAOdata                                                           #FAOdata    Topics   Animal Production & Health       ...
#FAOdata                                                              #FAOdata The Need Recommendations from Evaluations, ...
#FAOdata                                                         #FAOdata    Uniting Our Data    http://data.fao.org is a ...
#FAOdata                                                                      #FAOdata Overview of Deliverables  n   Web S...
#FAOdata                                                          #FAOdata Mantra1. Uniting our data – data.fao.org taglin...
#FAOdata                                                         #FAOdata The Website                Should be the home sc...
#FAOdata                                                         #FAOdata Website - Use cases1.   Browse and share2.   Fin...
#FAOdata                                                                    #FAOdata Site structure                       ...
#FAOdata                                                        #FAOdata Statistics, maps, pictures, documents and more.07...
#FAOdata                                                       #FAOdata Search07/06/2012   data.fao.org - European Data Fo...
#FAOdata                                                       #FAOdata Navigate07/06/2012   data.fao.org - European Data ...
#FAOdata                                                          #FAOdata Landing Page07/06/2012      data.fao.org - Euro...
#FAOdata                                                        #FAOdata Engage, not just disseminate07/06/2012    data.fa...
#FAOdata                                                                #FAOdata Engage, not just disseminate             ...
#FAOdata                                                         #FAOdata Website – Statistical Data07/06/2012     data.fa...
#FAOdata                                                        #FAOdata Mobile First - Responsive Design07/06/2012    dat...
#FAOdata                                                          #FAOdata Data Loading 10 of 16 knowledge divisions 356,0...
finding data will be this easy... Release One 12 December 2012For more information, visit us at data.fao.org              ...
#FAOdata                                                          #FAOdata Guiding Principles n   Custodian only providing...
#FAOdata                                                           #FAOdata Data Flow and Integration                 Data...
#FAOdata                                                                                      #FAOdata data.fao.org - The ...
#FAOdata                                                         #FAOdata Catalogue, UUIDs and APIs Every asset is registe...
#FAOdata                                                                        #FAOdata Types of Metadata Structural , t...
#FAOdata                                                       #FAOdata Metadata StandardsAsset            Standards / For...
#FAOdata                                                       #FAOdata AGROVOCUN-FAO corporate, multilingualthesaurus of ...
#FAOdata                                                         #FAOdata LOD2: Publink Consultation Program Experience, ...
#FAOdata                                                        #FAOdata FAO Geopolitical OntologyUN-FAO geopolitical onto...
#FAOdata                                                                 #FAOdata  FAO Geopolitical OntologyNames (short, ...
#FAOdata                                                          #FAOdata APIs, Widgets and Embedded Content             ...
#FAOdata                                                        #FAOdata Widgets – Country ProfilesCountryProfiles = FAO s...
#FAOdata                                                                 #FAOdata Country Information  Country Profiles w...
#FAOdata                                                         #FAOdataWeb Application Example - Key Water Indicator Por...
#FAOdata                                                            #FAOdata Global Agro-Ecological ZonesInformation at 3 ...
#FAOdata                                                                                              #FAOdata Global Agro...
#FAOdata                                                       #FAOdata             http://gaez.fao.org07/06/2012   data.f...
#FAOdata                                                                                      #FAOdata data.fao.org - The ...
#FAOdata                                                                                                           #FAOdat...
#FAOdata                                                                       #FAOdata Technologies – Just 1 or 2 ;-) Op...
#FAOdata                                                           #FAOdata Some Big Bits                   ESB and BPM   ...
#FAOdata                                                              #FAOdata Infrastructure Fine-Grained monitoring and...
#FAOdata                                                         #FAOdata Build Lifecycle – Continuous Integration07/06/20...
#FAOdata                                                       #FAOdata Thank you & QuestionsThank you for listening …    ...
Upcoming SlideShare
Loading in …5
×

EDF2012 Karl Morteo - data.fao.org

827 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
827
On SlideShare
0
From Embeds
0
Number of Embeds
193
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Food and Agriculture Organization United Nations Specialized AgencyHeadquarters in RomeRegional, Subregional, Country (approx 150) and Liaison Offices throughout the World
  • OFFICE OF THE DIRECTOR-GENERALOffice of EvaluationOffice of the Inspector-GeneralLegal OfficeEthics OfficeOffice of Corporate Communications and External RelationsOffice of Strategy, Planning and Resources ManagementAGRICULTURE AND CONSUMER PROTECTION DEPARTMENTAnimal Production and Health DivisionNutrition and Consumer Protection DivisionPlant Production and Protection DivisionRural Infrastructure and Agro-Industries DivisionJoint FAO/IAEA Division of Nuclear Techniques in Food and AgricultureECONOMIC AND SOCIAL DEVELOPMENT DEPARTMENTAgricultural Development Economics DivisionStatistics DivisionTrade and Markets DivisionGender, Equity and Rural Employment DivisionFISHERIES AND AQUACULTURE DEPARTMENTFisheries and Aquaculture Policy and Economics DivisionFisheries and Aquaculture Resources Use and Conservation DivisionFORESTRY DEPARTMENTForest Economics, Policy and Products DivisionForest Assessment, Management and Conservation Division NATURAL RESOURCES MANAGEMENT AND ENVIRONMENT DEPARTMENTClimate, Energy and Tenure DivisionLand and Water DivisionTECHNICAL COOPERATION DEPARTMENTPolicy and Programme Development Support DivisionEmergency Operations and Rehabilitation DivisionInvestment Centre DivisionCORPORATE SERVICES, HUMAN RESOURCES AND FINANCE DEPARTMENTShared Services Centre Human Resources Management DivisionFinance DivisionConference, Council,and Protocol Affairs DivisionAdministrative Services DivisionOFFICE OF THE DEPUTY DIRECTOR-GENERAL: KnowledgeOffice of Knowledge Exchange, Research and ExtensionOFFICE OF THE DEPUTY DIRECTOR-GENERAL: OperationsChief Information Officer DivisionOffice of Support to Decentralisation Animal Production & Health Animal Health Animal diseases & Control Animal genetic resources & Breeding Animal nutrition & Feeds Production & Livestock Economics & PolicyAgroindustry Economic development Policies Trade Education & Extension Communication for development Education Extension Training & Capacity building Engineering, Technology & Research Biotechnology Geographical information systems Postharvest technology Research Statistics Farming Practices & Systems Farm management Farming systems Land use Fisheries & AquacultureAquaculture Development Ecosystems Governance Issues Food Security Agricultural situation & Early warning Emergency relief & Food aid Ethics Food supply International cooperation Food safety & Human nutrition Diet & Nutrition Food additives Food composition Nutrition education Quality controls & Assurance Forestry Assessment & Monitoring Environment Forest management Forest products & Services Forest resources Geographical & Regional Information Africa America Asia Europe Oceania Government, Administration & Legislation Administration Agricultural and rural legislation Environmental legislation Fisheries Food legislation Information Management Databases & Information systems Documentation Early warning systems Geographical information systems Information science Natural Resources & Environment Biodiversity Climate change Desertification Drainage & Irrigation Ecology & Ecosystems Plant Production & Protection Crops & Crop management Fertilizers Integrated pest management Irrigation Pest control & Pesticides Rural & Social Development Gender HIV/AIDS Households Participation Poverty
  • http://aarhus11.jboye.com/news/responsive-web-design-in-practice/
  • Screen cast and RoamBI demo on the iPad ….UservoiceAsset focus switching
  • http://api.data.fao.org/resources/<type>/<database>/<resourcename>.<format>
  • FAO’s Office of Knowledge, Exchange, Research and Extension, in cooperation with a community of practitioners
  • Central Product Classification (CPC) http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=16SITC (Standard International Trade Classification)http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=28Species2000 http://www.sp2000.org/ITIS Catalogue of Life
  • FAO’s Office of Knowledge, Exchange, Research and Extension, in cooperation with a community of practitioners
  • Source Control Management (SVN)Dependency management (Maven2)Automatic Unit & Integration Tests (JUnit)Performance Analysis Test (Apache JMeter)Issue Management (JIRA)Continuous Integration (Jenkins)Binary Code Repository (Artifactory)Service documentation (WIKI)Monitoring System (JBoss RHQ)Job Scheduler (SOS JobScheduler): ETL jobs (TalendOS), Geospatial batches (Geobatch), Custom jobs (java, etc.)
  • EDF2012 Karl Morteo - data.fao.org

    1. 1. #FAOdata #FAOdata http://data.fao.org Why, What and How Karl Morteo , Project Manager, IT Division, UN-FAO Karl.Morteo@fao.org07/06/2012 data.fao.org - European Data Forum 2012 1
    2. 2. #FAOdata #FAOdata Food and Agriculture Organization of the United Nations For a world without hunger07/06/2012 data.fao.org - European Data Forum 2012 2
    3. 3. #FAOdata #FAOdata Topics Animal Production & Health  Forestry Economics & Policy  Natural Resources & Education & Extension Environment Engineering, Technology &  Plant Production & Research Protection Farming Practices & Systems  Rural & Social Development Fisheries & Aquaculture  Climate, Energy and Tenure Food Security  Land and Water Food safety & Human nutrition  Agriculture07/06/2012 data.fao.org - European Data Forum 2012 3
    4. 4. #FAOdata #FAOdata The Need Recommendations from Evaluations, Review and Consultations n Unifying the fragmented and “linear” information storage and dissemination in the area of the technical systems n Consolidating the fragmented and duplicated Information Systems and Information Management environments presently distributed across many organizational units n A top priority for the Information Technology Division is Consistent, Integrated, Authoritative and Easy to Access Corporate Data Repository07/06/2012 data.fao.org - European Data Forum 2012 4
    5. 5. #FAOdata #FAOdata Uniting Our Data http://data.fao.org is a one-stop shop that aggregates, integrates, and catalogues data from multiple sources within the Organization. These entries cover topics related to nutrition, food and agriculture and include data such as statistics, maps, pictures, documents and more.07/06/2012 data.fao.org - European Data Forum 2012 5
    6. 6. #FAOdata #FAOdata Overview of Deliverables n Web Site: http://data.fao.org for data consumers, managers and information systems integrators n Data Warehouse Repository (supporting statistics, maps and digital assets) n Tools for data and metadata management including support for data hotels n A platform for integration and exchange n Integration of existing information systems n Harmonisation and de-duplication of data n Widgets to embed services in websites, including charts, maps and content n Advocacy and communication n Project management and administration07/06/2012 data.fao.org - European Data Forum 2012 6
    7. 7. #FAOdata #FAOdata Mantra1. Uniting our data – data.fao.org tagline2. Engage not just disseminate (harness the crowd) #FAOdata3. Mobile First – J. Boye – Morgan Stanely4. Serve the data in the most convenient format for all users - Linked and Open Data5. Eat your own dog food - http://en.wikipedia.org/wiki/Eating_your_own_dog_food07/06/2012 data.fao.org - European Data Forum 2012 7
    8. 8. #FAOdata #FAOdata The Website Should be the home screen07/06/2012 data.fao.org - European Data Forum 2012 8
    9. 9. #FAOdata #FAOdata Website - Use cases1. Browse and share2. Find and use (e.g. journalists preparing reports)3. Query and analyze4. Just give me the data07/06/2012 data.fao.org - European Data Forum 2012 9
    10. 10. #FAOdata #FAOdata Site structure 6 official languages Login Help Full text search Structured search Filter by object Navigation by Topics, Concepts, Tags, Places, Catalogue types07/06/2012 data.fao.org - European Data Forum 2012 10
    11. 11. #FAOdata #FAOdata Statistics, maps, pictures, documents and more.07/06/2012 data.fao.org - European Data Forum 2012 11
    12. 12. #FAOdata #FAOdata Search07/06/2012 data.fao.org - European Data Forum 2012 12
    13. 13. #FAOdata #FAOdata Navigate07/06/2012 data.fao.org - European Data Forum 2012 13
    14. 14. #FAOdata #FAOdata Landing Page07/06/2012 data.fao.org - European Data Forum 2012 14
    15. 15. #FAOdata #FAOdata Engage, not just disseminate07/06/2012 data.fao.org - European Data Forum 2012 15
    16. 16. #FAOdata #FAOdata Engage, not just disseminate Login and share with popular Social Networks:  Google+  Facebook  Twitter Expertise at FAO:  Curate  Create  Link, Tag, Share  Comment07/06/2012 data.fao.org - European Data Forum 2012 16
    17. 17. #FAOdata #FAOdata Website – Statistical Data07/06/2012 data.fao.org - European Data Forum 2012 17
    18. 18. #FAOdata #FAOdata Mobile First - Responsive Design07/06/2012 data.fao.org - European Data Forum 2012 18
    19. 19. #FAOdata #FAOdata Data Loading 10 of 16 knowledge divisions 356,000,000 Statistical values 734,000 Geo Layers 1,500,000 Statistical Maps 435 Documents 90 Pictures 40+ Terabytes (2 Stats, 8 Maps online) 25 Information Systems07/06/2012 data.fao.org - European Data Forum 2012 19
    20. 20. finding data will be this easy... Release One 12 December 2012For more information, visit us at data.fao.org Picture from Goodfon
    21. 21. #FAOdata #FAOdata Guiding Principles n Custodian only providing the receptacle, the data ownership, rights and responsibilities stay with the contributor n Will not replace existing thematic ‘brands’ n Open to all FAO contributors n One stop shop single point of access and reference for all data07/06/2012 data.fao.org - European Data Forum 2012 21 21
    22. 22. #FAOdata #FAOdata Data Flow and Integration Data Source Ingest Harmonise Data Integrate Source Enrich Data Publish Source Data Source07/06/2012 data.fao.org - European Data Forum 2012 22
    23. 23. #FAOdata #FAOdata data.fao.org - The Big Picture Specialised Website Services and Widgets application(s) consume/provide Orchestration and Integration Search Catalogue Statistics Maps Content Infrastructure Statistical Data Full text Identity Warehouse Geospatial Documents Logging Structured Metadata Raster Pictures Caching Linked Data Time Series Vector Video Security ... Indicators Point Multimedia Audit Observations Pages ...07/06/2012 data.fao.org - European Data Forum 2012 23
    24. 24. #FAOdata #FAOdata Catalogue, UUIDs and APIs Every asset is registered in the Catalogue with a Universally Unique Identifier (UUID). UUID:11ff9540-be95-46c6-b2bc-c5874d21d4d1 With a corresponding URI http://ref.data.fao.org/UUID Assets can also be accessible and managed through an Application Programming Interfaces (API) http://api.data.fao.org/resources/ images/countryprofiles/flag-arg07/06/2012 data.fao.org - European Data Forum 2012 24
    25. 25. #FAOdata #FAOdata Types of Metadata Structural , typically derived from physical data, organization and/or structure. Examples include the dimensions of a data cube, number of facts in a data set or image file format. Reference, describes the data and is the information used to understand the data. Examples include title, author, subjects, keywords, publisher, method of collection and processing, data source and references. Administrative, technical information regarding the data storage and rights management including: ownership, permissions, visibility, security, rights back up, audit, and preservation. Social, describes how the community consume, use and categories the data. Examples include: number of citations, download counts, user rating, comments and tagging.07/06/2012 data.fao.org - European Data Forum 2012 25
    26. 26. #FAOdata #FAOdata Metadata StandardsAsset Standards / FormatsStatistics SDMX, DDIMaps OGC, ISO19115Documents Dublin Core, MODS, FRBRPictures IPTC, XMPTags …… more07/06/2012 data.fao.org - European Data Forum 2012 26
    27. 27. #FAOdata #FAOdata AGROVOCUN-FAO corporate, multilingualthesaurus of topics related toagriculture, forestry, fisheries, environment and more. Available in more than 20 languages, with an average of 40.000 terms.Published as Linked Data and aligned with twelve vocabularies. http://aims.fao.org/agrovoc/lod07/06/2012 data.fao.org - European Data Forum 2012 27
    28. 28. #FAOdata #FAOdata LOD2: Publink Consultation Program Experience, Guidance and Best Practices Linked Data Model to expose social metadata Alignment and LOD-ificationCentral Product Classification in RDF integrated in the data catalogue of data.fao.org07/06/2012 data.fao.org - European Data Forum 2012 28
    29. 29. #FAOdata #FAOdata FAO Geopolitical OntologyUN-FAO geopolitical ontology. Available in 6+1 languages. Includes FAO, UN and internationally recognized data sources. Records historical changes.Published as Linked Data, RDF and OWL http://www.fao.org/countryprofiles/geoinfo/geo political/resource/07/06/2012 data.fao.org - European Data Forum 2012 29
    30. 30. #FAOdata #FAOdata FAO Geopolitical OntologyNames (short, official) , Codes,Area types, Groups membershipSupporting data: coordinates, Land borders Currency names and codes, Adjectives of nationality,Basic statistical data: country area, land area, agricultural land area, population and GDP07/06/2012 data.fao.org - European Data Forum 2012 30
    31. 31. #FAOdata #FAOdata APIs, Widgets and Embedded Content What’s a widget? Program that performs simple function and can be embedded into a webpage (e.g. weather report or stock quote) http://review-data.fao.org/developers07/06/2012 data.fao.org - European Data Forum 2012 31
    32. 32. #FAOdata #FAOdata Widgets – Country ProfilesCountryProfiles = FAO site + Widgets07/06/2012 data.fao.org - European Data Forum 2012 32
    33. 33. #FAOdata #FAOdata Country Information  Country Profiles www.fao.org  Country Briefs www.fao.org  FAO Representations websites (Sub-regional Office for the Pacific Islands)  Country Office Information Network coin.fao.org  International Plant Protection Convention Country Profiles www.ippc.int  Names of Countries http://termportal.fao.org/faonocs/main/start.do07/06/2012 data.fao.org - European Data Forum 2012 33
    34. 34. #FAOdata #FAOdataWeb Application Example - Key Water Indicator Portal http://www.unwater.org/statistics_KWIP.html07/06/2012 data.fao.org - European Data Forum 2012 34
    35. 35. #FAOdata #FAOdata Global Agro-Ecological ZonesInformation at 3 inputs levels (high, intermediate, low) on:  agro-ecological zones  agro-climatically attainable yields  yield constraints  crop calendars  agro-ecological suitability and productivity assessment  potential production estimates  actual yield and production  yield and production gaps07/06/2012 data.fao.org - European Data Forum 2012 35
    36. 36. #FAOdata #FAOdata Global Agro-Ecological Zones  THEMES  5 thematic areas (> 300,000 global datasets at mainly 5 arc-minutes, also core layers at 30 arc-seconds)  CROPS  11 crop groups, 49 crops, 92 crop types and 280 Crop/LUTs  downscaling of 23 crops/commodities are available for 23 crop/commodities  yield and production gap analysis for 17 crops/commodities  WATER SUPPLY  5 water supply types (rain-fed, irrigation, gravity, sprinkler, drip)  INPUT LEVELS  4 Input levels (High, Intermediate, Low, Mixed)  TIME PERIOD  Historical 1961-2000, 30 year average (1961-1990) and Future, 2020s, 2050s, 2080s)07/06/2012 data.fao.org - European Data Forum 2012 36
    37. 37. #FAOdata #FAOdata http://gaez.fao.org07/06/2012 data.fao.org - European Data Forum 2012 37
    38. 38. #FAOdata #FAOdata data.fao.org - The Technology Specialised Website Services and Widgets application(s) consume/provide Orchestration and Integration Search Catalogue Statistics Maps Content Infrastructure Statistical Data Full text Identity Warehouse Geospatial Documents Logging Structured Metadata Raster Pictures Caching Linked Data Time Series Vector Video Security ... Indicators Point Multimedia Audit Observations Pages ...07/06/2012 data.fao.org - European Data Forum 2012 38
    39. 39. #FAOdata #FAOdata Resource Catalog Other data.fao.org Subsystems External Applications Enterprise Service Bus (ESB) Resource Catalog SOAP Web Service interface RESTful interface Service Layer Business Layer (JEE) Full-text SQL/JPA-QL SPARQL XQuery query RDBMS Full-text Triple Store XML-DB (Postgres) Index (SOLR) (Ontotext) 07/06/2012 data.fao.org - European Data Forum 2012KISAF: Linked Open Data – 24 November, 2011 39 39
    40. 40. #FAOdata #FAOdata Technologies – Just 1 or 2 ;-) Opscode –Chef, Red Hat- RHQ, Nagios- Enterprises Nagios, Jenkins-CI – Opensource – Jenkins, Apache Maven, Apache – SVN, Atlassian – JIRA, Apache – Jmeter, Sourceforge Opensource – Junit, Oracle- Java, Liferay - Liferay Portal, PostgreSQL – PostgreSQL PostGIS, Pgbouncer – pgbouncer, Pentaho - Pentaho BI, Analytical Labs – Saiku, Talend - Talend Open Studio, RedHat/Jboss - Application Server, Vmware - Server Virtualization, Exceliance – HAProxy, RedHat- Enterprise Linux Server, GeoNetwork OpenSource – GeoNetwork, OpenGeo, GeoSolutions, Fedora Commons, Refractions Research – GeoServer, Ontotext - OWLIM, RedHat/Jboss - JBoss AS / JEE, Alfresco - Activiti BPM Platform, jasig - CAS Client, Batchwork Software - Doc2Doc, Google Code Opensource - ZXing ("Zebra Crossing"), Highsoft Solutions AS – highcharts, Twitter – Bootstrap, Liferay - Alloy-UI, JQuery - JQuery-ui, Geert de Deckere - Graph Up and the there are all the SaaS products ….07/06/2012 data.fao.org - European Data Forum 2012 40
    41. 41. #FAOdata #FAOdata Some Big Bits ESB and BPM ~2TB07/06/2012 data.fao.org - European Data Forum 2012 41
    42. 42. #FAOdata #FAOdata Infrastructure Fine-Grained monitoring and alert – RHQ + Nagios Backup: VM images and DB redundancy 14* Physical Servers, 40+ Production VMs 100 TB SAN, 24 port 1 Gigabit switchDevOps: Developers and SysAdminsworking together as oneto deliver agile, dependable andresponsive solutions:Changing the way we do IT07/06/2012 data.fao.org - European Data Forum 2012 42
    43. 43. #FAOdata #FAOdata Build Lifecycle – Continuous Integration07/06/2012 data.fao.org - European Data Forum 2012 43
    44. 44. #FAOdata #FAOdata Thank you & QuestionsThank you for listening … Questions?07/06/2012 data.fao.org - European Data Forum 2012 44

    ×