Data retrieval basics_v1.0

469 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
469
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Open DataRussian Open Gov resources (Minobr, Roskomnadzor, Rosstat)Moscow Open Data Portal (Museums, Architecture, Budget)Private Resources (Openpolice, Apps4Russia, Gis-Lab)Online and Social Media Data SourcesFactiva, Medialogia, Integrum, Media Cloud, Russian Media CloudSocial API and tools (FB, VK, TW, Topsy, SocialBro, NodeXL)Multimedia ArchivesLibrary of Congress, Russian State Library, MAT, Tate, HermitagePractical Assignment
  • Open DataRussian Open Gov resources (Minobr, Roskomnadzor, Rosstat)Moscow Open Data Portal (Museums, Architecture, Budget)Private Resources (Openpolice, Apps4Russia, Gis-Lab)Online and Social Media Data SourcesFactiva, Medialogia, Integrum, Media Cloud, Russian Media CloudSocial API and tools (FB, VK, TW, Topsy, SocialBro, NodeXL)Multimedia ArchivesLibrary of Congress, Russian State Library, MAT, Tate, HermitagePractical Assignment
  • Open DataRussian Open Gov resources (Minobr, Roskomnadzor, Rosstat)Moscow Open Data Portal (Museums, Architecture, Budget)Private Resources (Openpolice, Apps4Russia, Gis-Lab)Online and Social Media Data SourcesFactiva, Medialogia, Integrum, Media Cloud, Russian Media CloudSocial API and tools (FB, VK, TW, Topsy, SocialBro, NodeXL)Multimedia ArchivesLibrary of Congress, Russian State Library, MAT, Tate, HermitagePractical Assignment
  • Open DataRussian Open Gov resources (Minobr, Roskomnadzor, Rosstat)Moscow Open Data Portal (Museums, Architecture, Budget)Private Resources (Openpolice, Apps4Russia, Gis-Lab)Online and Social Media Data SourcesFactiva, Medialogia, Integrum, Media Cloud, Russian Media CloudSocial API and tools (FB, VK, TW, Topsy, SocialBro, NodeXL)Multimedia ArchivesLibrary of Congress, Russian State Library, MAT, Tate, HermitagePractical Assignment
  • Open DataRussian Open Gov resources (Minobr, Roskomnadzor, Rosstat)Moscow Open Data Portal (Museums, Architecture, Budget)Private Resources (Openpolice, Apps4Russia, Gis-Lab)Online and Social Media Data SourcesFactiva, Medialogia, Integrum, Media Cloud, Russian Media CloudSocial API and tools (FB, VK, TW, Topsy, SocialBro, NodeXL)Multimedia ArchivesLibrary of Congress, Russian State Library, MAT, Tate, HermitagePractical Assignment
  • Data retrieval basics_v1.0

    1. 1. Center for the Study of New Media and Societywww.newmediacenter.ruData Retrieval BasicsSergey Chernov
    2. 2. Data Expert Evolution5/24/2013 Sergey Chernov, Information Retrieval Basics1995 1999 2013Everyone is a dataexpert today!
    3. 3. Outline Open Data Online and Social Media Data Sources Multimedia Archives Hands-on AssignmentSergey Chernov, Information Retrieval Basics
    4. 4. Outline Open Data Online and Social Media Data Sources Multimedia Archives Hands-on AssignmentSergey Chernov, Information Retrieval Basics
    5. 5. Open Data Open data is the idea that certain data should befreely available to everyone to use and republish asthey wish, without restrictions from copyright,patents or other mechanisms of control.5/24/2013 Sergey Chernov, Information Retrieval Basics
    6. 6. Open Gov Data Several national governments have created web sites todistribute a portion of the data they collect.5/24/2013 Sergey Chernov, Information Retrieval Basics
    7. 7. Data.gov – US Open Gov Data5/24/2013 Sergey Chernov, Information Retrieval Basics
    8. 8. Opengovdata.ru5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.facebook.com/pages/OpenGovDataru/139336139422921
    9. 9. Open Gov: Open Data5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://открытыеданные.большоеправительство.рф/
    10. 10. Roskomnadzor (Federal Service for Supervision ofCommunications, Information Technology and Mass Media)5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.rsoc.ru
    11. 11. Roskomnadzor Open Data5/24/2013 Sergey Chernov, Information Retrieval Basics
    12. 12. Roskomnadzor Open Data (cont)5/24/2013 Sergey Chernov, Information Retrieval Basics
    13. 13. Russian Ministry of Education and Science5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://минобрнауки.рф/%D0%BE%D1%82%D0%BA%D1%80%D1%8B%D1%82%D1%8B%D0%B5-%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D0%B5
    14. 14. Rosstat (Russian Federal StateStatistics Service)5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.gks.ru
    15. 15. Rosstat’s Interactive Database5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://cbsd.gks.ru/
    16. 16. Single Cross-Departament InfomationStatistical System (ЕМИСС)5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.fedstat.ru/indicators/start.do
    17. 17. Moscow Open Data Portal5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://data.mos.ru/
    18. 18. Example: Moscow Museums5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://data.mos.ru/datasets/426_muzei/map/
    19. 19. Moskomarchitecture Geoportal5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://egip.mka.mos.ru/egip/egip.nsf/va_GeoDataCatalogByCat?OpenView
    20. 20. Moscow Open Budget5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://budget.mos.ru/gp_xml
    21. 21. Openpolice.ru5/24/2013 Sergey Chernov, Information Retrieval Basics
    22. 22. Apps4Russia Annual contest ofopendata-based mobileapplications Last winner – projectSotskartochka (MySocial Card)5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.mysocialcard.ru/http://apps4russia.ru/
    23. 23. GIS-LAB5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://gis-lab.info/start.html
    24. 24. Outline Open Data Online and Social Media Data Sources Multimedia Archives Hands-on AssignmentSergey Chernov, Information Retrieval Basics
    25. 25. Factiva5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.dowjones.com/factiva/-28 languages from 200 countries over 35 years- Top news sources include The Wall Street Journal, Dow Jones Newswires, The NewYork Times, The Sydney Morning Herald and Le Monde.
    26. 26. Medialogia5/24/2013 Sergey Chernov, Information Retrieval Basicsmlg.ru
    27. 27. Integrum5/24/2013 Sergey Chernov, Information Retrieval BasicsIntegrum.ru
    28. 28. 5/24/2013 Sergey Chernov, Information Retrieval Basicswww.mediacloud.org
    29. 29. Media Cloud – Twitter vs LiveJournal5/24/2013 Sergey Chernov, Information Retrieval Basics
    30. 30. Russian Media Cloud5/24/2013 Sergey Chernov, Information Retrieval Basics
    31. 31. Russian Media Cloud (cont)5/24/2013 Sergey Chernov, Information Retrieval Basics
    32. 32. Russian Media Cloud: Association Golos5/24/2013 Sergey Chernov, Information Retrieval BasicsMedia / ClusterElectionsMonitoring andViolationsReportsElect-oralLegis-lationPressureon Golosobserversin regionsODIHRobser-versFine forillegalagitationNTV and"Surkovs Propa-ganda"Shiba-novasnote-bookRoskom-nadzorwarningDDOSattacksTOTALkommersant.ru 51 19 -- 9 2 -- -- -- -- 81golos-org LJ 29 24 7 3 -- -- -- -- -- 63gazeta.ru 22 2 -- 1 9 3 6 1 2 46echo.msk.ru/blog 13 18 1 -- 1 -- -- -- 1 34newsru.com 11 -- -- -- 3 3 1 2 2 22svpressa.ru 9 2 -- -- 1 -- 2 -- 3 17lenta.ru/ 2 -- -- -- 3 1 3 2 4 15www.kavkaz-uzel.ru 12 1 -- -- 1 -- -- -- 1 15aif.ru 11 3 -- -- -- -- -- -- -- 14anticompromat LJ 5 2 -- -- 1 3 2 -- 1 14vz.ru 3 -- -- -- 4 6 1 -- -- 14bfm.ru 2 -- -- -- 4 1 -- 1 4 12www.news2.ru 3 2 -- -- 3 1 1 1 1 12fontanka.ru 1 1 -- -- 3 2 2 1 1 11
    33. 33. VisualizingData withGephi5/24/2013 Sergey Chernov, Information Retrieval Basicsgephi.org
    34. 34. Facebook5/24/2013 Sergey Chernov, Information Retrieval BasicsCities Facebook Userscity_code1 city1 Абаза 602 Абакан 106603 Агидель 2804 Адыгейск 605 Азов 38206 Ак-Довурак 207 Алапаевск 5208 Алатырь 3209 Алейск 2010 Анапа 382011 Ангарск 640012Анжеро-Судженск 62013 Анива 16014 Апатиты 358015 Арамиль 38016 Арзамас 40
    35. 35. Vk.com5/24/2013 Sergey Chernov, Information Retrieval Basics- Server with 12 cores and 40Tb of space- Several weeks of crawling- PostgreSQL database- 120 mln user profilesCities VK Users Facebook Userscity_code1 city1 Абаза 4312 602 Абакан 89141 106603 Агидель 6336 2804 Адыгейск 1805 605 Азов 24208 38206 Ак-Довурак 3548 207 Алапаевск 12697 5208 Алатырь 11280 3209 Алейск 5918 2010 Анапа 42679 382011 Ангарск 68572 640012 Анжеро-Судженск 21929 62013 Анива 1603 16014 Апатиты 27148 358015 Арамиль 2368 380http://habrahabr.ru/post/123856/
    36. 36. Twitter API5/24/2013 Sergey Chernov, Information Retrieval Basics
    37. 37. Yandex Blog Search5/24/2013 Sergey Chernov, Information Retrieval Basics
    38. 38. Topsy5/24/2013 Sergey Chernov, Information Retrieval Basics
    39. 39. SocialBro5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://userguide.socialbro.com/post/40596560247/using-socialbro-to-reach-the-maximum-audience-with-your
    40. 40. Social Bro (cont)5/24/2013 Sergey Chernov, Information Retrieval Basics
    41. 41. NodeXL5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://nodexl.codeplex.com/
    42. 42. Outline Open Data Online and Social Media Data Sources Multimedia Archives Practical AssignmentSergey Chernov, Information Retrieval Basics
    43. 43. Library of Congress5/24/2013 Sergey Chernov, Information Retrieval Basics
    44. 44. Russian State Library5/24/2013 Sergey Chernov, Information Retrieval Basics
    45. 45. The MAT5/24/2013 Sergey Chernov, Information Retrieval Basics
    46. 46. Tate5/24/2013 Sergey Chernov, Information Retrieval Basics
    47. 47. The Hermitage5/24/2013 Sergey Chernov, Information Retrieval Basics
    48. 48. Outline Open Data Online and Social Media Data Sources Multimedia Archives Hands-on AssignmentSergey Chernov, Information Retrieval Basics
    49. 49. Types of OGD Apps 13 categories of appsidentified: health & safety, entertainment, sports, transportation, city services, real estate,5/24/2013 Sergey Chernov, Information Retrieval Basicshttp://www.slideshare.net/timoreilly/open-up-15195896 environment, education, public safety & lawenforcement, food & dining, development, business & finance, government & civics.
    50. 50. Russian Gov Data Applications Government tender system 3akupki.ru., bicotender.ru, initpro.ru, ist-budget.ru, is-zakupki.ru,goszakaz.ru, goszakupki-cpr.ru, marketing.interfax.ru,multitender.ru, my-tender.ru, regionzakaz.ru, seldon.ur.ru, tender-spb.ru Legal cases analysis rospravosudie.com, www.pravo.ru, www.intergrum.ru, spark-interfax.ru Income declarations declarator.org, publicprofit.ru Budget budget4me.ru, rosspending.ru Public transport rasp.yandex.ru, maps.google.ru, doroga.tv, Rusavtobus.ru5/24/2013 Sergey Chernov, Information Retrieval Basics
    51. 51. International OGD Apps http://www.fixmystreet.com/ http://seeclickfix.com/ http://codeforamerica.org/ http://open311.org/ www.recovery.gov http://data.seattle.gov/ (Notifire, MomMaps) https://nycopendata.socrata.com/ (WiFi) http://2011.nycbigapps.com/submissions http://daten.berlin.de/ (Wheelmap, Aircraft noise)5/24/2013 Sergey Chernov, Information Retrieval Basics
    52. 52. Hands-on assignment Working in teams of 2-3 people Search for motivating open data projects on the Web Explore data.mos.ru, openpolice.ru, opengovdata.ru orany other resource of your choice Propose a project based on some dataset. How the datalooks like? What is missing? Which skills and tools youmight need? Present your project5/24/2013 Sergey Chernov, Information Retrieval Basics

    ×