Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mapping Commodity Trading

1,138 views

Published on

Historical Trading Data by Team Ash at the Big Data InfoVis Summer School

members
Joe Wandy
Asma Malik
Michael Mauderer
Sadiq Sani
Benjamin Bach

  • Check the source ⇒ www.HelpWriting.net ⇐ This site is really helped me out gave me relief from headaches. Good luck!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I have always found it hard to meet the requirements of being a student. Ever since my years of high school, I really have no idea what professors are looking for to give good grades. After some google searching, I found this service ⇒ www.HelpWriting.net ⇐ who helped me write my research paper.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Mapping Commodity Trading

  1. 1. Mapping Commodity Trading in the 19th Century Benjamin Bach, INRIA, Paris Asma Malik, University of Strathclyde, Glasgow Michael Mauderer, University of St Andrews Sadiq Sani, Robert Gordon University, Aberdeen Joe Wandy, University of Glasgow
  2. 2. Outline ● Project Overview ● Data ● Technology ● Demo ● Future Work
  3. 3. Overview 19th Century Commodities Diseases Locations Disasters
  4. 4. Process
  5. 5. Tasks ● Retrieve documents mentioning ○ Commodities ○ Locations ○ Time range ● Relations between retrieved terms ○ Spatial relations ○ Temporal relations ○ Co-occurrence relations Users: Historians
  6. 6. Data ● Commodities: 1067 ● Time: 1600 - 1952 (452 years) ● Documents: 18 580 ● Location occurrences: 91 650 469 ● Commodity occurrences: 29 020 013
  7. 7. The Data ● PostgreSQL Database in Edinburgh ○ Not accessible ● PostgreSQL Database in St Andrews ○ Low Performance ● PostgreSQL Database Backup ○ 2.5GB compressed binary data ○ Cannot be imported into Amazon RDS
  8. 8. Solution 1 ● Create a more compatible SQL export to import into Amazon RDS ○ 24GB raw text file containing SQL statements ○ still incompatible ○ hard to correct errors in a timely manner
  9. 9. Solution 2 ● Create EC2 instance running a PostgreSQL database ○ Powerful enough ○ Enough storage ○ Accessible
  10. 10. Big Data Problems ● Simple things take a long time ● Incremental finding of errors/new problems
  11. 11. The Pipeline ● D3 for client-side presentation ● Java+SQL for server-side processing data Database Web Service Client Commodities, date range
  12. 12. Initial Sketches
  13. 13. Visualization - Space and time -> Finding related terms + documents - find related documents - what are documents talking about - Implicit knowledge: - Co-occurrences of terms in documents For every commodity: 1) Get top 10 documents, 2) Limit related terms to 6 3) Sum up co-occurrences
  14. 14. Demo
  15. 15. Future work - Query by Location - Time diagrams for term frequency over time - Encode information in matrix cells (#doc,collection..) - Show and browse documents - Handle big data: diseases, disasters, .. - Co-occurrences ?
  16. 16. Thank you for listening!

×