How Quick Can We Be? Data Visualization Techniques for Engineers.
Upcoming SlideShare
Loading in...5
×
 

How Quick Can We Be? Data Visualization Techniques for Engineers.

on

  • 2,945 views

 

Statistics

Views

Total Views
2,945
Slideshare-icon Views on SlideShare
2,896
Embed Views
49

Actions

Likes
3
Downloads
35
Comments
0

3 Embeds 49

http://lanyrd.com 47
http://stg.lanyrd.org 1
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • How quick can we be?\nData visualization techniques for FE Engineers\n
  • \n  Alice - Senior Engineer for Google's Crisis Response Team\n    Background - Person Finder, \n        Maps, Back-end engineering, C++, Python\n\n\n   Avni - Senior Front-end Engineer for Yahoo's Tiger Team\n    Background - DB backed web development, \n                    F2E for the last 2 years\n\n This is us at RHoK as we're very sleep deprived trying to get our hacks done. \n\n
  • \n  Alice - Senior Engineer for Google's Crisis Response Team\n    Background - Person Finder, \n        Maps, Back-end engineering, C++, Python\n\n\n   Avni - Senior Front-end Engineer for Yahoo's Tiger Team\n    Background - DB backed web development, \n                    F2E for the last 2 years\n\n This is us at RHoK as we're very sleep deprived trying to get our hacks done. \n\n
  • I want to first explain how we came to speaking about data visualization. \n\nAlice and I are both very passionate about Open Source technologies and bringing women into computing. \nAt the Grace Hopper Conference in October, the largest technical conference for women worldwide, \nwe both participated on the Open Source for Good Panel. The panel was hoping to motivate women there, who \nwere very new to Open Source, to become involved in Humanitarian FOSS projects. \n\nAlice had a spur of the moment thought to challenge the women at Grace Hopper to attend the next Random Hacks of Kindness event.\n\nRandom Hacks of Kindness or RHoK is a community of developers around the world working to develop software solutions that \nrespond to the challenges facing humanity today. Twice a year, the RHoK core team holds hack events around the world. \nAnd it just so happened there was hack taking place in December about two months after the Grace Hopper Conference.\n\n
  • - At previous RHoK events, there were only about 2-3 women attending each location. \n- Usually, the number of women in a technical community or work environment is very few and over the last 20 years, \n the number of women graduating with CS degrees has dropped drastically [1]. \n\n- So we decided we would set the challenge to an ambitious 20%. \n- We would aim to get the goal of having 20% of the participants be female at each of the 21 locations around the world. We announced the challenge at Grace Hopper and promoted it via FB, Twitter, and Linked in groups, mailing lists, and a scholarship for 1 female student.\n\n- After the event - we wanted to see how many locations reached the 20%\n- In several locations were were quite successful. For example, in Chicago where we hit 31%, the winning group was comprised 4/5 female.\n\n\n[1] http://portal.acm.org/citation.cfm?id=1461928.1461947&preflayout=flat&CFID=19515564&CFTOKEN=41185706  \n\n
  • \nWe also wanted to represent the data we collected in a easy to comprehend manner.  \nThe data from the RHoK event made it very clear that we are so far from getting to 50/50.\n\nWe wanted to present this data so people feel shocked when they see it and are \nperhaps compelled to raise awareness and take action regarding the gender disparity in open source and CS in general.\n\nData visualization is not something either one of us do every day and we have limited amount of time due to our day jobs.\nbut we wanted to see what we could do with the data we had. \n\nWe talked to colleagues and found out that this is actually an interesting problem. A lot of engineers, even F2Es, don't think DV is easy.\nWe wanted to see if this was true. So we gave ourselves a challenge to explore data visualization techniques and the idea of the talk formed.\n\nI want to first show you the raw data we had. \n
  • \nThe RHoK challenge was a spur of the moment idea by Alice that we presented at GHC. \nWe didn't take have a data collection process in place when we made the challenge. We kind of just did it. \n\nWe had online event registration - but at each of the location, people wrote their names when they signed in, so we didn’t have an online record of \nwho came to each hack or the gender of people who attended. \n\nThis is a snapshot of our original data set. We had one row per participant and Alice and I manually entered in the gender for every single person. \nKeep in mind this is over 1300 registrants for all of the locations. \n\nNow that we had all this data we wanted to figure out a way to present it in a manner that is easily digestible by the public and by the RHoK core team. \n\n\n
  • \nWe first aggregated the data so we could see how each location fared using spreadsheet formulas to help us.\n\nWe then wanted to start looking at different tools to help us visualize the data. \n\n
  • \nWe decided on the following process : \n\n- We would start with a set of tools to evaluate\n- We would take 10 minutes to see if the tool is viable option\n- If we found the tool to be viable, we would explore the tool further to reach our desired goals.\n
  • With our goals being to: \n- Plot number of people at each location\n- Plot percentage of women at each location\n- Be able to plot 2 dimensions - both the number of people and the percentage of women\n- Be able to clearly show the gender disparity\n- Be able to use this tool to easy visualize the data for future RHoKs. We want to continue this challenge and hopefully slowly \nincrease the goal percentage as more and more women attend. \n- We would take 1 hour with each tool to see if can reach our metric of success\n
  • With our goals being to: \n- Plot number of people at each location\n- Plot percentage of women at each location\n- Be able to plot 2 dimensions - both the number of people and the percentage of women\n- Be able to clearly show the gender disparity\n- Be able to use this tool to easy visualize the data for future RHoKs. We want to continue this challenge and hopefully slowly \nincrease the goal percentage as more and more women attend. \n- We would take 1 hour with each tool to see if can reach our metric of success\n
  • With our goals being to: \n- Plot number of people at each location\n- Plot percentage of women at each location\n- Be able to plot 2 dimensions - both the number of people and the percentage of women\n- Be able to clearly show the gender disparity\n- Be able to use this tool to easy visualize the data for future RHoKs. We want to continue this challenge and hopefully slowly \nincrease the goal percentage as more and more women attend. \n- We would take 1 hour with each tool to see if can reach our metric of success\n
  • With our goals being to: \n- Plot number of people at each location\n- Plot percentage of women at each location\n- Be able to plot 2 dimensions - both the number of people and the percentage of women\n- Be able to clearly show the gender disparity\n- Be able to use this tool to easy visualize the data for future RHoKs. We want to continue this challenge and hopefully slowly \nincrease the goal percentage as more and more women attend. \n- We would take 1 hour with each tool to see if can reach our metric of success\n
  • With our goals being to: \n- Plot number of people at each location\n- Plot percentage of women at each location\n- Be able to plot 2 dimensions - both the number of people and the percentage of women\n- Be able to clearly show the gender disparity\n- Be able to use this tool to easy visualize the data for future RHoKs. We want to continue this challenge and hopefully slowly \nincrease the goal percentage as more and more women attend. \n- We would take 1 hour with each tool to see if can reach our metric of success\n
  • The first tool we started with was openheatmap. \n\n\n
  • - OpenHeatMap is an opensource project, implemented by Pete Warder.\n - It used the OpenStreetMap tiles to draw maps. \n\n- The service designed to make it easy for non-specialists to create maps for communicating information. (dream tool for us?!)\n- For developers, there is a JQuery plugin that makes it easy to create an open source mapping component on any web page.\n\n - I want to show you a quick demo of us actually using the tool with our aggregated data so you can see how easy it is to use. \n
  • \n- As you can see the tool is very easy to use. \n- And I like that this map made it very clear which were the high percentage women areas. \nNortheast US and Zambia! AkiraChix - Nairobi\n\n- And so far, it is the dream tool BUT then we realized some limitations:\n- Once you create your heatmap and left the edit window, \nyou can not go back and change the parameters (there is no logging) after its creation\nAlso since we used a csv to upload the data - after uploading, we can’t edit the raw data.\n You can create a map directly from data in a google spreadsheet - and your map will then get updates periodically automatically\n\n- As we saw, the generated URL is predefined with some arbitrary name: http://www.openheatmap.com/view.html?map=JamieTriglyceridesMournings\n\n- The solution which gives you more flexibility, is to\n host the heatmap on your site using OpenHeatmap API, however, after couple of hours, \n we tried to use API, but lots of restrictions (need to geocode the data, need to be familiar with OSM) and documentation is really not clear.\n \n - Pete - the author is very responsive to github messages though. \n\n \n
  • We realized at this point the tools could be broken up into 2 paths: non programmatic and programatic. \nSo we thought we would look at several tools of each type. \n\n
  • For nonprogramatic we decided to look at: \nopenheatmap - which you just saw\ngoogle fusion tables\ngoogle charts\nsocrata\nchartsbin\nfactual\n\nChartsbin was not a good option because it doesn’t allow us to upload our own data. \nFactual - not a good option because doesn’t provide a map view of data\n
  • For nonprogramatic we decided to look at: \nopenheatmap - which you just saw\ngoogle fusion tables\ngoogle charts\nsocrata\nchartsbin\nfactual\n\nChartsbin was not a good option because it doesn’t allow us to upload our own data. \nFactual - not a good option because doesn’t provide a map view of data\n
  • For nonprogramatic we decided to look at: \nopenheatmap - which you just saw\ngoogle fusion tables\ngoogle charts\nsocrata\nchartsbin\nfactual\n\nChartsbin was not a good option because it doesn’t allow us to upload our own data. \nFactual - not a good option because doesn’t provide a map view of data\n
  • For programmatic - we decided to look at: \ngoogle maps js api\nyahoo maps web services\nbing maps ajax control\ngoogle charts api\n
  • \n
  • First - I want to do a demo so you can see how powerful Google Fusion Tables is. \n
  • Here again is the chart where we are using the aggregated counts to show number of people attending in each country. \n \n
  • In Google Charts, there is a heatmap gadget you can use to plot data. \nThis is a heatmap of the number of women aggregated by country. You can access this functionality directly from google spreadsheets.\nThere was a little bit of difficulty plotting the data.\n- You need ISO country code and it had to be done manually\n- Aggregation counts also had to be be done manually \n- For non programatic tools for heatmaps - fusion tables is more powerful than google charts\n\n
  • Socrata is a company working on Open Data Services for Government.\n It provides a platform to help you share data with others. \nUsed by city of Seattle for example, used by data.gov for examples.\n\n\n
  • They also have a way to upload your own data and visualize them using various Maps API\nIt is less trivial than openheatmap and it seems that socrata’s main usecase is to quickly allow visualization of exisiting data on the platform. \n\n- You can choose between Google/Bing and eSRI maps (specialized GIS software).\n- In 3 minutes, we were able to create an account and upload a csv to plot heatmap per country. \n- However you are required to Geocode your points if you’re interested in a location other than country or a US state.\n- Slow to render\n- Great way to visualize existing sets, not perfect tool for just a one time use.\n\n
  • We then moved on to the google maps api\nWe created a maps key to use in our application.\nWe found it pretty quickly that we could easily plot individual markers of data using aggregated data set\n\n\n
  • - With the Google Maps API it was very easy to take our summary data points \nand plot them on a map and include number of people and % of women in the  info window.\n\n\n
  • Initialize the map\n
  • Initialize the map\n
  • Initialize the map\n
  • Initialize the map\n
  • Data\nThe locations data was a simple array with the columns being \nlocation, latitude, longitude, number of people, percentage\n
  • Data\nThe locations data was a simple array with the columns being \nlocation, latitude, longitude, number of people, percentage\n
  • Plot the map\n
  • Plot the map\n
  • Plot the map\n
  • Plot the map\n
  • Plot the map\n
  • Plot the map\n
  • Google Maps API - add InfoWindow\n
  • Google Maps API - add InfoWindow\n
  • Google Maps API - add InfoWindow\n
  • Google Maps API - add InfoWindow\n
  • Google Maps API - add InfoWindow\n
  • Google Maps API - add InfoWindow\n
  • Then you get this map we looked at before.\n\nThe data points still don't make it very clear from one view the number and percentage. I can click on a point, \nSo we kept looking and we came upon Marker Clusterer.\n
  • Marker is an extension to the google maps api written by Frank Wu. \nThis is the plot of the number of women that attended each location using\nthe extension. \n\nIf I know that over 1300 people attended  the hacks I can quickly see with this map that women make up a small part of the total attendees. \nLove the graphics, but this chart still doesn't show me the ratio of men to women, so it's difficult to grasp the disparity. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • - To use the marker clusterer - just had to include one file\n- Data had to be one row per person though\n- The marker clusterer would cluster the data for you. \n
  • \n- Marker clusterer\n- No way for tool use different data fields (e.g. male, female, percentages ) to plot different colors or differentiate between data sets.\n
  • \n- Marker clusterer\n- No way for tool use different data fields (e.g. male, female, percentages ) to plot different colors or differentiate between data sets.\n
  • And as we saw, this was the result. The clusterer uses both the size of the marker and the color to plot\n\n
  • One nice feature of the marker clusterer is the drill down capability it provides\nAutomatically clusters based on your zoom level\n
  • We feel like we're making some progress, but there is still not clear visualization which shows the number of people who attended vs the percentage of women. \n\nSo we continued exploring. \nWe came across layers in the google maps api and found a layer for fusion table heatmaps. \nWe saw the possibility of integrating our fusion tables with the google maps api using the FusionTablesLayer class\n\nWe thought - oh cool - we can use the fusion tables we already have, and create a heat map with the google maps api and plot markers on top of it. \nThis is an Example Google Maps API with Fusion Table Heatmaps and what it is doing is plotting a single point for each location. \n\n- With our aggregated data\n- Integration of fusion table and google maps API to generate heat map doesn't fit our use case\n- Possible to do with individual data\n\n- And we were wondering why do the fusion table heatmaps look for different from the heatmap option \nMade use realize that fusion table heat maps were done with Flash. \n\n- It wasn’t the same thing like we thought. \n\n\nhttp://code.google.com/apis/maps/documentation/javascript/examples/layer-fusiontables-heatmap.html\n
  • - Here's how you do it \n- Valuable if you want to use your data directly from fusion table\n- Fusion table must be public \n- Heatmap option most likely  not so valuable with aggregated data\n
  • - Here's how you do it \n- Valuable if you want to use your data directly from fusion table\n- Fusion table must be public \n- Heatmap option most likely  not so valuable with aggregated data\n
  • - We then came across the Circle class in the Google Maps API\n- We started with an initial basic map with plots of all the cities\n- We used the same data set that we had in the basic map\n- We first plotted circles where the size was based on attendance\n
  • Creating circles was easy. \n
  • Creating circles was easy. \n
  • \n
  • \n
  • circle plot\n- again different sizes based on attendance, but now \n- circle color based on percentage\n- easy to view which places had a lot of people and also to see gauge the relative percentage of women \n
  • Add back in call to setMarkers and InfoWindow. \n- setMarkers is where the event listener is added to each marker. \n
  • Add back in call to setMarkers and InfoWindow. \n- setMarkers is where the event listener is added to each marker. \n
  • circle plot\ncircles automatically scale to your zoom level\n
  • \n
  • This is the first map we created with the Yahoo Maps API.\nWe had explored the Google Maps API extensively and we wanted to see what Yahoo and Bing had to offer. \nIt turns out that the APIs are very similar in structure to each other. The function names are different, but the structure is the same. \nCreate a map, plot the markers, add event listeners for the click to get the info windows. \nOne thing I didn’t like about the Yahoo! Maps was the default marker, so I went ahead and used the same icon that the google maps api provides by default.\n\n\n
  • \n
  • Create new icon, set the source, and size, and make sure you pass the icon to the marker \nand voila - \n
  • Create new icon, set the source, and size, and make sure you pass the icon to the marker \nand voila - \n
  • your map uses your custom icon.\nWhen we looked at the Bing Maps AJAX Control - it turns out that their API is also very similar in structure. \n\n
  • StreetView\n\n- One advantage that google maps does have is the streetview. \n\n- Streetview is available by default when you create a map using the Google Maps API\n\n- This might be useful in the next RHoK (coming up in June 2nd, 3rd) to use before the event so people can find the location of the hacks. \n
  • The Google Charts Tools \nYou can use the Google Charts Tools / Visualization API to embed an interactive chart, graph, or other graphic onto your web page. Visualizations are interactive, and also expose events, such as user mouse clicks, that enable you to write code to create great effects on your page—for example, to combine a map and a table that stay in sync when clicked.\n\nYou can embed a visualization either using JavaScript, or as an embedded gadget (some visualizations are provided as both).\n\nhttp://code.google.com/apis/visualization/documentation/\n
  • If you use JS, there are still the same limitations we encountered when building the gadged. You need ISO country code and it had to be done manually and Aggregation counts also had to be be done manually. Can’t plot cities.\nBut Google Charts are a great way to plot multiple charts synchronized with each other on the same page. It also offers a large variety of \nvisualization options - table, bar, line, and some interesting custom ones - I saw a speedometer in their example. They are constantly improving it also so it’s\ndefinitely something to look at. \n
  • In conclusion - if we take a look at all the tools together and their capabilities.\n\n- 2D visualization\ngoogle maps API\n\n- No Coding required\n- Legend?\n- First Map (Easy | Moderate | Hard)\n- Full control\n- Data points (individual, aggregate)\n\n- When you’re exploring - the non programmatic tools such as google fusion tables might be what you want.\n- When you know what you want - google maps api, yahoo maps web services, google charts api - something that gives you more control\n\nNotes/ Special Features\nScaling\nClustering based on zoom level\nFusion Tables\nCountries vs. Regions\nMap not zoomable\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Some more conclusions: \n \n - Very easy to plot flat numbers with most tools\n - Plotting multiple sets of data without code is not trivial\n - Flash graphics still ahead of JS graphics with respect to data visualization\n - Most map and charting tools need better overall legend support\n - Almost all tools provide an easy way to have individual data point information windows, but support for overall legend is missing\n - We've uncovered just the tip of the iceberg in terms of data visualization\n - A lot of tools out there\n - Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge\n - exception: Argentina - 120 people, only 9% women\n - We need to improve and automate our data collection process as much as possible\n - Accurate registration via eventbrite\n - Import dataset with gender and location directly from eventbrite\n - Even though we hit >= 20% in 8 (1/3) locations, we're not sure in what manner people participated in. \n - Our goal is to get 20% women hackers, so we should also make sure to collect what role people are playing at RHoK\n - We need more women hackers\n - When women do participate, they win!\n\n\n\n
  • Questions?\n
  • Please come participate!\n
  • Contact us with any questions. \n
  • \n
  • \n

How Quick Can We Be? Data Visualization Techniques for Engineers. How Quick Can We Be? Data Visualization Techniques for Engineers. Presentation Transcript

  • How quick can we be?(data visualization techniques for engineers)
  • Who we are
  • Who we areAlice Bonhomme-Biais
  • Who we areAlice Bonhomme-Biais Avni Khatri
  • DATA
  • OUR PROCESS
  • Goals
  • Goals1. Plot the number of people at each location
  • Goals1. Plot the number of people at each location2. Plot the percentage of women at each location
  • Goals1. Plot the number of people at each location2. Plot the percentage of women at each location3. Plot two dimensions
  • Goals1. Plot the number of people at each location2. Plot the percentage of women at each location3. Plot two dimensions4. Clearly show the gender disparity
  • Goals1. Plot the number of people at each location2. Plot the percentage of women at each location3. Plot two dimensions4. Clearly show the gender disparity5. Use the tool to visualize data for future RHoKs
  • OPENHEATMAP
  • OpenHeatMap Demo
  • TOOLS
  • Non-programmatic tools
  • Non-programmatic tools• OpenHeatMap• Google Fusion Tables• Google Charts• Socrata• ChartsBin• Factual
  • Non-programmatic tools• OpenHeatMap• Google Fusion Tables• Google Charts• Socrata• ChartsBin• Factual
  • Non-programmatic tools• OpenHeatMap• Google Fusion Tables• Google Charts• Socrata• ChartsBin• Factual
  • Programatic tools
  • Programatic tools• Google Maps JS API• Yahoo! Maps Web Services• Bing Maps AJAX Control• Google Charts Tools
  • GOOGLE FUSION TABLES
  • Fusion Tables Demo
  • SOCRATA
  • GOOGLE MAPS API
  • function initialize() { var latlng = new google.maps.LatLng(41.850033, -87.6500523); var myOptions = { zoom: 3, center: latlng, mapTypeId: google.maps.MapTypeId.ROADMAP};var map = new google.maps.Map( document.getElementById("map_canvas"), myOptions);}
  • function initialize() { var latlng = new google.maps.LatLng(41.850033, -87.6500523); var myOptions = { zoom: 3, center: latlng, mapTypeId: google.maps.MapTypeId.ROADMAP};var map = new google.maps.Map( document.getElementById("map_canvas"), myOptions);}
  • function initialize() { var latlng = new google.maps.LatLng(41.850033, -87.6500523); var myOptions = { zoom: 3, center: latlng, mapTypeId: google.maps.MapTypeId.ROADMAP};var map = new google.maps.Map( document.getElementById("map_canvas"), myOptions);}
  • var locations = [ ["Aarhus, Denmark",56.162939,10.203921,10,20], ["Bangalore, India",12.971599,77.594563,217,11.98], ["Chicago, Illinois",41.878114,-87.629798,107,31.78], ["Nairobi, Kenya",1.283333,36.816667,33,6.06].......];
  • var locations = [ ["Aarhus, Denmark",56.162939,10.203921,10,20], ["Bangalore, India",12.971599,77.594563,217,11.98], ["Chicago, Illinois",41.878114,-87.629798,107,31.78], ["Nairobi, Kenya",1.283333,36.816667,33,6.06].......];
  • function setMarkers(map, markers) { ... for (var i = 0; i < markers.length; i++) { markerLatLng = new google.maps.LatLng(markers[i][1],markers[i][2]); marker = new google.maps.Marker({ position: markerLatLng, map: map, title: markers[i][0], zIndex: 1 ); }}// add call to setMarkers in initializefunction initialize() { ... setMarkers (map, locations);}
  • function setMarkers(map, markers) { ... for (var i = 0; i < markers.length; i++) { markerLatLng = new google.maps.LatLng(markers[i][1],markers[i][2]); marker = new google.maps.Marker({ position: markerLatLng, map: map, title: markers[i][0], zIndex: 1 ); }}// add call to setMarkers in initializefunction initialize() { ... setMarkers (map, locations);}
  • function setMarkers(map, markers) { ... for (var i = 0; i < markers.length; i++) { markerLatLng = new google.maps.LatLng(markers[i][1],markers[i][2]); marker = new google.maps.Marker({ position: markerLatLng, map: map, title: markers[i][0], zIndex: 1 ); }}// add call to setMarkers in initializefunction initialize() { ... setMarkers (map, locations);}
  • function setMarkers(map, markers) { for (var i = 0; i < markers.length; i++) { ... iwContent = "<span class=”label”>" + markers[i][0] + "</span><br/>" + "Number of People: " + markers[i][3] + "<br/>" + "% of Women: " + markers[i][4]; marker = new google.maps.Marker({ ... html: iwContent }); google.maps.event.addListener(marker, "click", function() { infowindow.setContent(this.html); infowindow.open(map, this); }); }}function initialize() { ... infowindow = new google.maps.InfoWindow({ content: "loading..." });}
  • function setMarkers(map, markers) { for (var i = 0; i < markers.length; i++) { ... iwContent = "<span class=”label”>" + markers[i][0] + "</span><br/>" + "Number of People: " + markers[i][3] + "<br/>" + "% of Women: " + markers[i][4]; marker = new google.maps.Marker({ ... html: iwContent }); google.maps.event.addListener(marker, "click", function() { infowindow.setContent(this.html); infowindow.open(map, this); }); }}function initialize() { ... infowindow = new google.maps.InfoWindow({ content: "loading..." });}
  • function setMarkers(map, markers) { for (var i = 0; i < markers.length; i++) { ... iwContent = "<span class=”label”>" + markers[i][0] + "</span><br/>" + "Number of People: " + markers[i][3] + "<br/>" + "% of Women: " + markers[i][4]; marker = new google.maps.Marker({ ... html: iwContent }); google.maps.event.addListener(marker, "click", function() { infowindow.setContent(this.html); infowindow.open(map, this); }); }}function initialize() { ... infowindow = new google.maps.InfoWindow({ content: "loading..." });}
  • <script src="http://gmaps-utility-library.googlecode.com/svn/trunk/markerclusterer/1.0/src/markerclusterer.js"></script>var data = {"count": 232,"participants": [ { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }, { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }......]};
  • <script src="http://gmaps-utility-library.googlecode.com/svn/trunk/markerclusterer/1.0/src/markerclusterer.js"></script>var data = {"count": 232,"participants": [ { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }, { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }......]};
  • <script src="http://gmaps-utility-library.googlecode.com/svn/trunk/markerclusterer/1.0/src/markerclusterer.js"></script>var data = {"count": 232,"participants": [ { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }, { "location": "Aarhus, Denmark", "gender": "Female", "latitude": "56.162939", "longitude": "10.203921" }......]};
  • var markers = [];for (var i = 0; i < data.participants.length; ++i) { var latlng = new GLatLng(data.participants[i].latitude, data.participants[i].longitude); var marker = new GMarker(latlng, {icon: icon}); markers.push(marker);}var markerCluster = new MarkerClusterer(map, markers);
  • var markers = [];for (var i = 0; i < data.participants.length; ++i) { var latlng = new GLatLng(data.participants[i].latitude, data.participants[i].longitude); var marker = new GMarker(latlng, {icon: icon}); markers.push(marker);}var markerCluster = new MarkerClusterer(map, markers);
  • var layer = new google.maps.FusionTablesLayer(747060, { heatmap: true });layer.setMap(map);
  • var layer = new google.maps.FusionTablesLayer(747060, { heatmap: true });layer.setMap(map);
  • function plotCircles(map, dataPoints) { ... for (var i = 0; i < dataPoints.length; i++) { pointLatLng = new google.maps.LatLng(dataPoints[i][1],dataPoints[i][2]); circleOptions = { strokeColor: "#FF0000", strokeOpacity: 0.8, strokeWeight: 2, fillColor: “#FF0000”, fillOpacity: 0.40, map: map, center: pointLatLng, radius: dataPoints[i][3]*3000 }; circle = new google.maps.Circle(circleOptions); }
  • function plotCircles(map, dataPoints) { ... for (var i = 0; i < dataPoints.length; i++) { pointLatLng = new google.maps.LatLng(dataPoints[i][1],dataPoints[i][2]); circleOptions = { strokeColor: "#FF0000", strokeOpacity: 0.8, strokeWeight: 2, fillColor: “#FF0000”, fillOpacity: 0.40, map: map, center: pointLatLng, radius: dataPoints[i][3]*3000 }; circle = new google.maps.Circle(circleOptions); }
  • ...percentage = dataPoints[i][4];if (percentage <= 10) { color = "#FF0000";} else if (percentage >=20) { color = "#339900";} else { color = "#FFCC00";}circleOptions = { strokeColor: color,....
  • ...percentage = dataPoints[i][4];if (percentage <= 10) { color = "#FF0000";} else if (percentage >=20) { color = "#339900";} else { color = "#FFCC00";}circleOptions = { strokeColor: color,....
  • ...setMarkers (map, locations);infowindow = new google.maps.InfoWindow({ content: "loading..." });....
  • ...setMarkers (map, locations);infowindow = new google.maps.InfoWindow({ content: "loading..." });....
  • YAHOO! MAPS WEB SERVICES
  • ....icon = new YImage();icon.src = "marker_34_red.png";icon.size = new YSize(20,34);icon.offsetSmartWindow = new YCoordPoint(9,0);....var marker = new YMarker(point, icon);....
  • ....icon = new YImage();icon.src = "marker_34_red.png";icon.size = new YSize(20,34);icon.offsetSmartWindow = new YCoordPoint(9,0);....var marker = new YMarker(point, icon);....
  • GOOGLE CHARTS TOOLS
  • Conclusions Visualize 2 No Coding Legend First Map Full Control sets of data OpenHeatMap Easy Google Fusion Easy Tables Google Charts Moderate Socrata Moderate Google Maps API Moderate Yahoo Maps Moderate Web Services Google Charts Moderate Tools
  • Conclusions
  • Conclusions• Very easy to plot flat numbers with most tools
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure• Weve uncovered just the tip of the iceberg in terms of data visualization
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure• Weve uncovered just the tip of the iceberg in terms of data visualization• Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure• Weve uncovered just the tip of the iceberg in terms of data visualization• Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge• We need to improve and automate our data collection process as much as possible
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure• Weve uncovered just the tip of the iceberg in terms of data visualization• Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge• We need to improve and automate our data collection process as much as possible• We need more women hackers
  • Conclusions• Very easy to plot flat numbers with most tools• Plotting multiple sets of data without code is not trivial• Flash graphics still ahead of JS graphics with respect to data visualization• Most map and charting tools need better overall legend support• Maps API products very similar in structure• Weve uncovered just the tip of the iceberg in terms of data visualization• Our visualized data showed us that the more the people that attended the event, the more likely we were to meet the 20% challenge• We need to improve and automate our data collection process as much as possible• We need more women hackers• When women do participate, they win!
  • RHoK #3 on June 4th and 5th SIGN UP NOW!
  • Contact usaliceb@google.com avni@avni.net @avni321
  • Resources and further• http://msdn.microsoft.com/en-us/library/bb429619.aspx• http://code.google.com/apis/chart/• http://www.google.com/fusiontables/public/tour/index.html• http://code.google.com/apis/maps/documentation/javascript/• http://developer.yahoo.com/maps/• http://www.openheatmap.com/• http://googlegeodevelopers.blogspot.com/2009/04/markerclusterer-solution-to-too- many.html• http://opendata.socrata.com/
  • Photo Credits • http://www.flickr.com/photos/compassalba/5361528668/sizes/o/in/photostream/ • http://www.flickr.com/photos/rhok_chicago/5244434726/in/ set-72157625557470340 • http://www.flickr.com/photos/rhok_chicago/5233673064/in/ set-72157625529538216  • http://www.flickr.com/photos/10785432@N03/4123252604/ • http://www.flickr.com/photos/thehbunny/5055948378/ • http://www.flickr.com/photos/53941041@N00/4977883190/ • http://www.flickr.com/photos/rhok_chicago/5243866521/ • http://www.flickr.com/photos/rhok_chicago/5244461658/in/ set-72157625557470340 • http://www.flickr.com/photos/rhok_chicago/5233687260/in/ set-72157625529538216 • http://www.flickr.com/photos/ladypain/1950149100/in/photostream/ • http://www.flickr.com/photos/10687935@N04/3055314845/ • http://www.flickr.com/photos/42788859@N00/318947873/