Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learn D3.js in 90 minutes

2,263 views

Published on

Presentation / Workshop which will teach you the core patterns, concepts and visualisation options of D3.js (v4). Accompanying exercises can be found here: https://github.com/josdirksen/d3exercises

Published in: Data & Analytics
  • Be the first to comment

Learn D3.js in 90 minutes

  1. 1. LEARNING D3.JS IN 90 MINUTES
  2. 2. THIS SESSION ~ WHAT ARE WE GOING TO DO ~
  3. 3. BEFORE WE START Get the code from: You need a webserver for some of the examples: IntelliJ supports this (right click on HTML and run) Easiest is with python: check if you can run: https://github.com/josdirksen/d3exercises python -m SimpleHTTPServer (for Python 2) python3 -m http.server (for Python 3) If not download the mongoose webserver, and you have
  4. 4. WHO AM I Love writing, and talking about technology (frontend as well as backend). Writing about stuff for Packt and Manning Twitter: Github: Contact me: Blog at: @josdirksen http://www.github.com/josdirksen jos.dirksen@gmail.com www.smartjava.org Plug: Expert Data Visualization with D3.js (Q1 2017)
  5. 5. WHAT ARE WE GOING TO DO I'll introduce a subject (couple of minutes) I'll Explain the exercise You can fill in the blanks in the provided code. // Step 2. We can also select multiple elements. Use d3.select and // d3.selectAll to see the difference when selecting all the // <li> elements, and use either classed or attr to set the class // of these elements to 'step2'. // d3.selectAll(...).classed(..., ...) Halfway during the exercise I'll push an answer to git and Don't hesitate to ask questions!
  6. 6. WE'RE GOING TO TRY AND COVER... Learn how D3.js binds data to DOM elements. Make basic charts and diagrams Visualize hierarchical data Show connected graphs Combine gis/geo information with opendata Make art with voronois
  7. 7. WHAT AREN'T WE GOING TO COVER D3.js is big, very big. We'll explore the basics for data visualization, but there is a whole lot more. Most important stuff we skip: User interaction, mouse events, drag events. Animations Zooming & dragging Streaming data loading and queue data Typescript, ES6, Scale.js bindings ...
  8. 8. D3.JS INTRODUCTION
  9. 9. WHAT IS D3.JS "D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using . D3's emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful and a to DOM manipulation." --- d3js.org HTML, SVG, and CSS visualization components data-driven approach
  10. 10. WHAT IS IT NOT Not a charting library: But provides the building blocks (see nvd3.js, Rickshaw.js, c3.js). Not a drawing library: Doesn't draw data itself, uses SVG for visualization. Not an SVG abstraction layer: Manipulates the DOM and SVG directly. Not a Canvas/WebGL library: Minimal support for Canvas, for WebGL use Three.js. Not a framework: Can be used with Angular 1/2, React, Vue.js, , is just a (very extensive) library.
  11. 11. ONLINE INFORMATION Lots of online resources, Most for v3. current version is v4. API: Bit overwhelming.. But won't be in about 80 minutes https://github.com/d3/d3/blob/master/API.md
  12. 12. WORK BY EXAMPLE Usually easier to work from examples Best starting point: Most examples still use v3 however.. https://bl.ocks.org/
  13. 13. HELLOWORLD OF D3.JS
  14. 14. HELLOWORLD OF D3.JS Goal of this first example, is to learn how you can select elements with D3.js, append new ones, remove elements, and set basic properties on the elements.
  15. 15. SELECTIONS d3.select and d3.selectAll Uses the W3C selection syntax: uses querySelector and querySelectorAll Examples: d3.selectAll('p') // Select all p elements in the document. d3.select('p') // Select first p element in the document. // Selects 1st span element of all <p> in the document d3.selectAll('p').select('span') d3.select('#id') // select on id d3.select('.classname') // select on css class
  16. 16. WE CAN CHANGE THE PROPERTIES OFF THESE SELECTIONS attr(), style(), html(), text(), classed() Value can also be a function: d3.selectAll('img.thumb').attr('height', 50) d3.selectAll('span').style('color', 'steelblue') d3.selectAll('span').html('<strong>d3.js</strong>') d3.selectAll('span').text('content') // conditionally set or remove classes d3.select('div').classed('container', true) // d is bound data, i is the index d3.selectAll('img').attr('src', function(d, i) { return 'image' + i + '.png'; });
  17. 17. MODIFY THE SELECTION append(): Append child to each element of selection remove(): Remove all the elements in the selection insert(): Insert an element before another element d3.selectAll('p').append('span'). // remove the first li of every lu d3.selectAll('ul').select('li').remove() // insert a new span as the first element of the p d3.selectAll('p').insert('span', ':first-child'); Selections are immutable, these actions return new selections, so we can chain them!
  18. 18. BUT JQUERY CAN DO... With you manipulate elements from a selection, with you bind data (next example) has many data visualization utils and APIs and focusses on web app related plugins. But there is a big overlap both are libraries have and use a . Similar to D3.js: Raphael.js and Processing.js jQuery D3.js D3.js jQuery DOM manipulation CSS/W3C selectors fluent API
  19. 19. LET'S GET STARTED $ git clone https://github.com/josdirksen/d3-exercises $ python -m SimpleHTTPServer Once again, if you don't have python, you can use mongoose: View at http://localhost:8000/src/exercise-01/E01.html
  20. 20. EXERCISE 1: LEARNING D3.JS Follow steps described in: <ROOT>/src/exercise-01/js/E01.js Use d3.select(), d3.selectAll(), class(), attr(), append(), remove(), html(), text() If needed reference D3.js select API: You'll get approximately 5 minutes. https://github.com/d3/d3-selection Should you have question? Please ask.
  21. 21. USE DATABINDING WITH D3.JS
  22. 22. D3.JS DATA BINDING Learn how to create data driven document by usin the basic select(), enter(), merge(), remove() cycle when binding data to a selection
  23. 23. BASIC DATABINDING Bind data to a selection with data(...) <!-- Input Dom --> <span></span> <span></span> <span></span> <span></span> var values=['a', 'b', 'c']; d3.selectAll('span') .data(values) .text(function(d) {return d}); <!-- Output Dom --> <span>a</span> <span>b</span> <span>c</span> <span></span>
  24. 24. DATA DRIVEN ELEMENTS Assume an empty DOM var values=['a', 'b', 'c']; var selection = d3.selectAll('span').data(values) A selection is actually more than just the found elements enter(): placeholder selection for unbound data // for all unbound data append a span attribute selection.enter().append('span').text(function(d) {return d}); exit(): placeholder selection for le over elements // remove all the leftover elements selection.exit().remove()
  25. 25. LEADS TO THIS PATTERN function update(values) { // select all the current span elements and assign the data var selection = d3.selectAll('span').data(values) // for the unbound data, create a new span, and attach a class var newElements = selection.enter() .append('span') .attr('class', 'someClass'); // Merge combines two selections. For the combined selection // set the value of the span to the bound datum. newElements.merge(selection).text(function(d) {return d}); // and remove the now obsolete elements selection.exit().remove(); } This is the basic pattern you use to bind data, add new DOM elements, updated existing ones, and remove obsolete ones.
  26. 26. WHICH PROVIDES US WITH An easy way to let our data determine our elements. Reduces unnecessary creation of elements. Allow all kinds of hooks e.g. for animations. // assume values contains some random numbers function update(values) { var selection = d3.selectAll('circle').data(values) var newElements = selection.enter() .append('circle') .attr('class', 'someClass') .attr('cx', function(d, i) {return i * 50}) .attr('cy', function() {return 50}) .merge(selection) .transition().duration(2000) .attr('r', function(d, i) {return d}); // and remove the now obsolete elements selection.exit().remove(); }
  27. 27. EXERCISE 2: GOAL
  28. 28. EXERCISE 2: DATABINDING Follow steps described in: <ROOT>/src/exercise-02/js/E02.js Use (same as previous) and enter(), exit(), append(), remove() If needed reference D3.js select API: If time permits look at transistions API: You'll get approximately 5 minutes. https://github.com/d3/d3-selection https://github.com/d3/d3-selection Should you have question? Please ask.
  29. 29. BASIC CHARTS AND DIAGRAMS
  30. 30. BASIC CHARTS AND ELEMENTS Learn how to create basic charts and elements using scales and generators.
  31. 31. SVG, GROUPS AND POSITIONING Absolute positioning: <rect x="100", y="100"></rect> <circle cx="100", cy="100"></circle> Provides a g element for grouping g doesn't have an x or y like attribute, uses transform All attributes on this element apply on children <g transform="translate(50 200)">...</g> <!-- positioning --> <g transform="scale(0.5 0.3)">...</g> <!-- sizing --> <g transform="rotate(45)">...</g> <!-- rotate the g --> These actions can be combined But positioning is still difficult
  32. 32. POSITIONING WITH SCALES A scale translates an input domain to an output range Different scales ( ) : continuous input to continuous output scalePow, scaleLog, scaleTime : continuous input to interpolator scaleQuantize: continuous input to discrete range scaleQuantile: sampled input to discrete range scaleThreshold: scaleQuantize with thresholds scaleOrdinal: discrete input to discrete range , scalePoint Lots of useful helper functions https://github.com/d3/d3-scale scaleLinear scaleSequential scaleBand
  33. 33. SCALELINEAR: CONTINUOUS INPUT TO CONTINUOUS OUTPUT var x = d3.scaleLinear() .domain([10, 130]) // e.g the min and max of your data .range([0, 960]); // the width of your chart x(20); // 80 x(50); // 320 var color = d3.scaleLinear() .domain([10, 100]) .range(["brown", "steelblue"]); color(20); // "#9a3439" color(50); // "#7b5167"
  34. 34. WHAT CAN WE INTERPOLATE? The domain must be numbers, the range can be any of this: And if this doesn't match, you can create your own.
  35. 35. SCALESEQUENTIAL: CONTINUOUS INPUT TO INTERPOLATOR var color = d3.scaleSequential(d3.interpolatePlasma).domain([0, 100]); color(30); // #8f0da4 color(60); // #e16462 color(80); // #fca636 Many interpolators available:
  36. 36. HOW WOULD YOU USE THIS? // say our values contains data from 1984 to 2014 var min = d3.min(values, function (d) { return d.value; }); var max = d3.max(values, function (d) { return d.value; }); // evenly divide the years along the xAxis var xScale = d3.scaleLinear().domain([1984,2014]).range([0, 1080]); // evenly divide the values along the yAxis var yScale = d3.scaleLinear() .domain([min, max]).range([0, 700]); // get a color var col = d3.scaleSequential(d3.interpolatePlasma).domain([0, 100]); d3.selectAll("rect").data(values).enter() .append("rect") .attr("x", xScale) // xScale = (d: any) => Numeric .attr("y", function(d) {return yScale(d)}) // alternatively .attr("fill", col)
  37. 37. SCALES AND AXIS D3 provides easy way to create an axis: https://github.com/d3/d3-axis var min = d3.min(values, function (d) { return d.value; }); var max = d3.max(values, function (d) { return d.value; }); var yScale = d3.scaleLinear() .domain([min, max]).range([0, 700]); // s denotes that we want to use international system of units // to display axis values var bottomAxis = d3.axisBottom().scale(yScale).ticks(20, "s"); Results in:
  38. 38. EXERCISE 3: GOAL
  39. 39. BEFORE WE START D3.js support easy loading csv, tsv, json, xml name,sex,amount Emma,F,20355 Olivia,F,19553 ... d3.csv('data/yob2015.txt', function (d) { return { name: d.name, sex: d.sex, amount: +d.amount }; }, function (data) { }
  40. 40. ALSO FOR MULTIPLE FILES d3.queue() .defer(d3.json, "./data/world-110m.v1.json") .defer(d3.csv, "./data/worldbank_popular_2014.csv") .defer(d3.csv, "./data/iso-mapping.csv") .await(function (error, topoData, worldbank, mapping) { }
  41. 41. EXERCISE 3: WORKING WITH SCALES Follow steps described in: <ROOT>/src/exercise-03/js/E03.js If needed reference D3.js APIs: You'll get approximately 8 minutes. https://github.com/d3/d3-scale https://github.com/d3/d3-axis Should you have question? Please ask.
  42. 42. USING SVG FOR CHARTS SVG provides a large number of elements: But limited primitive shapes: <circle>, <ellipse>, <line>, <path>, <polygon>, <polyline>, <rect> https://developer.mozilla.org/en-US/docs/Web/SVG
  43. 43. THE PATH ELEMENT IS VERSATILE the d attribute describes a path: <path class="arc" d="M136.86141570725613,-17.69047457265137A138,138, 0,0,1,124.60150267951192,59.31665474390461L62.948628653284814,28. 257214134993035A69,69,0,0,0,68.61145448511735,-7.312203049469534Z" style="fill: rgb(252, 160, 130);"></path> M, L, A, C, A, Q, T ... for lines, arcs, curves Hard to determine the correct values yourself D3.js provides generators for complex shapes
  44. 44. THE ARC GENERATOR We'll create an arc segment, a part of a pie chart var arc = d3.arc() .outerRadius(height/2 * 0.6).innerRadius(height/2 * 0.3); // create the right half of a pie chart arc({ startAngle: 0, endAngle: Math.PI }); // "M1.469576158976824e-14,-240A240,240,0,1,1,1.469576158976824e-14, // 240L7.34788079488412e-15,120A120,120,0,1,0,7.34788079488412e-15, // -120Z"
  45. 45. COMBINED WITH THE PIE FUNCTION Piechart: d3.pie() to generate config for d3.arc var arc = d3.arc() .outerRadius(height/2 * 0.6).innerRadius(height/2 * 0.3); var data = [{count:10}, {count:20}, {count:30}] var pie = d3.pie() .padAngle(0.04) .value(function (d) { return d.count; }); var arcs = pie(data) // "[{"data":{"count":10},"index":2,"value":10, // "startAngle":5.215987755982988, // "endAngle":6.283185307179586,"padAngle":0.04}, // {"data":{"count":20},"index":1,"value":20, // "startAngle":3.121592653589793, // "endAngle":5.215987755982988,"padAngle":0.04} ... selectAll("path").data(arcs).enter() .append("path").attr("d", arc);
  46. 46. ANOTHER STANDARD PATTERN 1. Define generator which creates paths based on properties. (d3.arc) 2. Define generator which creates config for other generators (d3.pie) 3. Pass data to step 2, result is enriched data with config. 4. Use the normal selectAll(), enter(), merge(), exit() pattern
  47. 47. EXERCISE 4: GOAL
  48. 48. EXERCISE 4, CREATE A PIE CHART Follow steps described in: <ROOT>/src/exercise-04/js/E04.js If needed reference D3.js APIs: You'll get approximately 5 minutes. https://github.com/d3/d3-shape
  49. 49. VISUALIZING TREE STRUCTURES
  50. 50. TREE AND HIERARCHIES See what is possible in d3 to visualizes nested trees of data
  51. 51. D3.JS SUPPORTS TREE DATA API: Many standard visualizations: d3.tree() https://github.com/d3/d3-hierarchy
  52. 52. D3.CLUSTER() Same as the tree, but leaves are at the same position
  53. 53. D3.TREEMAP() Supports different clustering algorithms
  54. 54. D3.PACK() Same as d3.treemap, but with circles
  55. 55. OR YOUR CUSTOM IMPLEMENTATION Sample
  56. 56. ALL FOLLOW SAME APPROACH 1. Load the data (d3.csv,d3.tsv,d3.json etc.) 2. Convert the data into a tree stucture 3. Pass it through a generator 4. Use the output from the generator to draw the chart
  57. 57. NESTED DATA: D3.STRATISFY d3.stratisfy() can follow ids in your data id,parentId,name,description 180580,,Felidae,cats 180581,552363,Lynx,lynxes 180582,180581,rufus,Bobcat 180583,180582,rufus,bobcat 180584,180581,lynx,Eurasian Lynx 180585,180581,canadensis,Canada lynx` var stratify = d3.stratify(); var root = stratify(data);
  58. 58. NESTED: D3.NEST / D3.HIERARCHY d3.nest() can group data (multiple times) based on a key "Country (en)";"Country (de)";"Country (local)";"Country code";"Continent" "Afghanistan";"Afghanistan";"Afganistan/Afqanestan";"AF";"Asia"; "Egypt";"Ãgypten";"Misr";"EG";"Africa"; var entries = d3.nest() .key(function (d) {return d.Continent; }) .entries(data); var root = d3.hierarchy({values: entries}, function(d) { return d.values; })
  59. 59. USE A GENERATOR With data in the correct structure, we can use a generator // normal node tree var tree = d3.tree() .size([height, width]) .separation(function(a, b) { return (a.parent === b.parent ? 5 : 13) }); // create a treemap var tree = d3.treemap() .size([width, height]) .padding(2) .tile(d3.treemapSquarify.ratio(1)) // enrich the root tree(root)
  60. 60. WHICH ENRICHES THE STRUCTURE root.leaves() // return all the nodes without children root.descendants() // return array of this and all descendants chart.selectAll(".node") .data(root.descendants()) .enter(..)
  61. 61. EXERCISE 5: GOAL
  62. 62. EXERCISE 5, CREATE A TREEMAP Follow steps described in: <ROOT>/src/exercise-05/js/E05.js If needed reference D3.js APIs: You'll get approximately 8 minutes. https://github.com/d3/d3-hierarchy
  63. 63. VISUALIZES GRAPHS OF DATA
  64. 64. VISUALIZE GRAPHS Quick introduction on different ways how to use the force, hord, and matrix layouts to visualize a graph of data
  65. 65. FORCE LAYOUT FOR GRAPHS API: A graph is a set of nodes. Define forces which are applied: to nodes to links Run a simulation https://github.com/d3/d3-force
  66. 66. CAN APPLY DIFFERENT FORCES forceCenter: keeps nodes in the center forceCollide: nodes have a radius, avoid collisions forceLink: push/pull nodes together based on distance forceManyBody: simulate attraction or repulsion between nodes forceX: force nodes towards specific position forceY: force nodes towards specific position
  67. 67. BASIC SETUP var graph = ... // define forces var simulation = d3.forceSimulation() .force("link", d3.forceLink().id(function(d) { return d.id; })) .force("charge", d3.forceManyBody()) .force("center", d3.forceCenter(width / 2, height / 2)); // run a simulation simulation .nodes(graph.nodes) .on("tick", ticked); // do something on each tick function ticked() {...}
  68. 68. FORCE LAYOUT MULTIPLE FORCES ~ example ~
  69. 69. ALSO USE WITHOUT LINKS ~ example ~
  70. 70. ALTERNATIVE: CHORD DIAGRAM ~ example ~
  71. 71. ALTERNATIVE: MATRIX DIAGRAM ~ example ~
  72. 72. EXERCISE 6: GOAL Play around with the different forces and see the effect.
  73. 73. EXERCISE 6, APPLY FORCES Follow steps described in: <ROOT>/src/exercise-06/js/E06.js If needed reference D3.js APIs: You'll get approximately 5 minutes. https://github.com/d3/d3-force
  74. 74. GIS AND GEO DATA
  75. 75. GEO D3.js has extensive support for working with geo data. We'll explore different ways of visualizing and interacting with geo data.
  76. 76. BEST PART OF D3.JS At least for me..
  77. 77. CAUSE IT LOOKS NICE
  78. 78. AND INFORMATIVE
  79. 79. AND COLOURFUL
  80. 80. QUICK NOTE ON PROJECTIONS Projection defines how the coordinates from the source are projected to a flat canvas. The projection:Sinu Mollweide Example
  81. 81. CREATING A MAP Going to explain how to create this: Example
  82. 82. HOW TO LOAD DATA Most common format for GIS data is ArcGIS shapefile. binary format Use QGis, Mapshaper, OGR to convert to open formats D3.js can work with: GeoJSON: A standard supported by many applications. TopoJSON: A specific optimized format for smaller files.
  83. 83. GEOJSON AND TOPOJSON FORMATS Can contain multiple geometries. Each geometry is described (usually) as a path. Can contain additional properties population, unemployment rate, etc. one file for everything Try to work with TopoJSON, since it's much smaller {"type":"Topology","objects":{"cb_2015_us_county_500k": {"type": "GeometryCollection","geometries":[{"type":"Polygon","properties": {"GEOID":"01005","NAME":"Barbour"},"id":"01005","arcs":[[0,1,2,3 ,4,5,6,-7,6,7,8,9,10,11,12,13,14 ...
  84. 84. WORKING WITH TOPO DATA IN D3.JS (pretty much the same way as we did before) 1. Setup a path generator and projection 2. Load the data 3. Use the projection to generate path segments
  85. 85. LOAD THE DATA (this is prepared data, where value contains percentage of internet users) {"type":"Topology","objects":{"countries":{"type":"GeometryCollection" "geometries":[{"type":"Polygon","id":"004","arcs":[[0,1,2,3,4,5]], "properties":{"value":"7","countryA":"AFG","name":"Afghanistan"}}, {"type":"MultiPolygon","id":"024","arcs":[[[6,7,8,9 .... d3.json("./data/world-110m-inet.json", function(loadedTopo) { // by using topojson.feature, we convert the topoJson to geojson, // where each country is a single feature in the features array. countries = topojson.feature(loadedTopo, loadedTopo.objects.countries).features; });
  86. 86. WHEN LOADED THE DATA LOOKS LIKE THIS
  87. 87. SETUP THE PATH GENERATOR var projection = d3.geoNaturalEarth() var path = d3.geoPath().projection(projection) For all the supported projections see: Pretty much any projection you can think off Relatively easy to create your own one https://github.com/d3/d3-geo/ https://github.com/d3/d3-geo-projection
  88. 88. WITH A GENERATOR AND THE DATA var projection = d3.geoNaturalEarth() var path = d3.geoPath().projection(projection) d3.json("./data/world-110m-inet.json", function(loadedTopo) { countries = topojson.feature(loadedTopo, loadedTopo.objects.countries).features; svg.selectAll('.country').data(countries).enter() .append("path") .classed('country', true) .attr("d", path); } And that's it..
  89. 89. AND YOU'RE DONE! But colors?
  90. 90. ADD A SIMPLE SCALE var color = d3.scaleSequential(d3.interpolateGreens).domain([0,100]) var projection = d3.geoNaturalEarth() var path = d3.geoPath().projection(projection) d3.json("./data/world-110m-inet.json", function(loadedTopo) { countries = topojson.feature(loadedTopo, loadedTopo.objects.countries).features; svg.selectAll('.country').data(countries).enter() .append("path") .classed('country', true) .attr("d", path); .attr("fill", function(d) {return d.properties.value ? color(+d.properties.value) : '#ccc'}); }
  91. 91. EASY RIGHT?
  92. 92. EXERCISE 7: GOAL Render the US election
  93. 93. EXERCISE 7, RENDER THE US ELECTION RESULTS Follow steps described in: <ROOT>/src/exercise-07/js/E07.js If needed reference D3.js APIs: You'll get approximately 10 minutes. https://github.com/d3/d3-geo/ https://github.com/d3/d3-geo-projection
  94. 94. D3.JS AND GENERATIVE ART
  95. 95. NOT EVERYTHING SHOULD HAVE A MEANING
  96. 96. D3.JS SUPPORTS VORONOIS In mathematics, a Voronoi diagram is a partitioning of a plane into regions based on distance to points in a specific subset of the plane. That set of points (called seeds, sites, or generators) is specified beforehand, and for each seed there is a corresponding region consisting of all points closer to that seed than to any other.
  97. 97. VORONOI
  98. 98. MUST GO DEEPER
  99. 99. AND DEEPER
  100. 100. ANY DEEPER BREAKS MY BROWSER This is rendered as SVG elements We could output to canvas as well
  101. 101. EXERCISE 7, PLAY AROUND WITH THE VORONOI CODE Not really an exercise, but allows you to experiment with voronois. Follow steps described in: <ROOT>/src/exercise-08/js/E08.js
  102. 102. @josdirken ~ Exercises for this session: https://github.com/josdirksen/d3exercises THANK YOU!

×