Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Affordable REST - Coder Camp Hamilton 2019


Published on

This presentation talks about the need for municipalities to take more of an interest in presenting their constituents with information, especially around affordable housing. How much work and money can be saved if they invest a little time in disseminating information on a website, rather than hoarding it? I was able to produce this project in 5 days as a student. What's their excuse?

Published in: Technology
  • Be the first to comment

  • Be the first to like this

An Affordable REST - Coder Camp Hamilton 2019

  2. 2. ABOUT ME  Quality Assurance Supervisor, Weever Apps  Former Instructor, Sheridan College & Mohawk College in Web Design (including WordPress) and Capstone Project  Co-organizer, WordCamp Hamilton, 2015-18  “Lead Dudette” defined as: "a young woman or girl who is very popular or admired by her peers”. Thanks @modmatt! SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 2
  3. 3. AN AFFORDABLE REST  The slides have been posted to  The live project can be found at (wilt = What I Learned Today)  The API is at  The Github Repository is at REST SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 3
  4. 4. THE CASE: CITY OF TORONTO – TORONTO COMMUNITY HOUSING The City of Toronto has a 10-year waiting list for subsidized housing. In order to determine which development that a seeker can apply to, they must speak with someone at Toronto Community Housing. All of the Organizational Unit Managers email addresses and phone numbers are identical. Can you imagine 82,000+1 households calling one number and sending to a single email address? SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 41. Source:
  5. 5. WHAT’S THE PROBLEM? When searching for housing, the primary need is the unit size (how many bedrooms) and special needs (such as wheelchair accessible). The criteria is less about location, and more about need. I called the offices of three city councillors to test if what I was going for made sense. I never got a response. Also, the open data was unusable, given the above criteria. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 5
  6. 6. WHAT’S THE SOLUTION? What if seekers had the ability to find out which units in the City are available to them, based on unit size and speciality? That would shorten the time to serve these people and reduce the amount of work on an already burdened system. Note: Since I did the project, they have made some updates to the website, but it’s still pretty bad. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 6
  7. 7. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 7 Snapshot of data from the Open Data portal for the City of Toronto
  8. 8. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 8 071d60f89RCRD&vgnextchannel=1a66e03bb8d1e310VgnVCM10000071d60f89RCRD
  12. 12. MY PROCESS I wrote the entire app in React and Node and deployed on AWS in a week. 1. Create mock data based on the information that I needed to see, such as unit size and speciality. 2. Scrape the data from the existing site to determine how many org units I need to scrape 3. Feed the previous step into another scrape to grab the names of the buildings, etc., along with the org unit info. 4. Push the mock data into the scraped data to create a single JSON file and serve it up in an API 5. Serve the data from the API to a front end SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 12
  13. 13. 1. CREATE THE MOCK DATA  I used Mockaroo  This is a sample of a single building’s data  I created 1,000 records and just reused them as needed {"unitType": {"bachelor":4,"oneBr":142, "twoBr":204,"threeBr":10, "fourBr":90}, "special": {"seniors":false, "supportive":false, "accessible":false, "aboriginal":false}} SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 13
  14. 14. 2. DETERMINE NUMBER OF ORG UNITS BY SCRAPING <li class="static selected"><a class="static selected menu-item ms-core-listMenu-item ms-displayInline ms-bold ms-core-listMenu- selected ms-navedit-linkNode" tabindex="0" href="/about/our- housing/Pages/Don-Valley-Beaches.aspx"><span class="additional-background ms-navedit-flyoutArrow"><span class="menu-item-text">Don Valley Beaches</span><span class="ms-hidden">Currently selected</span></span></a></li> SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 14
  15. 15. 2. DETERMINE NUMBER OF ORG UNITS BY SCRAPING (CON’T) axios.get(' housing/Pages/default.aspx') .then(res => { const $ = cheerio.load(; let sidebarData = '#zz4_RootAspMenu'; // This targets the sidebar menu that contains the organizational names and corresponding links. let numOfOrgs = $(sidebarData).children().length; // Should be 11 items. for(let i=2; i<numOfOrgs; i++){ let orgRow = $(sidebarData).children()[i]; let orgObject = $(orgRow).children()[0]; SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 15 let orgLinkRaw = $(orgObject)[0]; let orgLink = (''+ orgLinkRaw.attribs.href); // urlArray.push(axios.get(req[i].orgLink)); // This one doesn't work yet urlArray.push(orgLink); // This will create the array correctly. } }) .catch(err => { console.log(err); });
  16. 16. 2. DETERMINE NUMBER OF ORG UNITS BY SCRAPING (CON’T) Ideally, I should have been able to hand off the created array to Axios to repeat the following steps for as many items as are in the array. Unfortunately, one of the items didn’t work, so I had to manually hand over the remaining 10 items. axios.all([axios.get('https://www.toront housing/Pages/Wexton- Rexdale.aspx'),… SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 16
  17. 17. 3. CAPTURE THE ORG UNIT INFO AND BUILDINGS <strong>Operating Unit Manager: </strong> <br>John Perkovic<br><br> <strong>Office Location and hours: </strong> <br>145 Strathmore Blvd. <br>Monday to Friday 8:30am to 4:30pm<br><br> <strong>Tel:</strong> (416) 981-5500<br> <strong>Email:</strong> <a href=""></a>… SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 17
  18. 18. 3. CAPTURE THE ORG UNIT INFO AND BUILDINGS (CON’T) Grab the organizational unit information and put it into an object, … 1. .then(axios.spread (function (res)… 2. ...const orgData = ' field'; let orgManager = $(orgData).children()[2]; … SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 18
  19. 19. 3. CAPTURE THE ORG UNIT INFO AND BUILDINGS (CON’T) <table align="left" width="90%" cellpadding="10" border="1" style="width:100%;" class="table table- hover"><tbody><tr><td width="8%"> <strong>Dev ID</strong></td><td align="left" width="21%"> <strong>Development Name</strong></td><td align="left" width="71%"> <strong>Address(es)</strong></td></tr> <tr><td>701</td><td>Asquith Park</td><td>40 Asquith Ave. <br> </td></tr> SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 19 …then each building.
  20. 20. 4. PUSH IN THE MOCK DATA & SERVE IN API As each building is built, the mock data is pushed into the object. let tempAddress = $(tempRow).children()[2]; //The third column in the table. let bldgAddress = $(tempAddress).text(); let bldgData = { "DEV_ID" : bldgID, "DEV_NAME" : bldgName, "BLDG_ADDRESS" : bldgAddress, "Unit_Type" : mockData[i].unitType, "Speciality" : mockData[i].special }SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 20
  21. 21. 4. PUSH IN THE MOCK DATA & SERVE IN API (CON’T) {"orgUnit": "Weston Rexdale", "orgManager": "Joan White", "orgAddress": "2765 Islington Ave.", "orgPhone": "(416) 981-5500", "orgHours": "Monday to Friday 8:30am to 4:30pm", "orgEmail": "", "orgBuildings": [{ "DEV_ID": "47", "DEV_NAME": "Albion Shendale", "BLDG_ADDRESS": "275 Albion Rd.;1 Shendale Dr. ", SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 21 "Unit_Type": { "bachelor": 5, "oneBr": 172, "twoBr": 47, "threeBr": 171, "fourBr": 5 }, "Speciality": { "seniors": true, "supportive": true, "accessible": true, "aboriginal": false}},
  22. 22. 4. PUSH IN THE MOCK DATA & SERVE IN API (CON’T) SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 22 fs.writeFile('./data/webscrapeResults.JSON', JSON.stringify(orgArr, null, 2), () => { console.log('File written'); })
  23. 23. 5. SERVE THE DATA TO THE FRONT END Now, a seeker can select the most pertinent information that they can use to make informed decisions. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 23
  24. 24. 5. SERVE THE DATA TO THE FRONT END (CON’T) Shows the relevant results. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 24
  25. 25. NEXT STEPS Rebuild the project to bring it up to date: 1. Fix the first webscrape and feed it to the second 2. Connect it to React 3. Build the front end, without Bootstrap New Features: 1. Produce the details page, that will also use location data to show a map 2. Give users the option to choose the region from the home page so they can select where they might like to live, broken down by those organizational units 3. Provide a map on the front page that shows the locations of all of the buildings, and make them ”clickable” 4. Take the new data that I’ve found and use real data, rather than mock data and web scrape it. SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 25
  27. 27. A BIG THANK YOU TO MY COLLEAGUES THAT HELPED  James King  For giving me the idea to give this talk last month  Aya Myhr  For encouraging me when I thought I was crazy  Jenifer Champagne  For helping fix some of the code on her time off  Adam Sutch  For giving such sage advice and also fixing stuff SHANTA R. NATHWANI - @SHANTADOTCA - HTTP://SHANTA.CA 27