This document discusses using web scraping to create a more user-friendly interface for websites with poor designs. It describes web scraping as extracting data from websites and recommends tools like Python, BeautifulSoup and Spectre CSS. The document then demonstrates how to scrape timetable data from the MyCiti transit website by inspecting page elements, investigating encoded URLs, and writing code to retrieve the raw data. Finally, it suggests creating a new interface with the scraped data to "Make UI Great Again."

1. Web Scrape to “Make UI
Great Again”
6 December 2017Gavin Wiener

2. Goal
Using web-scraping, you seldom have to be forced into using
an unfriendly user interface

3. What is Web-Scraping
“Web scraping, web harvesting, or web data extraction is data
scraping used for extracting data from websites”
https://en.wikipedia.org/wiki/Web_scraping

4. Tools
● Python + BeautifulSoup (min. previous knowledge required)
○ https://www.python.org/
○ https://www.crummy.com/software/BeautifulSoup/bs4/doc/
● Mobile-Friendly CSS - Spectre.css
○ https://picturepan2.github.io/spectre/
● 1 or more badly designed websites
○ https://myciti.org.za/en/home/
○ https://myciti.org.za/en/timetables/route-stop-timetables/
● (Optional) Hosting
○ I essentially created a website

5. MyCiti: Initial
Simulator: iPhone 5

6. MyCiti: Goal
Simulator: iPhone 5

7. Identify Structure: Inspecting Elements

8. Get Data: Investigate URLs
Encoded
https://myciti.org.za/en/timetables/route-stop-timetables/?timetable%5Bweekday%5D=sunday&timetable
%5Bstation%5D=493&timetable%5Broute%5D=&timetable%5Bdirection%5D=
Decoded
https://myciti.org.za/en/timetables/route-stop-timetables/?timetable[weekday]=su
nday&timetable[station]=493&timetable[route]=&timetable[direction]=
Tool: https://meyerweb.com/eric/tools/dencoder/

9. I Can Haz Your Data - Code
Getting the timetable of a stop

10. I Can Haz Your Data - Code
Getting the timetable of a stop

11. I Can Haz Your Data - Raw
Getting the timetable of a stop

12. Create a New Interface

13. Summary
1. Find a website e.g. MyCiti
2. Identify the structure, and interesting components e.g. <table>
3. Identify how to reach the data e.g. urls
4. ‘Scrape’ the data with code e.g. code
5. Create your new interface

14. And You Have a Website
gavinwiener@gmail.com
http://github.com/divisionMax/