OTC Start Thinking BIG Data 2018 10-18
•Download as PPTX, PDF•
1 like•188 views
Whatever the size or type of organization, Big Data has permeated our transportation industry. It is no longer a question of IF Big Data will be useful, but instead WHY is it useful and HOW can we best apply it. This presentation aims to address how we can leverage existing services and available partnerships in transportation, consider new and emerging technologies, and determine strategy for what’s to come in transportation, including connected and autonomous vehicles. While it may be a huge challenge to solve transportation problems with Big Data, it can help us make better travel decisions today and plan for better infrastructure tomorrow.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to OTC Start Thinking BIG Data 2018 10-18
Similar to OTC Start Thinking BIG Data 2018 10-18 (20)
Recently uploaded
Recently uploaded (20)
OTC Start Thinking BIG Data 2018 10-18
- 1. london.ca Start Thinking BIG Data OTC Traffic Engineering Workshop October 18, 2018
- 2. london.ca 2 • Jon Kostyniuk, P.Eng. • Primarily transportation modelling and forecasting background. • All models are wrong, but some are useful. • TripChain.Org project lead, trip generation data using blockchain technology. • Interested in data, its value, and how we can better apply its usefulness. Who am I?
- 3. london.ca 3 • It’s not just coming, it’s here. • It will demand more attention with: oConnected and Autonomous Vehicles (CAVs) oMobility-as-a-Service (MaaS) oDistributed Systems • We’re not all going to be data experts, but we should have some basic data literacy. • Help increase data-driven decision making (DDDM). • Data like a currency, can provide value when timely transmitted; therefore, communication is key. Why should I care about Big Data?
- 4. london.ca 4 • Volume: Vast amounts generated every second. • Velocity: Not only speed generated, but speed at which it moves around. • Variety: Types of data to work with, not just structured data anymore. • Veracity: What is the trustworthiness of the data? • Value: How does the data bring business value? Defining “Big Data”
- 5. london.ca Existing Data and Services 5
- 6. london.ca 6 Existing Traffic Signal System • Existing system from 2005, updated regularly, becoming dated with limited abilities with RT aspirations. • 401 existing signal locations connected to a central system at City Hall. • Limited real-time awareness at intersections without context.
- 7. london.ca 7 • Video data collection – TMCs, ATRs. • Bluetooth readers for travel time and OD studies, limited use. • Radar detection, including many of over 40 permanent ATRs. • Traffic data via Google Maps and APIs, limited use/confidence • Limitations to “snapshot in time” data collection, but demand is dynamic. Existing Data Services
- 8. london.ca 8 • Our TIMMS project, may include: oTSP to support RT, o“Adaptive” Signals at key corridors, oA modern TMC, oCCTV monitoring, and/or oTravel time monitoring and feedback. • Need to manage expectations for new system, not a panacea. • Goals and objectives to include development of a Data Strategy. New System Planned
- 9. london.ca 9 • Public web app, road closure and disruption information. • Lots of detailed information, used by many business units. • Need to “get” information from website, not everyone knows about. • Custom-build, used for past decade, but somewhat dated. • Currently reviewing its use cases and future direction. Renew London Web App
- 10. london.ca Emerging Data and Services 10
- 11. london.ca 11 • Better strategy to “push” data to popular apps and services. • Enhanced Renew London, added data feed. oWaze’s CIFS XML format • Agnostic on third-party usage of feed. • Joined Waze’s CCP in April 2018, now live. • Use Waze’s web interfaces, third party tools, or build your own. Renew London Integration with Waze Construction Road Closures Collisions Road Hazards
- 12. london.ca 12 WARP Project • Open-source traffic data solution by Louisville KY, supported by OGC – QR to GitHub repo. • Polls traffic data, dumps in database for historical use. • Processor features, in-development: oHosted cloud service, database oAPI endpoints, integration oTraffic study tool, analytics oInteractive map, visualization
- 13. london.ca 13 Train-Delay Warning System • Testing sensors at 3 crossing locations. • Participating in BCIP pilot program with TRAINFO in 2019. • Provide crossing insights along with Bluetooth sensors. • Gathers data, predicts when a train may be present • Early warning system via VMS to allow drivers to choose better routes or other “push” services.
- 14. london.ca 14 A blockchain is a distributed database that is updated in near real-time, stored in decentralized locations, and easy to monitor. It has a level of security that insures that no one party can modify a database entry, because, in a sense, everyone is watching. Blockchain Technology
- 16. london.ca Process Characteristics Blockchain Benefits Consensus between Multiple Parties Enhanced coordination and choreography between parties through a shared view of the latest data status. Reconciliation Master source of data instead of disparate data sources that require constant validation and reconciliation. Data Lineage (Temporal or 4D Data) Complete traceability, ensuring integrity of data that is continuously updated and maintained by multiple parties. Auditability Reliable and accurate audit trail with transparency of the party responsible for each data change. 16 Blockchain Technology
- 17. london.ca 17 • Potential Use Cases… Blockchain Technology
- 18. london.ca • CAV Technical Background report, May 2018 oQR Code report link above. • Council resolution, develop a CAV Strategic Plan. 18 Connected and Autonomous Vehicles
- 19. london.ca 19 • Some municipalities early pilots in SPaT and MAP V2I connectivity. oSignal phase and timing oPhysical geometry of intersection • Data partnerships with OEMs, other non-traditional stakeholders. oE.g., vehicle manufacturers, communications, ride sharing, etc. • Each CAV a valuable data source. oHow can municipalities leverage the data? oImpacts on liability, responsibility, etc.? Connected and Autonomous Vehicles
- 20. london.ca 20 • Key points on data: oThere are significant privacy issues. oTechnology expertise is urgently needed. oFocus on digital infrastructure. Connected and Autonomous Vehicles • PPSC Report, The Future of Automated Vehicles in Canada, January 2018 – QR Code report link above. • Governments should build data expertise and capacity.
- 21. london.ca Data Strategy and Approach 21
- 22. london.ca 22 A plan designed to improve all of the ways you acquire, store, manage, share, and use data. What is a Data Strategy?
- 23. london.ca 23 • Each component independent and can evolve as needed. • Each component has an individual set of skills and capabilities. • “Enterprise-class” strategy not always needed. • Complexity increases with organizational scope. • Establish goals for each component. Essential Data Strategy Components Source: The 5 essential Components of a Data Strategy, SAS Institute Inc.
- 24. london.ca 24 Identify data and understand its meaning. Provision data to be made available while respecting rules and access guidelines. Govern through policies and mechanisms to ensure effective data usage. Store persistent data in a structure and location that supports access and processing. Integrate by moving and combining data to provide a unified view. Core Components Defined
- 25. london.ca 25 • Do we currently partner with the right people? • What are our data bottlenecks? • How do we optimize our data collection and usage? • Are we getting the most out of our equipment / services? • Which data services are the most unreliable and why? • Do we have the right IT systems in place? • What parts of our operations could be more efficient? • What are our core data competencies? • What data skills gaps exist in our municipality? • What are our key skills needed in the next two years? Example Strategic Questions
- 26. london.ca 26 Big Data Skills • Business Skills: Understanding transportation services, including communication and interpersonal. • Analytical Skills: Spot patterns, cause and effect, build models, etc. • Computer Science: Hardware, software, AI, programming, etc. • Statistics and Mathematics: Determine relevant data, sample sizes, algorithms, etc. • Creativity: Ability to convey insights effectively to an audience.
- 27. london.ca 27 • Well, maybe not completely… but, hello visualizations!! • Consider best way to communicate data with audience. • Insightful to add 4th dimension… time. Goodbye Spreadsheets
- 28. london.ca 28 • Challenging to have staff and expertise. • Organizational challenges – operations, structure, size. • BDaaS viable alternative to do-it-yourself. • Collect, store, analyze, and provide access • Does service provider support your strategy? • Cost of Big Data vs. traditional approach. Big Data as a Service (BDaaS)
- 29. london.ca 29 Key Takeaways • Like promises of the past, Big Data is not a panacea, manage expectations. • Plan ahead, use Big Data effectively. • Beware of too many purpose-built apps. • Informed travellers = better decisions and less frustration. “Fewer red lights, less stop and start and a minimum of delays, those are the goals of a computerized traffic control system in which London may participate” ~ London Free Press, 1962
- 30. london.ca 30 Jon Kostyniuk, P.Eng. jkostyniuk@london.ca Thank You!!
Editor's Notes
- Thank You I appreciate you all attending and listening to this talk. I’d like to extend a special thank you to the OTC staff and directors, and in particular Scott Godwin and Doug Green, for giving me the opportunity to speak today. Ask Questions to the Audience How do you feel when you just get a green light only to be stopped again at a red light at the next intersection? How do you feel when you turn the corner and see those flashing lights and railroad crossing bells ringing, not knowing how long you’ll be stuck there?
- Who am I? [Cover bullets on slide] Why am I telling you about Big Data? Big Data really is this undefined “blob” that means many things to many people – ongoing challenge. Essentially, I wish to advocate that: All municipalities are different (e.g. size, staffing, abilities, needs, etc.), so we should consider our Strategic Needs; When data insights are provided to travellers in near real-time, better decisions can be made to reduce frustration; and When presented effectively, DDDM can help us make better transportation planning and design decisions. I will briefly cover: Some of the conventional approaches we are taking at the City of London (which I’m sure is similar with many of us); Look at some emerging and even fanciful technologies; and Most importantly, overview how we can pursue developing a Data Strategy for our own municipalities.
- Why should I care about Big Data? It’s not just coming, it’s here. It’s only going to demand more attention with the advent of: Connected and Autonomous Vehicles (CAVs); Mobility-as-a-Service (MaaS); and Distributed Systems (i.e. Blockchain and Smart Contracts). We’re not all going to be data experts (don’t be scared), but we should have some basic data literacy. If applied effectively, Big Data can help increase data-driven decision making. Data is like a currency and can provide value when it is timely transmitted to where it’s needed. Therefore, communication of data is key.
- Defining “Big Data” Generally, big data can be defined by one or more of these 5 “Vs”. Volume [Read slide…] We’re not just talking about gigabytes – but terabytes, petabytes, and even zettabytes. How do we comprehend let alone work with this? Velocity [Read slide…] Data, and especially real time data, is most useful when it gets to where it’s needed in a timely manner. Variety [Read slide…] While structured data, like SQL databases, will continue to retain importance, unstructured data such as audio, video, and photos are becoming more important. Veracity [Read slide…] Given the other Vs, we need to have a certain level of trust in our data sources to help us drive appropriate decisions. Value [Read slide…] If the data we are collecting does not help us gain relevant insights, why are we doing it?
- Existing Traffic Signal System Existing traffic signal system in-place since 2005 and updated regularly. We have 401 traffic signal locations in current system, including both full signals and IPS. All signals connected to a central system, can change timings from City Hall. System becoming dated, coming to end of lifespan, and limited considering RT aspirations. Limited real-time awareness at each intersection without context. [Talk about February 2018 flood.]
- Existing Data Services Typical practices compared with many medium-sized cities currently. Video data collection for most common TMCs and ATRs. Use four (4) “portable” Bluetooth units to perform travel time and OD studies, but this is of limited use. In recent years, radar detection has become our “go-to” technology, including many of our over 40 permanent ATR locations. On occasion we have used Google Maps and APIs to obtain traffic data, but still limited use and confidence in this approach. While these services provide a good, basic data background, there are still limitations to the “snapshot in time” approach whereas travel demands are dynamic.
- New System Planned We do have a new system planned called our Transportation Integrated Mobility Management System (TIMMS) project. Still under development, but may include TSP, “Adaptive” Signals, TMC, CCTVs, and/or Travel Time Sensors. Despite this we need to manage expectations for the new system. One of the main components of this new system I wish to discuss today will be the development of a Data Strategy.
- Renew London Web App The City curates a web app that provides up-to-date road closure and disruption information. Lots of detailed information available within the app, used may many business units within the City. However, one needs to “get” the information from the website. What if you don’t know about the website or are a visitor to the City? This is frustrating to travellers and goes to the point that data is like a currency that needs to provide timely value when it is needed. While there are similar third-party services available, Renew London is a custom-build, but is becoming somewhat dated. We are currently assessing the use cases and future direction of Renew London.
- Renew London Integration First step: Instead to going to the Renew London website to “get” data, we’ll create a “push” data feed any third-party can use. Our feed uses Waze’s Closure and Incident Feed Specification (CIFS) format as de-facto standard. Generally, we have an agnostic stance on third-party usage: Working with internal stakeholders such as London Transit Commission, Emergency Services, and Corporate Security. Open to third-party external apps and services, e.g. TomTom, Garmin, Apple Maps, Navmii, We joined Waze’s Connected Citizen’s Program (CCP) as of April 2018. Free, two-way agreement. Provide CIFS data every 2 minutes and obtain crowdsource and congestion data from Waze. Push common data such as construction, collisions, road closures, and road hazards. Ability to push out Emergency Shelter information in event of emergency. Can access Waze via their web interface, third-party tools, or build/integrate your own.
- WARP Project Unique, less conventional ways are being explored to capture traffic data. Open-source Waze Analytics Relational-database Platform (WARP) is on project currently under development by Louisville KY, the Open Government Coalition, and various other partners. Includes four man areas of development: A database to process and store historical data; Data hooks via an API to integrate with other third-party products; A traffic study tool to provide analytics to assist DDDM; and An interactive map to visualize events and patterns over periods of time. I’ve heard claims this could effectively provide data for FREE* traffic studies. Time will tell, but caution that FREE likely means “Some conditions may apply”. However, crowdsourced data could provide a preliminary snapshot of problematic areas of interest. Source https://docs.google.com/presentation/d/1loAV4BDAUyXdrn44QoLmYiwZdLmL59C4jvJGlZ1a-AY/edit
- Train-Delay Warning System Who likes getting stuck at train crossings? We are currently testing train presence sensors at three (3) London railway crossing locations. In 2019, we will be participating in a pilot program with TRAINFO through the Build in Canada Innovation Program (BCIP). Our current project with this system is to gather train disruption data and determining the impacts to traffic patterns. Over time, the algorithms build a crossing profile to “predict” when likely train disruption events will occur. The intent is to provide timely, early warning systems to inform drivers and allow them to choose alternate routes. Use Variable Message Signs (VMS) and/or travel apps, such as Waze. Still limitations in propagation time from sensor detection to “push” notifications in apps - the train disruption could be over.
- Blockchain Technology Who here has heard of blockchain technology or smart contracts? Who here has heard about Bitcoin or Ethereum? Many people seem to know Bitcoin, but don’t (literally) buy into the hype! Many people are similarly less familiar with Blockchain Technology. I would argue that this technology is more important and has more of a potential future in Big Data and transportation. Let’s start with a definition… [Read Definition]. In other words, a blockchain is distributed, immutable, and auditable.
- Blockchain Technology To visualize how blockchains interact, consider a centralized system, such as a central traffic signal system or ATMS. Blockchain generally acts as a distributed system, but has characteristics of a decentralized system – a hybrid between the two. Effectively, new data propagates through the network in near real-time as in the distributed model. However, individuals in isolation can access the network as in the decentralized model. Redundancy in the network – several, if not most, of the network nodes could fail while continuing to maintain functionality.
- Potential Use Cases Link together Mobility-as-a-Service (MaaS) partners for seamless data exchange. Transparency and incentive tracking for Public-Private-Partnerships (P3s) – MOBI. Help the AI in Autonomous Vehicle algorithms “learn” quicker by propagating new data in near real-time. Break down data silos and give credit and value to data producers – Ocean Protocol. Shameless plug for my pet project, TripChain.Org – propagate trip generation data to improve transportation planning decisions. Many other Blockchain projects, constant churn, still emerging. Prognosis for Blockchain Overall still an emerging, promising technology, but watch progress in coming 5-10 years. Technology still not proven in production, no “killer app” yet to emerge. Although data storage getting cheaper and cheaper, still beware of data bloat.
- Connected and Autonomous Vehicles Briefly touch upon Connected and Autonomous Vehicles (CAVs). While still several years out, we wish to take a more proactive approach to preparing for CAVs as opposed to reacting when they show up on our roads. Developed a CAV Technical Background report in May 2018 and obtained a Council resolution to prepare a CAV strategy for the City of London. Focus on recommendations including Infrastructure, Land Use, Transit, Parking, Accessibility, Safety, Privacy and Security, and Public Awareness and Education.
- Connected and Autonomous Vehicles Getting an early start, some municipalities are already piloting Vehicle-to-Infrastructure (V2I) communication, including SPaT and MAP data. To pull CAVs off, we will need to explore and leverage non-traditional data partnerships. Each CAV itself will likely be a valuable data source. How can municipalities leverage this data for planning and operations? Will vehicle self-report pot holes, road conditions, etc.? How will municipalities respond? What are the impacts on liability. May questions to explore in the near future.
- Connected and Autonomous Vehicles The report included several key points related to data and municipalities: There are significant privacy issues: As more and more vehicle data is generated, collected and shared, governments must act to protect the privacy rights and security of individuals. Technology expertise is urgently needed: It will be crucial for regulators to develop expertise in data science and computer science. CAVs will generate large volumes of data which currently have no clear ownership rights. Governments will need to effectively address these concerns and protect the public interest. Physical infrastructure modifications can wait, focus on digital infrastructure: CAVs are being designed to work with the physical or “hard” infrastructure that exists today. However, governments will need to be aware of which technologies automakers and suppliers are using. Digital interfaces such as sensor data, maps, etc. warrant more immediate attention to help support CAV systems. Recommends governments should build data expertise and capacity as part of the interdisciplinary effort to make CAVs a reality in the coming years and decades.
- Example Strategic Questions Thinking at a high level, here are some example strategic questions municipalities could consider. [Review question list]
- Big Data Skills Five (5) essential data science skills to develop. [Review slides]
- Goodbye Spreadsheets I have a love/hate relationship with spreadsheets. Tables and charts will always be valuable to gain insights, but depending on the audience, data visualizations are a great tool. Observing data over time also gives a sense to guide decision making.
- Big Data as a Service (BDaaS) Of course, it can be challenging for many municipalities to have staff and expertise move from data to insights. Organizational challenges such as operations, structure, and size may provide hurdles to gaining insights. BDaaS is a growing and viable alternative to do-it-yourself data analysis. Depending on the service can collect, store, analyze, and provide access to data. Ultimately need to consider whether a BDaaS provider supports your strategy and look at the cost vs. traditional approaches. These are not an endorsement nor an extensive list, but here are several companies in our industry which may offer BDaaS to varying degrees. I’m sure there are many others that I have not yet considered.
- Key Takeaways I like this quote… [Read Quote]. Does this ring true today, nearly 60 years later? Like promises of the past, Big Data is not a panacea, so manage expectations with key stakeholders and the public. By creating a Data Strategy and planning ahead, your municipality can use Big Data effectively. Beware of too many purpose-built apps. Look integration opportunities with prominent or popular apps or services to maximize your impact. Not just serving your municipality, but also visitors and travellers passing through. And lastly, informed travellers have the ability to make better travel decisions, hopefully experiencing less frustration. Help to use the transportation network more effectively. Knowledge is power and can help minimize the frustration the comes from the unexpected.