Download to read offline
Uploaded byOpen Knowledge Belgium
The story behind SNCB alerts
The document discusses the creation of an open-source tool called sncbalerts, designed to inform train users about delays in real-time. It outlines the development process, including data gathering from live boards and the use of platforms like Twitter and Telegram for alerts. The project leverages technologies such as Symfony 4, Node.js, and Neo4j to enhance its functionality.
The story behind SNCB alerts
- 1. Open Belgium 2018 ByPol Dellaiera
- 2. Please raise yourhand if...
- 4.
- 5. From door todoor: ● In train: ~1h50 ● In car: ~1h15
- 6. 4h “lost” inpublic transportation
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13. In my case... Iwork.
- 15.
- 16.
- 17. I had theidea to create a tool that would “alert” trains users of the delays.
- 18. SNCBAlerts (it just anugly name) ● Open Source ● Using Symfony 4 ● Must rely on Git ● Must be pluggable
- 19. What is adelay ?
- 20. 1. the actof delaying; procrastination; loitering. 2. an instance of being delayed: there were many delays during the train trip. 3. the period or amount of time during which something is delayed: The ballet performance began after a half-hour delay. Definition of “delay”
- 21. Where to getthe data ?
- 23. Liveboard ● Destinations ● Departures ●Delays ● Cancellations ● Line number
- 24. The idea behind SNCBAlertsis to provide informations about delays and alerts in a standardized way.
- 25.
- 26. Data to provide ●Departure station 🆗 ● Destination station 🆗 ● Train line ❎ ● Delay (in minutes) 🆗
- 27. My first ideawas to create a matrix of 600 columns on 600 rows… and to do it manually.
- 29.
- 30. After some talkswith Pieter Colpaert from irail.be, he showed how I could use wikidata to complete train stations data.
- 31. So I startedto manually edit the ~600 stations in Belgium...
- 32. Added informations ● Irail.be identifier ●Adjacent stations ● Connected lines ● Name normalization
- 33. I created atemporary tool to show me the percentage of completion of that work.
- 34.
- 35.
- 36.
- 37.
- 38. A NodeJS libraryhas been written as well, just to feed the map.
- 40. Neo4J, a graphdatabase platform, was used to display the relationships in between stations.
- 41.
- 42.
- 43.
- 44. 3 gateways ● Twitter(delays >= 15 min) ● Telegram (public channel) ● Telegram Bot (custom alerts)(for now)
- 45. Twitter ● @sncbalerts ●28k tweets
- 46.
- 47.
- 48. Telegram ● Publicchannel ● Bot
- 49.
- 50.
- 51.
- 52. ● Symfony 4 ●NodeJS ● Wikidata ● SparQL ● Heroku ● Neo4J What I learned ?
- 53. ● A Twitteraccount spamming its subscribers with trains delays greater than 15 minutes. ● A Telegram channel spamming subscribers with delays greater than 1 minute ● A Telegram bot that subscribers are able to customize to receive their delays and alerts in “real time”. What I give back
- 54.
- 55. I’m taking mycar to go to work.
Editor's Notes
- #2 Hello everyone, welcome to this session. First I’d like to thanks to the organizers for letting me present a small project I’ve been working on the past 3 months.
- #3 Before going deep into the subject, I would like to start by asking a question to everyone here… Who’s relying on public transportation to go to work ?
- #4 If some of you are using the trains in Belgium, I guess, sadly, that you are used to this kind of liveboard… If not, let me tell you about my own experience.
- #5 I used to live in Mons for 36 years and as you can see, and since a couple of days ago, I was living in the worst part of Belgium when it comes to punctuality of trains.
- #6 It used to take me almost 2 hours per day to go to work with trains… and 1h15 by car if I leave home early in the morning around 6h00.
- #7 If you do the math, it’s 4 hours lost in public transportation.
- #8 If some of you are like me and hate wasting time doing nothing, then the question that I’d like to ask is… What to do when you waste 4h every day to go to work ?
- #9 Some of you might say: Sleeping. Which is a very good idea, I love sleeping. But in my train, it’s most of the time, impossible. The line 97/96 which is the one I was using are equipped with old trains, without automatic doors and not well isolated agains all kind of noises.
- #10 The second choice is reading. Either to learn stuff or to just relax.
- #11 The third option is to listen to music or podcasts. Of course with a reasonable volume to not disturb your neighbors. Trust me, this never happend.
- #12 The fourth option is to work, this is my favorite option, because it allows you to focus on something else quickly.
- #13 The last option is to learn stuff by reading.
- #14 In my case, my favorite activity in the train besides sleeping when it’s not too noisy, is working on my personal projects.
- #15 Being a long time contributor to the Drupal OpenSource community, I usually maintain and develop my personal modules. Drupal is a framework/cms written in PHP and created by Dries Buytaert a fellow belgian.
- #16 So, how the story of my project began ?
- #17 Just because of this. I was tired to not have a practical way to check if my trains were late or not. Tired of the lack of consistency in these data. We are at the edge of 2020 and we still do not have a way to check if a train will be late or not in a consistent way.
- #18 So, I had the idea to create a tool that would send users the trains delays and alerts.
- #19 So I created SNCBAlerts which is a platform that gather delays and alerts of belgian trains, then this send those data to different gateways. SNCBAlerts is OpenSource, using Symfony 4, relies on Git for many things, and is pluggable.
- #20 Before going further, the main question that everyone should ask himself is, “what, in the end, is a delay” ?
- #21 According to the internet, this is the definition of a delay. But in our case, my case, a train delay is when the train doesn’t leave at the right time. According to SNCB, a delay is when your train has more than 15 minutes of late. Which is completely unfair, but it’s not the main subject of the presentation.
- #22 Where to get those delays ? Are they available somewhere ?
- #23 Of course, you can find them on the liveboard of each stations in Belgium.
- #24 A liveboard provides information about destinations, departures, delays, cancellations… and unfortunately, it doesn’t provides the train line.
- #25 But… the idea of SNCBAlerts is to provide informations about delays and alerts in a standardized way.
- #26 This is why it relies on irail.be from Pieter Colpaert to get its data.
- #27 Irail.be provides information about the departure, the destination, the delays, the cancellations, but not the train line. So, the first version of SNCBAlerts was not having those train line information.
- #28 In order to get these information, as I didn’t find it anywhere on the internet, was to create a matrix of ~600 columns on ~600 rows, which makes about ~36000 cells and to complete that grid manually. Of course, if you think about it, it would only require the half of the cells because the train line from a station A to B is the same as from B to A. But still … 18000 cells.
- #29 To be honest, I started the document on Google Docs… and it was completely unmaintainable. I had to find another way to do this.
- #30 So, I created an issue on the irail project on Github to have some feedback.
- #31 And after a talk with Pieter Colpaert, I deleted the Google Doc I was creating to use Wikidata. A tool that I didn’t even know it existed.
- #32 So, I started to edit the ~600 stations in Belgium on Wikidata… even if it’s a lot, it much better than filling 18000 cells in a spreadsheet.
- #33 The added informations on Wikidata are the following… As each station on irail.be has an identifier, I filled a property on each station on wikidata with the station identifier from irail. I also added adjacent stations, connected lines and also normalized some stations names.
- #34 I had no way to have the big picture of what is done and what is left to do… so I created a small map with informations from irail and wikidata. The map is showing belgian stations on a map. Based on the state of completion, the markers were red meaning that most of the informations were missing. Yellow, meaning that there were missing informations, and green meaning that everything was ok.
- #35 This is a screenshot of the tool… as you can see, most of the stations were already done.
- #36 Today the map looks like this. Most of the belgian stations are green, some of them are still in yellow. In this case, it doesn’t means that there are missing information, it just means that they only have one adjacent station, they are the last stations on a line.
- #37 Here’s a glimpse of the information that you can get when you click on a station with a yellow marker. As you can see, there is only one adjacent station.
- #38 If you click on a green marker, you can see that all the information are there.
- #39 In order to display this live map, I wrote a nodejs library just to feed the map. This library has been extended since and can be used to query irail.be and cross those data with wikidata quite easily.
- #40 The nodejs/npm package is Opensource and available for everyone, the name is irail-api.
- #41 The tools you’ve seen were great, but it was not enough to visually check the consistency of the “adjacent stations” property.
- #42 What you see here is a map showing belgian railway stations and their link in between each others. To draw the map, Neo4J and D3 were used.
- #43 I really enjoyed doing that part, it’s actually the part I liked the most. I liked it so much that I did the same for the metro stations in London. It’s not possible to do it for the Brussels metro, a lot of informations are missing on Wikidata.
- #44 So, enough talking, what’s the result of all of this ?
- #45 3 gateways or plugins are available and Open Source: The twitter account, the Telegram channel and a Telegram bot.
- #46 The twitter account SNCBAlerts is up since the 6 of December and now has more than 28 thousands tweets so far.
- #47 Here’s a message from @sncbalerts on Twitter when it was the first version.
- #48 And thanks to the data on Wikidata, we can now get the line information in the message.
- #49 The telegram gateway provides a public channel that everyone can join and see “real time” alerts and delays. This is very hard to follow what is going on there because there’s so much information that it’s unusable. This is why I created a bot that users can customize at will.
- #50 Here’s a screenshot of the Telegram web application and the SNCB Alerts public channel.
- #51 And here’s a screenshot of the Telegram web application and the SNCB Alerts bot. As you can see, you can query it inline.
- #52 If you’re using Telegram, you can use the bot today by following the url that you can find on the Twitter bio.
- #53 What are the things that I learned by doing this ?
- #55 What about the future ?