Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Delivering near real-time mobility
insights at Swisscom
François Garillot
francois.garillot@swisscom.com
@huitseeker
Agenda
Intro
Smart-Data
Big Data Architecture
Streaming
Data challenges
Introduction : Positioning
Positioning users in a modern
network
no radio-goniometer at scale
cell of attachment has position, beam characteristics
o...
Positioning at specific locations
handovers at specific cell-to-cell location
phone needs to be active
Positioning with more precision
better positioning with excellent data sources:
3G : GPEH
4G: LTE-CTR
Trajectory
data mining
time series reconstruction
trajectory segmentation
map matching, clustering
mode of transport detec...
How to create value with
positioning at Swisscom ?
with competitive analytics & data sources,
and by making sure it embodi...
Smart Data
On (not) tracking (any users)
"Swisscom strictly complies with all applicable legislations, in
particular with the telecom...
Smart Data : Big Data without Big Brother
Privacy preservation is an asset
It makes sense to care as much about your custo...
Swisscom mobile subscribers
source: xavierstuder.com, MD&A reports
Our choices
public good applications: making Switzerland run better,
understanding places, not individuals,
all results pr...
Markets
A first product : City
"It's a dream for civil engineers" -- Alexandre Machu, Urban
systems engineer, Pully
Demo time
Usages
New roads to divert transit traffic out of downtown (informs a 50M$
project)
Parking lot expansion and transformati...
Big Data architecture
In the backend
Spark configuration essentials for enterprise
jobs
spark.executor.memory="not the default 1g"
spark.kryo.registrator="some...
See Also
In the front-end
Scala (1/2)
typeChronoHistory = List[UEupdate] @@ Chronological
typeAnteChronoHistory = List[UEupdate] @@ AnteChronologica...
Scala (2/2)
implicit def reverseChrono(l: ChronoHistory): AnteChronoHistory = l.reve
implicit def reverseAnteChrono(l: Ant...
Streaming Analytics
Selecting users on a path of Interest
Massive discrepancy between # of users (2-3E6)
and # of interesting users (1.5E3 on ...
Graph matching
Locality-sensitive hashing short histories
A family H of hashing functions is -sensitive if:(r, cr, , )p1 p2
if then
if th...
Computing speeds: Solving graph
constraints
a speed comes from a user well-positioned, twice
plus route knowledge
given a ...
Solving graph constraints
just a few users left in computation at this stage
so a lot invested in > linear complexity algo...
Data Challenges
Crucial elements
Quality, reliability of data sources
Automated ground truth checking
sensors
TEMS fleet
What's the ground...
In the works
Accuracy improvements
More features (see you Spark Summit EU!)
Streaming for city
Thank you
Upcoming SlideShare
Loading in …5
×

Delivering near real time mobility insights at swisscom

142 views

Published on

A use case developed at Swisscom to help urban planners understand their cities and to measure speeds on Swiss highways.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Delivering near real time mobility insights at swisscom

  1. 1. Delivering near real-time mobility insights at Swisscom François Garillot francois.garillot@swisscom.com @huitseeker
  2. 2. Agenda Intro Smart-Data Big Data Architecture Streaming Data challenges
  3. 3. Introduction : Positioning
  4. 4. Positioning users in a modern network no radio-goniometer at scale cell of attachment has position, beam characteristics over history, best position ~200m
  5. 5. Positioning at specific locations handovers at specific cell-to-cell location phone needs to be active
  6. 6. Positioning with more precision better positioning with excellent data sources: 3G : GPEH 4G: LTE-CTR
  7. 7. Trajectory data mining time series reconstruction trajectory segmentation map matching, clustering mode of transport detection ...
  8. 8. How to create value with positioning at Swisscom ? with competitive analytics & data sources, and by making sure it embodies the right values.
  9. 9. Smart Data
  10. 10. On (not) tracking (any users) "Swisscom strictly complies with all applicable legislations, in particular with the telecommunications law and the data protection initiative." Jürg Studerus, Swisscom Senior Manager, Corporate Responsibility
  11. 11. Smart Data : Big Data without Big Brother Privacy preservation is an asset It makes sense to care as much about your customer as they do about you. We technically enforce this answering only synoptic questions, no individual ones, with data flow control : we neutralize quasi-identifiers at every stage
  12. 12. Swisscom mobile subscribers source: xavierstuder.com, MD&A reports
  13. 13. Our choices public good applications: making Switzerland run better, understanding places, not individuals, all results presented aggregated, anonymized.
  14. 14. Markets
  15. 15. A first product : City "It's a dream for civil engineers" -- Alexandre Machu, Urban systems engineer, Pully
  16. 16. Demo time
  17. 17. Usages New roads to divert transit traffic out of downtown (informs a 50M$ project) Parking lot expansion and transformation (informs a 10M$ project) Electric car charging station deployment
  18. 18. Big Data architecture
  19. 19. In the backend
  20. 20. Spark configuration essentials for enterprise jobs spark.executor.memory="not the default 1g" spark.kryo.registrator="something custom"// and companions spark.shuffle.service.enabled="true" spark.dynamicAllocation.enabled="true" spark.deploy.recoveryMode="ZOOKEEPER" spark.deploy.recoveryDirectory="/path/to/state" spark.deploy.zookeeper.url="quorumMachine1:2181, ..." NOT the only valuable settings, see https://techsuppdiva.github.io for more
  21. 21. See Also
  22. 22. In the front-end
  23. 23. Scala (1/2) typeChronoHistory = List[UEupdate] @@ Chronological typeAnteChronoHistory = List[UEupdate] @@ AnteChronological implicit classChrono(l: List[UEupdate]){ def asChrono: ChronoHistory = { chronoCheck(l) l.asInstanceOf[ChronoHistory] } def asAnteChrono: AnteChronoHistory = { anteChronoCheck(l) l.asInstanceOf[AnteChronoHistory] } }
  24. 24. Scala (2/2) implicit def reverseChrono(l: ChronoHistory): AnteChronoHistory = l.reve implicit def reverseAnteChrono(l: AnteChronoHistory): ChronoHistory = l.
  25. 25. Streaming Analytics
  26. 26. Selecting users on a path of Interest Massive discrepancy between # of users (2-3E6) and # of interesting users (1.5E3 on test segments) Filtering interesting time series.
  27. 27. Graph matching
  28. 28. Locality-sensitive hashing short histories A family H of hashing functions is -sensitive if:(r, cr, , )p1 p2 if then if then p–q ≤ r P [h(q) = h(p)] ≥rH p1 p–q ≥ cr P [h(q) = h(p)] ≤rH p2 More : Locality Sensitive Hashing By Spark, Uber, Spark Summit A Gentle Introduction to Locality-Sensitive Hashing with Apache Spark, Scala by The Bay
  29. 29. Computing speeds: Solving graph constraints a speed comes from a user well-positioned, twice plus route knowledge given a history of cells, where was the user, exactly ?
  30. 30. Solving graph constraints just a few users left in computation at this stage so a lot invested in > linear complexity algorithms
  31. 31. Data Challenges
  32. 32. Crucial elements Quality, reliability of data sources Automated ground truth checking sensors TEMS fleet What's the ground truth for mode of transport, domicile, etc ? Colleagues and friends volunteers
  33. 33. In the works Accuracy improvements More features (see you Spark Summit EU!) Streaming for city Thank you

×