Successfully reported this slideshow.
Your SlideShare is downloading. ×

How Apache Spark Is Helping Tame the Wild West of Wi-Fi

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 9 Ad
Advertisement

More Related Content

Slideshows for you (20)

Viewers also liked (19)

Advertisement

Similar to How Apache Spark Is Helping Tame the Wild West of Wi-Fi (20)

More from Spark Summit (20)

Advertisement

Recently uploaded (20)

How Apache Spark Is Helping Tame the Wild West of Wi-Fi

  1. 1. HOW APACHE SPARK IS HELPING TAME THE WILD WEST OF WI-FI Tomasz Magdanski Director, Big Data and Analytics, iPass
  2. 2. Who Are We? iPass: the world’s largest Wi-Fi network • Global operations, Silicon Valley headquarters • On Nasdaq since 2003 • 40+ patents • 800 of the Fortune 2000 • Launched iPass SmartConnect™ in Fall 2015 2 57M+ HOTSPOTS 160+ NETWORK PROVIDERS 120+ COUNTRIES
  3. 3. Wi-Fi Is Unpredictable 3
  4. 4. So what’s the solution?
  5. 5. Spark & Databricks ● 21B scans -> 500M records -> 100M hotspots ● Spark helped us make sense of the data ● We needed a solution that can automatically scale and handle real time analytics
  6. 6. Spark: From Concept to Production • Past: – in-house prototyping – Spark 1.3 – RDD • Present: – AWS and Databricks – Spark 2.0 – Datasets – UDFs – Window aggregations – Full advantage of Tungsten and Catalyst
  7. 7. Building Wi-Fi Network Characteristics • Future: – Moving Hotspot – Changing SSID – Grouping and Graphframes - to find relationships • Most of our code is written in Scala notebooks • Ready to switch to structure streaming
  8. 8. Conclusions • Now we know • Thanks to Databricks platform – Smaller team - big result – Focus on building scalable business logic, not infrastructure • Small companies can successfully run big data projects without breaking the bank
  9. 9. THANK YOU. tmagdanski@ipass.com

×