Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 1 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 2 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 3 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 4 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 5 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 6 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 7 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 8 How Apache Spark Is Helping Tame the Wild West of Wi-Fi Slide 9
Upcoming SlideShare
Democratizing AI with Apache Spark
Next
Download to read offline and view in fullscreen.

1 Like

Share

Download to read offline

How Apache Spark Is Helping Tame the Wild West of Wi-Fi

Download to read offline

Spark Summit EU talk by Tomasz Magdanski

Related Books

Free with a 30 day trial from Scribd

See all

How Apache Spark Is Helping Tame the Wild West of Wi-Fi

  1. 1. HOW APACHE SPARK IS HELPING TAME THE WILD WEST OF WI-FI Tomasz Magdanski Director, Big Data and Analytics, iPass
  2. 2. Who Are We? iPass: the world’s largest Wi-Fi network • Global operations, Silicon Valley headquarters • On Nasdaq since 2003 • 40+ patents • 800 of the Fortune 2000 • Launched iPass SmartConnect™ in Fall 2015 2 57M+ HOTSPOTS 160+ NETWORK PROVIDERS 120+ COUNTRIES
  3. 3. Wi-Fi Is Unpredictable 3
  4. 4. So what’s the solution?
  5. 5. Spark & Databricks ● 21B scans -> 500M records -> 100M hotspots ● Spark helped us make sense of the data ● We needed a solution that can automatically scale and handle real time analytics
  6. 6. Spark: From Concept to Production • Past: – in-house prototyping – Spark 1.3 – RDD • Present: – AWS and Databricks – Spark 2.0 – Datasets – UDFs – Window aggregations – Full advantage of Tungsten and Catalyst
  7. 7. Building Wi-Fi Network Characteristics • Future: – Moving Hotspot – Changing SSID – Grouping and Graphframes - to find relationships • Most of our code is written in Scala notebooks • Ready to switch to structure streaming
  8. 8. Conclusions • Now we know • Thanks to Databricks platform – Smaller team - big result – Focus on building scalable business logic, not infrastructure • Small companies can successfully run big data projects without breaking the bank
  9. 9. THANK YOU. tmagdanski@ipass.com
  • bunkertor

    Nov. 9, 2016

Spark Summit EU talk by Tomasz Magdanski

Views

Total views

1,040

On Slideshare

0

From embeds

0

Number of embeds

119

Actions

Downloads

48

Shares

0

Comments

0

Likes

1

×