Sharing Slides Securely with 10,000 People in Real-Time : Socket.IO and Node.JS in Production

8,847

Published on

Cinchcast is an innovative new platform that enables large-scale conference calls and webcasts for your business. Our webcasting solution must be able to securely push out slides to tens of thousands of people while synchronizing with the audio stream. In this talk, we will discuss how we used NodeJS and Socket.IO to achieve this goal. Socket.IO has tremendous capabilities to overcome limitations of the web sockets in the browser ecosystem today. However, leveraging and scaling this framework can be both challenging and interesting across multiple servers: How do you keep state? Why do connections begin getting dropped? Is it really secure? We will answer all of these questions in this talk. We will focus on understanding Socket.IO scalability and high availability, and discuss some security pitfalls that you need to watch out for.


Aleksandr Yampolskiy is a CTO of Cinchcast, cloud-based conferencing solution for enterprises, and BlogTalkRadio, the world's largest online radio network averaging 36 million unique visitors a month. Prior to joining Cinchcast, Alex was a Head of Security and Compliance at Gilt Groupe companies, building their team from scratch to a team serving over 1300 employees. Before that, he has worked at Goldman Sachs, Oracle, and Microsoft, where he was a lead technologist building large-scale enterprise software focused on IDM, SSO, authentication and authorization. Aleksandr has been cited in New York Times, ComputerWorld, Observer, and other media. He speaks regularly about software development processes and security. He has a B.A. in Mathematics and Computer Science from New York University, and a Ph.D. in Computer Science from Yale University. He is also an organizer of NYC Technology Startup (a group of over 1400 entrepreneurs and developers) and NYC REDIS NOSQL meetups. In his spare time, he enjoys wandering New York museums, playing chess, martial arts, and public speaking.


Danny Gershman is a Principal Engineer at Cinchcast. He's spent over 16 years experience developing software and now focusing on R&D. Worked with various technologies, but currently I'm working with .NET, SQL Server, Redis, NodeJS, Socket.IO, jwPlayer. Also teller of bad jokes.

Published in: Technology

Sharing Slides Securely with 10,000 People in Real-Time : Socket.IO and Node.JS in Production

  1. 1. Sharing Slides Securely with 10,000 People in Real-Time : Socket.IO and Node.JS in ProductionAleksandr Yampolskiy and Danny Gershman
  2. 2. Who Are We? • Aleksandr Yampolskiy, CTO @ayampolskiy (alexyampolskiy@cinchcast.com) • Previously head of security and compliance for Gilt Groupe companies, in charge of securing IT infrastructure, secure architecture, PCI/SOX compliance, etc. • Various leadership roles in Goldman Sachs, Oracle, Microsoft building scalable, enterprise software for IDM, SSO, AuthN/AuthZ. • Ph.D. in Distributed Computing • Hobbies: chess, Edward Hopper, Ray Bradbury, martial arts, lately foosball and coffee. • Danny Gershman @dannygnj, Principal Engineer (dannygershman@cinchcast.com) • Hes spent over 16 years experience developing software and now focusing on R&D. • Worked with various technologies such .NET, SQL Server, Redis, NodeJS, Socket.IO, jwPlayer, Liquid Office, Teleform, Ascent Capture, Classic ASP, and GWBASIC and DOS • Paintballer, DJ, Cat-Lover, and from New Jersey (hold the applause please)
  3. 3. Cinchcast, Inc. Patented, Cloud-Based PlatformCinchcast, an enterprise BlogTalkRadio, atechnology company, consumer mediaprovides a cloud-based company, is the largestsolution for conference online radio network incalls and webcasts. the world.
  4. 4. Cinchcast Connect Enhancing internal and external corporate communications while significantly reducing associated costs Marketing Events Earnings/Analyst Calls Executive Communications Employee Townhalls All-Hands Meetings Team Meetings Training4
  5. 5. DEMO TIME
  6. 6. Challenge• Security for sensitive conversations• Real-time update of slides and analytics• 10,000s or more participants on various devices, including older browsers• No browser plugins + Minimal bandwidth (12 MB/hr)
  7. 7. How Does It All Work? Cinchcast SaaS NodeJS server cluster
  8. 8. What Do We Use Node.JS+Socket.IO for?• Keeping track of real-time listeners on the permalink page• Pushing the slide notifications to thousands of viewers in real-time!
  9. 9. Node.JS• Node.JS = Javascript on your server.• Asynchronous event loop.
  10. 10. NodeJS Security Issues• Perennial input validation issues – Rulle #1 – validate thy input – Never assume the input is well-formed. Think like a hacker!• JSON eval – JSON.parse(str) vs eval(str) – var queryData = url.parse(req.url, true).query; – Eval(“console.log(„”+queryData.log+”‟)”); – what if I call http://127.0.0.1/?log=1‟);var sys=require(„sys‟); var exec=require(„child_process‟).exec;function puts=….• An unhandled exception can crash your server
  11. 11. Example - XSS
  12. 12. Socket.IO• Socket.IO = Persistent client-server connection, cross- browser compatible. handshake Handshake accepted , transports, connection id, config
  13. 13. Socket.IO Security Issues• Communication in ws:// protocol is unencrypted.• Don‟t trust the client! All origins are allowed by default.• Have to build your own authentication/authorization (https://github.com/LearnBoost/socket.io/wiki/Authorizing )
  14. 14. Example – Origin• Malicious client by Krysztof Kotowicz (https://github.com/koto/socket_io_client)• It can handshake with socket.io server, ignore origin restrictions, handle heartbeats, fuzz messages
  15. 15. What’s Different About Node.JS+Socket.IOSecurity?• More code and complexity in Node.JS/Socket.IO apps.• We now need to review client-side and server-sidecode.• Dynamic, agile development approach results in codethat’s not thoroughly tested• Complicated UI frameworks may contain their ownsubtle security bugs• New security attacks
  16. 16. What’s Different About Web 2.0 Security?• Web 2.0 has completely new app security threats – Malicious AJAX code execution – WSDL scanning and enumeration – RSS injection – XML poisoning – CSRF attacks
  17. 17. Relax, it’s not that bad!
  18. 18. Web 2.0 Security Reality• Fundamentals are still the same, for Web 1.0 and Web 2.0, and fornode.js+socket.io apps.• Multilayered “onion security”.• None of the “new” attacks appear on OWASP top 10 list of securitybugs.• In fact, Verizon 2009 data breach report lists top data breach causesas - Weak or default passwords - SQL injection attacks - Improper access rights - XSS attacks
  19. 19. Our Approach• Security decisions are based on risk, not just threats and vulnerabilities (risk = threat*vulnerability*cost).• Don‟t chase hot vulnerabilities of the day. Instead, mitigate top risks.• AAA and least privilege principle.• Heavily based on policy and user education.• “Onion security” – multiple protections at each layer.• Achieve “essential”, then worry about “excellent”.• Be a “how team” instead of a “no team”.• Build security into the software development lifecycle.
  20. 20. What Do We Do To Protect?HMAC-SHA1 digest authentication based off rooms anduser type. ACLs are applied one authenticated.
  21. 21. What Do We Do To Protect?• Secure Web-sockets
  22. 22. Multi-core
  23. 23. Multi-core
  24. 24. Multi-server• Wait…how will we share data?
  25. 25. Session Data in Socket.IO
  26. 26. Of CourseThe Greatest Session Store of All Time Is…
  27. 27. Sharing Session
  28. 28. Redis Store for Socket.IO
  29. 29. Storing / Retrieving Data in Session withRedis
  30. 30. This can start getting out of hand
  31. 31. Matryoshka Code
  32. 32. Async (https://github.com/caolan/async)
  33. 33. Eventing Across Nodes (Pub-Sub)
  34. 34. Multi-serverEach process gets its own port, then individually exposed via a load balancerwith a virtual IP. Uses Layer 4 level proxying and SSL certificate is on the loadbalancer. 3001 3002 3001 3002 3003 3004 3003 3004 192.168.1.100 192.168.1.101 Load Balancer 129.186.73.100
  35. 35. Time Sync
  36. 36. Failback with Upstart and Monit
  37. 37. Gotchas
  38. 38. Conclusion• Security problems may be new but old principles apply• Validate thy input• HMAC-SHA1 digest authentication• Know the gotchas for multitasking (time sync, ulimits, data sharing, etc.)• We will tweet the slides link.• Talk to us @ayampolskiy or @dannygnj
  39. 39. PITCH: Use Us For Large-Scale Conference CallsContact us at http://cinchcast.com/contact/

×