Infrastructure of the social highway Preview

315 views

Published on

Published in: Technology, Sports
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
315
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Begin Dabbling with All-Things-Internet in 1993Unix, Linux, gopher, WAIS, telnet, ftp, MUDsFirst Webmaster job in 1994Co-founded an ISP in 1995Co-founded a Web consulting company in 1997Spent 8 years in the trenches at Classmates.comCurrently VP of Technology at Salad Labs (creator of the Pop Salad celebrity stock market game)Please note, if you have, or would like to download a QR reader, I will be sharing a URL at the end of the talk via QR code to the notes of all of the topics and references I make in this talk.
  • So to provide the greatest value, focus on what matters.
  • This goes contrary to “what are you, personally, and your organization generally, really good at?”How are you providing the utmost in value if you’re focusing limited resources on the full stack?
  • + You can be up and running in minutes+Low initial cost of entry (compared to building out your own data center and babysitting your own hardware)+ Spin up/down nodes as you need+Scaling out by adding nodes is easy (No waiting 4 weeks for the hardware to arrive and then spend time racking and installing software)+ Scaling up by adding disk/CPU can be easy (depending on the provider)No control over hosting, network, disk storage (hey, maybe this is a plus?)Over the long term, the big boys (aka Zynga) use public clouds for flash crowds, and use a hybrid/private cloud for cost management of baseline traffic- Shared components means less than optimal performance characteristics
  • But, for all the benefits of ACID, there are equivalent tradeoffs.Did you make your tradeoffs consciously?To scale traditional ACID databases requires sophisticated database technologies, and these come with a cost.Meanwhile, technologies have emerged over the past few years that rival the maturity and stability of ACID databases.
  • Some applications are better suited for graph structures.For example, a database storing all the evacuation routes for all major cities for the Federal Government… Key Value Architecture?Column Store Architecture?Document Store Architecture?EXAMPLE:Neo4J, VertexDB: A Graph Database architected around Nodes and Edges (objects and their relations)
  • Infrastructure of the social highway Preview

    1. 1. Infrastructure of the Social HighwayCreators of Khan Klatt, VP of Technology, Salad Labs, Inc.
    2. 2. #emmeconhttp://i.khan.org/emmecon-khan
    3. 3. AgendaAdding ValueTechnology Then & NowA Deeper Dive Technology Process PeopleParting Thoughts Photo Credit: cafeconlecheporfavor @ Flickr
    4. 4. The good news is that more than ever,value accrues to those that show up, those that make a difference, those that do work that matters. -Seth Godin Photo Credit: eat more toast@ Flickr
    5. 5. Key Differences between now & 19951. You had to do the full stack, or at least huge part of it.2. Computer hardware was expensive3. Huge lead times to order, receive, install and deploy.4. Programming languages, architectures and frameworks were not as mature and not as capable. Photo Credit: jacksnell707 @ Flickr
    6. 6. full stack
    7. 7. Logarithmic Scale! Storage vs. Memory/CPUMagnitude Time (Years)
    8. 8. full stack revisited SaaS = Software as a Service PaaS = Platform as a ServiceIaaS = Infrastructure as a Service Photo Credit: captpiper @ Flickr
    9. 9. expensive revisited A Few Thoughts on the Cloud (IaaS)Benefits: Detriments:• Speed • Less Control• Cost • Costly at large scale• Flexibility • Lower Performance• Scalability• Elasticity Photo Credit: paul-vallejo @ Flickr
    10. 10. “I Meant To Do That!” Photo Credit: estherase @ Flickr
    11. 11. If all you’ve got is a hammer… Ask yourself: What classes of data do you have? How consistent does it have to be? How available does it have to be? How up-to-date does it have to be? What’s the breakdown of reads to writes? What’s your tolerance for operations and DBA overhead? What’s your tolerance for engineering and development overhead? How much scalability do you need? Photo Credit: m2w2 @ Flickr
    12. 12. Scalability & The Art of Engine Maintenance at 20,000 feetAs the size and transaction volume of the database grows linearly,response times tend to grow logarithmically.
    13. 13. Typically (but not always) linear growth tends to be fairlyflat, meaning logarithmic scale is expensive.Cost Scale
    14. 14. Consider the Graph… Photo Credit: iconolith @ Flickr
    15. 15. Putting It All Together Web Server (nginex)Application Cache Web Server (memcached) (apache) Application Server (JVM, Ruby on Rails, Symfony) Batch Processing & Machine Learning (Hadoop, MapReduce, Mahout) Data Persistence (ACID, BASE, or Graph) Photo Credit: maynard @ Flickr
    16. 16. Sprint, Don’t Walk, to Agile*“The chemistry-building manager takes pains to divide the work into pieces and makes surethat each piece has some substantive demonstration of its own completion.Such a manager may contrive to deliver a product in twenty versions, even though two aresufficient for upper management and the user. It may even be necessary to conceal some ofthese interim versions from the client, and build them only for internal confirmation andsatisfaction.Each new version is an opportunity for closure. Team members get warmed up as the momentapproaches, they sprint near the very end. They get a high from success. It suffuses them withrenewed energy for the next step. It makes them feel closer together.” Tom DeMarco & Timothy Lister, PeopleWare, 1987 * Unless you are literally going to Mars! Photo Credit: lowercolumbiacollege @ Flickr
    17. 17. VP of Technology at Salad Labs, Inc. http://www.saladlabs.com http://www.popsalad.comPersonal Contact:• Twitter: @khanklatt• Blog: http://www.khan.org Creators of• LinkedIn: khanklatt Photo Credit: vernhart @ Flickr

    ×