The document summarizes key differences in technology infrastructure between 1995 and the present day. It notes that in 1995, developers had to build full technology stacks themselves as computer hardware was expensive and took a long time to deploy. Programming languages and frameworks were less mature then compared to now. The document then discusses how cloud computing (IaaS), platform as a service (PaaS), and software as a service (SaaS) have changed the technology landscape by reducing costs and deployment times while increasing flexibility and scalability. It emphasizes choosing the right data storage solution based on factors like data consistency needs and read/write volumes. The document concludes by discussing agile development practices.
3. Agenda
Adding Value
Technology Then & Now
A Deeper Dive
Technology
Process
People
Parting Thoughts
Photo Credit:
cafeconlecheporfavor @ Flickr
4. The good news is that more than ever,
value accrues to those that show up,
those that make a difference,
those that do work that matters.
-Seth Godin
Photo Credit:
eat more toast@ Flickr
5. Key Differences between now & 1995
1. You had to do the full stack, or at least huge part of it.
2. Computer hardware was expensive
3. Huge lead times to order, receive, install and deploy.
4. Programming languages, architectures and frameworks were not as mature
and not as capable.
Photo Credit:
jacksnell707 @ Flickr
8. full stack revisited
SaaS = Software as a Service
PaaS = Platform as a Service
IaaS = Infrastructure as a Service
Photo Credit:
captpiper @ Flickr
9. expensive revisited
A Few Thoughts on the Cloud (IaaS)
Benefits: Detriments:
• Speed • Less Control
• Cost • Costly at large scale
• Flexibility • Lower Performance
• Scalability
• Elasticity
Photo Credit:
paul-vallejo @ Flickr
10. “I Meant To Do That!”
Photo Credit:
estherase @ Flickr
11. If all you’ve got is a hammer…
Ask yourself:
What classes of data do you have?
How consistent does it have to be?
How available does it have to be?
How up-to-date does it have to be?
What’s the breakdown of reads to writes?
What’s your tolerance for operations and DBA overhead?
What’s your tolerance for engineering and development overhead?
How much scalability do you need?
Photo Credit:
m2w2 @ Flickr
12. Scalability & The Art of Engine
Maintenance at 20,000 feet
As the size and transaction volume of the database grows linearly,
response times tend to grow logarithmically.
13. Typically (but not always) linear growth tends to be fairly
flat, meaning logarithmic scale is expensive.
Cost
Scale
15. Putting It All Together
Web Server
(nginex)
Application Cache
Web Server
(memcached)
(apache)
Application Server
(JVM, Ruby on Rails, Symfony)
Batch Processing & Machine Learning
(Hadoop, MapReduce, Mahout)
Data Persistence
(ACID, BASE, or Graph)
Photo Credit:
maynard @ Flickr
16. Sprint, Don’t Walk, to Agile*
“The chemistry-building manager takes pains to divide the work into pieces and makes sure
that each piece has some substantive demonstration of its own completion.
Such a manager may contrive to deliver a product in twenty versions, even though two are
sufficient for upper management and the user. It may even be necessary to conceal some of
these interim versions from the client, and build them only for internal confirmation and
satisfaction.
Each new version is an opportunity for closure. Team members get warmed up as the moment
approaches, they sprint near the very end. They get a high from success. It suffuses them with
renewed energy for the next step. It makes them feel closer together.”
Tom DeMarco & Timothy Lister, PeopleWare, 1987
* Unless you are literally going to Mars! Photo Credit:
lowercolumbiacollege @ Flickr
17. VP of Technology at Salad Labs, Inc.
http://www.saladlabs.com
http://www.popsalad.com
Personal Contact:
• Twitter: @khanklatt
• Blog: http://www.khan.org Creators of
• LinkedIn: khanklatt
Photo Credit:
vernhart @ Flickr
Editor's Notes
Begin Dabbling with All-Things-Internet in 1993Unix, Linux, gopher, WAIS, telnet, ftp, MUDsFirst Webmaster job in 1994Co-founded an ISP in 1995Co-founded a Web consulting company in 1997Spent 8 years in the trenches at Classmates.comCurrently VP of Technology at Salad Labs (creator of the Pop Salad celebrity stock market game)Please note, if you have, or would like to download a QR reader, I will be sharing a URL at the end of the talk via QR code to the notes of all of the topics and references I make in this talk.
So to provide the greatest value, focus on what matters.
This goes contrary to “what are you, personally, and your organization generally, really good at?”How are you providing the utmost in value if you’re focusing limited resources on the full stack?
+ You can be up and running in minutes+Low initial cost of entry (compared to building out your own data center and babysitting your own hardware)+ Spin up/down nodes as you need+Scaling out by adding nodes is easy (No waiting 4 weeks for the hardware to arrive and then spend time racking and installing software)+ Scaling up by adding disk/CPU can be easy (depending on the provider)No control over hosting, network, disk storage (hey, maybe this is a plus?)Over the long term, the big boys (aka Zynga) use public clouds for flash crowds, and use a hybrid/private cloud for cost management of baseline traffic- Shared components means less than optimal performance characteristics
But, for all the benefits of ACID, there are equivalent tradeoffs.Did you make your tradeoffs consciously?To scale traditional ACID databases requires sophisticated database technologies, and these come with a cost.Meanwhile, technologies have emerged over the past few years that rival the maturity and stability of ACID databases.
Some applications are better suited for graph structures.For example, a database storing all the evacuation routes for all major cities for the Federal Government… Key Value Architecture?Column Store Architecture?Document Store Architecture?EXAMPLE:Neo4J, VertexDB: A Graph Database architected around Nodes and Edges (objects and their relations)