Your SlideShare is downloading. ×
From concept to cloud a look at modern software development
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

From concept to cloud a look at modern software development


Published on

La computadora HAL 9000 es uno de los personajes centrales de "2001 Odisea del Espacio". Considerando que esta historia fue escrita en 1968, podemos decir que la tecnología ha tenido un tremendo …

La computadora HAL 9000 es uno de los personajes centrales de "2001 Odisea del Espacio". Considerando que esta historia fue escrita en 1968, podemos decir que la tecnología ha tenido un tremendo avance desde entonces. Y nuestra habilidad para crear software también ha mejorado un poco. Pero todavía estamos muy lejos de poder construir algo como HAL 9000. En esta conferencia echaremos un vistazo a las principales tendencias en desarrollo de software hacia los próximos años.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. From concept to cloud:A look at modern software development From concept to cloud: Chris Richardson A look Author original atthe of POJOs in Action Founder of modern software development @crichardson Chris Richardson Author of POJOs in Action, Founder of the original, @crichardson
  • 2. About Chris
  • 3. (About Chris)
  • 4. About Chris()
  • 5. About Chris
  • 6. About Chris
  • 7. vmc push About-Chris Developer Advocate forSignup at Promo code: sgce2012
  • 8. Dreams of my childhood Published in 1968
  • 9. The vision HAL 9000 .... is an artificial intelligence became operational on 12 January 1992.....So how are we doing?
  • 10. The reality Floor cleaning robot
  • 11. The reality cluster of ninety IBM Power 750 servers with a total of 2880 POWER7 processor cores and 16 Terabytes of RAM.
  • 12. We still have a long way to go before we can build HAL But despite that....
  • 13. Today, a small team of people can easily build an application that’s used by millions of people world-wide system-of-the-world-for-it.html
  • 14. Endpoints everywhere
  • 15. The web
  • 16. Cloud Computing
  • 17. How did we get here?
  • 18. Many of the fundamentals of computing wereinvented before many of us were born
  • 19. Computer hardware
  • 20. 1948 - First computer2010 30 tons2000 167 m21990 150 KW1980 40 divs/second197019601950
  • 21. 1956 - Hard drive 3.7 Mbit Weighed over a ton Designed to fit through a door
  • 22. 1971 - microprocessor 4 bit 108-740 kHz 2,300 transistors
  • 23. The power of Moore’s law
  • 24. Hard disk capacity
  • 25. In my career 1982 RM 380Z 4 Mhz 8-bit Z80 32K RAM 100K floppy2012MacBookPro2.5 Ghz Quad core 64 bit Intel i7256K/core + 8M cache,16G RAM512G SSD drive
  • 26. Machrone’s law “...the price of thecomputer you want to buy will always be $5000*...” * Today it’s $3000
  • 27. Wirth’s law“Software is getting slower more rapidly than hardware becomes faster.”
  • 28. Programming languages
  • 29. 1958 - Lisp2010 (defun factorial (n) garbage collection2000 (if (<= n 1) dynamic typing1990 1 self-hosting compiler1980 (* n (factorial (- n 1))))) tree data structures197019601950 Lisp_(programming_language)1940
  • 30. 1960 - Algol 60 procedure Absmax(a) Size:(n, m) Result:(y) Subscripts:(i, k);2010 value n, m; Many array a; integer n, m, i, k; real y;2000 begin integer p, q; languages y := 0; i := k := 1; for p:=1 step 1 until n do including Java1990 for q:=1 step 1 until m do are derived if abs(a[p, q]) > y then1980 begin y := abs(a[p, q]); from Algol i := p; k := q1970 end end Absmax1960
  • 31. 1967 - Simula20102000 class-based object-1990 oriented programming198019701960
  • 32. So what have programming language designers been doing since then?
  • 33. 1995 - Java Not innovative BUT201020001990 Brought garbage collection, object- oriented programming, exception1980 handling, safety ... to mainstream1970 developers19601950 Gosling “Java is a blue collar google “Gosling the feel of Java” language”1940
  • 34. Tools and techniques
  • 35. 1980s - Cool IDEs for Lisp and Smalltalk2010 Editor2000 Class1990 List1980 Repl19701960 Class hierarchy1950 Debugger1940
  • 36. 1990s2010 • CVS - Lockless version management2000 • DVCS - distributed version control1990 • Refactoring1980 • Agile/XP starts to become popular1970 • Mosaic web browser1960 • Web crawlers19501940
  • 37. 2000s - online developer community • Explosion of open-source2010 • Binary artifact repositories - 41K maven.org200019901980 • social coding - 1M projects!1970196019501940
  • 38. 2000s - DevOps, NoOps, Continuous Deployment, etc2010 • Developers responsible for testing,2000 operations, ...1990 • Continuous deployment: update production1980 many times a day • Chaos monkeys - constantly verify that the19701960 system can tolerate failure1950
  • 39. William Gibson “The future is already here –its just not evenly distributed.” Ideas take a long time to diffuse through the software community
  • 40. Where is here?
  • 41. Polyglot applications NoSQL databases Cloud Computing
  • 42. Until ~2004 Java wasthe one language to rule them all
  • 43. Some developersbecame frustrated with the perceived complexity of Java
  • 44. Ruby on Rails became popular• Web application framework for Ruby• Significantly more productive• Simplicity of Convention over Configuration• Motivated the Java community to improve: • e.g. Spring became even easier to use But...
  • 45. …Highlighted problems with Java • Tedious collection processing • Painful object construction code • Tedious XML processing • Types: verboseness and complexity of generics • Limited support for DSLsReopened the debate about programming languages
  • 46. Alternative languages• Ruby• Groovy• Clojure• Scala• JavaScript/NodeJS
  • 47. Closuresscala> val numbers = Array(1,2,3,4,5)numbers: Array[Int] = Array(1, 2, 3, 4, 5)scala> val oddNumbers = numbers.filter((x => x % 2 == 1))oddNumbers: Array[Int] = Array(1, 3, 5)scala> val doubled = * 2)doubled: Array[Int] = Array(2, 4, 6, 8, 10)
  • 48. XML processingscala> val xml = <foo><bar>baz</bar></foo>xml: scala.xml.Elem = <foo><bar>baz</bar></foo>scala> (xml "bar").textres9: String = “baz”
  • 49. Meta-programmingComputer programs that modify themselves
  • 50. Grails persistence methodsclass Customer { String name} Customer c = new Customer("John Doe")   if (! fail "validation failed: ${c.errors}"   Customer c2 = Customer.get(   def customers = Customer.findAllByName(“Fred”)
  • 51. Sinatra DSL example require sinatra get /hi do "Hello World!" endA Ruby-based Domain Specific Language for web programming
  • 52. So what should we dowith these languages?
  • 53. Traditional web application architecture WAR StoreFrontUI Billing Service Customer MySQLBrowser Apache Management Database Inventory ServiceSimple to Shipping Service develop Tomcat test deploy scale
  • 54. But there are problems• Inadequate for modern, real-time web applications (need NodeJS)• Obstacle to frequent deployments• Slows down IDE and dev/test cycle• Obstacle to scaling development• Requires long term commitment to technology stack
  • 55. Modular, polyglot application architecture Desktop Browser Native Mobile application HTML5 mobile application StoreUI StoreUI StoreUI Asynchronous, NodeJS Javascript scalable StoreUI communication RabbitMQ Spring/ Rails Inventory Scala web Shipping Service Service application Customer Inventory Order Billing Service Management Database Database Standalone “headless” Customer Spring/Java Database applications
  • 56. Real world examples Between 100-150 services are accessed to build a page. eBaySDForum2006-11-29.pdf
  • 57. There are drawbacks• Complexity: Architectural, development, deployment• Deciding when to use it • In the beginning: you don’t need it and it will slow you down • When you do need it: refactoring existing code is painful
  • 58. But there are many benefits• Scales development: focused two pizza devops teams • Deploy services independently • Scale services independently• Improves fault isolation• Enforces well defined interfaces between components• Eliminates long-term commitment to a single technology stackModular, polyglot applications
  • 59. If services are small...• Regularly rewrite using a better technology stack• Pick the best developers rather than best <pick a language> developers polyglot culture• Adapt system to changing requirements and better technology without a total rewrite
  • 60. Moreover: you are not the same you ...• 50 to 70 billion of your cells die each day Can we build software• Cell lifetimes: systems with these • hours - some white blood cells characteristics? • days - stomach lining cells • years - bone cells Too much technical debt • lifetime - brain cells component death?• Yet you (the system) remains intact DesignBeyondHumanAbilitiesSimp.pdf
  • 61. Polyglot applications NoSQL databases Cloud Computing
  • 62. Relational databaseshave been the place to store your data
  • 63. Limitations of relational databases• Scaling • Despite Moore’s law vertical scaling is limited/expensive • Horizontal scaling is limited• Distribution• Updating schema• O/R impedance mismatch• Handling semi-structured data
  • 64. Solution: Use NoSQL Benefits Drawbacks• Higher performance • Limited transactions• Higher scalability • Relaxed consistency• Richer data-model • Unconstrained data• Schema-less
  • 65. NoSQL databasesData Model ExamplesKey-value Redis,VoldemortDocument MongoDB, CouchDbExtensible columns/Column- Cassandra, Hbase, SimpleDB,oriented DynamoDBGraph Neo4j lists 122+ NoSQL databases
  • 66. Redis• Advanced key-value store K1 V1• Very fast K2 V2• Optional persistence K3 V2
  • 67. MongoDB: fast, scalable, document oriented Server Database: Food To Go Collection: Restaurants { "_id" : ObjectId("4bddc2f49d1505567c6220a0") "name": "Ajanta", "serviceArea": ["94619", "99999"], BSON = binary "openingHours": [ { JSON "dayOfWeek": 1, "open": 1130, "close": 1430 }, Sequence of { bytes on disk "dayOfWeek": 2, "open": 1130, è fast i/o "close": 1430 }, … ] }
  • 68. Cassandra: very scalable Application ApplicationCassandra cluster Cassandra cluster Node 1 Node 1 Node 4 Node 2 Node 4 Node 2 Node 3 Node 3 Datacenter 1 Datacenter 2
  • 69. The future is polyglot persistence e.g. Netflix • RDBMS • SimpleDB • Cassandra • Hadoop/HbaseIEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg
  • 70. Polyglot applications NoSQL databases Cloud Computing
  • 71. Let’s imagine that you want to deploy an app...• Do you know how much hardware to buy?• Can you afford it?• How long does it take to approve, buy and install?• Who is going to set it up?• What happens if the traffic increases 10x?• Can we afford a test lab?
  • 72. Cloud computing empowers us to dealwith these challenges...
  • 73. Cloud computing defined IT delivered as a service Over the internet Self-service Pay per use
  • 74. The three layers of cloud computingSaaS PaaS IaaS
  • 75. Wednesday October 18, 2006....AWS evangelist .... spoke at the Oakland JUG
  • 76. Sign up and deploy yourapplication a few minutes later • Login using your existing Amazon account • Select the web
  • 77. Benefits of IaaS • Agility • Pay per use • Elasticity • Focus on deploying your applicationBut you need to configure and maintain the operating systems, the app servers and the databases etc.
  • 78. Need to move up the stack SaaS PaaS IaaS
  • 79. PaaS =Application deployment and management +Service provisioning
  • 80. The Open Platform as a Service Deploy and scale applications in seconds,without locking yourself into a single cloud Flex ible, pen, le, O lable Simp Sca
  • 81. Easy polyglot application deployment and service provisioning OSS community vFabricPostgres Private   Ap Clouds   p Data Services lica o n  S vFabric Public erv RabbitMQTM ice Msg Services Clouds  In ter fac Micro e Other Services Clouds Additional partners services …
  • 82. CloudFoundry.COM - Multi-tenant PaaS operated by VMware CloudFoundry.COM (beta) Services Runtimes & Frameworks vCenter / vSphere Infrastructure
  • 83. Micro Cloud Foundry – Industry first TM downloadable PaaS Micro Cloud Foundry Services Runtimes & Frameworks Your Laptop/PC Single VM instance of Cloud Foundry that runs on a developer’s MAC or PC
  • 84. CloudFoundry.ORG - Community open-source project Apache2 license CloudFoundry.ORG Your Infrastructure Download Setup Deploy Behind Code Environment BOSH Firewall
  • 85. Sinatra + Redisrequire sinatrarequire redis Connect to Redisconfigure do $r = =>, :port => 6379) if !$rendget / do "Hello World! " + $r.incr("hitcounter").to_send Increment hit counter
  • 86. Consuming 3rd party cloud services
  • 87. Thousands of 3rd party services Number of APIs increasing exponentially
  • 88. Cloud service trends• Predominantly REST• Predominantly JSON• > billion API calls/day: Twitter, Google, Facebook, Netflix, Accuweather, ...• Increasing number of API-only companies
  • 89. Diverse Paypal SendGrid GoogleCybersource Twilio GeoNames YahooAlchemyAPI Tropo WorldAddresses Amazon Mashery
  • 90. Amazon Simple Storage Service (S3)• Bucket • has a globally unique name, e.g. • contains objects• Objects - 1 byte – 5 Tbytes• Pay per use: storage + bandwidth• Accessed via RESTful API • PUT – create bucket or object • GET – retrieve object 1T objects • DELETE – delete an object June 2012
  • 91. - face detection and recognition /detect.json?api_key=&api_secret=&urls=<url>&detector=Aggressive&attributes=all
  • 92. Twilio - Telephony and SMS as a service • REST API • Allocate phone numbers • Make and receive phone calls • Send and receive SMS messages • Pay per use: • Phone number – per month • Phone calls - per-minute • SMS – per SMS sent or received • Example: Salesforce – SMS-based voting for 19,000 conference attendees
  • 93. Expose your APIs andbuild an ecosystem
  • 94. Where are we going?
  • 95. Going beyond today’s pop culture But love is blind and lovers cannot see The pretty follies that themselves commit; For if they could, Cupid himself would blush To see me thus transformed to a boy. William Shakespeare, The Mechant of Venice
  • 96. Gartner hype curve
  • 97. Concurrency
  • 98. Clock speeds have plateaued
  • 99. Multi core is the future• Dual core A5 for iPhone 4S• Quad core Intel i7 for laptops• 10 core Intel Xeon Processor• 3072 core NVidia Tesla Kepler
  • 100. Sometimes we can use request- level parallelism and let the container worry about it BUT not always...
  • 101. But concurrentprogramming is difficult
  • 102. The problem = Mutable state that’sshared between multiple threads (using locks)
  • 103. Not thread safepublic class InventoryTracker { private int inStock; private int sold = 0; public InventoryTracker(int initialInventory) { this.inStock = initialInventory; } public int getInStock() { return inStock; } public int sell(int units) { inStock -= units; sold += units; return inStock; }}
  • 104. Thread safepublic class InventoryTracker { private int inStock; private int sold = 0; public InventoryTracker(int initialInventory) { this.inStock = initialInventory; } public synchronized int getInStock() { return inStock; } public synchronized int sell(int units) { inStock -= units; sold += units; return inStock; }}
  • 105. NodeJS - Event driven• Single threaded runtime no concurrency• Runs an event loop that waits for i/o events• i/o event callback function • Updates state • Publishes events triggers more callbacks) • Initiates i/o, registering callbacks Mutable state that’s • Returns shared between multiple threads
  • 106. NodeJS examplevar http = require(http);var fs = require(fs); HTTP request handlervar url = require(url);var inStock = 100;var sold = 0;http.createServer(function (req, res) { var quantity = parseInt(url.parse(req.url, true).query.quantity); inStock -= quantity; sold += quantity; var message = "inStock=" + inStock + ", sold=" + sold; res.end(message);}).listen(1337, "");console.log(Server running at;
  • 107. Scala Actors• Actor = object that asynchronously exchanges immutable messages with other actors• Actor processes one message at a time • Sends messages • Updates its private mutable state • Creates new actors• No shared state Mutable state that’s shared between multiple threads
  • 108. Scala examplecase class Sell(quantity: Int) Message typescase class Sale(inStock : Int, sold : Int)class InventoryTracker(initialInventory : Int) extends Actor { var inStock = initialInventory var sold = 0 def receive = { case Sell(quantity) => Update state inStock -= quantity sold += quantity sender ! Sale(inStock, sold)} } Send reply case class Order(quantity: Int) class StoreFront(inventoryTracker: ActorRef) extends Actor { var numberOfOrders = 0 def receive = { case Order(quantity) => Send message numberOfOrders += 1 inventoryTracker ! Sell(quantity) case Sale(inStock, sold) => println("numberOfOrders=" + numberOfOrders + ",inStock=" + inStock + ", sold=" + sold) Handle reply } }
  • 109. Pure functional programming• Eliminates mutable state BUT• I/O and state management require Monads• Monads are extremely difficult to learn Mutable state that’s shared between multiple threads
  • 110. Clojure - a modern LISP• Practical functional programming language• Runs on the JVM• Core data structures are immutable BUT• Ref = object that holds a mutable reference to a value• Refs can only be updated within a transaction • Software Transaction Memory • Atomic, Consistent, Isolated Mutable state that’s • No explicit locks shared between multiple threads
  • 111. Clojure STM example Create refs(defrecord InventoryTracker [in-stock sold])(defn make-inventory-tracker [initial-inventory] (InventoryTracker. (ref initial-inventory) (ref 0))) Update refs(defn sale [tracker quantity] (dosync (let [new-in-stock (alter (:in-stock tracker) - quantity) new-sold (alter (:sold tracker) + quantity) ] [new-in-stock new-sold] ))) dosync(def it (make-inventory-tracker 100)) = a transaction(sale it 25)
  • 112. Final thoughtsSoftware development is an old... yet immature professionMulti-core programming is a challenge BUTThe cloud enables anyone with a good idea tocreate an application that touches the lives ofmillions of people
  • 113. Thank you! Chris Richardson Author of POJOs in Action Founder of the original @crichardson plainoldobjects.comSignup @ Promo code: sgce2012