You're Going To Need A Bigger Toolbox

3,128 views
3,025 views

Published on

Talk from Design It Build It conference in Newcastle. Convincing people to look outside their current development tools for inspiration

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,128
On SlideShare
0
From Embeds
0
Number of Embeds
384
Actions
Shares
0
Downloads
60
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

You're Going To Need A Bigger Toolbox

  1. 1. You’re Going To Need A Bigger Toolbox DIBI 28th April 2010 gareth rushgrove | morethanseven.net http://www.flickr.com/photos/booleansplit/2376359338/
  2. 2. morethanseven.net Gareth Rushgrove gareth rushgrove | morethanseven.net
  3. 3. We Used To Just Build Websites gareth rushgrove | morethanseven.net
  4. 4. Probably Simple Websites gareth rushgrove | morethanseven.net
  5. 5. Maybe the odd Ecommerce Site gareth rushgrove | morethanseven.net
  6. 6. Then We Built Web Applications gareth rushgrove | morethanseven.net
  7. 7. Now We Just Build. gareth rushgrove | morethanseven.net
  8. 8. Your Toolbox is Growing gareth rushgrove | morethanseven.net http://www.flickr.com/photos/gordonr/42555739
  9. 9. 1 Embrace Heterogeneous Environments gareth rushgrove | morethanseven.net http://www.flickr.com/photos/mk1971/2548492513/
  10. 10. 1 No Development Tool Silver Bullet gareth rushgrove | morethanseven.net http://www.flickr.com/photos/mk1971/2548492513/
  11. 11. We Build Websites With... gareth rushgrove | morethanseven.net http://www.flickr.com/photos/freefoto/3436970425/
  12. 12. PHP, MySQL, Apache gareth rushgrove | morethanseven.net
  13. 13. .NET, MSSQL, IIS gareth rushgrove | morethanseven.net
  14. 14. Java, Oracle, Tomcat gareth rushgrove | morethanseven.net
  15. 15. Lots of Others gareth rushgrove | morethanseven.net
  16. 16. Just One Stack? gareth rushgrove | morethanseven.net http://www.flickr.com/photos/harrygoldenfeld/4263710200/
  17. 17. The Guardian Java, Python, Oracle, App Engine gareth rushgrove | morethanseven.net
  18. 18. LastFM PHP, C++, Java, Hadoop, Python gareth rushgrove | morethanseven.net
  19. 19. GitHub Ruby, Erlang, MySQL, Redis, Sinatra gareth rushgrove | morethanseven.net
  20. 20. Twitter Ruby, Scala, Java, C/C++ gareth rushgrove | morethanseven.net
  21. 21. Facebook PHP, Erlang, C, MySQL, Cassandra gareth rushgrove | morethanseven.net
  22. 22. 2 2. Know When Your Stack is out of its Depth gareth rushgrove | morethanseven.net http://www.flickr.com/photos/dk_spook/2421009077/
  23. 23. What Should You Know About? gareth rushgrove | morethanseven.net http://www.flickr.com/photos/takomabibelot/220265100/
  24. 24. wiki.nginx.org Serve Web Pages With Nginx gareth rushgrove | morethanseven.net
  25. 25. “Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and it does five of them 50 times faster than Apache. Chris Lea Why Nginx gareth rushgrove | morethanseven.net
  26. 26. server { listen 80; server_name www.example.com; location / { root /var/www/example.com; } } Nginx Example gareth rushgrove | morethanseven.net
  27. 27. http { upstream php { server localhost:8002; } upstream python { server localhost:8003; } } server { server_name www.example.com; location / { proxy_pass http://python; } location ~ /basket/* { proxy_pass http://php; } } Nginx Example gareth rushgrove | morethanseven.net
  28. 28. - Thin - http://code.macournoyer.com/thin/ - Mongrel - http://github.com/fauna/mongrel/ - Spawning - http://pypi.python.org/pypi/Spawning/ - Unicorn - http://github.com/defunkt/unicorn/ Also See gareth rushgrove | morethanseven.net
  29. 29. memcached.org Caching with Memcached gareth rushgrove | morethanseven.net
  30. 30. from django.core.cache import cache key = "/about/" content = cache.get(key) if not content: # expensive query to get content cache.set(key, content, 300) Memcached Example gareth rushgrove | morethanseven.net
  31. 31. - Squid - http://www.squid-cache.org - Varnish - http://varnish-cache.org Also See gareth rushgrove | morethanseven.net
  32. 32. lucene.apache.org/solr/ Search with Solr gareth rushgrove | morethanseven.net
  33. 33. http://solr:8983/solr/products/select/ ?q=colour:red &sort=price%20desc &rows=60 &wt=json Solr Example gareth rushgrove | morethanseven.net
  34. 34. - Xapian - http://xapian.org - Sphinx - http://sphinxsearch.com - Nutch - http://lucene.apache.org/nutch/ - Whoosh - http://whoosh.ca Also See gareth rushgrove | morethanseven.net
  35. 35. rabbitmq.com Asynchronous Processing with RabbitMQ gareth rushgrove | morethanseven.net
  36. 36. require 'carrot' q = Carrot.queue('carrot', :durable => true) 10.times do |num| q.publish(num.to_s) end RabbitMQ Enqueue Example gareth rushgrove | morethanseven.net
  37. 37. puts "count: #{q.message_count}" while msg = q.pop(:ack => true) puts msg q.ack end Carrot.stop RabbitMQ Dequeue Example gareth rushgrove | morethanseven.net
  38. 38. - Apache ActiveMQ - http://activemq.apache.org - Beanstalk - http://kr.github.com/beanstalkd/ - Stomp Server - http://stomp.codehaus.org - MemcacheQ - http://memcachedb.org/memcacheq/ Also See gareth rushgrove | morethanseven.net
  39. 39. couchdb.apache.org Data Storage With CouchDB gareth rushgrove | morethanseven.net
  40. 40. <?php $options['host'] = "localhost"; $options['port'] = 5984; $couch = new CouchSimple($options); $resp = $couch->send("PUT", "/test"); $resp = $couch->send("PUT", "/test/123", '{"_id":"123","data":"Foo"}'); ?> CouchDB Example gareth rushgrove | morethanseven.net
  41. 41. <?php $options['host'] = "localhost"; $options['port'] = 5984; $couch = new CouchSimple($options); $resp = $couch->send("GET", "/test/_all_docs"); $resp = $couch->send("GET", "/test/123"); $resp = $couch->send("DELETE", "/test/"); ?> CouchDB Example gareth rushgrove | morethanseven.net
  42. 42. - MongoDB - http://www.mongodb.org - Tokyo Tyrant - http://1978th.net/tokyotyrant/ - Cassandra - http://cassandra.apache.org - Redis - http://code.google.com/p/redis/ Also See gareth rushgrove | morethanseven.net
  43. 43. hadoop.apache.org Data Mining With Hive gareth rushgrove | morethanseven.net
  44. 44. CREATE TABLE u_data ( userid INT, movieid INT, rating INT, unixtime STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY 't' STORED AS TEXTFILE; Hive Create Example Cucumber DSL Example gareth rushgrove | morethanseven.net
  45. 45. LOAD DATA LOCAL INPATH 'data.txt' OVERWRITE INTO TABLE u_data; SELECT COUNT(1) FROM u_data; Hive Load DSL Example Cucumber Example gareth rushgrove | morethanseven.net
  46. 46. add FILE weekday_mapper.py; INSERT OVERWRITE TABLE u_data_new SELECT TRANSFORM (userid, movieid, rating, unixtime) USING 'python weekday_mapper.py' AS (userid, movieid, rating, weekday) FROM u_data; SELECT weekday, COUNT(1) FROM u_data_new GROUP BY weekday; Hive Map Reduce Example Cucumber DSL Example gareth rushgrove | morethanseven.net
  47. 47. - Hadoop - http://hadoop.apache.org - Hive - http://wiki.apache.org/hadoop/Hive/ - Pig - http://hadoop.apache.org/pig/ - Dumbo - http://lastfm.com/dumbo/ - Disco - http://discoproject.org Also See gareth rushgrove | morethanseven.net
  48. 48. cukes.info Testing with Cucumber gareth rushgrove | morethanseven.net
  49. 49. Feature: google.co.uk To broaden their knowledge A user should be able To search for things Scenario: Searching for things Given I visit "http://www.google.co.uk" When I fill in "q" with "wikipedia" And I press "Google Search" Then I should see "www.wikipedia.org" Cucumber DSL Example gareth rushgrove | morethanseven.net
  50. 50. Feature: google.co.uk To broaden their knowledge A user should be able To search for things Scenario: Searching for things Given I visit "http://www.google.co.uk" When I fill in "q" with "wikipedia" And I press "Google Search" Then I should see "www.wikipedia.org" 1 scenario (1 failed) 4 steps (1 failed, 2 skipped, 1 passed) 0m0.332s Cucumber Results Example gareth rushgrove | morethanseven.net
  51. 51. puppetlabs.com Server Provisioning with Puppet gareth rushgrove | morethanseven.net
  52. 52. class baseclass { $packagelist = ["sudo", "openssh-server"] package { $packagelist: ensure => installed } service { sshd: name => "ssh", enable => true, ensure => running } } Puppet Class Example gareth rushgrove | morethanseven.net
  53. 53. node 'example.com' inherits basenode { $packagelist = ["nginx"] package { $packagelist: ensure => installed } service { "nginx": ensure => running, require => Package["nginx"] } } Puppet Node Example gareth rushgrove | morethanseven.net
  54. 54. - Chef - http://wiki.opscode.com/display/chef/Home/ Also See gareth rushgrove | morethanseven.net
  55. 55. Conclusions gareth rushgrove | morethanseven.net http://www.flickr.com/photos/batega/1596898776/
  56. 56. 1 Know What's Possible gareth rushgrove | morethanseven.net http://www.flickr.com/photos/docman/36125185/
  57. 57. 2 Look for Opportunities gareth rushgrove | morethanseven.net http://www.flickr.com/photos/kgregory/500456103/
  58. 58. 3 Experiment gareth rushgrove | morethanseven.net http://www.flickr.com/photos/seeminglee/3967329241/
  59. 59. 4 Manage Complexity gareth rushgrove | morethanseven.net http://www.flickr.com/photos/30890318@N06/3510161637/
  60. 60. Questions? gareth rushgrove | morethanseven.net http://flickr.com/photos/psd/102332391/

×