Escalando Aplicaciones Web

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    Notes on slide 1

    Map (Java) = dict (Python) = associative array (PHP) = Hash (Ruby) = Object (Javascript)

    1 Favorite

    Escalando Aplicaciones Web - Presentation Transcript

    1. Escalando aplicaciones web BarCamp Buenos Aires 2009 Santiago Coffey - Popego
    2. Simplest webapp
    3. Scaling with job queues
    4. Scaling with alternative databases
    5. YouTube recipe for rapid growth #!/usr/bin/env python while True: identify_bottlenecks() fix_bottlenecks() drink() sleep() notice_new_bottleneck()
    6. What can I do today to make it better?
    7. Web server?
    8. Ability to take the small...
    9. ... to the big
    10. SCALING SUCKS
    11. RDBMS vs. simple key-value stores ACID: Atomicity, High availability Consistency, Isolation, Durability Replication, distributed Transactions Fault tolerance SQL: Joins, Group-By Scalability
    12. ACID? What for?
    13. memcached Distributed memory caching system Works as a large hash table distributed across multiple machines Used to speed up applications by caching data and objects in memory, thus saving database or external data source access
    14. memcached Lacks authentication and security -> firewall Lacks atomicity Lacks durability -> memcachedb (whose BerkeleyDB backend is easily replicable) Fast, provides high availability
    15. Other key-value stores... Redis Project Voldemort CouchDB Scalaris HyperTable Cassandra HBase OpenChord / DHT
    16. MapReduce Soft ware framework Introduced by Google To support distributed computing On large data sets Using clusters of computers
    17. Word count example: Mapper #!/usr/bin/env python import sys def mapper(stream=sys.stdin): for line in stream: for word in line.split(): print '%st%d' % (word, 1) if __name__ == "__main__": mapper()
    18. Word count example: Reducer #!/usr/bin/env python import sys def reducer(stream=sys.stdin): counter = {} for line in stream: word, count = line.split('t', 1) counter[word] = counter.get(word, 0) + int(count) for word, count in sorted(counter, key=lambda i: i[0]) print '%st%d' % (word, count) if __name__ == "__main__": reducer()
    19. Framework implementations Hadoop BashReduce: http://github.com/erikfrey/ bashreduce/tree/master br [-m host1 [host2...]] [-c column] [-r reduce] [-i input] [-o output]
    20. HBase/BigTable HBase: Open source implementation of Google’s BigTable Not a RDBMS
    21. A BigTable is a sparse, distributed, persistent multidimensional sorted map
    22. HBase: map At its core it’s a map a collection of keys and values where each key has a value associated
    23. HBase: persistent Map data is permanently stored e.g.: file on a disk filesystem
    24. HBase: distributed Built upon distributed filesystems Underlying storage can be spread out Among an array of independent machines (like in a RAID system) e.g.: HDFS, Amazon S3, GFS Replicated across a number of nodes
    25. HBase: sorted Key-value pairs kept in strict order Keys are indexed so that similar items are nearer (avoiding table scans)
    26. HBase: multidimensional Actually it’s a map of maps: Row (root key) Column family (specified upon creaton, cannot be modified) Qualifier (unlimited, can be modified) Timestamp (defaults to latest) Value
    27. HBase: sparse A row can have any number of columns in each column family or none at all Also: sparseness in key gaps
    28. Hey! I’m just a developer. Do I need to understand the details of this super filesystem? NO!
    29. Amazon Web Services Accessed over HTTP Using REST and SOAP protocols Also: GUI clients Billed on “usage” (usage form depends on exact service)
    30. Amazon Web Services: Elastic Map Reduce Enables to process vast amounts of data Utilizes a hosted Hadoop framework running on the web-scale infrastructure EC2 and S3 Easy and cost-effective Reliable and fault-tolerant
    31. Amazon Web Services: Elastic Map Reduce Why Elastic? You choose how many computer instances (and how much powerful) You launch job flows on demand and they are started within minutes Computer instances automatically torn down on completion
    32. Amazon Web Services: EC2 EC2: Elastic Compute Cloud Provides scalable virtual private ser vers using Xen Machine instances launched on demand Charges per machine: Hourly + data transfer
    33. When is my system elastic? When infrastructure can easily grow and shrink
    34. Amazon Web Services: S3 S3: Simple Storage System Stores arbitrary objects (< 5 GB) in unique user-assigned keys Organized into buckets (each owned by an AWS account)
    35. Amazon Web Services: S3 Scalability, high availability and low latency At commodity costs: $0.15 per GB- month + bandwith charges
    36. Amazon Web Services: SQS SQS: Simple Queue Ser vice Distributed queue messaging ser vice Supports programmatic sending of messages via web ser vice applications Resolves common producer-consumer problems among distributed hosts
    37. Other Amazon Web Services... Amazon SimpleDB: operates in concert with EC2 and S3 to provide the core functionality of a database running queries on structured data Amazon CloudFront, a content delivery net work (CDN) for distributing objects stored in S3 to so- called "edge locations" near the requester AWS Management Console: A web-based interface to manage and monitor the Amazon infrastructure suite
    38. ¡Gracias! santiago.coffey@popego.com http:/ witter.com/scoffey /t http://scoffey.popego.com

    + Santiago CoffeySantiago Coffey, 4 weeks ago

    custom

    183 views, 1 favs, 1 embeds more stats

    Presentation at BarCamp Buenos Aires 2009 on Scalin more

    More info about this document

    CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

    Go to text version

    • Total Views 183
      • 182 on SlideShare
      • 1 from embeds
    • Comments 0
    • Favorites 1
    • Downloads 2
    Most viewed embeds
    • 1 views on http://www.coffey.com.ar

    more

    All embeds
    • 1 views on http://www.coffey.com.ar

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories