• Save

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Cloud Computing in PHP With the Amazon Web Services

on

  • 8,587 views

Presentation delivered on April 30, 2010 to the Linux Fest in Bellingham, Washington.

Presentation delivered on April 30, 2010 to the Linux Fest in Bellingham, Washington.

Statistics

Views

Total Views
8,587
Views on SlideShare
8,586
Embed Views
1

Actions

Likes
7
Downloads
73
Comments
0

1 Embed 1

http://192.168.33.10:3000 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • First, it’s useful to provide the context that the way we think about what Amazon.com is, the way we think about it at the highest levels of the Company, is that we have three macro and distinct businesses: our Consumer/Retail business, our Seller business, and our Developer business.
  • Amazon Web Services is a set of primitive building blocks to enable developers to build applications and, ultimately, businesses, on our battle-tested infrastructure. So what kinds of building blocks do developers need? First, you need compute resources. The Elastic Compute Cloud, or EC2, allows developers to rent virtual computers, running in our data centers, and pay for them by the hour. Provision as many as you need – 1, 10, 1000, for as long as you need them, and you pay only for what you use (starting at 10 cents/hour). Next, you probably need somewhere to durably store and retrieve data. The Simple Storage Service, is, well, a simple storage service that allows to store and retrieve objects in the cloud using simple put and get API calls.You might need to index and query for that data, or create other structured data that you want to query, which is what we built SimpleDB.
  • On the application platforms point, good to mention:Microsoft .NETIBM WebSphereRubyPHPOracle Fusion MiddlewareJBossAlso a good place to mention IBM AMIs (bring your own license or pay by the hour)
  • SimpleDB is a very easy to use database service that can be used for a variety of tasks, including indexing information stored in S3, maintaining access and other types of logs for your applications, and storing all kinds of structured data in a reliable, scalable database.You can see a quick example of how easy SimpleDB is to use. Here’s your classic products table with an item number, description, and other attributes for each product. You can see the queries we’d use to store and retrieve data from SimpleDB. With SimpleDB, the API for putting data into the database is very flexible. I can add as much data as I’d like to each of the attributes for a given product. Querying the database is just as easy. I specify the “Domain” (or table) and the fields on which I’d like to execute the query. In this case, I’d like to obtain all items in the table whose description matches the given parameter.
  • SQS is an easy to use queueing service. The most typical use case for SQS is to coordinate activity between multiple EC2 instances. Take, for example, a typical web application with a presentation tier, application server or business logic tier, and database tier. You’d typically have multiple EC2 instances on the presentation and application server tiers, which would be working independently to process requests coming to your site.Amazon SQS is a queueing service that can provide the glue between your web server and your application server. The most common setup will involve configuring two queues. The first queue will accept messages from the web server hosted on Amazon EC2. Application servers, also hosted on Amazon EC2, will pluck those messages off the queue, process data based on the contents of the message, and then place the equivalent of an “I’m done! Here are the results.” message on the second queue. The web server would then pluck the message off the second queue and return results back to the client that made the initial request. In this way, your Amazon EC2 instances can grow or shrink, startup and fail with impunity, while you can rest assured that all of your data processing happens reliably.

Cloud Computing in PHP With the Amazon Web Services Cloud Computing in PHP With the Amazon Web Services Presentation Transcript

  • Cloud Computing in PHP With the Amazon Web Services
    Jeff Barr, Senior Web Services Evangelist
    jbarr@amazon.com
    @jeffbarr on Twitter
  • Based in Seattle
    Unix since 1982; Linux since 1995
    Career path:
    Startups
    Microsoft
    Consultant to VCs and startups
    Amazon Web Services
    Lead AWS Blogger
    Author of “Host Your Website in the Cloud”
    Introduction
  • Amazon / Cloud / AWS Intro
    AWS Overview
    Programmable Infrastructure
    S3, EC2, SDB, SQS
    Programming AWS in PHP
    CloudFusion library
    Scalable Image Processing
    Visualizing Infrastructure
    Dynamic Infrastructure
    Wrapup / Q & A
    Session Outline
  • AMAZON’S THREE BUSINESSES
    Consumer (Retail)Business
    SellerBusiness
    Developers &IT Professionals
    Tens of millions of active customer accounts
    Seven countries: US, UK, Germany, Japan, France, Canada, China
    Sell on Amazon websites
    Use Amazon technology for your own retail website
    Leverage Amazon’s massive fulfillment center network
    On-demand infrastructure for hosting web-scale solutions
    Hundreds of thousands of registered customers
  • What is Cloud Computing?
    First, think of your electricity service…
    Power is available to you on-demand, you pay only for what you use…
    …and you plug into a vast electrical grid managed by professionals to get you the lowest cost, most reliable power with much greater efficiency and safety than you could probably do on your own.
  • Introducing Amazon Web Services
    AWS provides flexible, scalable, secure, and cost-effective IT infrastructure for businesses of all sizes around the world.
    Compute power and storage is available to you on-demand, you pay only for the resources you use…
    …running on scalable, reliable, and secure infrastructure operated by Amazon Web Services, based on the knowledge gleaned from over a decade of building efficient and dependable infrastructure for Amazon.com.
  • Amazon Web Services
    Your Custom Applications and Services
    Monitoring
    Amazon CloudWatch
    Management
    AWS Management Console
    Tools
    AWS Toolkit for Eclipse
    Java, PHP, Ruby, Python, .Net Developer Centers
    Isolated Network
    Amazon Virtual Private Cloud
    Content Delivery
    Amazon CloudFront
    Messaging
    Amazon Simple Queue Service (SQS)
    Amazon Simple Notification Service (SNS)
    Payments
    Amazon Flexible Payments Service (FPS)
    On-Demand Workforce
    Amazon Mechanical Turk
    Parallel Processing
    Amazon Elastic MapReduce
    Compute
    Amazon Elastic Compute Cloud (EC2)
    • Elastic Load Balancing
    • Auto Scaling
    Storage
    Amazon Simple Storage Service (S3)
    • AWS Import/Export
    Database
    Amazon RDS
    Amazon SimpleDB
  • All functionality accessed by APIs
    Amazon and third-party libraries
    Command-line tools
    AWS Management Console
    Third-party Tools
    Programmable Infrastructure
    // Run an instance
    $EC2 = new AmazonEC2();
    $Options = array('KeyName' => "Jeff's Keys",
    'InstanceType' => "m1.small");
    $Res = $EC2->run_instances("ami-db7b9db2", 1, 1, $Options);
  • Scalable data storage in-the-cloud
    Highly available and durable
    Pay-as-you-go pricing:
    Storage: tiered $0.15/GB to $0.055/GB
    Data Transfer Out: tiered $0.15/GB to $0.080/GB
    Data Transfer In:Free until June 30, 2010
    Requests: nominal charges
    Big and busy:
    102 billion objects
    100K requests/second
    Amazon Simple Storage Service (S3)
  • Amazon EC2: on-demand compute power
    Obtain and boot new server instances in minutes
    Quickly scale capacity up or down
    Key features:
    Support for Linux, Windows, and OpenSolaris
    Supports all major web and application platforms
    Deploy across Availability Zones for reliability
    Elastic IPs provide greater flexibility
    Persistent storage with Amazon Elastic Block Store
    Monitoring (CloudWatch), Load Balancing, Auto-Scaling
    Amazon Elastic Compute Cloud (EC2)
  • EC2 Programming Interface (API)
    Images:
    RegisterImage
    DescribeImages
    DeregisterImage
    ModifyImageAttribute
    DescribeImageAttribute
    ResetImageAttribute
    Instances:
    RunInstances
    DescribeInstances
    TerminateInstances
    StopInstances
    GetConsoleOutput
    RebootInstances
    IP Addresses:
    AllocateAddress
    ReleaseAddress
    AssociateAddress
    DisassociateAddress
    DescribeAddresses
    Keypairs:
    CreateKeyPair
    DescribeKeyPairs
    DeleteKeyPair
    Security Groups:
    CreateSecurityGroup
    DescribeSecurityGroups
    DeleteSecurityGroup
    AuthorizeSecurityGroupIngress
    RevokeSecurityGroupIngress
    Block Storage Volumes:
    CreateVolume
    DeleteVolume
    DescribeVolumes
    AttachVolume
    DetachVolume
    CreateSnapshot
    DescribeSnapshots
    DeleteSnapshot
    • VPC:
    • CreateCustomerGateway
    • DeleteCustomerGateway
    • DescribeCustomerGateways
    • AssociateDhcpOptions
    • CreateDhcpOptions
    • DeleteDhcpOptions
    • DescribeDhcpOptions
    • CreateSubnet
    • DeleteSubnet
    • DescribeSubnets
    • CreateVpc
    • DeleteVpc
    • DescribeVpcs
    • CreateVpnConnection
    • DeleteVpnConnection
    • DescribeVpnConnections
    • AttachVpnGateway
    • CreateVpnGateway
    • DeleteVpnGateway
    • DescribeVpnGateways
    • DetachVpnGateway
  • Simple, scalable storage solution for structured data
    Provides core database functionality for data storage and querying
    No schema, no data modeling, no DBA
    SQL queries
    Amazon SimpleDB
    Store:
    PUT (item, 123), (description, Sweater), (color, Blue), (color, Red)
    Query:
    SELECT * FROM Inventory WHERE material='Leather'
  • Reliable, highly scalable, hosted queue for messaging
    Build automated workflows for all applications
    Coordinate multiple Amazon EC2 instances
    Amazon Simple Queue Service
    Consumer
    Producer
    Queue
    Consumer
    Producer
    Producer
  • Getting Started With AWS
    Create
    Developer Account
    Retrieve Private and Public Keys
    Enter Payment
    Information
    Build & Deploy Application
    Sign Up for Desired Services
    Monitor and Scale Application
  • Moving Right Along!
    "Enough with the talk, show us some code, Jeff!"
  • Programming AWS With PHP and CloudFusion
    We’ll use the CloudFusion libraries from www.getcloudfusion.com
  • CloudFusion Basics
    • Download from SVN
    • Add directory to PHP’s include_path
    • Add AWS keys to config.inc.php
    define('AWS_KEY', 'J35NTGFCQOIUY3OMNSQQ');
    define('AWS_SECRET_KEY', '99pizu2vVOK11rk9UAgWVj7PBGzWwertqJlgLV0c');
    • Include one file:
    require_once('cloudfusion.class.php');
  • CloudFusion XML / PHP SimpleXML
    CloudFusion methods return a SimpleXML Object:
    [body] => SimpleXMLElement Object
    (
    [Name] => sitepoint-aws-cloud-book
    [Prefix] => SimpleXMLElement Object ()
    [Marker] => SimpleXMLElement Object ()
    [MaxKeys] => 1000
    [IsTruncated] => false
    [Contents] => Array
    (
    [0] => SimpleXMLElement Object
    (
    [Key] => images/2008_shiller_housing_projection.jpg
    • Access as $Res->body->Contents[0]->Key
  • Example 1 – Scalable Processing Pipeline
  • Building a Scalable Image Processing Pipeline
    Fetch a web page, store in Amazon S3
    Parse page and extract links to images
    Fetch first 16 images on page, store in Amazon S3
    Render images as composite image
  • SQS-Based Application Architecture
    S3
    URL Queue
    Fetch Images
    Fetch & Store Page
    Render Queue
    S3
    S3
    Parse Queue
    Render Images & Pages
    Parse Page
    Image Queue
  • Architecture Attributes
    Simple
    Each stage is easy to understand
    AWS reduces low-level coding
    Scalable
    Add more processes
    Add more instances
    Add more stages
    Storage any amount of data
    Fault Tolerant
    Messages remain in queues until processed
    Messages reappear in queues if process dies
    Asynchronous
    Each stage runs at its own speed
    Build, test,run one stage at a time
  • Define Queues
    // Queues
    define('URL_QUEUE', 'c_url');
    define('PARSE_QUEUE', 'c_parse');
    define('IMAGE_QUEUE', 'c_image');
    define('RENDER_QUEUE', 'c_render');
    define('FEED_QUEUE', 'c_feed');
  • Create Queues
    // Create the SQS access object
    $SQS = new AmazonSQS();
    for ($i = 1; $i < count($argv); $i++)
    {
    $Queue = $argv[$i];
    $Res = $SQS->create_queue($Queue);
    if ($Res->isOK())
    {
    print("Created queue '${Queue}' ");
    }
    }
  • Create Queues & Check Status
    $ ./create_queues.php c_urlc_parsec_imagec_renderc_feed
    $ ./crawl_queue_status.php
    c_urlc_parsec_imagec_render
    ----- ------- ------- --------
    0 0 0 0
  • Locate a Queue's URL
    // Create the SQS access object
    $SQS = new AmazonSQS();
    // Locate the queue
    $QueueURL = FindQueueURL($SQS, URL_QUEUE);
  • Post Message to Queue
    // Create message
    $HistItem = array('Posted by ' . $argv[0] . ' at ' . date('c'));
    $Message = json_encode(array('Action' => 'FetchPage',
    'Origin' => $argv[0],
    'Data' => $argv[$i],
    'History' => $HistItem));
    // Post message
    $Res = $SQS->send_message($QueueURL, $Message);
    if ($Res->isOK())
    {
    print("Posted '${Message}' to QueueURL '${QueueURL}' ");
    }
  • Receive Message From Queue
    function PullMessage($SQS, $QueueURL)
    {
    while (true)
    {
    $Res = $SQS->receive_message($QueueURL);
    if ($Res->isOk())
    {
    if (IsSet($Res->body->ReceiveMessageResult->Message))
    {
    }
    else
    {
    sleep(1);
    }
    }
    }
  • Return Received Message
    $Message = $Res->body->ReceiveMessageResult->Message;
    $MessageBody = $Message->Body;
    $MessageDetail = json_decode($MessageBody, true);
    $ReceiptHandle = $Message->ReceiptHandle;
    return array('QueueURL' => $QueueURL,
    'Timestamp' => date('c'),
    'Message' => $Message,
    'MessageBody' => $MessageBody,
    'MessageDetail' => $MessageDetail,
    'ReceiptHandle' => $ReceiptHandle);
  • Sample Processing Stage
    // Pull, process, post
    while (true)
    {
    // Pull the message from the queue
    $Message = PullMessage($SQS, $QueueURL);
    if ($Message != null)
    {
    // Extract message detail
    $MessageDetail = $Message['MessageDetail'];
    $ReceiptHandle = $Message['ReceiptHandle'];
    $PageURL = $MessageDetail['Data'];

    }
    }
  • Detailed Processing
    // Fetch the page
    print("Processing URL '${PageURL}': ");
    $HTML = file_get_contents($PageURL);
    print(" Retrieved " . strlen($HTML) .
    " bytes of HTML ");
    // Store the page in S3
    $Key = 'page_' . md5($PageURL) . '.html';
    if (UploadObject($S3, BOOK_BUCKET, $Key, $HTML, S3_ACL_PUBLIC))
    {
    // Get URL in S3
    $S3URL = $S3->get_object_url(BOOK_BUCKET, $Key);
    print(" Uploaded page to S3 as '${Key}' ");
  • Detailed Processing
    // Form message to pass page along to parser
    $Origin = $MessageDetail['Origin'];
    $History = $MessageDetail['History'];
    $History[] = 'Fetched by ' . $argv[0] . ' at ' . date('c');
    $Message = json_encode(array('Action' => 'ParsePage',
    'Origin' => $Origin,
    'Data' => $S3URL,
    'PageURL' => $PageURL,
    'History' => $History));
    // Pass the page along to the parser
    $Res = $SQS->send_message($QueueParse, $Message);
  • Render Images
    foreach ($ImageKeys as $ImageKey)
    {
    // Fetch the image
    print(" Fetch image '${ImageKey}' ");
    $Image = $S3->get_object(BOOK_BUCKET, $ImageKey);
    // Convert it to GD format
    $ImageBits = ImageCreateFromString($Image->body);
    // Copy it to proper spot in the destination
    print(" Render image at ${NextX}, ${NextY} ");
    ImageCopy($ImageOut, $ImageBits, $NextX, $NextY,
    0, 0, ImageSx($ImageBits), ImageSy($ImageBits));
    // Update position for next image
    $NextX += THUMB_SIZE + GAP_SIZE;
    if (($NextX + THUMB_SIZE) > $OutX)
    {
    $NextX = BORDER_LEFT;
    $NextY += THUMB_SIZE + GAP_SIZE;
    }
    }
  • Example 2 – Create Infrastructure Graph
  • Processing Flow
    Get AWS Metadata
    Build Object Model
    Render Object Model as DOT Graph
    Render Dot Graph as PDF
  • Get AWS Metadata
    // Create the service access objects
    $Service_EC2 = new AmazonEC2);
    $Service_S3 = new AmazonS3);
    $Service_SDB = new AmazonSDB);
    $Service_CF = new AmazonCloudFrontetKey);
    // Fetch information about all of the EC2 objects
    $ResElasticIP = $Service_EC2->describe_addresses();
    $ResAvailabilityZones = $Service_EC2->describe_availability_zones();
    $ResInstances = $Service_EC2->describe_instances();
    $ResVolumes = $Service_EC2->describe_volumes();
    $ResSnapshots = $Service_EC2->describe_snapshots();
  • Build Object Model
    // EC2 Instances
    foreach ($ResInstances->body->reservationSet->item as $ItemSet)
    {
    foreach ($ItemSet->instancesSet->item as $Item)
    {
    $InstanceId = (string) $Item->instanceId;
    $ImageId = (string) $Item->imageId;
    $State = (string) $Item->instanceState->name;
    $InstanceType = (string) $Item->instanceType;
    $AvailabilityZone = (string) $Item->placement->availabilityZone;
    $LaunchTime = (string) $Item->launchTime;
    $Region->AddInstance(new Instance($AvailabilityZone, $InstanceId, $ImageId, $State, $InstanceType, $LaunchTime));
    }
    }
  • Sample Object (EC2 Instance)
    class Instance
    {
    var $State;
    var $ImageId;
    var $InstanceId;
    var $LaunchTime;
    var $InstanceType;
    var $AvailabilityZone;
    function __construct(…) {}
    public function GetAvailabilityZone() {}
    private function GetLabel() {}
    function Render() {}
    function RenderEdges() {}
    }
  • Render Object Model as DOT Graph
    function WriteNode($FP, $IndentLevel, $Node, $Label, $NodeType)
    {
    fwrite($FP, Indent($IndentLevel) . Quote($Node) .
    ' [' .
    'label=' . Quote($Label) . ', ' .
    GetNodeStyle($NodeType) .
    "]; ");
    }
  • DOT Text
    graph aws
    {
    // Region
    subgraph "cluster_us-east-1"
    {
    label="Region us-east-1";
    style=filled; fillcolor=yellow; color=black;
    // Elastic IP Addresses
    subgraph "cluster_us-east-1_ips"
    {
    label="Elastic IP Addresses";
    style=filled; fillcolor=darksalmon; color=black;
    "75.101.154.199" [label="Elastic IP 75.101.154.199 ec2-75-101-154-199.compute-1.amazonaws.com", color=black, style=filled, fillcolor=wheat, shape=rect];
    "us-east-1_phantom_ips" [label="", shape=point, style=invis];
    }
  • Render Dot Text to PDF
    $ dot –Tpdf aws_meta.dot > ~jeff/public_html/aws_meta.pdf
  • Finished Infrastructure Graph
  • Example 3 – Dynamically Instantiate Infrastructure
    10 GB
    EBS
    Volume
    EC2 Instance
    Elastic IP
    Address
    100 GB
    EBS
    Volume
  • Run an EC2 Instance
    // Create the EC2 access object
    $EC2 = new AmazonEC2(AWS_PUBLIC_KEY, AWS_SECRET_KEY);
    // Run an instance
    $Options = array('KeyName' => "Jeff's Keys",
    'InstanceType' => "m1.small");
    $Res = $EC2->run_instances("ami-db7b9db2", 1, 1, $Options);
  • Get Instance Info
    // Get the Id and Availability Zone of the instance
    $Instances = $Res->body->instancesSet;
    $InstanceId = $Instances->item->instanceId;
    $AvailabilityZone = $Instances->item->placement ->availabilityZone;
    print("Launched instance ${InstanceId} " .
    "in availability zone ${AvailabilityZone}. ");
  • Allocate Elastic IP Address
    // Allocate an Elastic IP address
    $Res = $EC2->allocate_address();
    if (!$Res->isOK())
    {
    exit("Could not allocate public IP address. ");
    }
    // Get the allocated Elastic IP address
    $PublicIP = $Res->body->publicIp;
    print("Assigned IP address ${PublicIP}. ");
  • Attach IP Address to Instance
    // Associate the Elastic IP address with the instance
    $Res = $EC2->associate_address($InstanceId, $PublicIP);
    if (!$Res->IsOK())
    {
    exit("Could not associate IP address ${PublicIP} " .
    "with instance ${InstanceId}. ");
    }
    print("Associated IP address ${PublicIP} " .
    "with instance ${InstanceId}. ");
  • Create 2 EBS Virtual Disk Volumes
    // Create two EBS volumes in the instance's availability zone
    $Res1 = $EC2->create_volume(10, $AvailabilityZone);
    $Res2 = $EC2->create_volume(100, $AvailabilityZone);
    if (!$Res1->isOK() || !$Res2->isOK())
    {
    exit("Could not create EBS volumes. ");
    }
    // Get the volume Ids
    $VolumeId1 = $Res1->body->volumeId;
    $VolumeId2 = $Res2->body->volumeId;
  • Attach The Volumes To the Instance
    $Res1 = $EC2->attach_volume($VolumeId1, $InstanceId, '/dev/sdf');
    $Res2 = $EC2->attach_volume($VolumeId2, $InstanceId, '/dev/sdg');
    if (!$Res1->isOK() || !$Res2->isOK())
    {
    exit("Could not attach EBS volumes " .
    "${VolumeId1} and ${VolumeId2} " .
    "to instance ${InstanceId}. ");
    }
  • Now Hiring
    Northern Virginia:
    SDE – Virtual Private Cloud
    Front End Web Developer – Virtual Private Cloud
    Seattle:
    SDE
    Program Management
    Developer Support
    More Info:
    AWS booth in lobby
    http://aws.amazon.com/jobs
  • Q & A
  • Amazon Web Services: http://aws.amazon.com
    Virtual Private Cloud: http://aws.amazon.com/vpc
    Relational Database Service: http://aws.amazon.com/rds
    jbarr@amazon.com
    twitter.com/jeffbarr
    For MORE INFORMATION
  • Thanks!
    jbarr@amazozn.com