Spitogatos.gr & Amazon Web Services
         High level architecture & lessons learned

                                   AWSUG GR meetup #1
                                          16 Feb 2012




                                 Andreas Chatzakis
                  co-founder / IT Director @ Spitogatos.gr
By David Fletcher - www.cloudtweaks.com
#about_us
Helping you find a property

Finding a property in Greece is complex, lacks transparency.
We make life easier for househunters via:
     Powerful search functionality
          Web & Mobile
          Location & Criteria
     Quality content
          Listings (we love photos)
          Articles
     mySpitogatos
          Email alerts
          Save your search
          Favorite listings & notes
          Contact the realtors


                                                                          4
Realtors love us too!

Professionals need help in those turbulent times.
We add value in multiple ways:
     Cost effective promotion & high quality leads
          Targeted channel (very)
          Leads already filtered (we ve seen the fotos!)
     Technology services for realtors
          Turnkey web site solution
          Listing synchronization web service
     B2B via Spitogatos Network (SpiN) business
      network / collaboration tool for realtors
     Channel for foreign buyers via the English version




                                                                                    5
Not just “αγγελίες”

Our vision is to become a one-stop-shop for all things house related




                                                                         6
#why_aws
The need for change
Infrastructure change dictated by our business needs
     Ongoing traffic increase
        A nice headache
     Growing Data
        S3 to the rescue
     Major new functionality
        Higher CPU & RAM requirements
     Uneven traffic pattern
          Why should I pay for this server during the night?
                                                                ...and constraints
                                 Let's not spend our time managing systems:
                                     AMIs? Yes thank you!
                                     Zero management services like ELB?   √ Like


                                                                                     8
#architecture
Your typical multi-tier web architecture




Storage Layer

                                                      10
Tools of the trade

Currently utilizing the following AWS services
   EC2



   S3
        Property photos
        Thumbnails
        Online backups
   Elastic Load Balancer (ELB)
   CloudWatch
   Route53


                                                                  11
A bit about Scalr

Cloud management solution reduced project risks & ITOPS effort




                                        Server Templates          Config & Bootstrap




                                         Backups             Scripts            DNS




         Autoscaling                 Master          Slave       Recover       Control

        Convention over Configuration - out-of-the-box automation and scalability        12
Services under our radar

We will utilize additional AWS services whenever that helps us:
    Reduce in-house ITOPS overheads
    Scale capacity and performance
    Enable us to bring new added value services to the market faster

    Cloudfront
         Milan node makes it attractive vs our current CDN solution
    Simple Email Service
    ElastiCache
    SimpleDB / DynamoDB
         Sessions, Logs, Aggregate stats, Personalization etc
         EMR / HiveQL
    Xeround or Amazon RDS
         MySQL as a service

                                                                                        13
#learnings
Not a walk in the park

From a single server to a scalable solution
   Deal with sessions
        Sticky sessions no good when autoscaling (scale downs)
        Memcache (nodes will die, sessions will be lost)
        Shared file system (maybe...) or Key-value store
        Zend Server cluster
   User uploads
        Rewrite code to utilize S3
        Or just plug in a POSIX compliant NAS solution
   Database
        Master-slave needs code changes (Replication lag)
              You thought splitting SELECTs from INSERTs/UPDATEs is sufficient?
   Individual nodes will fail – dont' take them for granted
        Holy grail: share nothing architecture

                                                                                               15
Continuous improvement

It is an iterative process
   Incrementally add robbustness
        Measure and collect data (Newrelic, MONyog, sysstat...)
        Alert (Cloudwatch, monit)
        Self healing (beware of false positives)
        Fall gracefully (MRT vs MTBF)
   Optimize
        Test out different EC2 Instance Types
            Once sure, reserve them for significant savings

        Use Cloudwatch in front of S3
        Test autoscaling strategies
            Scaleup fast, scaledown slow




                                                                                16
#questions

AWS case study: real estate portal

  • 1.
    Spitogatos.gr & AmazonWeb Services High level architecture & lessons learned AWSUG GR meetup #1 16 Feb 2012 Andreas Chatzakis co-founder / IT Director @ Spitogatos.gr
  • 2.
    By David Fletcher- www.cloudtweaks.com
  • 3.
  • 4.
    Helping you finda property Finding a property in Greece is complex, lacks transparency. We make life easier for househunters via:  Powerful search functionality  Web & Mobile  Location & Criteria  Quality content  Listings (we love photos)  Articles  mySpitogatos  Email alerts  Save your search  Favorite listings & notes  Contact the realtors 4
  • 5.
    Realtors love ustoo! Professionals need help in those turbulent times. We add value in multiple ways:  Cost effective promotion & high quality leads  Targeted channel (very)  Leads already filtered (we ve seen the fotos!)  Technology services for realtors  Turnkey web site solution  Listing synchronization web service  B2B via Spitogatos Network (SpiN) business network / collaboration tool for realtors  Channel for foreign buyers via the English version 5
  • 6.
    Not just “αγγελίες” Ourvision is to become a one-stop-shop for all things house related 6
  • 7.
  • 8.
    The need forchange Infrastructure change dictated by our business needs  Ongoing traffic increase  A nice headache  Growing Data  S3 to the rescue  Major new functionality  Higher CPU & RAM requirements  Uneven traffic pattern  Why should I pay for this server during the night? ...and constraints Let's not spend our time managing systems:  AMIs? Yes thank you!  Zero management services like ELB? √ Like 8
  • 9.
  • 10.
    Your typical multi-tierweb architecture Storage Layer 10
  • 11.
    Tools of thetrade Currently utilizing the following AWS services  EC2  S3  Property photos  Thumbnails  Online backups  Elastic Load Balancer (ELB)  CloudWatch  Route53 11
  • 12.
    A bit aboutScalr Cloud management solution reduced project risks & ITOPS effort Server Templates Config & Bootstrap Backups Scripts DNS Autoscaling Master Slave Recover Control Convention over Configuration - out-of-the-box automation and scalability 12
  • 13.
    Services under ourradar We will utilize additional AWS services whenever that helps us:  Reduce in-house ITOPS overheads  Scale capacity and performance  Enable us to bring new added value services to the market faster  Cloudfront  Milan node makes it attractive vs our current CDN solution  Simple Email Service  ElastiCache  SimpleDB / DynamoDB  Sessions, Logs, Aggregate stats, Personalization etc  EMR / HiveQL  Xeround or Amazon RDS  MySQL as a service 13
  • 14.
  • 15.
    Not a walkin the park From a single server to a scalable solution  Deal with sessions  Sticky sessions no good when autoscaling (scale downs)  Memcache (nodes will die, sessions will be lost)  Shared file system (maybe...) or Key-value store  Zend Server cluster  User uploads  Rewrite code to utilize S3  Or just plug in a POSIX compliant NAS solution  Database  Master-slave needs code changes (Replication lag)  You thought splitting SELECTs from INSERTs/UPDATEs is sufficient?  Individual nodes will fail – dont' take them for granted  Holy grail: share nothing architecture 15
  • 16.
    Continuous improvement It isan iterative process  Incrementally add robbustness  Measure and collect data (Newrelic, MONyog, sysstat...)  Alert (Cloudwatch, monit)  Self healing (beware of false positives)  Fall gracefully (MRT vs MTBF)  Optimize  Test out different EC2 Instance Types  Once sure, reserve them for significant savings  Use Cloudwatch in front of S3  Test autoscaling strategies  Scaleup fast, scaledown slow 16
  • 17.