SlideShare a Scribd company logo
Cloud	
  Architecture	
  Tutorial	
  
How	
  Ne3lix	
  Built	
  a	
  Scalable	
  Java	
  oriented	
  PaaS	
  running	
  on	
  AWS	
  
                                      Part	
  1	
  of	
  3	
  


             Qcon	
  London	
  March	
  5th,	
  2012	
  
                      Adrian	
  Cockcro6	
  
                      @adrianco	
  #ne:lixcloud	
  
              h>p://www.linkedin.com/in/adriancockcro6	
  
Tutorial	
  Abstract	
  –	
  Set	
  Context	
  
•    StarJng	
  with	
  the	
  usual	
  quesJons:	
  “Why	
  Ne:lix,	
  why	
  cloud,	
  why	
  AWS?”	
  
•    This	
  tutorial	
  explains	
  which	
  business	
  models	
  and	
  applicaJons	
  benefit	
  most	
  from	
  cloud,	
  what	
  to	
  
     look	
  for	
  in	
  a	
  cloud	
  provider,	
  and	
  how	
  the	
  tradiJonal	
  enterprise	
  compuJng	
  marketplace	
  is	
  being	
  
     disrupted.	
  
•    Moving	
  on	
  to	
  the	
  next	
  quesJon:	
  “What	
  can	
  run	
  in	
  the	
  cloud?”	
  a	
  step	
  by	
  step	
  approach	
  to	
  cloud	
  
     migraJon	
  is	
  described,	
  along	
  with	
  a	
  varied	
  set	
  of	
  use	
  cases	
  for	
  both	
  customer	
  facing	
  and	
  internal	
  
     web	
  services,	
  big	
  data	
  analyJcs	
  and	
  bulk	
  computaJon.	
  Cloud	
  migraJon	
  starts	
  by	
  moving	
  
     developers	
  to	
  work	
  on	
  cloud	
  using	
  “boot	
  camp”	
  training	
  sessions,	
  then	
  a6er	
  building	
  out	
  the	
  iniJal	
  
     core	
  pla:orm,	
  the	
  first	
  applicaJons	
  are	
  launched.	
  
•    The	
  real	
  meat	
  of	
  the	
  tutorial	
  comes	
  when	
  we	
  look	
  at	
  how	
  to	
  construct	
  an	
  applicaJon	
  with	
  a	
  host	
  of	
  
     important	
  properJes:	
  elasJc,	
  dynamic,	
  scalable,	
  agile,	
  fast,	
  cheap,	
  robust,	
  durable,	
  observable,	
  
     secure.	
  Over	
  the	
  last	
  three	
  years	
  Ne:lix	
  has	
  figured	
  out	
  cloud	
  based	
  soluJons	
  with	
  these	
  
     properJes,	
  deployed	
  them	
  globally	
  at	
  large	
  scale	
  and	
  refined	
  them	
  into	
  a	
  global	
  Java	
  oriented	
  
     Pla:orm	
  as	
  a	
  Service.	
  The	
  PaaS	
  is	
  based	
  on	
  low	
  cost	
  open	
  source	
  building	
  blocks	
  such	
  as	
  Apache	
  
     Tomcat,	
  Apache	
  Cassandra,	
  and	
  Memcached.	
  Components	
  of	
  this	
  pla:orm	
  are	
  in	
  the	
  process	
  of	
  
     being	
  open-­‐sourced	
  by	
  Ne:lix,	
  so	
  that	
  other	
  companies	
  can	
  get	
  a	
  start	
  on	
  building	
  their	
  own	
  
     customized	
  PaaS	
  that	
  leverages	
  advanced	
  features	
  of	
  AWS	
  and	
  supports	
  rapid	
  agile	
  development.	
  
•    The	
  architecture	
  is	
  described	
  in	
  terms	
  of	
  anJ-­‐pa>erns	
  -­‐	
  things	
  to	
  avoid	
  in	
  the	
  datacenter	
  to	
  cloud	
  
     transiJon.	
  A	
  scalable	
  global	
  persistence	
  Jer	
  based	
  on	
  Cassandra	
  provides	
  a	
  highly	
  available	
  and	
  
     durable	
  under-­‐pinning.	
  Lessons	
  learned	
  will	
  cover	
  soluJons	
  to	
  common	
  problems,	
  availability	
  and	
  
     robustness,	
  observability.	
  A>endees	
  should	
  leave	
  the	
  tutorial	
  with	
  a	
  clear	
  understanding	
  of	
  what	
  is	
  
     different	
  about	
  cloud	
  architectures,	
  why,	
  what	
  and	
  how	
  to	
  make	
  the	
  transiJon,	
  and	
  a	
  set	
  of	
  flexible	
  
     and	
  scalable	
  open	
  source	
  building	
  blocks	
  that	
  can	
  be	
  used	
  to	
  construct	
  their	
  own	
  cloud	
  pla:orm.	
  
PresentaJon	
  vs.	
  Tutorial	
  
•  PresentaJon	
  
   –  Short	
  duraJon,	
  focused	
  subject	
  
   –  One	
  presenter	
  to	
  many	
  anonymous	
  audience	
  
   –  A	
  few	
  quesJons	
  at	
  the	
  end	
  
•  Tutorial	
  
   –  Time	
  to	
  explore	
  in	
  and	
  around	
  the	
  subject	
  
   –  Tutor	
  gets	
  to	
  know	
  the	
  audience	
  
   –  Discussion,	
  rat-­‐holes,	
  “bring	
  out	
  your	
  dead”	
  
Tutorial	
  SecJons	
  
  Intro:	
  Who	
  are	
  you,	
  what	
  are	
  your	
  quesJons?	
  
	
  
Part	
  1	
  -­‐	
  Why	
  use	
  cloud,	
  what	
  runs	
  in	
  the	
  cloud	
  
Part	
  2	
  -­‐	
  Pla:orm	
  component	
  architecture	
  
Part	
  3	
  -­‐	
  Running	
  in	
  the	
  cloud	
  
Adrian	
  Cockcro6	
  
•  Director,	
  Architecture	
  for	
  Cloud	
  Systems,	
  Ne:lix	
  Inc.	
  
      –  Previously	
  Director	
  for	
  PersonalizaJon	
  Pla:orm	
  

•  DisJnguished	
  Availability	
  Engineer,	
  eBay	
  Inc.	
  2004-­‐7	
  
      –  Founding	
  member	
  of	
  eBay	
  Research	
  Labs	
  

•  DisJnguished	
  Engineer,	
  Sun	
  Microsystems	
  Inc.	
  1988-­‐2004	
  
      –    2003-­‐4	
  Chief	
  Architect	
  High	
  Performance	
  Technical	
  CompuJng	
  
      –    2001	
  Author:	
  Capacity	
  Planning	
  for	
  Web	
  Services	
  
      –    1999	
  Author:	
  Resource	
  Management	
  
      –    1995	
  &	
  1998	
  Author:	
  Sun	
  Performance	
  and	
  Tuning	
  
      –    1996	
  Japanese	
  EdiJon	
  of	
  Sun	
  Performance	
  and	
  Tuning	
  
             •  	
  SPARC	
  &	
  Solarisパフォーマンスチューニング (サンソフトプレスシリーズ)	
  


•  More	
  
      –  Twi>er	
  @adrianco	
  –	
  Blog	
  h>p://perfcap.blogspot.com	
  
      –  PresentaJons	
  at	
  h>p://www.slideshare.net/adrianco	
  
A>endee	
  IntroducJons	
  
•  Who	
  are	
  you,	
  where	
  do	
  you	
  work	
  
•  Why	
  are	
  you	
  here	
  today,	
  what	
  do	
  you	
  need	
  
•  “Bring	
  out	
  your	
  dead”	
  
       –  Do	
  you	
  have	
  a	
  specific	
  problem	
  or	
  quesJon?	
  
       –  One	
  sentence	
  elevator	
  pitch	
  
	
  
Why	
  Ne:lix,	
  Why	
  Cloud,	
  Why	
  
               AWS	
  
               Part	
  1	
  of	
  3	
  
Ne:lix	
  Inc.	
  


   With	
  more	
  than	
  23	
  million	
  streaming	
  members	
  in	
  the	
  
      United	
  States,	
  Canada,	
  LaBn	
  America,	
  the	
  United	
  
       Kingdom	
  and	
  Ireland,	
  NeGlix,	
  Inc.	
  is	
  the	
  world's	
  
      leading	
  internet	
  subscripBon	
  service	
  for	
  enjoying	
  
                           movies	
  and	
  TV	
  series..	
  




Source:	
  h>p://ir.ne:lix.com	
  
What	
  kind	
  of	
  Cloud?	
  
•  So6ware	
  as	
  a	
  Service	
  –	
  SaaS	
  
     –  Replaces	
  in	
  house	
  applicaJons	
  
     –  Targets	
  end	
  users	
  
•  Pla:orm	
  as	
  a	
  Service	
  –	
  PaaS	
  
     –  Replaces	
  in	
  house	
  operaJons	
  funcJons	
  
     –  Targets	
  developers	
  
•  Infrastructure	
  as	
  a	
  Service	
  –	
  IaaS	
  
     –  Replaces	
  in	
  house	
  datacenter	
  capacity	
  
     –  Targets	
  developers	
  and	
  ITops	
  
What	
  Ne:lix	
  Did	
  
•  Moved	
  to	
  SaaS	
  
    –  Corporate	
  IT	
  –	
  Workday	
  etc.	
  
    –  Tools	
  –	
  Pagerduty,	
  AppDynamics,	
  ElasJc	
  MapReduce	
  
•  Built	
  our	
  own	
  PaaS	
  <-­‐	
  today’s	
  focus	
  
    –  Customized	
  to	
  make	
  our	
  developers	
  producJve	
  
    –  When	
  we	
  started,	
  we	
  had	
  li>le	
  choice	
  
•  Moved	
  incremental	
  capacity	
  to	
  IaaS	
  
    –  No	
  new	
  datacenter	
  space	
  since	
  2008	
  as	
  we	
  grew	
  
    –  Moved	
  our	
  streaming	
  apps	
  to	
  the	
  cloud	
  
Why	
  Use	
  Public	
  Cloud?	
  
Things	
  We	
  Don’t	
  Do	
  
Be>er	
  Business	
  Agility	
  
Data	
  Center	
                   Ne:lix	
  could	
  not	
  
                                      build	
  new	
  
                                   datacenters	
  fast	
  
                                       enough	
  

    Capacity	
  growth	
  is	
  acceleraJng,	
  unpredictable	
  
    Product	
  launch	
  spikes	
  -­‐	
  iPhone,	
  Wii,	
  PS3,	
  Xbox	
  
  InternaJonal	
  –	
  Canada,	
  LaJn	
  America,	
  UK/Ireland	
  
Out-­‐Growing	
  Data	
  Center	
  
             h>p://techblog.ne:lix.com/2011/02/redesigning-­‐ne:lix-­‐api.html   	
  


                               37x	
  Growth	
  Jan	
  
                               2010-­‐Jan	
  2011	
  


Datacenter	
  
Capacity	
  
Ne:lix.com	
  is	
  now	
  ~100%	
  Cloud	
  
  A	
  few	
  small	
  back	
  end	
  data	
  sources	
  sJll	
  in	
  progress	
  
          All	
  internaJonal	
  product	
  is	
  cloud	
  based	
  
   USA	
  specific	
  logisJcs	
  remains	
  in	
  the	
  Datacenter	
  
  Working	
  on	
  SOX,	
  PCI	
  as	
  scope	
  starts	
  to	
  include	
  AWS	
  
Ne:lix	
  Choice	
  was	
  AWS	
  with	
  our	
  
   own	
  pla:orm	
  and	
  tools	
  
     Unique	
  pla:orm	
  requirements	
  and	
  
     extreme	
  scale,	
  agility	
  and	
  flexibility	
  
Leverage	
  AWS	
  Scale	
  
   “the	
  biggest	
  public	
  cloud”	
  
 AWS	
  investment	
  in	
  features	
  and	
  automaJon	
  
Use	
  AWS	
  zones	
  and	
  regions	
  for	
  high	
  availability,	
  
         scalability	
  and	
  global	
  deployment	
  
But	
  isn’t	
  Amazon	
  a	
  compeJtor?	
  
Many	
  products	
  that	
  compete	
  with	
  Amazon	
  run	
  on	
  AWS	
  
  We	
  are	
  a	
  “poster	
  child”	
  for	
  the	
  AWS	
  Architecture	
  
      Ne:lix	
  is	
  one	
  of	
  the	
  biggest	
  AWS	
  customers	
  
    Co-­‐opeJJon	
  -­‐	
  compeJtors	
  are	
  also	
  partners	
  
Could	
  Ne:lix	
  use	
  another	
  cloud?	
  
 Would	
  be	
  nice,	
  we	
  use	
  three	
  interchangeable	
  CDN	
  Vendors	
  
    But	
  no-­‐one	
  else	
  has	
  the	
  scale	
  and	
  features	
  of	
  AWS	
  
            You	
  have	
  to	
  be	
  this	
  tall	
  to	
  ride	
  this	
  ride…	
  
                               Maybe	
  in	
  2-­‐3	
  years?	
  
We	
  want	
  to	
  use	
  clouds,	
  
     we	
  don’t	
  have	
  Jme	
  to	
  build	
  them	
  
                             Public	
  cloud	
  for	
  agility	
  and	
  scale	
  
We	
  use	
  electricity	
  too,	
  but	
  don’t	
  want	
  to	
  build	
  our	
  own	
  power	
  staJon…	
  
AWS	
  because	
  they	
  are	
  big	
  enough	
  to	
  allocate	
  thousands	
  of	
  instances	
  per	
  
                                     hour	
  when	
  we	
  need	
  to	
  
Amazon Cloud Terminology Reference
     See http://aws.amazon.com/ This is not a full list of Amazon Web Service features

•    AWS	
  –	
  Amazon	
  Web	
  Services	
  (common	
  name	
  for	
  Amazon	
  cloud)	
  
•    AMI	
  –	
  Amazon	
  Machine	
  Image	
  (archived	
  boot	
  disk,	
  Linux,	
  Windows	
  etc.	
  plus	
  applicaJon	
  code)	
  
•    EC2	
  –	
  ElasJc	
  Compute	
  Cloud	
  
       –    Range	
  of	
  virtual	
  machine	
  types	
  m1,	
  m2,	
  c1,	
  cc,	
  cg.	
  Varying	
  memory,	
  CPU	
  and	
  disk	
  configuraJons.	
  
       –    Instance	
  –	
  a	
  running	
  computer	
  system.	
  Ephemeral,	
  when	
  it	
  is	
  de-­‐allocated	
  nothing	
  is	
  kept.	
  
       –    Reserved	
  Instances	
  –	
  pre-­‐paid	
  to	
  reduce	
  cost	
  for	
  long	
  term	
  usage	
  
       –    Availability	
  Zone	
  –	
  datacenter	
  with	
  own	
  power	
  and	
  cooling	
  hosJng	
  cloud	
  instances	
  
       –    Region	
  –	
  group	
  of	
  Avail	
  Zones	
  –	
  US-­‐East,	
  US-­‐West,	
  EU-­‐Eire,	
  Asia-­‐Singapore,	
  Asia-­‐Japan,	
  SA-­‐Brazil,	
  US-­‐Gov	
  
•    ASG	
  –	
  Auto	
  Scaling	
  Group	
  (instances	
  booJng	
  from	
  the	
  same	
  AMI)	
  
•    S3	
  –	
  Simple	
  Storage	
  Service	
  (h>p	
  access)	
  
•    EBS	
  –	
  ElasJc	
  Block	
  Storage	
  (network	
  disk	
  filesystem	
  can	
  be	
  mounted	
  on	
  an	
  instance)	
  
•    RDS	
  –	
  RelaJonal	
  Database	
  Service	
  (managed	
  MySQL	
  master	
  and	
  slaves)	
  
•    DynamoDB/SDB	
  –	
  Simple	
  Data	
  Base	
  (hosted	
  h>p	
  based	
  NoSQL	
  datastore,	
  DynamoDB	
  replaces	
  SDB)	
  
•    SQS	
  –	
  Simple	
  Queue	
  Service	
  (h>p	
  based	
  message	
  queue)	
  
•    SNS	
  –	
  Simple	
  NoJficaJon	
  Service	
  (h>p	
  and	
  email	
  based	
  topics	
  and	
  messages)	
  
•    EMR	
  –	
  ElasJc	
  Map	
  Reduce	
  (automaJcally	
  managed	
  Hadoop	
  cluster)	
  
•    ELB	
  –	
  ElasJc	
  Load	
  Balancer	
  
•    EIP	
  –	
  ElasJc	
  IP	
  (stable	
  IP	
  address	
  mapping	
  assigned	
  to	
  instance	
  or	
  ELB)	
  
•    VPC	
  –	
  Virtual	
  Private	
  Cloud	
  (single	
  tenant,	
  more	
  flexible	
  network	
  and	
  security	
  constructs)	
  
•    DirectConnect	
  –	
  secure	
  pipe	
  from	
  AWS	
  VPC	
  to	
  external	
  datacenter	
  
•    IAM	
  –	
  IdenJty	
  and	
  Access	
  Management	
  (fine	
  grain	
  role	
  based	
  security	
  keys)	
  
AWS	
  and	
  the	
  Seven	
  Dwarfs	
  
•  Public	
  Cloud	
  AlternaJves	
  to	
  AWS	
  
    –  Far	
  fewer	
  features,	
  much	
  smaller	
  scale	
  
    –  Less	
  mature	
  APIs,	
  many	
  variants	
  of	
  APIs	
  
    –  Some	
  have	
  addiJonal	
  features	
  or	
  performance	
  
•  Private	
  Cloud	
  AlternaJves	
  
    –  O6en	
  harder	
  to	
  build	
  and	
  run	
  than	
  you	
  think	
  
    –  Without	
  scale	
  and	
  mulJ-­‐tenancy,	
  much	
  higher	
  costs	
  
    –  O6en	
  driven	
  by	
  ITops	
  needs	
  rather	
  than	
  developers	
  
Some	
  AlternaJve	
  Public	
  Clouds	
  
     IaaS	
  that	
  you	
  could	
  build	
  your	
  own	
  PaaS	
  architecture	
  on   	
  
•  OpenStack	
  Based	
  
    –  Rackspace	
  
    –  HP	
  Cloud	
  
    –  ATT	
  Cloud	
  
•  GoGrid	
  
•  Joyent	
  –	
  Solaris	
  in	
  the	
  cloud	
  
•  Memset	
  –	
  UK	
  based	
  
What	
  about	
  other	
  PaaS?	
  
•  CloudFoundry	
  –	
  Open	
  Source	
  by	
  VMWare	
  
    –  Developer-­‐friendly,	
  easy	
  to	
  get	
  started	
  
    –  Missing	
  scale	
  and	
  some	
  enterprise	
  features	
  
•  Rightscale	
  
    –  Widely	
  used	
  to	
  abstract	
  away	
  from	
  AWS	
  
    –  Creates	
  it’s	
  own	
  lock-­‐in	
  problem…	
  
•  AWS	
  is	
  growing	
  into	
  this	
  space	
  
    –  We	
  didn’t	
  want	
  a	
  vendor	
  between	
  us	
  and	
  AWS	
  
    –  We	
  wanted	
  to	
  build	
  a	
  thin	
  PaaS,	
  that	
  gets	
  thinner	
  
Enterprise	
  Market	
  DisrupJon	
  
•  Enterprise	
  CompuJng	
  Vendors	
  $$$$$$	
  
    –  IBM,	
  HP,	
  Dell,	
  Oracle,	
  EMC,	
  NetApp…	
  
    –  CIO/ITOps	
  integrates	
  and	
  provisions	
  
    –  TradiJonal,	
  and	
  moving	
  towards	
  private	
  clouds	
  

•  IaaS	
  Vendors	
  Sell	
  Directly	
  to	
  Developers	
  $	
  
    –  Bypassing	
  ITOps	
  with	
  stealth	
  cloud	
  based	
  projects	
  
    –  Bypassing	
  enterprise	
  vendor	
  supply	
  chain	
  
    –  Low	
  margin,	
  low	
  fricJon,	
  dollar	
  at	
  a	
  Jme	
  
What	
  Runs	
  in	
  the	
  Cloud?	
  

   Step	
  by	
  Step	
  Ne:lix	
  Product	
  
                  TransiJon	
  
Ne:lix	
  Deployed	
  on	
  AWS	
  
  2009	
           2009	
              2010	
              2010	
            2010	
             2011	
  

Content	
          Logs	
              Play	
              WWW	
             API	
                CS	
  
    Video	
                                                                                     InternaJonal	
  
   Masters	
             S3	
              DRM	
             Sign-­‐Up	
      Metadata	
          CS	
  lookup	
  


                                                                                Device	
         DiagnosJcs	
  
     EC2	
         EMR	
  Hadoop	
     CDN	
  rouJng	
        Search	
          Config	
           &	
  AcJons	
  


                                                              Movie	
         TV	
  Movie	
       Customer	
  
      S3	
              Hive	
         Bookmarks	
           Choosing	
       Choosing	
           Call	
  Log	
  


                     Business	
                                                 Social	
  
    CDNs	
                                Logging	
           RaJngs	
        Facebook	
        CS	
  AnalyJcs	
  
                   Intelligence	
  
Movie	
  Encoding	
  farm	
  (2009)	
  
                •    Tens	
  of	
  thousands	
  of	
  videos	
  
Content	
       •    Thousands	
  of	
  EC2	
  instances	
  
   Video	
      •    Encoding	
  apps	
  on	
  Windows/Linux	
  
  Masters	
     •    ~100	
  files	
  per	
  video	
  
                •    Petabytes	
  of	
  S3	
  
    EC2	
  
                •    Content	
  Delivery	
  Networks	
  
                     	
  
     S3	
            “NeGlix	
  is	
  one	
  of	
  the	
  largest	
  customers	
  
                          of	
  the	
  biggest	
  CDNs	
  Level3,	
  Akamai	
  
                          and	
  Limelight”	
  
   CDNs	
  
Cloud	
  Encoding	
  Pipeline	
  

                                                                   Encode	
       S3	
       Encode	
              S3	
  
Movie	
       Master	
                  Network	
      S3	
                                                                   Copy	
  to	
      CDN	
       Stream	
  
Studios	
                  Ne:lix	
                   Master	
     Mezza-­‐     Mezza-­‐    to	
  	
  ~100	
     Origin	
  
                                                                                                                                               Origin	
  
               Tape	
                   Upload	
                                 nine	
                           files	
       CDN	
                         to	
  TV	
  
                                                                    nine	
                    files	
  




     Licensed	
  content	
  is	
  provided	
  to	
  Ne:lix	
  via	
  Post	
  ProducJon	
  Companies	
  
     Many	
  formats	
  are	
  reduced	
  to	
  a	
  single	
  high	
  quality	
  mezzanine	
  format	
  on	
  S3	
  
     Individual	
  formats	
  and	
  speeds	
  etc.	
  are	
  encoded	
  in	
  ~100	
  files	
  
          	
  Many	
  formats	
  for	
  older	
  and	
  newer	
  hardware	
  and	
  various	
  game	
  consoles	
  
          	
  Many	
  speeds	
  from	
  mobile	
  through	
  standard	
  and	
  high	
  definiJon	
  
          	
  SubJtles	
  for	
  many	
  languages,	
  sJll	
  frames	
  etc.	
  
     StaJc	
  files	
  are	
  copied	
  to	
  each	
  Content	
  Delivery	
  Network’s	
  “origin	
  server”	
  
     CDNs	
  migrate	
  files	
  to	
  “edge	
  servers”	
  near	
  the	
  end	
  user	
  as	
  needed	
  
     Files	
  stream	
  to	
  PC/Mac/iPad	
  or	
  TV	
  over	
  HTTP	
  using	
  “range	
  get”	
  to	
  move	
  chunks	
  
Ne:lix	
  EC2	
  Instances	
  per	
  Account	
  
          (summer	
  2010,	
  producJon	
  is	
  much	
  higher	
  now…)	
  
“Many	
  Thousands”	
  




           Content	
  Encoding	
  




          Test	
  and	
  ProducJon	
  
                                             Log	
  Analysis	
  

                                         “Several	
  Months”	
  
Hadoop	
  -­‐	
  ElasJc	
  Map-­‐Reduce	
  (2009)	
  
                     •    Web	
  Access	
  Logs	
  
 Logs	
  
                     •    Streaming	
  Service	
  Logs	
  
        S3	
         •    Terabytes	
  per	
  day	
  scale	
  
     EMR	
  
                     •    Easy	
  Hadoop	
  via	
  Amazon	
  EMR	
  
    Hadoop	
  
                     •    Hive	
  SQL	
  “Data	
  Mart”	
  
      Hive	
         •    Gateway	
  to	
  Datacenter	
  BI	
  with	
  Pig	
  
                     	
  
                     See	
  www.slideshare.net/Ne:lix	
  for	
  more	
  details	
  
    Business	
  
  Intelligence	
  
Streaming	
  Service	
  Back-­‐end	
  
             (early	
  2010)	
  
                •    PC/Mac	
  Silverlight	
  Player	
  Support	
  
Play	
  
                •    Highly	
  available	
  “play	
  bu>on”	
  
   DRM	
        •    DRM	
  Key	
  Management	
  
   CDN	
  
                •    Generate	
  route	
  to	
  stream	
  on	
  CDN	
  
  rouJng	
  
                •    Lookup	
  bookmark	
  for	
  user/movie	
  
Bookmarks	
     •    Update	
  bookmark	
  for	
  user/movie	
  
                •    Log	
  quality	
  of	
  service	
  
 Logging	
  
Web	
  site,	
  a	
  page	
  at	
  a	
  Jme	
  
         (through	
  2010	
  and	
  2011)	
  
                •      Clean	
  presentaJon	
  layer	
  rewrite	
  
WWW	
           •      Search	
  auto-­‐complete	
  
  Signup	
  
                •      Search	
  backend	
  and	
  landing	
  page	
  
                •      Movie	
  and	
  genre	
  choosing	
  
  Search	
      •      Star	
  raJngs	
  and	
  recommendaJons	
  
                •      Similar	
  movies	
  
  Movie	
       •      Page	
  by	
  page	
  to	
  100%	
  of	
  views	
  
 Choosing	
  
                	
  
                         (Account	
  signup	
  parJally	
  migrated	
  in	
  2011)	
  
  RaJngs	
  
API	
  for	
  TV	
  devices	
  and	
  iPhone	
  etc.	
  
                       (2010)	
  
                    •     Public	
  API:	
  developer.ne:lix.com	
  
 API	
              •     Interfaces	
  to	
  everything	
  else	
  
  Metadata	
        •     TV	
  Device	
  ConfiguraJon	
  
                    •     Personalized	
  movie	
  choosing	
  
    Device	
  
    Config	
         •     Facebook	
  integraJon	
  
  TV	
  Movie	
  
                    •     See	
  presentaJons	
  on	
  slideshare	
  
  Choosing	
             “NeGlix	
  is	
  an	
  API	
  for	
  streaming	
  to	
  TVs	
  
    Social	
                   	
  (we	
  also	
  do	
  DVD’s	
  and	
  a	
  web	
  site)”	
  
  Facebook	
  
Customer	
  Service	
  Tools	
  (2011)	
  
                      •  Support	
  external	
  contract	
  CS	
  
  CS	
                    –  Canada	
  –	
  French	
  Canadian	
  
InternaJonal	
  
  CS	
  lookup	
  
                          –  LaJn	
  America	
  –	
  Spanish/Portuguese	
  
                          –  Future	
  flexibility	
  worldwide	
  
 DiagnosJcs	
  
  &	
  AcJons	
       •  Needed	
  CS	
  tools	
  for	
  Partners	
  
                          –  Rewrote	
  for	
  cloud	
  architecture	
  
  Customer	
  
   Call	
  Log	
          –  Migrated	
  from	
  Oracle	
  to	
  Cassandra	
  
                          –  Web	
  based	
  SOA	
  tools	
  
 CS	
  AnalyJcs	
  
Takeaway	
  
                                                     	
  
 NeGlix	
  has	
  built	
  and	
  deployed	
  a	
  scalable	
  global	
  PlaGorm	
  as	
  a	
  Service.	
  
                                                     	
  
Key	
  components	
  of	
  the	
  NeGlix	
  PaaS	
  are	
  being	
  released	
  as	
  Open	
  Source	
  
                   projects	
  so	
  you	
  can	
  build	
  your	
  own	
  custom	
  PaaS.	
  
                                                     	
  
                                  h>p://github.com/Ne:lix	
  
                                 h>p://techblog.ne:lix.com	
  
                                 h>p://slideshare.net/Ne:lix	
  
                                               	
  
                          h>p://www.linkedin.com/in/adriancockcro6	
  
                                  @adrianco	
  #ne:lixcloud	
  
                                               	
  

                                  End	
  of	
  Part	
  1	
  of	
  3	
  

More Related Content

What's hot

AWS networking fundamentals
AWS networking fundamentalsAWS networking fundamentals
AWS networking fundamentals
Amazon Web Services
 
Aws 101 A walk-through the aws cloud (2013)
Aws 101  A walk-through the aws cloud (2013)Aws 101  A walk-through the aws cloud (2013)
Aws 101 A walk-through the aws cloud (2013)
Martin Yan
 
IaC on AWS Cloud
IaC on AWS CloudIaC on AWS Cloud
IaC on AWS Cloud
Bhuvaneswari Subramani
 
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Emerging Trends in Hybrid-Cloud & Multi-Cloud StrategiesEmerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Chaitanya Atreya
 
Cost Optimization on AWS
Cost Optimization on AWSCost Optimization on AWS
Cost Optimization on AWS
Amazon Web Services
 
AWS Direct Connect
AWS Direct ConnectAWS Direct Connect
AWS Direct Connect
Amazon Web Services
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
Karthik Gaekwad
 
(NET406) Deep Dive: AWS Direct Connect and VPNs
(NET406) Deep Dive: AWS Direct Connect and VPNs(NET406) Deep Dive: AWS Direct Connect and VPNs
(NET406) Deep Dive: AWS Direct Connect and VPNs
Amazon Web Services
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
Amazon Web Services
 
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트:: AWS Summit Online Korea 2020
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트::  AWS Summit Online Korea 2020천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트::  AWS Summit Online Korea 2020
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트:: AWS Summit Online Korea 2020
Amazon Web Services Korea
 
AWS Business Essentials Day
AWS Business Essentials DayAWS Business Essentials Day
AWS Business Essentials Day
Amazon Web Services
 
AWS 101 - An Introduction to the Amazon Cloud
AWS 101  - An Introduction to the Amazon CloudAWS 101  - An Introduction to the Amazon Cloud
AWS 101 - An Introduction to the Amazon Cloud
CloudHesive
 
AWS Cloud Practitioner Tutorial | Edureka
AWS Cloud Practitioner Tutorial | EdurekaAWS Cloud Practitioner Tutorial | Edureka
AWS Cloud Practitioner Tutorial | Edureka
Edureka!
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
Amazon Web Services
 
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Amazon Web Services
 
Aws introduction
Aws introductionAws introduction
Aws introduction
MouryaKumar Reddy Rajala
 
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
Amazon Web Services Korea
 
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
Amazon Web Services Korea
 
App Modernization Pitch Deck.pptx
App Modernization Pitch Deck.pptxApp Modernization Pitch Deck.pptx
App Modernization Pitch Deck.pptx
MONISH407209
 
AWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
AWS Networking Fundamentals - SVC304 - Anaheim AWS SummitAWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
AWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
Amazon Web Services
 

What's hot (20)

AWS networking fundamentals
AWS networking fundamentalsAWS networking fundamentals
AWS networking fundamentals
 
Aws 101 A walk-through the aws cloud (2013)
Aws 101  A walk-through the aws cloud (2013)Aws 101  A walk-through the aws cloud (2013)
Aws 101 A walk-through the aws cloud (2013)
 
IaC on AWS Cloud
IaC on AWS CloudIaC on AWS Cloud
IaC on AWS Cloud
 
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Emerging Trends in Hybrid-Cloud & Multi-Cloud StrategiesEmerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
Emerging Trends in Hybrid-Cloud & Multi-Cloud Strategies
 
Cost Optimization on AWS
Cost Optimization on AWSCost Optimization on AWS
Cost Optimization on AWS
 
AWS Direct Connect
AWS Direct ConnectAWS Direct Connect
AWS Direct Connect
 
Why to Cloud Native
Why to Cloud NativeWhy to Cloud Native
Why to Cloud Native
 
(NET406) Deep Dive: AWS Direct Connect and VPNs
(NET406) Deep Dive: AWS Direct Connect and VPNs(NET406) Deep Dive: AWS Direct Connect and VPNs
(NET406) Deep Dive: AWS Direct Connect and VPNs
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
 
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트:: AWS Summit Online Korea 2020
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트::  AWS Summit Online Korea 2020천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트::  AWS Summit Online Korea 2020
천만사용자를 위한 AWS 클라우드 아키텍처 진화하기 – 문종민, AWS솔루션즈 아키텍트:: AWS Summit Online Korea 2020
 
AWS Business Essentials Day
AWS Business Essentials DayAWS Business Essentials Day
AWS Business Essentials Day
 
AWS 101 - An Introduction to the Amazon Cloud
AWS 101  - An Introduction to the Amazon CloudAWS 101  - An Introduction to the Amazon Cloud
AWS 101 - An Introduction to the Amazon Cloud
 
AWS Cloud Practitioner Tutorial | Edureka
AWS Cloud Practitioner Tutorial | EdurekaAWS Cloud Practitioner Tutorial | Edureka
AWS Cloud Practitioner Tutorial | Edureka
 
DevOps on AWS
DevOps on AWSDevOps on AWS
DevOps on AWS
 
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
Introduction to the Well-Architected Framework and Tool - SVC208 - Anaheim AW...
 
Aws introduction
Aws introductionAws introduction
Aws introduction
 
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
마이그레이션과 함께 시작되는 Cloud Financial Management 전략 세우기-곽내인, AWS Cloud Financial Ma...
 
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
고급 클라우드 아키텍처 방법론- 양승도 솔루션즈 아키텍트:: AWS Cloud Track 2 Advanced
 
App Modernization Pitch Deck.pptx
App Modernization Pitch Deck.pptxApp Modernization Pitch Deck.pptx
App Modernization Pitch Deck.pptx
 
AWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
AWS Networking Fundamentals - SVC304 - Anaheim AWS SummitAWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
AWS Networking Fundamentals - SVC304 - Anaheim AWS Summit
 

Similar to Cloud Architecture Tutorial - Why and What (1of 3)

Netflix in the Cloud at SV Forum
Netflix in the Cloud at SV ForumNetflix in the Cloud at SV Forum
Netflix in the Cloud at SV Forum
Adrian Cockcroft
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Adrian Cockcroft
 
Migrating to Public Cloud
Migrating to Public CloudMigrating to Public Cloud
Migrating to Public Cloud
Adrian Cockcroft
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
Srijan Technologies
 
Netflix in the Cloud
Netflix in the CloudNetflix in the Cloud
Netflix in the Cloud
Adrian Cockcroft
 
Netflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at GlueconNetflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at Gluecon
Adrian Cockcroft
 
Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09
Rex Wang
 
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
IndicThreads
 
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial IntroductionGluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Adrian Cockcroft
 
Oracle cloud oagi
Oracle cloud oagiOracle cloud oagi
Oracle cloud oagi
Mathews Job
 
Continuously Design your Continuous Deployment
Continuously Design your Continuous DeploymentContinuously Design your Continuous Deployment
Continuously Design your Continuous Deployment
Michael Elder
 
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
Riccardo Romani
 
Openstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platformOpenstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platform
Nagaraj Shenoy
 
FLUX - Crash Course in Cloud 2.0
FLUX - Crash Course in Cloud 2.0 FLUX - Crash Course in Cloud 2.0
FLUX - Crash Course in Cloud 2.0
Mark Hinkle
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
Alicja Sieminska
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
MongoDB
 
Cloud computing What Why How
Cloud computing What Why HowCloud computing What Why How
Cloud computing What Why How
Asian Institute of Technology (AIT)
 
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, SmileOCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware
 
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Marc Dutoo
 
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick HamonOpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
eNovance
 

Similar to Cloud Architecture Tutorial - Why and What (1of 3) (20)

Netflix in the Cloud at SV Forum
Netflix in the Cloud at SV ForumNetflix in the Cloud at SV Forum
Netflix in the Cloud at SV Forum
 
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with NotesYow Conference Dec 2013 Netflix Workshop Slides with Notes
Yow Conference Dec 2013 Netflix Workshop Slides with Notes
 
Migrating to Public Cloud
Migrating to Public CloudMigrating to Public Cloud
Migrating to Public Cloud
 
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
[Srijan Wednesday Webinars] How to Build a Cloud Native Platform for Enterpri...
 
Netflix in the Cloud
Netflix in the CloudNetflix in the Cloud
Netflix in the Cloud
 
Netflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at GlueconNetflix Architecture Tutorial at Gluecon
Netflix Architecture Tutorial at Gluecon
 
Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09Oracle Keynote Cloud Expo 11-04-09
Oracle Keynote Cloud Expo 11-04-09
 
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
Current State of Affairs – Cloud Computing - Indicthreads Cloud Computing Con...
 
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial IntroductionGluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
Gluecon 2013 - NetflixOSS Cloud Native Tutorial Introduction
 
Oracle cloud oagi
Oracle cloud oagiOracle cloud oagi
Oracle cloud oagi
 
Continuously Design your Continuous Deployment
Continuously Design your Continuous DeploymentContinuously Design your Continuous Deployment
Continuously Design your Continuous Deployment
 
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
Software Defined IT @ Evento SOIEL Roma 6 Aprile 2017
 
Openstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platformOpenstack - Enterprise cloud management platform
Openstack - Enterprise cloud management platform
 
FLUX - Crash Course in Cloud 2.0
FLUX - Crash Course in Cloud 2.0 FLUX - Crash Course in Cloud 2.0
FLUX - Crash Course in Cloud 2.0
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
Java Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the CloudJava Agile ALM: OTAP and DevOps in the Cloud
Java Agile ALM: OTAP and DevOps in the Cloud
 
Cloud computing What Why How
Cloud computing What Why HowCloud computing What Why How
Cloud computing What Why How
 
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, SmileOCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
 
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
 
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick HamonOpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
OpenStack in Action! 5 - Dell - OpenStack powered solutions - Patrick Hamon
 

More from Adrian Cockcroft

Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
Adrian Cockcroft
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
Adrian Cockcroft
 
Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013
Adrian Cockcroft
 
Netflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search RoadshowNetflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search Roadshow
Adrian Cockcroft
 
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Adrian Cockcroft
 
Gluecon keynote
Gluecon keynoteGluecon keynote
Gluecon keynote
Adrian Cockcroft
 
Dystopia as a Service
Dystopia as a ServiceDystopia as a Service
Dystopia as a Service
Adrian Cockcroft
 
Netflix and Open Source
Netflix and Open SourceNetflix and Open Source
Netflix and Open Source
Adrian Cockcroft
 
NetflixOSS Meetup
NetflixOSS MeetupNetflixOSS Meetup
NetflixOSS Meetup
Adrian Cockcroft
 
AWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at NetflixAWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at Netflix
Adrian Cockcroft
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSF
Adrian Cockcroft
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source Platform
Adrian Cockcroft
 
Cassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWSCassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWS
Adrian Cockcroft
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Adrian Cockcroft
 
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Adrian Cockcroft
 
Global Netflix Platform
Global Netflix PlatformGlobal Netflix Platform
Global Netflix Platform
Adrian Cockcroft
 
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Adrian Cockcroft
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global Cassandra
Adrian Cockcroft
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
Adrian Cockcroft
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
Adrian Cockcroft
 

More from Adrian Cockcroft (20)

Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
Flowcon (added to for CMG) Keynote talk on how Speed Wins and how Netflix is ...
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
 
Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013Bottleneck analysis - Devopsdays Silicon Valley 2013
Bottleneck analysis - Devopsdays Silicon Valley 2013
 
Netflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search RoadshowNetflix Global Applications - NoSQL Search Roadshow
Netflix Global Applications - NoSQL Search Roadshow
 
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)
 
Gluecon keynote
Gluecon keynoteGluecon keynote
Gluecon keynote
 
Dystopia as a Service
Dystopia as a ServiceDystopia as a Service
Dystopia as a Service
 
Netflix and Open Source
Netflix and Open SourceNetflix and Open Source
Netflix and Open Source
 
NetflixOSS Meetup
NetflixOSS MeetupNetflixOSS Meetup
NetflixOSS Meetup
 
AWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at NetflixAWS Re:Invent - High Availability Architecture at Netflix
AWS Re:Invent - High Availability Architecture at Netflix
 
Architectures for High Availability - QConSF
Architectures for High Availability - QConSFArchitectures for High Availability - QConSF
Architectures for High Availability - QConSF
 
SV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source PlatformSV Forum Platform Architecture SIG - Netflix Open Source Platform
SV Forum Platform Architecture SIG - Netflix Open Source Platform
 
Cassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWSCassandra Performance and Scalability on AWS
Cassandra Performance and Scalability on AWS
 
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)Cloud Architecture Tutorial - Platform Component Architecture (2of3)
Cloud Architecture Tutorial - Platform Component Architecture (2of3)
 
Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)Cloud Architecture Tutorial - Running in the Cloud (3of3)
Cloud Architecture Tutorial - Running in the Cloud (3of3)
 
Global Netflix Platform
Global Netflix PlatformGlobal Netflix Platform
Global Netflix Platform
 
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
Global Netflix - HPTS Workshop - Scaling Cassandra benchmark to over 1M write...
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global Cassandra
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
 
Performance architecture for cloud connect
Performance architecture for cloud connectPerformance architecture for cloud connect
Performance architecture for cloud connect
 

Recently uploaded

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
Ivanti
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
Adam Dunkels
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 

Recently uploaded (20)

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
Patch Tuesday de julio
Patch Tuesday de julioPatch Tuesday de julio
Patch Tuesday de julio
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 

Cloud Architecture Tutorial - Why and What (1of 3)

  • 1. Cloud  Architecture  Tutorial   How  Ne3lix  Built  a  Scalable  Java  oriented  PaaS  running  on  AWS   Part  1  of  3   Qcon  London  March  5th,  2012   Adrian  Cockcro6   @adrianco  #ne:lixcloud   h>p://www.linkedin.com/in/adriancockcro6  
  • 2. Tutorial  Abstract  –  Set  Context   •  StarJng  with  the  usual  quesJons:  “Why  Ne:lix,  why  cloud,  why  AWS?”   •  This  tutorial  explains  which  business  models  and  applicaJons  benefit  most  from  cloud,  what  to   look  for  in  a  cloud  provider,  and  how  the  tradiJonal  enterprise  compuJng  marketplace  is  being   disrupted.   •  Moving  on  to  the  next  quesJon:  “What  can  run  in  the  cloud?”  a  step  by  step  approach  to  cloud   migraJon  is  described,  along  with  a  varied  set  of  use  cases  for  both  customer  facing  and  internal   web  services,  big  data  analyJcs  and  bulk  computaJon.  Cloud  migraJon  starts  by  moving   developers  to  work  on  cloud  using  “boot  camp”  training  sessions,  then  a6er  building  out  the  iniJal   core  pla:orm,  the  first  applicaJons  are  launched.   •  The  real  meat  of  the  tutorial  comes  when  we  look  at  how  to  construct  an  applicaJon  with  a  host  of   important  properJes:  elasJc,  dynamic,  scalable,  agile,  fast,  cheap,  robust,  durable,  observable,   secure.  Over  the  last  three  years  Ne:lix  has  figured  out  cloud  based  soluJons  with  these   properJes,  deployed  them  globally  at  large  scale  and  refined  them  into  a  global  Java  oriented   Pla:orm  as  a  Service.  The  PaaS  is  based  on  low  cost  open  source  building  blocks  such  as  Apache   Tomcat,  Apache  Cassandra,  and  Memcached.  Components  of  this  pla:orm  are  in  the  process  of   being  open-­‐sourced  by  Ne:lix,  so  that  other  companies  can  get  a  start  on  building  their  own   customized  PaaS  that  leverages  advanced  features  of  AWS  and  supports  rapid  agile  development.   •  The  architecture  is  described  in  terms  of  anJ-­‐pa>erns  -­‐  things  to  avoid  in  the  datacenter  to  cloud   transiJon.  A  scalable  global  persistence  Jer  based  on  Cassandra  provides  a  highly  available  and   durable  under-­‐pinning.  Lessons  learned  will  cover  soluJons  to  common  problems,  availability  and   robustness,  observability.  A>endees  should  leave  the  tutorial  with  a  clear  understanding  of  what  is   different  about  cloud  architectures,  why,  what  and  how  to  make  the  transiJon,  and  a  set  of  flexible   and  scalable  open  source  building  blocks  that  can  be  used  to  construct  their  own  cloud  pla:orm.  
  • 3. PresentaJon  vs.  Tutorial   •  PresentaJon   –  Short  duraJon,  focused  subject   –  One  presenter  to  many  anonymous  audience   –  A  few  quesJons  at  the  end   •  Tutorial   –  Time  to  explore  in  and  around  the  subject   –  Tutor  gets  to  know  the  audience   –  Discussion,  rat-­‐holes,  “bring  out  your  dead”  
  • 4. Tutorial  SecJons   Intro:  Who  are  you,  what  are  your  quesJons?     Part  1  -­‐  Why  use  cloud,  what  runs  in  the  cloud   Part  2  -­‐  Pla:orm  component  architecture   Part  3  -­‐  Running  in  the  cloud  
  • 5. Adrian  Cockcro6   •  Director,  Architecture  for  Cloud  Systems,  Ne:lix  Inc.   –  Previously  Director  for  PersonalizaJon  Pla:orm   •  DisJnguished  Availability  Engineer,  eBay  Inc.  2004-­‐7   –  Founding  member  of  eBay  Research  Labs   •  DisJnguished  Engineer,  Sun  Microsystems  Inc.  1988-­‐2004   –  2003-­‐4  Chief  Architect  High  Performance  Technical  CompuJng   –  2001  Author:  Capacity  Planning  for  Web  Services   –  1999  Author:  Resource  Management   –  1995  &  1998  Author:  Sun  Performance  and  Tuning   –  1996  Japanese  EdiJon  of  Sun  Performance  and  Tuning   •   SPARC  &  Solarisパフォーマンスチューニング (サンソフトプレスシリーズ)   •  More   –  Twi>er  @adrianco  –  Blog  h>p://perfcap.blogspot.com   –  PresentaJons  at  h>p://www.slideshare.net/adrianco  
  • 6. A>endee  IntroducJons   •  Who  are  you,  where  do  you  work   •  Why  are  you  here  today,  what  do  you  need   •  “Bring  out  your  dead”   –  Do  you  have  a  specific  problem  or  quesJon?   –  One  sentence  elevator  pitch    
  • 7. Why  Ne:lix,  Why  Cloud,  Why   AWS   Part  1  of  3  
  • 8. Ne:lix  Inc.   With  more  than  23  million  streaming  members  in  the   United  States,  Canada,  LaBn  America,  the  United   Kingdom  and  Ireland,  NeGlix,  Inc.  is  the  world's   leading  internet  subscripBon  service  for  enjoying   movies  and  TV  series..   Source:  h>p://ir.ne:lix.com  
  • 9. What  kind  of  Cloud?   •  So6ware  as  a  Service  –  SaaS   –  Replaces  in  house  applicaJons   –  Targets  end  users   •  Pla:orm  as  a  Service  –  PaaS   –  Replaces  in  house  operaJons  funcJons   –  Targets  developers   •  Infrastructure  as  a  Service  –  IaaS   –  Replaces  in  house  datacenter  capacity   –  Targets  developers  and  ITops  
  • 10. What  Ne:lix  Did   •  Moved  to  SaaS   –  Corporate  IT  –  Workday  etc.   –  Tools  –  Pagerduty,  AppDynamics,  ElasJc  MapReduce   •  Built  our  own  PaaS  <-­‐  today’s  focus   –  Customized  to  make  our  developers  producJve   –  When  we  started,  we  had  li>le  choice   •  Moved  incremental  capacity  to  IaaS   –  No  new  datacenter  space  since  2008  as  we  grew   –  Moved  our  streaming  apps  to  the  cloud  
  • 11. Why  Use  Public  Cloud?  
  • 14. Data  Center   Ne:lix  could  not   build  new   datacenters  fast   enough   Capacity  growth  is  acceleraJng,  unpredictable   Product  launch  spikes  -­‐  iPhone,  Wii,  PS3,  Xbox   InternaJonal  –  Canada,  LaJn  America,  UK/Ireland  
  • 15. Out-­‐Growing  Data  Center   h>p://techblog.ne:lix.com/2011/02/redesigning-­‐ne:lix-­‐api.html   37x  Growth  Jan   2010-­‐Jan  2011   Datacenter   Capacity  
  • 16. Ne:lix.com  is  now  ~100%  Cloud   A  few  small  back  end  data  sources  sJll  in  progress   All  internaJonal  product  is  cloud  based   USA  specific  logisJcs  remains  in  the  Datacenter   Working  on  SOX,  PCI  as  scope  starts  to  include  AWS  
  • 17. Ne:lix  Choice  was  AWS  with  our   own  pla:orm  and  tools   Unique  pla:orm  requirements  and   extreme  scale,  agility  and  flexibility  
  • 18. Leverage  AWS  Scale   “the  biggest  public  cloud”   AWS  investment  in  features  and  automaJon   Use  AWS  zones  and  regions  for  high  availability,   scalability  and  global  deployment  
  • 19. But  isn’t  Amazon  a  compeJtor?   Many  products  that  compete  with  Amazon  run  on  AWS   We  are  a  “poster  child”  for  the  AWS  Architecture   Ne:lix  is  one  of  the  biggest  AWS  customers   Co-­‐opeJJon  -­‐  compeJtors  are  also  partners  
  • 20. Could  Ne:lix  use  another  cloud?   Would  be  nice,  we  use  three  interchangeable  CDN  Vendors   But  no-­‐one  else  has  the  scale  and  features  of  AWS   You  have  to  be  this  tall  to  ride  this  ride…   Maybe  in  2-­‐3  years?  
  • 21. We  want  to  use  clouds,   we  don’t  have  Jme  to  build  them   Public  cloud  for  agility  and  scale   We  use  electricity  too,  but  don’t  want  to  build  our  own  power  staJon…   AWS  because  they  are  big  enough  to  allocate  thousands  of  instances  per   hour  when  we  need  to  
  • 22. Amazon Cloud Terminology Reference See http://aws.amazon.com/ This is not a full list of Amazon Web Service features •  AWS  –  Amazon  Web  Services  (common  name  for  Amazon  cloud)   •  AMI  –  Amazon  Machine  Image  (archived  boot  disk,  Linux,  Windows  etc.  plus  applicaJon  code)   •  EC2  –  ElasJc  Compute  Cloud   –  Range  of  virtual  machine  types  m1,  m2,  c1,  cc,  cg.  Varying  memory,  CPU  and  disk  configuraJons.   –  Instance  –  a  running  computer  system.  Ephemeral,  when  it  is  de-­‐allocated  nothing  is  kept.   –  Reserved  Instances  –  pre-­‐paid  to  reduce  cost  for  long  term  usage   –  Availability  Zone  –  datacenter  with  own  power  and  cooling  hosJng  cloud  instances   –  Region  –  group  of  Avail  Zones  –  US-­‐East,  US-­‐West,  EU-­‐Eire,  Asia-­‐Singapore,  Asia-­‐Japan,  SA-­‐Brazil,  US-­‐Gov   •  ASG  –  Auto  Scaling  Group  (instances  booJng  from  the  same  AMI)   •  S3  –  Simple  Storage  Service  (h>p  access)   •  EBS  –  ElasJc  Block  Storage  (network  disk  filesystem  can  be  mounted  on  an  instance)   •  RDS  –  RelaJonal  Database  Service  (managed  MySQL  master  and  slaves)   •  DynamoDB/SDB  –  Simple  Data  Base  (hosted  h>p  based  NoSQL  datastore,  DynamoDB  replaces  SDB)   •  SQS  –  Simple  Queue  Service  (h>p  based  message  queue)   •  SNS  –  Simple  NoJficaJon  Service  (h>p  and  email  based  topics  and  messages)   •  EMR  –  ElasJc  Map  Reduce  (automaJcally  managed  Hadoop  cluster)   •  ELB  –  ElasJc  Load  Balancer   •  EIP  –  ElasJc  IP  (stable  IP  address  mapping  assigned  to  instance  or  ELB)   •  VPC  –  Virtual  Private  Cloud  (single  tenant,  more  flexible  network  and  security  constructs)   •  DirectConnect  –  secure  pipe  from  AWS  VPC  to  external  datacenter   •  IAM  –  IdenJty  and  Access  Management  (fine  grain  role  based  security  keys)  
  • 23. AWS  and  the  Seven  Dwarfs   •  Public  Cloud  AlternaJves  to  AWS   –  Far  fewer  features,  much  smaller  scale   –  Less  mature  APIs,  many  variants  of  APIs   –  Some  have  addiJonal  features  or  performance   •  Private  Cloud  AlternaJves   –  O6en  harder  to  build  and  run  than  you  think   –  Without  scale  and  mulJ-­‐tenancy,  much  higher  costs   –  O6en  driven  by  ITops  needs  rather  than  developers  
  • 24. Some  AlternaJve  Public  Clouds   IaaS  that  you  could  build  your  own  PaaS  architecture  on   •  OpenStack  Based   –  Rackspace   –  HP  Cloud   –  ATT  Cloud   •  GoGrid   •  Joyent  –  Solaris  in  the  cloud   •  Memset  –  UK  based  
  • 25. What  about  other  PaaS?   •  CloudFoundry  –  Open  Source  by  VMWare   –  Developer-­‐friendly,  easy  to  get  started   –  Missing  scale  and  some  enterprise  features   •  Rightscale   –  Widely  used  to  abstract  away  from  AWS   –  Creates  it’s  own  lock-­‐in  problem…   •  AWS  is  growing  into  this  space   –  We  didn’t  want  a  vendor  between  us  and  AWS   –  We  wanted  to  build  a  thin  PaaS,  that  gets  thinner  
  • 26. Enterprise  Market  DisrupJon   •  Enterprise  CompuJng  Vendors  $$$$$$   –  IBM,  HP,  Dell,  Oracle,  EMC,  NetApp…   –  CIO/ITOps  integrates  and  provisions   –  TradiJonal,  and  moving  towards  private  clouds   •  IaaS  Vendors  Sell  Directly  to  Developers  $   –  Bypassing  ITOps  with  stealth  cloud  based  projects   –  Bypassing  enterprise  vendor  supply  chain   –  Low  margin,  low  fricJon,  dollar  at  a  Jme  
  • 27. What  Runs  in  the  Cloud?   Step  by  Step  Ne:lix  Product   TransiJon  
  • 28. Ne:lix  Deployed  on  AWS   2009   2009   2010   2010   2010   2011   Content   Logs   Play   WWW   API   CS   Video   InternaJonal   Masters   S3   DRM   Sign-­‐Up   Metadata   CS  lookup   Device   DiagnosJcs   EC2   EMR  Hadoop   CDN  rouJng   Search   Config   &  AcJons   Movie   TV  Movie   Customer   S3   Hive   Bookmarks   Choosing   Choosing   Call  Log   Business   Social   CDNs   Logging   RaJngs   Facebook   CS  AnalyJcs   Intelligence  
  • 29. Movie  Encoding  farm  (2009)   •  Tens  of  thousands  of  videos   Content   •  Thousands  of  EC2  instances   Video   •  Encoding  apps  on  Windows/Linux   Masters   •  ~100  files  per  video   •  Petabytes  of  S3   EC2   •  Content  Delivery  Networks     S3   “NeGlix  is  one  of  the  largest  customers   of  the  biggest  CDNs  Level3,  Akamai   and  Limelight”   CDNs  
  • 30. Cloud  Encoding  Pipeline   Encode   S3   Encode   S3   Movie   Master   Network   S3   Copy  to   CDN   Stream   Studios   Ne:lix   Master   Mezza-­‐ Mezza-­‐ to    ~100   Origin   Origin   Tape   Upload   nine   files   CDN   to  TV   nine   files   Licensed  content  is  provided  to  Ne:lix  via  Post  ProducJon  Companies   Many  formats  are  reduced  to  a  single  high  quality  mezzanine  format  on  S3   Individual  formats  and  speeds  etc.  are  encoded  in  ~100  files    Many  formats  for  older  and  newer  hardware  and  various  game  consoles    Many  speeds  from  mobile  through  standard  and  high  definiJon    SubJtles  for  many  languages,  sJll  frames  etc.   StaJc  files  are  copied  to  each  Content  Delivery  Network’s  “origin  server”   CDNs  migrate  files  to  “edge  servers”  near  the  end  user  as  needed   Files  stream  to  PC/Mac/iPad  or  TV  over  HTTP  using  “range  get”  to  move  chunks  
  • 31. Ne:lix  EC2  Instances  per  Account   (summer  2010,  producJon  is  much  higher  now…)   “Many  Thousands”   Content  Encoding   Test  and  ProducJon   Log  Analysis   “Several  Months”  
  • 32. Hadoop  -­‐  ElasJc  Map-­‐Reduce  (2009)   •  Web  Access  Logs   Logs   •  Streaming  Service  Logs   S3   •  Terabytes  per  day  scale   EMR   •  Easy  Hadoop  via  Amazon  EMR   Hadoop   •  Hive  SQL  “Data  Mart”   Hive   •  Gateway  to  Datacenter  BI  with  Pig     See  www.slideshare.net/Ne:lix  for  more  details   Business   Intelligence  
  • 33. Streaming  Service  Back-­‐end   (early  2010)   •  PC/Mac  Silverlight  Player  Support   Play   •  Highly  available  “play  bu>on”   DRM   •  DRM  Key  Management   CDN   •  Generate  route  to  stream  on  CDN   rouJng   •  Lookup  bookmark  for  user/movie   Bookmarks   •  Update  bookmark  for  user/movie   •  Log  quality  of  service   Logging  
  • 34. Web  site,  a  page  at  a  Jme   (through  2010  and  2011)   •  Clean  presentaJon  layer  rewrite   WWW   •  Search  auto-­‐complete   Signup   •  Search  backend  and  landing  page   •  Movie  and  genre  choosing   Search   •  Star  raJngs  and  recommendaJons   •  Similar  movies   Movie   •  Page  by  page  to  100%  of  views   Choosing     (Account  signup  parJally  migrated  in  2011)   RaJngs  
  • 35. API  for  TV  devices  and  iPhone  etc.   (2010)   •  Public  API:  developer.ne:lix.com   API   •  Interfaces  to  everything  else   Metadata   •  TV  Device  ConfiguraJon   •  Personalized  movie  choosing   Device   Config   •  Facebook  integraJon   TV  Movie   •  See  presentaJons  on  slideshare   Choosing   “NeGlix  is  an  API  for  streaming  to  TVs   Social    (we  also  do  DVD’s  and  a  web  site)”   Facebook  
  • 36. Customer  Service  Tools  (2011)   •  Support  external  contract  CS   CS   –  Canada  –  French  Canadian   InternaJonal   CS  lookup   –  LaJn  America  –  Spanish/Portuguese   –  Future  flexibility  worldwide   DiagnosJcs   &  AcJons   •  Needed  CS  tools  for  Partners   –  Rewrote  for  cloud  architecture   Customer   Call  Log   –  Migrated  from  Oracle  to  Cassandra   –  Web  based  SOA  tools   CS  AnalyJcs  
  • 37. Takeaway     NeGlix  has  built  and  deployed  a  scalable  global  PlaGorm  as  a  Service.     Key  components  of  the  NeGlix  PaaS  are  being  released  as  Open  Source   projects  so  you  can  build  your  own  custom  PaaS.     h>p://github.com/Ne:lix   h>p://techblog.ne:lix.com   h>p://slideshare.net/Ne:lix     h>p://www.linkedin.com/in/adriancockcro6   @adrianco  #ne:lixcloud     End  of  Part  1  of  3