Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek)
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek)



Presented at JAX London 2013 ...

Presented at JAX London 2013

Imagine if, when your applications weren't in use, they could go to sleep, just like your laptop does when idle. Just think how much money you could save on your infrastructure. The problem with many resource-intensive Java applications is that they are far more difficult to redeploy than they are to take down. Consequently applications tend to be left running whether they are being used or not.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek) Presentation Transcript

  • 1. Put  your  Java  Apps  to  Sleep?   Find  out  How…     John  Ma(hew  Holt  |  JAX  London  2013   ! © Copyright Waratek 2013
  • 2. John Matthew Holt! •  CTO  and  co-­‐Founder  of  Waratek   •  Been  doing  JVM  engineering  for  10  years   •  Invented  “Java  VirtualizaCon”  to  transform   how  JVMs  operate  in  virtualized  and  cloud-­‐ compuCng  environments   •  Lead  the  design  of  the  first  Type-­‐1  VMM  for  a   JVM   © Copyright Waratek 2013
  • 3. What will we cover?! •  First  some  background…   – Why  virtualize  INSIDE  the  JVM?   – Think  “VMware  for  Java”   •  PuSng  you  Java  Apps  to  Sleep…   – Why  are  Java  apps  Cred?     (or  restated:  why  should  they  be  slept?)   – Let  your  apps  nap:  “wake-­‐on-­‐LAN  for  JVCs”     © Copyright Waratek 2013
  • 4. First some background…!     •  Why  virtualize  INSIDE  the  JVM?   © Copyright Waratek 2013
  • 5. Because…! •  Because  JVMs  make  bad  bedfellows  with  HW   hypervisors  (HV)   •  A  JVM  thwarts  HW  hypervisor  features  like:   –  The  GC  in  a  JVM  thwarts  memory-­‐overcommit  in  a  HW   HV   –  The  GC’s  heap  memory,  and  the  JIT’s  permgen/ codecache,  thwarts  ‘transparent  page   sharing’  (memory  de-­‐dup)  in  a  HW  HV   –  Dedicated  GC  and  JIT  for  EVERY  Java  applicaCon  wastes   lots  and  lots  of  CPU  Cme  and  memory  capacity  doing   the  same  thing  over,  and  over,  and  over,  and…     © Copyright Waratek 2013
  • 6. Today’s JVMs are OLD! •  Were  designed  ~20  years  ago,  and  haven’t   changed  design  since  then   •  JVM  design  PREDATES  the  major  2  trends  of   last  decade:  virtualizaCon  and  cloud-­‐compuCng   •  Only  2  innovaCons  in  JVMs  in  20  years:     – Beder  JIT  compilers   – Concurrent  GCs   – ….all  blindly  focused  on  single-­‐tenant  performance   at  the  expense  of  mulC-­‐tenant  efficiency   © Copyright Waratek 2013
  • 7. Server Java has problems! •  JVMs  are  grossly  inefficient  users  of  CPU  and  memory   –  A  JVM  assumes  it  can  use  all  available  CPU  and  memory  capacity   for  itself  and  its  single  applicaCon,  eg:   –  1  JBoss  can  consume  up  to  500MB  RSS  with  no  apps   –  1  Tomcat  can  consume  up  to  250MB  RSS  with  no  webapps   –  Running  mulCple  JVMs  on  a  server  rapidly  exhausts  physically   memory  while  CPU  uClisaCon  averages  5%   •  >100  different  JVM  versions  makes  compliance/support  a   NIGHTMARE   •  Security  vulnerabiliCes  and  compromises  are  EVERYWHERE   inside  Java   © Copyright Waratek 2013
  • 8. VMWare does not virtualize 
 Java well! •  Realworld  stats  using  VMware:   –  >80%  of  Server  JVMs  have  <=2GB  of  heap  memory   •  Yet  actual  applicaCon  heap  uClisaCon  averages  <20%   –  A  2GB  JVM  heap  needs  ~3GB  of  OS  RSS  memory,  which   in  turn  needs  ~4GB  of  VMWare  tenant  memory   •  …reducing  actual  applicaCon  memory  uClisaCon  to  <10%   –  CPU  uClisaCon  running  Java  on  Vmware  averages  <7%   •  Yet  physical  memory  is  exhausted  because  of  running   mulCple  JVMs   © Copyright Waratek 2013
  • 9. Java is a compliance nightmare! •  Realworld  stats:   –  93%  of  organisaCons  are  running  versions  of  Java  at   least  5  years  old   –  The  average  organisaCon  has  51  disCnct  versions  of   Java  installed   –  <1%  of  installed  Java  is  upgraded  to  the  latest  version   •  Recurring  enterprise  conversaCon:   –  QuesCon:  “what  version  of  Java  do  you  run?”   –  Answer:  “every  version  except  the  latest  version!”   © Copyright Waratek 2013
  • 10. Java is a security nightmare! •  Realworld  stats:   –  In  last  12  months  alone,  >250  CVE  alerts  issued  by  US-­‐ CERT/NIST  for  Java   –  The  most  prevalent  version  of  Java  has  96  “perfect  10”   vulnerabiliCes   –  “There  are  virtually  no  modern  versions  of  Java  without   any  known  severe  vulnerabiliCes”  (source:  Bit9)   •  CriCcal  Security  Patch  Updates  are  released  so   oven  that  developers  and  admins  can’t  keep  up     © Copyright Waratek 2013
  • 11. Virtualization is the answer…! •  …but  HIGHER  up  the  Server  Stack   •  VirtualizaCon  is  creeping  up  the  Server  Stack   already   –  x86  HW  got  virtualized  in  1999  (VMware  et  al)   –  x86  OS  got  virtualized  in  2006  (OpenVZ,  LXC,  et  al)   –  …now  it’s  Java’s  turn  in  2013   •  Virtualizing  Java  means  virtualizing  at  the  TOP  of   the  Server  Stack,  whereas  VMware  virtualizes  at   the  BOTTOM   © Copyright Waratek 2013
  • 12. Cost-per-tentant to zero ! •  The  higher  up  virtualizaCon  goes  up  the  Server   Stack,  the  cheaper  a  “tenant”  becomes   •  Virtualizing  INSIDE  the  JVM  gets  close  to  “zero-­‐ cost  tenants”:   – Helloworld  in  <1MB  of  tenant  memory   – Examples.war  (Tomcat  demo  webapps)  in  <5MB  of   tenant  memory   – …Java  becomes  CHEAPER  than  python/ruby/perl   when  the  JVM  itself  gets  virtualized!     © Copyright Waratek 2013
  • 13. Virtualization == Security! •  Virtualizing  the  JVM  makes  Server  Java  secure  and   compliant   –  Virtualized  Java  Apps  run  as  “tenants”  in  a  shared  JVM   –  Tenants  run  in  Java  Virtual  Container  (JVC)  “jails”   –  Tenants  are  isolated  and  “locked-­‐down”   –  Tenants  can  run  any  Java  version   •  Doesn’t  mader  what  “Java  version”  a  tenant  uses,  it  gets  the  latest   JVM   •  Means  a  “Java  version”  becomes  just  a  tenant  API  for  the   convenience  of  developers   •  Admins  control  the  REAL  version  at  the  JVM  level  so  that  tenants   (developers)  can  stop  caring  about  “versions”  altogether   © Copyright Waratek 2013
  • 14. First some background…!   •  Think  “VMWare  for  Java”   © Copyright Waratek 2013
  • 15. A hypervisor INSIDE a JVM! •  Waratek  has  wriden  a  type-­‐1  VMM  that  runs  INSIDE  the   Oracle  HotSpot  and  OpenJDK  JVMs   –  >300,000  lines  of  Java  code,  <5K  lines  of  C-­‐code   –  Can  easily  run  on  any  OS/CPU  pair  (only  Linux/x86  supported  so   far)   –  Does  EVERYTHING  VMware  does,  but  INSIDE  the  JVM   •  •  •  •  •  •  Virtual  FS   Virtual  networks/IPs/NICs   CPU  pinning  and  quotas   Memory  overcommit,  elasCc  memory,  memory  quotas   I/O  QoS,  rate-­‐limiCng   Etc…   –  …all  with  ZERO  code  change  to  Java  applicaCons   © Copyright Waratek 2013
  • 16. A real VMM inside the JVM! © Copyright Waratek 2013
  • 17. VMWare for Java! Java Code JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs •  Waratek  virtualizes  Java  exactly   like  VMware  virtualizes  HW   –  Waratek  inserts  a  “Java   Hypervisor”  between  the  JVM  and   the  Java  App  +  java.*  APIs  +   SecurityManager   –  The  Java  app  PLUS  java.*  APIs  and   SecurityManager  run  inside  a  Java   Virtual  Container  (JVC)  “jail”   –  For  a  Java  App  to  go  outside  of  its   JVC  “jail”  it  has  to  “trap”  to  the   Java  Hypervisor   Waratek JVM © Copyright Waratek 2013
  • 18. VMWare for Java! Java Code JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs Waratek JVM •  Every  incoming  and  outgoing  access   by  a  JVC  is  intercepted  and  checked  by   the  Java  Hypervisor,  e.g:   –  –  –  –  –  –  –  –  –  –  –  –  –  –  Class  loading  intercepted  and  checked   JNI  invocaCon  intercepted   Parameter  and  return  value  checks   File  access  (delete/copy/move)  checks   Network  operaCon  checks   IO  rate-­‐limiCng  and  flow-­‐control   CPU  Cme  control   Memory  consumpCon  control   Buffer  overflow  checks   Null  pointer  checks   Type  safety  checks   OS  access  checks     JNI  Libraries  isolated   …   © Copyright Waratek 2013
  • 19. Slots into existing VM tools! •  You  can  manage  the  Java  hypervisor  exactly  like  you  manage   exisCng  HW  hypervisors:   –  –  –  –  SSH  CLI   RedHat  LibVirt/KVM  integraCon  (beta)   VMWare  vCAC  integraCon  shortly   OpenStack  integraCon  shortly   •  Plus  other  management  interfaces   –  –  –  –  –  HTTP  REST  API   Local  and  remote  JMX  APIs   /proc/PID/*  pseudo-­‐filesystem  for  JVM  and  each  JVC   Vmware  CloudFoundry  integraCon  (beta)   RedHat  OpenShiv  integraCon  soon   © Copyright Waratek 2013
  • 20. A HelloWorld JVC in LibVirt XML! © Copyright Waratek 2013
  • 21. JVCs make Java secure! Java Code JVC Java App java.* APIs SecurityManager Waratek Java Hypervisor Process C/C++ Code JNI Libs Waratek JVM •  If  a  Java  App  inside  a  JVC   compromises  either/both  of   java.*  APIs  or  SecurityManager,   then  only  that  JVC  is  effected   and  NOT  the  Java  Hypervisor  or   JVM  or  other  JVCs   •  JNI  Libraries  are  isolatable  in   their  own  OS  process   •  Thus  a  compromised  JVC   cannot:   –  –  –  –  –  –  –  Insert  malicious  code  outside  its  JVC   Delete/move/copy  files  outside  its  VROOT   Trigger  buffer  overflows   Do  null-­‐pointer  segmentaCon  faults   Perform  unsafe  heap  memory  access   Fork  OS  processes   etc…   © Copyright Waratek 2013
  • 22. “Come in, we’re open”! •  The  Java  Hypervisor  has  an  open  API  for  unlimited   customizaCon  and  extensibility   •  Anyone  can  write  their  own  PURE-­‐JAVA  plugins  to   the  Java  hypervisor  to  customize  ANYTHING  about   JVCs,  eg:   –  Make  a  DropBox.com  account  the  virtual  filesystem  for   a  JVC   –  Create  a  virtual  LAN  between  distributed  JVCs   anywhere  in  the  world   –  Clone  live  network  I/O  of  an  acCve  JVC  to  a  test  JVC  for   side-­‐by-­‐side  applicaCon  tesCng   © Copyright Waratek 2013
  • 23. Example hypervisor plugin! •  Waratek  wrote  ElasCcat  as  one  example  of  a  Java   Hypervisor  plugin   •  ElasCcat  is  open  source,  is  <1000  lines  of  Java  code,  and   loads  into  the  Java  hypervisor  at  startup   •  ElasCcat  virtualizes  Apache  Tomcat  so  it  becomes  the   world’s  first  mulC-­‐tenant  JavaEE  container:   –  Tomcat  runs  in  one  JVC  and  each  webapp  runs  in  it’s  own   dedicated  JVC   –  If  one  webapp  crashes  it  won’t  crash  any  other  JVC   –  …all  this  achieved  with  ZERO  code  change  to  Tomcat  or   webapps,  and  without  them  knowing  they’ve  been   virtualized!   © Copyright Waratek 2013
  • 24. © Copyright Waratek 2013
  • 25. Putting Java to jSleep…!     •  Why  are  Java  apps  Cred?   (or:  why  should  they  be  slept?)   © Copyright Waratek 2013
  • 26. Java Apps are tired!! •  Realworld  stats:   – >50%  of  PaaS  apps  are  “dead  bone  idle”  at  any   given  Cme   – The  average  Dev/Test  environment  runs  at  <10%   sustained  uClizaCon   – Lots  and  lots  of  enterprise  Java  apps  are  only  used   periodically,  e.g.  end-­‐of-­‐week,  end-­‐of-­‐month…   •  But  these  apps  have  SLAs  and  can’t  be  shut-­‐off!   © Copyright Waratek 2013
  • 27. Fact from the field:! •  So  frustrated  was  one  Fortune500  admin  that  he   shut-­‐off  a  bunch  of  Java  apps  that  had  been  idle  for   2  weeks  (no  TCP  traffic)  and  waited  to  see  what   would  happen…  the  results  were:   –  …a  small  number  of  app-­‐owners  noCced  immediately   (1-­‐2  days)   –  …the  largest  number  of  app-­‐owners  noCced  within  1-­‐2   weeks   –  …and  a  small  number  of  app-­‐owners  NEVER  noCced  at   all!   © Copyright Waratek 2013
  • 28. Wouldn’t it be great if…! •  …you  could  “compress”  an  idle  app  and  re-­‐ awaken  it  instantly  WITHOUT  THE  APP   KNOWING?   •  …and  without  the  END-­‐POINTS  knowing?   •  …and  without  ever  RESTARTING  the  app  later?   © Copyright Waratek 2013
  • 29. Actually it exists…almost!! •  The  technology  for  this  nearly  already  exists:   –  It’s  called  “wake-­‐on-­‐LAN”  (WoL)   •  …but  there’s  a  problem:  WoL  only  works  at  the   HARDWARE  level,  so  is  of  NO  USE  in  mulCtenant   virtualized  cloud-­‐compuCng  environments   •  Think  about  it:   –  As  long  as  ONE  tenant  on  a  server  is  acCve  then  the  server   HAS  to  stay  alive  and  WoL  CANNOT  be  used!   © Copyright Waratek 2013
  • 30. Putting Java to jSleep…!     •  Let  your  apps  nap:  wake-­‐on-­‐LAN   for  JVCs   © Copyright Waratek 2013
  • 31. “Wake-on-LAN for JVCs”! •  What’s  needed  is  a  NEW  kind  of  WoL  that   “naps”  single  idle  tenants  on  a  mulCtenant   server…   •  …and  reclaims  the  memory/CPU/IO  quotas   that  were  provisioned  for  the  now-­‐napped  app,   to  be  used  to  run  other  new  apps   •  This  is  what  Waratek  jSleep  does  for  Java  apps!   © Copyright Waratek 2013
  • 32. jSleep in a Slide! •  Waratek  jSleep  works  inside  the  Java  Hypervisor  to   do  four  things:   –  1)  monitor  the  acCvity  of  individual  apps  (JVCs)   –  2)  put  idle  JVCs  to  sleep  aver  defined  inacCvity  thresholds   and  reclaim  their  CPU/mem/IO  quotas   –  3)  monitor  the  I/O  connecCons  (TCP/UDP  Sockets,   ServerSockets,  Files,  etc)  of  a  sleeping  JVC   –  4)  and  awaken  a  sleeping  JVC  when  I/O  acCvity  is   detected,  reinstaCng  reclaimed  quotas   •  …it’s  wake-­‐on-­‐LAN  for  JVCs!   © Copyright Waratek 2013
  • 33. jSleep is Very Powerful! •  Lots  of  “knobs  and  dials”  make  it  possible  to  define   sophisCcated  sleep/awake  profiles  that  respond  to:   –  CPU  acCvity   –  Network/IO  acCvity   –  IO  Cmeouts   –  Wall-­‐clock  Cme   –  …or  just  about  anything  else   •  THERE’S  ALSO  a  built-­‐in  network  profile  that  allows   for  immediate  configuraCon  via  the  JIRSH  CLI   © Copyright Waratek 2013
  • 34. Simple jSleep Example! •  Set  a  jSleep  policy  based  on  network  traffic  of  defined   sockets/ports,  eg:   –  jirsh # sleepinfo 1 –-set network=120000,include_local=:8080, –  Above  command:   •  Suspends  JVC  1  aver  the  network  interface  is  idle  120s   •  The  include_local param  is  a  series  of  host:port  combinaCons   that  specify  what  traffic  of  remote  hosts  and/or  local  ports  is  allowed   to  wake  the  JVC,  so  in  above:   –  Traffic  from  any  remote  host  to  local  port  8080  will  wake  the  JVC   –  Traffic  from  remote  host  to  local  port  4444  will  wake   the  JVC   © Copyright Waratek 2013
  • 35. jSleep API in 60 Seconds! •  2  packages,  8  classes:   –  cloudvm.jsleep.event.* •  •  •  •  BytesReceivedEvent CpuEvent NetworkIdleEvent TimeoutEvent –  cloudvm.jsleep.profile.* •  •  •  •  JSleepProfile JSleepProfile.MemoryType JSleepProfileState NetworkProfile © Copyright Waratek 2013
  • 36. jSleep API in 60 Seconds! •  Main  class:  cloudvm.jsleep.profile.JSleepProfile –  Every  JVC  can  have  one  of  these   –  Defines  the  two  CORE  methods:   •  checkSuspend( VirtualContainerMXBean ) •  checkResume( VirtualContainerMXBean ) –  Is  an  abstract  class  so  anyone  can  write  their  own  custom   sleep/resume  profile   –  Loaded  and  configured  using  JMX  (local  or  remote)   © Copyright Waratek 2013
  • 37. Custom jSleep API Example! •  Here’s  a  custom  jSleep  profile…     public  boolean  checkSuspend(VirtualContainerMXBean  paramVirtualContainerMXBean)  {          Date  currentDate  =  new  Date();          if(currentDate.aver(sleepDate)  &&  currentDate.before(resumeDate))  {                  return  true;          }  else  {                  return  false;          }   }     public  boolean  checkResume(VirtualContainerMXBean  paramVirtualContainerMXBean)  {          Date  currentDate  =  new  Date();          if(currentDate.before(sleepDate)  ||  currentDate.aver(resumeDate))  {                  return  true;          }  else  {                  return  false;          }             }     © Copyright Waratek 2013
  • 38. What happens when…! •  …a  JVC  sleeps?:   –  The  threads  of  the  JVC  are  suspended  on  the  OS  kernel   (but  NOT  destroyed)   –  The  JVC’s  dead  objects  are  asynchronously  reclaimed  by   the  GC   –  If  the  JSleepProfile  was  configured  with   MemoryType.VARIABLE,  the  JVCs  unused  memory  quota   will  be  reclaimed  for  use  by  other  JVCs   –  All  open  file  descriptors  will  be  kept  open   –  …all  this  in  <1  second   © Copyright Waratek 2013
  • 39. What happens when…! •  …a  JVC  wakes  up?:   – The  threads  of  the  JVC  are  resumed  on  the  OS   kernel   – If  the  JSleepProfile  was  configured  with   MemoryType.VARIABLE,  the  JVM  will  reinstate   the  prior  memory  quota  of  the  JVC     – Any  file  descriptors  with  received  but  unread  data   will  now  be  “readable”  by  the  applicaCon   – …all  this  in  <1  second   © Copyright Waratek 2013
  • 40. Why you should care…! •  Dev/Test,  PaaS,  and  middleware  environments  are   characterized  by  LOTS  of  “applicaCon  idleness”     •  Idle  apps  today  are  treated  NO  DIFFERENTLY  than  acCve,   busy  apps   •  Years  ago  H/W  wake-­‐on-­‐LAN  was  an  effecCve  technology  for   powering  down  idle  servers…  but  it  no  longer  works  in   mulCtenant  cloud  infrastructure   •  jSleep  is  “wake-­‐on-­‐LAN  for  JVCs”  and  can  increase  the   number  of  Java  apps-­‐per-­‐server  by  up  to  10x   –  …this  means  a  reducCon  in  server  sprawl  and  OpEx  up  to  90%   © Copyright Waratek 2013
  • 41. Summary!     •  Recap  and  links…   © Copyright Waratek 2013
  • 42. Biggest change to Java in yrs! •  A  hypervisor  inside  the  JVM  transforms  Java   with:   – Ultra-­‐liteweight  JVCs  for  near  zero-­‐cost  tenants   – JVC  “jails”  for  Java  security  and  isolaCon   – Mixed-­‐version  JVCs  for  version  consolidaCon  and   legacy  compliance   – jSleep  (“Wake-­‐On-­‐LAN  for  JVCs”)  for  near-­‐zero   footprint  of  idle  tenants   © Copyright Waratek 2013
  • 43. Links! •  General  informaCon  about  the  Waratek  JVM   – hdp://www.waratek.com   •  jSleep,  JVI  (Java  VirtualizaCon  Interface),  and   JMX  APIs   – hdp://javadoc.waratek.com   •  ElasCcat  source  code   – hdps://github.com/waratek/elasCcat-­‐driver   © Copyright Waratek 2013
  • 44. ! © Copyright Waratek 2013