Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
DICE	
  Horizon	
  2020	
  Project	
  	
  
Grant	
  Agreement	
  no.	
  644869	
  
h>p://www.dice-­‐h2020.eu	
   Funded	
 ...
MiSE	
  2015@ICSE,	
  Florence,	
  Italy,	
  May	
  17	
  2015	
  	
  
Overview	
  and	
  goals	
  
o  MDE	
  oYen	
  feat...
o  SoYware	
  market	
  rapidly	
  shiYing	
  to	
  Big	
  Data	
  
§  32%	
  compound	
  annual	
  growth	
  rate	
  in	...
DataInc	
  example	
  
o  DataInc	
  is	
  a	
  small	
  soYware	
  vendor	
  selling	
  cloud-­‐based	
  environmental	
 ...
DataInc	
  example	
  
o The	
  contract	
  requires	
  delivering	
  an	
  iniPal	
  version	
  
of	
  DICEnv	
  within	
...
Plalorm-­‐Indep.	
  
Model	
  
Domain	
  	
  
Models	
  
Quality-­‐Aware	
  MDE	
  Today	
  
6	
  ©DICE	
  
QA
Models
Arch...
Plalorm-­‐Indep.	
  
Model	
  
Domain	
  	
  
Models	
  
Quality-­‐Aware	
  MDE	
  Today	
  
7	
  ©DICE	
  
Architecture	
...
Plalorm-­‐Indep.	
  
Model	
  
Domain	
  	
  
Models	
  
Quality-­‐Aware	
  MDE	
  Today	
  
8	
  ©DICE	
  
Architecture	
...
Plalorm-­‐Indep.	
  
Model	
  
Domain	
  	
  
Models	
  
An	
  HolisPc	
  Approach:	
  DICE	
  
9	
  ©DICE	
  
ConPnuous	
...
Embracing	
  DevOps	
  
o SoYware	
  development	
  process	
  is	
  evolving	
  
§  	
  Developer:	
  “I	
  want	
  to	
...
Embracing	
  DevOps	
  
11	
  ©DICE	
  
o QA	
  must	
  become	
  lean	
  as	
  well	
  
§  ConPnuous	
  quality	
  check...
Benefits	
  
o  Tackling	
  skill	
  shortage	
  and	
  steep	
  learning	
  curves	
  
§  Data-­‐aware	
  methods,	
  mod...
DICE	
  Plalorm	
  Independent	
  Model	
  (DPIM)	
  
13	
  ©DICE	
   MiSE	
  2015@ICSE,	
  Florence,	
  Italy,	
  May	
  ...
DICE	
  Plalorm	
  and	
  Technology	
  Specific	
  Model	
  
(DTSM)	
  
14	
  ©DICE	
   MiSE	
  2015@ICSE,	
  Florence,	
 ...
DICE	
  Plalorm,	
  Technology	
  and	
  
Deployment	
  Specific	
  Model	
  (DDSM)	
  
15	
  ©DICE	
   MiSE	
  2015@ICSE,	...
DICE	
  Profile:	
  PIM	
  Level	
  
o FuncPonal	
  approach	
  to	
  data	
  to	
  be	
  expanded	
  
o Data	
  dependenci...
DICE	
  Profile:	
  PSM	
  Level	
  
o Need	
  for	
  technology-­‐specific	
  abstracPons	
  	
  
§  Hadoop:	
  Number	
  ...
§  Risk	
  of	
  harm	
  
§  Privacy	
  &	
  data	
  protecPon	
  
	
  
DICE	
  QA:	
  Quality	
  Dimensions	
  
o Relia...
DICE	
  QA:	
  Tools	
  
o  Discrete-­‐event	
  simulaNon:	
  assess	
  reliability	
  and	
  efficiency	
  
in	
  Big	
  Da...
DICE	
  QA:	
  Tools	
  
o  Architecture	
  opNmizaNon	
  tool:	
  find	
  architectural	
  
improvements	
  to	
  opPmise	...
DICE	
  Project	
  h>p://www.dice-­‐h2020.eu	
  
o Horizon	
  2020	
  Research	
  &	
  InnovaPon	
  AcPon	
  
§  Quality-...
Upcoming SlideShare
Loading in …5
×

MISE2015

202 views

Published on

DICE: Quality-Driven Development of Data- Intensive Cloud ApplicaPons

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

MISE2015

  1. 1. DICE  Horizon  2020  Project     Grant  Agreement  no.  644869   h>p://www.dice-­‐h2020.eu   Funded  by  the  Horizon  2020   Framework  Programme  of  the  European  Union   DICE:  Quality-­‐Driven   Development  of  Data-­‐ Intensive  Cloud   ApplicaPons     G.  Casale,  D.  Ardagna,  M.  Artac,  F.  Barbier,   E.  Di  Ni6o,  A.  Henry,  G.  Iuhasz,  C.  Joubert,       J.  Merseguer,  V.  I.  Munteanu,  J.  F.  Pérez,         D.  Petcu,  M.  Rossi,  C.  Sheridan,  I.  Spais,               D.  Vladušič    
  2. 2. MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015     Overview  and  goals   o  MDE  oYen  features  quality  assurance  (QA)   techniques  for  developers     o  How  should    quality-­‐aware  MDE  support  data-­‐ intensive  soYware  systems?   o ExisPng  models  and  QA  techniques  largely  ignore   properPes  of  data     o Characterize  the  behavior  of  new  technologies   o  DICE:  a  quality-­‐aware  MDE  methodology  inspired  by   DevOps  for  data-­‐intensive  cloud  applicaPons   2  ©DICE  
  3. 3. o  SoYware  market  rapidly  shiYing  to  Big  Data   §  32%  compound  annual  growth  rate  in  EU  through  2016   §  35%  Big  data  projects  are  successful  [CapGemini  2015]   o  European  call  for  soYware  quality  assurance  (QA)   §   ISTAG:  call  to  define  environments  “for  understanding  the   consequences  of  different  implementaNon  alternaNves  (e.g.   quality,  robustness,  performance,  maintenance,   evolvability,  ...)”   o  QA  evolving  too  slowly  compared  to  the  trends  in   soYware  development  (Big  data,  Cloud,  DevOps  ...)     MoPvaPon   3  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  4. 4. DataInc  example   o  DataInc  is  a  small  soYware  vendor  selling  cloud-­‐based  environmental   soYware   o  DICEnv,  a  warning  system  for  floods  in  rural  regions   o  monitoring  local  environmental  condiPons   o  fetching  precipitaPons  data  from  satellite  image  stream   o  DICEnv  exploits  Big  Data  technologies  and  cloud  capacity  for  online   water  simulaPons  and  MapReduce  for  batch  processing  of  historical   data       o  DICEnv  is  a  criPcal  system:   o  is  expected  to  remain  up  24/7   o  should  quickly  ramp  up  data  intake  rates,  as  well  as  memory  and  compute   capaciPes,  to  update  more  frequently  the  hazard  management  control   room   4  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  5. 5. DataInc  example   o The  contract  requires  delivering  an  iniPal  version   of  DICEnv  within  3  months  serving  a  small  area,   increasing  coverage  on  a  monthly  basis     o Challenges:   o How  to  implement  a  complex  cloud  applicaPon  in   such  a  short  Pme?     o How  to  saPsfy  all  the  quality  requirements?     o What  architecture  should  be  adopted?   5  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  6. 6. Plalorm-­‐Indep.   Model   Domain     Models   Quality-­‐Aware  MDE  Today   6  ©DICE   QA Models Architecture   Model   Plalorm-­‐Specific   Model   C#JavaC++ Plalorm   DescripPon   MARTE AnalyPcal  Models   Cost-­‐Quality  Models   Code   generaPon   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  7. 7. Plalorm-­‐Indep.   Model   Domain     Models   Quality-­‐Aware  MDE  Today   7  ©DICE   Architecture   Model   Plalorm-­‐Specific   Model   Code   generaPon   C#JavaC++ Plalorm   DescripPon   MARTE Issues  PIM  layer:       •  staNc  characterisNcs  of   data   •  dynamic  characterisNcs   of  data   •  data  dependencies   DICEnv    modeling  issues:       •  individual  dependencies   between  components  and  data   streams   •  relaPonships  between   compute  and  memory   requirements     •  lack  of  an  explicit  annotaPon   for  data  characterisPcs     MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  8. 8. Plalorm-­‐Indep.   Model   Domain     Models   Quality-­‐Aware  MDE  Today   8  ©DICE   Architecture   Model   Plalorm-­‐Specific   Model   Code   generaPon   C#JavaC++ Plalorm   DescripPon   MARTE Issues  at  PSM  layer:       •  heterogeneity  of  Big     Data  technologies     •  automaNc  translaNon  of   PSM  models  into   deployment  plans     QA  tools  limitaPons:     •  contenPon  at  processing   resources  with  limited  features   for  memory  consumpPon     •  fork  and  joining  are  complex   to  be  described  analyPcally   preserving  tractability     MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  9. 9. Plalorm-­‐Indep.   Model   Domain     Models   An  HolisPc  Approach:  DICE   9  ©DICE   ConPnuous   ValidaPon   ConPnuous   Monitoring   Data   Awareness   Architecture   Model   Plalorm-­‐Specific   Model   Plalorm   DescripPon   DICE MARTE Deployment  &   ConPnuous   IntegraPon   DICE IDE Big Data QA Models MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  10. 10. Embracing  DevOps   o SoYware  development  process  is  evolving   §   Developer:  “I  want  to  change  my  code”   §   Operator:  “I  want  systems  to  be  stable”   o ...but  code  changes  are  the  cause  of  most  instabiliPes!   o   DevOps  closes  the  gap  between  Dev  and  Ops   §  Lean  release  cycles  with  automated  tests  and  tools   §  Deep  modelling  of  systems  is  the  key  to  automaPon   10  ©DICE   Agile   Development   DevOps   Business   Dev   Ops   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  11. 11. Embracing  DevOps   11  ©DICE   o QA  must  become  lean  as  well   §  ConPnuous  quality  checks  and  model  versioning   o Modelling  of  the  operaPons   §  Dev  needs  awareness  of  infrastructure  and  costs   o ConPnuous  feedback   §  Forward  and  backward  model  synchronisaPon   §  Tracking  of  self-­‐adaptaPon  events  (e.g.  auto-­‐scaling)   o Big  data  coming  from  conPnuous  monitoring   §  QA  has  its  own  Big  data,  use  machine  learning?   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  12. 12. Benefits   o  Tackling  skill  shortage  and  steep  learning  curves   §  Data-­‐aware  methods,  models,  and  OSS  tools   o  Shorter  Pme  to  market  for  Big  Data  applicaPons   §  Cost  reducPon,  without  sacrificing  product  quality   o  Decrease  development  and  tesPng  costs   §  Select  opPmal  architectures  that  can  meet  SLAs   o  Reduce  number  and  severity  of  quality  incidents   §  IteraPve  refinement  of  applicaPon  design     12  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  13. 13. DICE  Plalorm  Independent  Model  (DPIM)   13  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  14. 14. DICE  Plalorm  and  Technology  Specific  Model   (DTSM)   14  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  15. 15. DICE  Plalorm,  Technology  and   Deployment  Specific  Model  (DDSM)   15  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  16. 16. DICE  Profile:  PIM  Level   o FuncPonal  approach  to  data  to  be  expanded   o Data  dependencies   §  graph  relaPonships  between  data,  archives  and  streams   o QA  focuses  on  quanPtaPve  aspects  of  data   o StaPc  characterisPcs  of  data   §  volumes,  value,  storage  locaPon,  replicaPon  pa>ern,   consistency  policies,  data  access  costs,  known  schedules  of   data  transfers,  data  access  control  /  privacy,  ...   o Dynamic  characterisPcs  of  data   §  cache  hit/miss  probabiliPes,  read/write/update  rates,   bursPness,  ...   16  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  17. 17. DICE  Profile:  PSM  Level   o Need  for  technology-­‐specific  abstracPons     §  Hadoop:  Number  of  mappers  and  reducers  ,  ...   §  In-­‐memory  DBs:  Peak  memory  and  variable  threading   §  Streaming:  merge/split/operators,  networking,  ...   §  Storage:  Supported  operaPons,  cost/byte  ,  ...   §  NoSQL:  Consistency  policies  ,  ...   o GeneraPon  of  deployment  plan   §  Proposed  Chef    +    TOSCA  extension   o Interest  is  both  on  private  and  public  clouds   17  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  18. 18. §  Risk  of  harm   §  Privacy  &  data  protecPon     DICE  QA:  Quality  Dimensions   o Reliability   o Efficiency   o Safety     18  ©DICE     §  Performance   §  Time  behaviour   §  Costs   §  Availability   §  Fault-­‐tolerance     MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  19. 19. DICE  QA:  Tools   o  Discrete-­‐event  simulaNon:  assess  reliability  and  efficiency   in  Big  Data  applicaPons,  accounPng  for  stochasPc   evoluPon  of  the  environment     o  stochasPc  Petri  nets  or  queueing  networks,  rely  on  simulaPon   o  Formal  verificaNon  tools:  assess  safety  risks  in  Big  Data   applicaPons,  e.g.  find  design  flaws  causing  order  and   Pming  violaPons  in  message  and  state  sequences   o  temporal  logic  formulae  and  bounded  model  checking,   saPsfiability  modulo  theories  solvers   o  quanPfier-­‐eliminaPon  techniques  to  extend  temporal  logic-­‐ based  verificaPon   19  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  20. 20. DICE  QA:  Tools   o  Architecture  opNmizaNon  tool:  find  architectural   improvements  to  opPmise  costs  and  quality   o  decomposiPon-­‐based  analysis  approach   o  resort  to  fluid  approximaPon  of  stochasPc  models   o  Feedback  analysis:  automated  extracPon  from  the   monitored  data  of  key  parameters  required  to  define   simulaPon  and  verificaPon  models   o  extract  model  parameters  through  log  mining  and  staPsPcal   esPmaPon  methods     o  breakdown  resource  consumpPon  into  its  atomic  components   on  the  end-­‐to-­‐end  path  of  requests   20  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    
  21. 21. DICE  Project  h>p://www.dice-­‐h2020.eu   o Horizon  2020  Research  &  InnovaPon  AcPon   §  Quality-­‐Aware  Development  for  Big  Data  applicaPons   §  Feb  2015  -­‐  Jan  2019,  4M  Euros  budget   §  9  partners  (Academia  &  SMEs),  7  EU  countries   21  ©DICE   MiSE  2015@ICSE,  Florence,  Italy,  May  17  2015    

×