• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
NREN strategies for Data-Intensive Science in a Carbon Constrained World
 

NREN strategies for Data-Intensive Science in a Carbon Constrained World

on

  • 1,018 views

Bill St. Arnaud

Bill St. Arnaud

Statistics

Views

Total Views
1,018
Views on SlideShare
1,018
Embed Views
0

Actions

Likes
0
Downloads
8
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    NREN strategies for Data-Intensive Science in a Carbon Constrained World NREN strategies for Data-Intensive Science in a Carbon Constrained World Presentation Transcript

    •      NREN  strategies  for  Data-­‐Intensive   Science    in  a  Carbon  Constrained   World         Bill  St.  Arnaud     Bill.st.arnaud@gmail.com     Unless  otherwise  noted  all  material  in  this  slide  deck  may  be  reproduced,   modified  or  distributed  without  prior  permission  of  the  author  
    • Theme  of  this  talk    •  We  have  already  lost  the  baIle  to  save  the  planet  from  extreme   climate  change.  Rather  than  focusing  on  reducing  energy   consumpKon,  (MiKgaKon)  we  now  need  to  focus  on  surviving   climate  change  (AdaptaKon)  •  Explosion  of  data  and  energy  consumpKon  by  computers  and   networks  is  contribuKng  to  energy  demand  and  CO2  emissions  •  But  big  data  and  science  will  be  criKcal  as  move  to  focus  on   adapKng  to  climate  change      •  How  can  Internet  and  IT  help  us  build  NRENs  and  support  science     and  educaKon  that  can  adapt  to  global  warming?  
    • Changing  NREN   networking  environment  •  Global  Virtual  Research  CommuniKes  •  Increasing  co-­‐operaKon  between  public  and  private   researchers  •  Rapidly  changing  users  demands  •  Increasing  potenKal  of  commercial  ICT-­‐service  providers  •  EducaKon:  any  Kme,  any  place,  any  device  •  CiKzen  Science  and  M2M  communicaKons  and  sensors  •  The  disappearing  campus  IT  &  diminishing  experKse  in  ICT   centres  of  connected  insKtuKons  
    • Although  there  is  less  news  coverage  global  warming  has  not  disappeared  
    • Half  of  US  experienced  record   droughts  or  deluges  in  2011     2010  warmest  year  ever  –  we   are  only  at  the  start  of  the   curve  of  the  hockey  s7ck.  The   worst  is  yet  to  come  
    • Blame  it  on  Canada   How  warming  in  the  ArcKc  affects  weather  in  Louisiana  •  Warming  ArcKc  slowing  down  jet  stream    •   Basic  Thermodynamics  -­‐  polar  temperatures   drive  the  jet  stream,     –  There’s  been  a  20  percent  drop  in  the  zonal   wind  speeds.  •  As  get  stream    slows  down,  it  leads  to  those   bigger  kinks  in  the  jet  stream.   –  That  amplificaKon  is  associated  with   persistent  weather  paIerns  that  lead  to   “extremes”  like  drought,  flooding  and  heat   waves.  •  Those  slow-­‐moving,  persistent  waves  of   weather  energy  may  have  played  a  role  in  the   big  snows  that  hammered  parts  of  the  West   last  winter,  as  well  as  some  of  the  extreme   winter  weather  that  hit    South  West  US  and   Europe  •  hIp://summitcountyvoice.com/2012/01/14/ global-­‐warming-­‐revenge-­‐of-­‐the-­‐atmosphere/     6  
    • Climate  Forecasts  •  MIT  report  predicts  median   temperature  forecast  of  5.2°C   –  11°C  increase  in  Northern  Canada   &  Europe   –  hIp://globalchange.mit.edu/pubs/ abstract.php?publicaKon_id=990   MIT  •  Last  Ice  age  average  global   temperature  was  5-­‐6°C  cooler  than   today   –  Most  of  Canada  &  Europe  was   under  2-­‐3  km  ice  •  Nearly  90  per  cent  of  new  scienKfic   findings  reveal  global  climate   disrupKon  to  be  worse,  and   progressing  more  rapidly,  than   expected.   •  hIp://www.skepKcalscience.com/pics/ Freudenburg_2010_ASC.pdf  
    • Future  Droughts  •   Palmer  Drought  Severity  Index,  or  PDSI.    •   The  most  severe  drought  in  recent  history,  in  the  Sahel  region  of  western  Africa  in  the  1970s,  had  a  PDSI  of  -­‐3  or  -­‐4.  •   By  2030  Western  USA  could  see    -­‐4  to  -­‐6.  Drought  in  Texas  clearly  caused  by  global  warming:  hIp://goo.gl/QjHRS  •   By  2100  some  parts  of  the  U.S.  and  LaKn  America  could  see  -­‐8  to  -­‐10  PDSI,  while  Mediterranean  areas  could  see  drought  in  the  -­‐15   hIp://www.msnbc.msn.com/id/39741525/ns/or  -­‐20  range.   us_news-­‐environment/  
    • DramaKc  changes  in  precipitaKon   •  Every  conKnent  has  suffered  record  rainfalls   •  Rains  submerged  one-­‐fioh  of  Pakistan,  a   thousand-­‐year  deluge  swamped  Nashville  and   storms  just  north  of  Rio  caused  the  deadliest   landslides  Brazil  has  ever  seen.   •  Observed  increase  in  precipitaKon  in  the  last  few   decades  has  been  due  in  large  part  to  a   disproporKonate  increase  in  heavy  and  extreme   precipitaKon  rates  which  are  exceeding   predicKons  made  in  models  
    • New  Challenge:  Climate   AdaptaKon      •  Obama’s  NaKonal  Science  Advisor  John  Holdren   “MiKgaKon  alone  won’t  work,  because  the  climate  is   already  changing,  we’re  already  experiencing   impacts….A  miKgaKon  only  strategy  would  be   insanity,”        •  Equal  emphasis  given  to  adaptaKon  –  avoiding  the   unmanageable,  and  adaptaKon  –  managing  the   unavoidable.”    •  Obama’s  Climate  AdaptaKon  ExecuKve  Order   –  hIp://www.stumbleupon.com/su/1tU8go/www.good.is/post/obama-­‐s-­‐secret-­‐climate-­‐adaptaKon-­‐plan/  
    • Climate  Change  Impact  on  Internet   and  NRENs  •  UK    Government  study  Climate  Change  could  ruin  the  Internet   –  hIp://www.grist.org/list/2011-­‐05-­‐09-­‐climate-­‐change-­‐could-­‐ruin-­‐the-­‐internet    •  California  aims  to  have  30%  renewable  power   –  Impact  on  reliability  of  power  systems  •  Last  year  Nuclear  power  plants  in  France  were  forced  to  shut  down   because  cooling  water  was  too  warm    •  Germany  is  commiIed  to  shuvng  down  all  of  its  nuclear  plants    •  Droughts  will  restrict  producKon  of  hydro-­‐electric  power  •  Energy  shortages  and  disrupKons  are  predicted  to  increase  in  the   coming  years    
    • Impact  on  ICT  sector  According  to  IEA  ICT  will  represent  40%  of  all  energy  consump7on  by  2030   www.smart2020.org   ICT  represent    8%  of  global  electricity  consumpKon     Future  Broadband-­‐  Internet  alone  is  expected  to  consume  5%  of  all  electricity   hIp://www.ee.unimelb.edu.au/people/rst/talks/files/Tucker_Green_Plenary.pdf    
    • R&E  biggest  consumer!!  Per  employee   Per  sector  Australian  Computer  Society  Study  hIp://www.acs.org.au/aIachments/ICFACSV4100412.pdf      
    • The  ICT  energy  consumpKon  in   higher-­‐  ed  •  Campus  compuKng  20-­‐40%  electrical  energy  consumpKon  on  most   campuses   –  Studies  in  UK  and  The  Netherlands   –  hIp://goo.gl/k9Kib    •  Closet  clusters  represent  up  to  15%  of  electrical  consumpKon   –  hIp://isis.sauder.ubc.ca/research/clean-­‐technology-­‐and-­‐energy/green-­‐it/  •  Campus  data  center  alone  represents  8-­‐20%  of  electrical  consumpKon   –  hIp://www.iisd.org/publicaKons/pub.aspx?pno=1341    •  IISD  study  demonstrated  that  moving  Canadian  research  to  cloud  would   pay  for  itself  in  energy  savings  and  CO2  reducKon   –  hIp://www.iisd.org/publicaKons/pub.aspx?pno=1341    
    • The  real  cost  of  campus  compuKng  •  Land  -­‐  2%  •  Core  and  shell  costs  –  9%     Belady,  C.,  “In  the  Data  Center,  Power  and  Cooling  Costs   More  than  IT  Equipment  it  Supports”,  Electronics  Cooling  •  Architectural  –  7%   Magazine  (February  2007)  •  Mechanical/Electrical  –  82%   –  16%  increase/year  since  2004   Source:  ChrisKan  Belady  
    • The  Data  Deluge   2004:  36  TB   2012:  2,300  TB  Genomic  sequencing  output  x2  every   Climate    9  month   model  intercomparison     project  (CMIP)  of  the  IPCC   MACHO  et  al.:  1  TB   Palomar:  3  TB   2MASS:  10  TB   GALEX:  30  TB   Sloan:  40  TB   Pan-­‐STARRS:     40,000  TB   1330  molec.  bio  databases     Nucleic  Acids  Research  (96  in  Jan  2001)   Source:  Ian  Foster,  UoChicago  
    • Big  science  has  achieved  big  successes   OSG:  1.4M  CPU-­‐hours/day,   >90  sites,  >3000  users,     >260  pubs  in  2010  LIGO:  1  PB  data  in  last  science  run,  distributed  worldwide   Robust  producKon  soluKons   SubstanKal  teams  and  expense   Sustained,  mulK-­‐year  effort   ApplicaKon-­‐specific  soluKons,      built  on  common  technology  ESG:  1.2  PB  climate  data  delivered  to  23,000  users;  600+  pubs   Source:  Ian  Foster,  UoChicago  
    • But  small  science  is  struggling  More  data,  more  complex  data  Ad-­‐hoc  soluKons  Inadequate  sooware,  hardware  Data  plan  mandates   Source:  Ian  Foster,  UoChicago  
    • Growth  in  sensor  networks  and  CiKzen   Science   Glacier  Tracking   Real  Time  Health  Monitoring   Smart  Trash   19  
    • THE  CHALLENGE  We  need  soluKons  to  address  climate  change,  data  deluge,  needs  of  scienKsts,  global  collaboraKon,  the  evolving  network  of  any  Kme,  any  place,  any  device  and  yet  addresses  the  challenge  of  disappearing  IT  on  campus  while  sKll  providing  a  leadership  role  in  next  generaKon  Internet  and  broadband,  and  find  ways  to  pay  for  it  all  in  an  era  of  severe  fiscal  constraint.  
    • THE  SOLUTION   1.  Brokered  Green  Clouds    and  off  site  campus  IT   2.  Sooware  Defined  Networks  (OpenFlow)   3.  NREN  naKonal  wireless  network     4.  Global  Interconnected  Dynamic  OpKcal  Networks   5.  eScience  Pla|orms  with  next  gen  IdM   6.  Community  anchor  IXPs  with  CDN  and  M2M  hosKng   7.  New  billion  dollar    revolving  green  energy  funds  at  many   universiKes   21  
    • 1.  Brokered  Green  Clouds  and  off  site  CAMPUS  IT   22  
    • UniversiKes  moving  to  eliminate  IT   departments  •  Already  many  primary  funcKons  of  IT  department  are  being  outsourced  to   the  cloud   –  E-­‐mail,  web,  DNS,  research  compuKng,  etc   –  University  of  Western  Australia  has  outsourced  virtually  all  campus  servers  to   an  external  private  cloud    •  Even  rouKng,  network  and  firewall  funcKons  being  outsourced  to  NREN   –  AARnet,  SUnet  and  other  NRENs  offering  border  gateway  rouKng  services  with   collapsed  IP  backbones   –  Sooware  Defined  Networks  makes  it  easy  to  configure  outsourced  LAN   –  Network  faciliKes  can  be  located    •  Increasingly  most  traffic  is  in/out  of  campus,  instead  of  within   –  Social  networking,  P2P,  Clouds,  Kuali,  Blackboard   –  Future  of  Campus  IT  –  high  speed  opKcal  network  connected  to  WiFi/5G  hot   spots  with  tablets   –  No  servers,  no  LAN    
    • MIT  to  build  zero  carbon  data  center   in  Holyoke  MA  •  The  data  center  will  be  managed  and  funded  by   the  four  main  partners  in  the  facility:  the   MassachuseIs  InsKtute  of  Technology,   Cisco  Systems,  the  University  of  MassachuseIs   and  EMC.  •  It  will  be  a  high-­‐performance  compuKng   environment  that  will  help  expand  the  research   and  development  capabiliKes  of  the  companies   and  schools  in  Holyoke   –  hIp://www.greenercompuKng.com/news/2009/06/11/ cisco-­‐emc-­‐team-­‐mit-­‐launch-­‐100m-­‐green-­‐data-­‐center      
    • NREN  Brokered  Cloud  for  IT   departments  and  Researchers  •  Internet  2  Net  +   –  Provisioning    of    mulK  vendor    cloud    services    leveraging    the    Internet2     Network    and    InCommon    Federated    AuthenKcaKon       –  Interoperable    marketplace    for    services    where    individual  insKtuKons     might  procure    services    from    a  wide    range    of    cloud    services     providers.          •  HEFCE  and  JISC  to  Deliver  Cloud-­‐Based  Services  for  UK  Research   –  Besides  providing  brokered  cloud  services  they  are  also  providing  cloud   “soluKons”  for  IT  departments  and  researchers   –  hIp://www.hpcinthecloud.com/hpccloud/2011-­‐06-­‐27/hefce_and_jisc_to_deliver_cloud-­‐ based_services_for_uk_research.html?utm_medium=twiIer&utm_source=twiIerfeed    •  SURFnet:  Community  Cloud  Models  and  the  Role  of  the  R&E  network  as  a   broker  for  cloud  services   –  hIp://www.slideshare.net/haroldteunissen/community-­‐clouds-­‐shared-­‐infrastructure-­‐as-­‐a-­‐service    
    • 2.  Soware  Defined  networks   26  
    • GreenStar  Network  World’s  First  Zero  Carbon  Cloud/Internet  
    • OpenFlow    Follow  the  wind/Follow  the  sun   Canadian  GSN     European  GSN     Domain   Domain   Export  VM   NoKfy  EU     Cloud  Manager   Cloud  Manager   Cloud  Manager   Internet   Dynamically  Configure   IP  Tunnel   Host   Network   Host   Resource   Manager   Resource   •     Shudown  VM   •     Copy  Image   •     Update  VM  Context   Mantychore2   •     Start  VM   Shared   VM   storage   Shared   storage   VM   Lightpath   OpKcal  switch   OpKcal  switch  Host   Cloud  Proxy     Cloud  Proxy     Host  
    • OpenFlow-­‐based  cloud     OpenFlow  Network  A   OpenFlow  Network  B   VM   VM   VM   VM   VM   VM   VM   VM   eth1   eth0   eth1   eth0   eth1   eth0   eth1  eth0   Open  Virtual   Open  Virtual   Open  Virtual   Open  Virtual   Switch  (OVS)   Switch  (OVS)   Switch  (OVS)   Switch  (OVS)   Host   Host   Host   Host   OF   Controller   Ethernet  Switch   OpenFlow  Control  plane   Internet   OpenFlow  Data  plane   OVS   OVS   eth0   eth1   eth0   eth1  
    • Green  Clouds  InternaKonal  •  GreenLight  explores  how  researchers  can  take  advantage  of  data  centers  linked  by   high-­‐speed  networking  in  an  era  of  carbon-­‐thrioy  compuKng    •  Recent  studies  migraKng  virtual  machines  to  green  energy  sites  indicate  that  100   Gb/s  networks  are  far  superior  to  10  Gb/s  to  make  this  transparent.  •  SURFnet  7  lightpath  connecKon  to  GreenQCloud  in  Iceland   SURFconecxt  control  of  lightpath  to   Future  Global  Network  of  Green  Clouds   GreenQCloud  in  Iceland   interconnected  by  GLIF  
    • Science  Cloud  CommunicaKon  Services  Network  •  Enterprise  clouds  use  commodity  internet;  computaKonal  clouds  for  data-­‐intensive   science  require  dynamic  cloud  provisioning  integrated  with  dynamic  high   performance.  •  TransCloud:  example  of  dynamic  networking  &  dynamic  cloud  provisioning   Example  of  working  in  the  TransCloud   [1]  Build  trans-­‐con7nental  applica7ons  spanning  clouds:   •   Distributed  query  applica7on  based  on  Hadoop/Pig   •   Store  archived  Network  trace  data  using  HDFS   •   Query  data  using  Pig  over  Hadoop  clusters   [2]  Perform  distributed  query  on  TransCloud,  which  currently  spans  the  following   sites:   •   HP  OpenCirrus   •   Northwestern  OpenCloud   •   UC  San  Diego   •   Kaiserslautern   Source:  Maxine  Brown  
    • 3.0  NREN  Na7onal  Wireless  Network       32  
    • Building  a  NREN  wireless  network  •  Vision:  to  allow  students,  researchers  and  employees  to  collaborate,   research,  learn  anyKme  and  anywhere  they  seem  fit!  •  Also  Internet  of  Things  –  Machine  to  Machine  communicaKons    •  ExisKng  3G  and  4G  networks  cannot  handle  data  load   –  Roaming  gateways  prevent  global  seamless  access   –  Voice  centric  architectures    •  New  mobile  networks  seamlessly  integrate  with  WiFi  on  campus   –  New  Wifi    2.0  standards  802.11u  allow  for  data  handoff  from  3G   networks   –  Eduroam  can  be  the  global  authorizaKon  tool   –  OpenFlow  can  be  used  to  architect  integrated  soluKons  from  wireless   node  across  opKcal  network  
    • Impact  of  NREN  wireless  networks  •  The  phone  is  a  also    a  sensor  pla|orm    •  Processing  is  done  in  real  Kme  in  the  cloud   –  Allowing  processing  that  can’t  be  done  on  the  device   –  Big  data  analysis  •  New  campus  or  hot  spot  centric  architectures  integraKng  LTE  and  Wifi   –  See  SURFnet  pilot   hIp://www.surfnet.nl/en/nieuws/Pages/BackgroundarKcle.aspx  •  WiFi  nodes  can  be  powered  by  renewable  sources  such  as  roof  top  solar   panel  over  400Hz  power  systems  or  ethernet  power     34  
    • The  Regulatory  Challenge  •  Today’s  SIM-­‐card  locks  user  to  the  network    •  If  NREN  becomes  a  MVNO  with  own  SIM-­‐cards,  users  could   roam  seamlessly  around  the  globe    •  Only  public  service  providers  have  access  to  IMSI-­‐numbers  for   SIM-­‐cards  •  One  opKon  is  to  lobby  regulators  to  give  R&E  networks  access   to  IMSI-­‐numbers  
    • 4.  Global  Interconnected  Dynamic  Op7cal  Networks       36  
    • GLIF   37  
    • e-­‐Research  Scenario   GOLE   38   Source:  SURFnet  
    • Importance  of  GOLEs  •  Increasingly  more  research  and  educaKon  is  internaKonal  collaboraKon   –  Cornell-­‐  Technion  announcement   –  US  overseas  university  campuses  in  UK  and  elsewhere   –  GOLES  enable  direct  peering  of  regional  networks  or  even  insKtuKons    •  Many  researchers  need  access  to  commercial  clouds  and  data  specialists   –  AUP  issues  ooen  prevent  NRENs  from  directly  connecKng  up  these  insKtuKons   –  Genomics  and  bio-­‐informaKcs  processing  and  climate  modeling    •  Many  commercial  research  insKtuKons  need  access  to  lightpaths   –  GOLES  provide  neutral  access  points  for  interconnect  to  AUP  free  lightpaths    •  Enables  new  services   –  Sooware  Defined  Network  using  Switched  lambdas   39  
    • 5.  eScience  Plaiorms  with  next  gen  IdM       40  
    • Towards  “research  IT  as  a  service”     Scientific data management as a service GO-Store GO-Collaborate GO-Galaxy GO-Transfer GO-Compute GO-Catalog GO-Team GO-User Source:  Ian  Foster,  UoChicago   41  
    • SaaS  services  in  acKon:  The  XSEDE   vision  Academic institution = Standard interface XUAS   Globus Online: Hosted persistent services User Team Catalog Transfer Compute ... 2InCommon ... Open Commercial Data Science XSEDE service provider provider provider Grid 42  
    • Virtual  OrganisaKons   CollaboraKon  Infrastructure     Netherlands BioInformatics Centre (NBIC) (SURFconext)   GuestsNBI N=6 N=10 N=30 N=20 C   A6ri Gro grou provisioni b.   N=66 upAAI   ps     ng   mgm …     t   Generic  Broker   Supporting Services Network   Storage   Compute   Instrument   •  SURFfederatie Virt Broker   Broker   Broker   Broker   •  SURFteamsual Comput Instrum •  OpenSocialNetwork   Storage   e   ent   IdPServices   Services   Services   Services   Experiment Publisher Grid res. PubMed My Apps.NB IC.nl Source:  SURFnet  
    • 6.    eScience  and  Big  Data  for  CiKzen  Science  and  Community       44  
    • Extending  science  and  educaKon  to   the  community    •  Community  anchor  Internet  Exchange  Points  help  clear  the  boIleneck  of   content  peering   –  Co-­‐hosKng  of  CDN  caching  boxes   –  Managed  by  NREN   –  Examples  include  KAREN  (New  Zealand),  BCnet  and  UNINETT  (Norway)  •  Minimize  tromboning  of  R&E  traffic  to  homes  and  schools  •  Can  support  extension  of  Eduroam  to  community  WiFi  spots  and/or   community  last  mile  networks  •  Allows  for  M2M  traffic  and  anywhere,  anyKme  traffic  to  propagate  through   the  community   Community  IXP  managed  by  NREN   45  
    • 7.  How  to  pay  for  it  all   46  
    • $1  billion  funding  program  •  Green  revolving  funds  are  either  part  of  a    university  endowment  program  or  publicly  traded   enKKes.   –  hIp://www.sustainablebusiness.com/index.cfm/go/news.display/id/23028    •  They  make  investments  in  energy  efficiency  and  GHG  reducKon  iniKaKves.  Payback    typically   32%    •  ICT  can  represent  up  to  40%  of  the  electrical  energy  consumpKon  at  university  and  growing  •  The  obvious  low  hanging  fruit  is  to  move,  as  much  as  possible  the  closet  clusters  and  campus   data  center  faciliKes  to  commercial  clouds.    Next  is  network  infrastructure  such  as  rouKng   and  servers    •  Other  obvious  money  saving  pracKces  are  to  power  laptop  and  cell  phone  charging  staKons   with  roof  top  solar  panels  or  micro  windmills,  deploy  solar/wind  powered  WiFi  nodes,  and   use  on  the  move  electric  charging  for  campus  uKlity  vehicles,  etc  •  Campus  IT  folk  and  NRENs  need  to  educate  managers  of  such  funds  the  IT  and  networking   can  play  a  much  more  significant  role  in  reducing  energy  consumpKon  and  GHG  emissions   47   then  tradiKonal  faciliKes  based  soluKons      
    • Cyber-­‐infrastructure  in  a  Carbon  Constrained  World   hIp://net.educause.edu/ir/library/pdf/ERM0960.pdf  
    • Let’s  Keep  The  ConversaKon  Going   E-­‐mail  list   Bill.St.Arnaud@gmail.com   Blogspot   Bill  St.  Arnaud   hIp://green-­‐broadband.blogspot.com   TwiIer   hIp://twiIer.com/BillStArnaud