Cesar working document 3 urban strategy experiment 2

  • 187 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
187
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.     CESAR  WORKING  DOCUMENT  SERIES   Working  document  no.3                 Support  for  planning  process  and  content     Urban  Strategy  Experiment  No2       M.  te  Brömmelstroet     13  April  2014             This  working  document  series  is  a  joint  initiative  of  the  University  of  Amsterdam,    Utrecht  University,  Wageningen  University  and   Research  centre  and  TNO               The  research  that  is  presented  in  this  series  is  financed  by  the  NWO  program  on  Sustainable  Accessibility  of  the  Randstad:   http://www.nwo.nl/nwohome.nsf/pages/nwoa_79vlym_eng          
  • 2. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  2     TABLE  OF  CONTENT   1.   INTRODUCTION  ...............................................................................................................  3   2.   SETUP  OF  THE  EXPERIMENT  ............................................................................................  4   2.1   Intervention:  Urban  Strategy  PSS  .........................................................................................  4   2.2   Mechanisms:  how  does  Urban  Strategy  bridge  the  PSS  implementation  gap?  ...................  4   2.3   Setup  of  the  controlled  randomized  trial  .............................................................................  5   Treatment  .....................................................................................................................................  6   Evaluation  .....................................................................................................................................  7   3.   EVALUATION  RESULTS  .....................................................................................................  8   3.1   Perceived  quality  of  the  planning  process  ............................................................................  8   3.2   Perceived  quality  of  the  planning  outcome  ..........................................................................  8   3.3   Usability  characteristics  of  Urban  Strategy  ..........................................................................  8   4.   CONCLUSIONS  AND  IMPLICATIONS  ...............................................................................  10   4.1   Reflection  ...........................................................................................................................  10   5.   CONCLUSIONS  AND  DISCUSSION  ...................................................................................  11   5.1   General  conclusions  ...........................................................................................................  11   REFERENCES  ..........................................................................................................................  12   APPENDIX  I:  COMPETITION  SHEET  (DUTCH)  .........................................................................  13   Introductie  ..................................................................................................................................  13   Herinrichting  Waalhaven  oostzijde  .............................................................................................  14   Opdrachtomschrijving:  ...............................................................................................................  15   Uitgangspunten  en  randvoorwaarden  ........................................................................................  15   APPENDIX  II:  EVALUATION  FORM  FOR  QUALITY  OF  THE  PROCES  ........................................  16   APPENDIX  III:  EVALUATION  FORM  FOR  QUALITY  OF  THE  OUTCOME  ...................................  18   APPENDIX  IV:  EVALUATION  FORM  FOR  USABILITY  OF  URBAN  STRATEGY  ...........................  19    
  • 3. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  3     1. INTRODUCTION   In  this  working  document  we  report  on  a  second  randomized  controlled  trial  with  Urban  Strategy  to   test  the  added  value  of  this  Planning  Support  System  (PSS).  Urban  Strategy  is  a  software  package   developed  by  TNO  that  aims  to  improve  the  planning  process  and  planning  outcomes  of  strategic   planning.   It   does   so   by   offering   a   range   of   quick   models   that   show   the   effects   of   planning   interventions   in   an   easy   to   understand   visual   environment.   To   gain   more   insight   into   these   potential   improvements,   we   have   conducted   an   experiment   with   a   group   of   master   students   in   Urban   Planning   of   the   University   of   Amsterdam.   We   make   use   of   the   measuring   framework   as   introduced  and  discussed  in  CESAR  Working  Document  No.  1.       First,  we  describe  the  setup  of  the  experiment  (section  2).  Then,  the  findings  of  the  experiment  are   presented  (section  3).  In  the  fourth  section  we  will  briefly  discuss  the  implications  of  these  findings   and  further  research.        
  • 4. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  4     2. SETUP  OF  THE  EXPERIMENT   2.1 Intervention:  Urban  Strategy  PSS   TNO  started  around  2005  with  the  development  of  a  PSS  –  Urban  Strategy  (Borst  et  al.  2007;  2009a;   2009b)  –  specifically  aiming  to  bridge  the  existing  flexibility-­‐  and  communication  bottlenecks.  Urban   Strategy  aims  to  improve  complex  spatial  planning  processes  on  the  urban-­‐  and  regional  level.  To   do   this,   different   computer   models   are   linked   to   a   central   database   and   interface   to   provide   insights  in  a  wide  area  of  urban  indicators  and  maps.  The  effects  of  interventions  in  infrastructure,   land  use,  build  objects  and  their  functions  can  be  calculated  and  visualized.  Because  the  PSS  is  able   to  calculate  fast  and  present  the  results  in  an  attractive  1D,  2D  and  3D  visualisation  this  can  be  used   in  interactive  sessions  with  planning  actors.       Starting  point  for  Urban  Strategy  is  the  use  of  existing  state-­‐of-­‐the-­‐art  and  legally  accepted  models.   To  link  these  existing  models  a  number  of  new  elements  were  developed:   - a  database  with  an  uniform  datamodel;   - interfaces  that  show  a  3D  image  of  the  modeled  situation,  indicators  and  that  offer  functionality   to  add  interventions;   - a  framework  that  structures  the  communication  between  the  models  and  the  interfaces.       2.2 Mechanisms:  how  does  Urban  Strategy  bridge  the  PSS  implementation  gap?   The  goal  of  Urban  Strategy  is  to  enable  planning  actors  in  workshop  sessions  to  communicate  their   ideas  and  strategies  to  the  PSS  and  to  learn  from  the  effects  that  are  shown.  This  interactivity  calls   for  fast  calculations  of  all  the  model  and  fast  communication  between  all  elements.  For  this,  the   models   were   enabled   to   respond   on   events   (urban   interventions   from   the   participants   in   the   workshops.  A  new  software  architecture  was  developed  to  have  all  these  elements  communicate   (figure  2).     Through  this  increases  speed  and  the  wide  variety  of  models  that  are  linked  together,  the  PSS  aims   to   be   highly   flexible   in   offering   answers   to   a   large   number   of   questions   that   a   group   of   urban   planning  actors  can  have.     Figure  2      Schematic  overview  of  communication  architecture  of  Urban  Strategy.         The  3D  interface  generates,  based  on  objects  in  the  database,  a  3D  digital  maquette  of  the  urban   environment.  To  this,  different  information  layers  can  be  addes,  such  as  air  quality  contours,  noise   contours  and  groundwater  levels.  Also,  the  objects  can  be  colored  according  to  their  characteristics   (function,  energy  use,  CO2  emissions,  number  of  inhabitants,  etc).  The  2D  interface  can  be  used  by   the  end  user  (or  operator)  to  add  changes  to  the  database.  Objects  can  be  added  or  removed,  their  
  • 5. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  5     location  can  be  changed  and  the  characteristics  of  the  object  can  be  changed.  The  1D  interface   shows   indicators   that   are   calculated   by   all   the   models   that   are   included.   Examples   are   the   percentage  of  noise  hindrance,  group  risk  in  an  area  or  the  contribution  of  types  of  objects  to  CO2   emission.       Figure  3      The  three  interfaces  of  Urban  Strategy             2.3 Setup  of  the  controlled  randomized  trial     Together  with  TNO  we  organized  a  number  of  sessions  in  which  the  usability  of  Urban  Strategy  was   tested.   For   this   we   set   up   a   typical   urban   planning   challenge:   Optimize   existing   ideas   for   new   housing  and  working  functions  in  an  old  harbor  area  of  Rotterdam  (figure  4).  The  existing  ideas  face   problems  due  to  restrictions  on  noise,  external  safety  and  air  quality  (figure  5).       Figure  4      The  harbor  area  in  the  larger  Rotterdam  region                
  • 6. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  6     Figure  5      Current  strategy  for  the  area  (top  left),  air  quality  norms  (top  right),  noise  norms  (bottom  left)  and   external  safety  (bottom  right)                           We  invited  24  students  from  the  course  Metropolitan  Transportation  Planning,  which  is  part  of  the   one-­‐year   Master   on   Urban   Planning   of   the   University   of   Amsterdam.   They   were   told   that   they   would   take   part   in   a   strategy   making   competition   and   were   not   aware   of   the   treatment.   They   represent   future   planning   practitioners   that   deal   with   planning   challenges   such   as   the   abovementioned.  The  students  were  randomly  split  into  six  groups  of  each  three  students.  Each   group   was   then   asked   to   perform   the   planning   task   as   a   group.   One   week   before   the   strategy   making   sessions,   they   were   introduced   to   the   planning   problem   and   criteria   on   which   the   developed  strategies  were  going  to  be  judged:  innovation,  workability,  relevance  and  specificity.   See  appendix  I  for  the  (Dutch)  form  with  information  that  each  participant  received.       The   six   groups   were,   again   randomly,   divided   into   control   and   treatment   groups.   Three   groups   performed  the  planning  task  without  support  of  Urban  Strategy,  while  three  other  groups  received   this  planning  support.  The  control  groups  worked  simultaneously,  while  the  treatment  groups  did   their  work  consecutively  (due  to  constraints  of  the  model  interaction).  All  groups  got  45  minutes  to   perform  the  task  and  were  asked  to  finalize  their  strategy  on  a  sheet  of  paper  and  an  empty  map  10   minutes  before  the  deadline.  The  control  groups  were  then  left  to  perform  their  task.  They  were   supported  by  the  maps  from  figure  5  and  empty  maps  to  draw  their  plans.  Also,  pens  and  markers   were  provided  (figure  6  left).         Treatment   The  groups  that  were  supported  with  Urban  Strategy  followed  a  fixed  protocol.  On  arrival  in  the   room,  Ralph  Klerkx  (TNO)  introduced  Urban  Strategy  to  the  group.  In  10  minutes,  he  showed  the  
  • 7. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  7     functionality  of  the  instrument  and  what  it  can  and  cannot  do.  After  this  introduction  Ralph  and   Sander  Schaminée  (TNO)  guided  the  groups  in  finding  solutions  for  the  planning  problem.  First,  they   were   asked   to   come   up   with   an   improved   design   of   the   location,   especially   the   place   and   orientation  of  the  housing  and  working  blocks.  With  this,  the  groups  started  to  interact  with  Urban   Strategy  to  see  the  effects  of  their  interventions.  During  this  interacting,  the  design  was  optimized   to   minimize   the   amount   of   housing   and   working   under   the   noise,   pollution   and   external   safety   norms.  Also,  strategies  were  developed  to  cope  with  other  negative  effects,  such  as  the  congestion   and  noise  of  the  road  adjacent  to  the  study  area.  During  these  design-­‐analysis  iterations,  the  groups   also  were  asked  to  write  down  the  strategies  and  their  reasoning.       The  strategy  making  took  place  in  a  classroom  in  which  the  group  had  a  table  to  discuss  and  draw   and  could  interact  with  the  different  interfaces  of  Urban  Strategy  (figure  6  right).  Sander  Schaminée   supported  this  by  drawing  the  proposed  interventions  into  the  model.       Figure  6      The  setting  of  the  control  group  (left)  and  the  treatment  group  (right)         Evaluation   Two  evaluation  instruments  were  used  to  test  the  effects  of  the  support  by  Urban  Strategy.  Using  a   7-­‐points  Likert  scale,  all  participants  were  asked  to  rate  a  number  of  statements  related  to  the   quality  of  the  strategy  making  process  (Appendix  II).  These  perceptions  were  aggregated  to  the   level  of  the  evaluation  dimensions  and  compared  between  the  control-­‐  and  treatment  groups.  Next   to  that,  the  resulting  strategies  were  collected  and  presented  to  two  external  raters.  Using  a  7-­‐ point  Likert  scale,  they  were  asked  to  rate  a  number  of  statements  related  to  the  quality  of  the   strategy  making  outcome  (Appendix  III).  The  strategies  were  presented  to  the  raters  without  the   hypotheses  or  indication  of  treatment/control.  These  perceptions  were  combined  and  aggregated   to  the  level  of  the  evaluation  dimensions  and  compared  between  treatment  and  control.     A  final  evaluation  form  was  provided  only  to  the  participants  of  the  treatment  groups.  They  were   asked  to  reflect  on  usability  characteristics  of  Urban  Strategy.  Again,  a  7-­‐point  Likert  scale  was  used   to  gather  their  perceptions.  
  • 8. CESAR  Working  Document  Series  no.  3     Urban  Strategy  experiment  no.  2   Page  8     3. EVALUATION  RESULTS   3.1 Perceived  quality  of  the  planning  process   Due  to  the  small  number  of  observations  we  cannot  assume  to  have  controlled  for  all  possible   other  influential  variables.  Also,  we  would  like  to  raise  awareness  for  the  fact  that  we  worked  with   students,  for  which  it  was  often  the  first  time  to  engage  in  such  a  strategy  making  process.  There   are  however  some  remarkable  outcomes.       On  the  overall  score  (at  right  of  table  1)  we  see  no  effect  of  the  support  by  Urban  Strategy.  On  the   level  of  the  dimensions,  we  see  a  meaningful  positive  effect  on  ‘effectiveness’,  although  this  effect   is  not  significant  due  to  the  large  variation  within  the  control-­‐  and  treatment  group.  Apparently,   perceptions  of  this  vary  wildly.  A  very  negative  effect  is  found  on  ‘consensus  on  goals’,  but  again   the  found  difference  is  not  significant.  Most  effects  on  the  other  dimensions  are  close  to  zero.  We   can  therefore  not  confirm  a  positive  effect  of  Urban  Strategy  on  the  strategy  making  process.       3.2 Perceived  quality  of  the  planning  outcome   The  overall  quality  of  the  planning  outcome  (at  right  in  table  2)  is  affected  negatively  by  the  support   of   Urban   Strategy.   This   effect   is   not   significant,   but   still   noteworthy.   Only   ‘implementability’   is   positively   affected,   while   all   other   dimensions   show   a   negative   effect   of   Urban   Strategy.   This   is   most   remarkable   for   ‘applicability’   and   the   related   grouped   dimension   of   ‘relevance’   where   negative  scores  of  respectively  1,5  and  0,78  are  found.  Again,  we  can  conclude  from  these  scores   that  we  cannot  confirm  a  positive  effect  of  Urban  Strategy  on  the  strategy  making  outcome.       3.3 Usability  characteristics  of  Urban  Strategy   Although  a  positive  effect  of  the  instrument  could  not  be  confirmed,  the  participants  were  very   positive  about  its  usability.  The  scores  in  table  3  (on  a  7  point  scale  and  arranged  from  high  to  low)   indicate  that  all  usability  dimensions  score  over  4,6.  Most  positive  elements  are  the  ‘support  for   evaluating  alternatives’,  ‘communicative  value’  and  ‘supporting  the  creation  of  ideas’.