Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ushahidi Toolbox - Real-time Evaluation


Published on

The Kenya Ushahidi Evaluation Project was 9-month Ushahidi evaluation project in partnership with the Harvard Humanitarian Initiative supported by the Knight Foundation. Jennifer Chan and Melissa Tully conducted research which lead to the creation of case studies and toolboxes. (2011) This is Toolbox #3: Real-Time Evaluation.

  • Be the first to comment

  • Be the first to like this

Ushahidi Toolbox - Real-time Evaluation

  1. 1. Toolbox  #3    Real-­‐Time  Evalua-on  
  2. 2. So  you’ve  done  all  of  the  planning  to  implement  your  Ushahidi/Crowdmap   pla?orm.  We’re  here  to  help  you  self-­‐evaluate  how  it’s  working!   Video   Audio   Image   Text  TWO  WAYS  TO  USE  THIS  TOOLBOX    1)  The  project  is  happening  now….I’m  ready  to  learn  about  how  the  project  is   ✔   A   doing  to  make  immediate  changes  to  improve  the  project!    2)  Well…  My  project  is  over,  but  I  want  to  learn  about  the  successes  and   ✔   B   challenges!      
  3. 3. The  first  step  is  to  ask  the  following  big  ques-ons….  •  Is  the  project  fulfilling  its  goals?   Video   Audio   Image  •  Is  the  project  fulfilling  its  objec-ves?   Text  
  4. 4. Video   Answer  the  following  big  ques-ons….   Audio   Image   Text   Goals:  Do  you  agree  or  disagree  with  the  following  statement?   The  project  has  :  “Goals  from  implementa5on  toolbox,  panel  #4,  imported  here”   Strongly    disagree   undecided    agree   strongly   disagree   Agree   Why?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       more   goals….   Objec<ves:  Do  you  agree  or  disagree  with  the  following  statement?   The  project  has  :  “Objec5ves  from  implementa5on  toolbox,  panel  #4,  imported  here”   Strongly    disagree   undecided    agree   strongly   disagree   Agree  Why?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       More   objec-ves….  
  5. 5. Video   How  are  you  achieving  your   Audio   program  objec-ves?   Image   Text  Which  parts  of  your  project  is  working  well?  and  helping  to  achieve  your  objec-ves?      Partnerships       Communica-on     Technology      Maps/Visualiza-on     Campaign/Messaging     Informa-on  Security       Informa-on  Flows      Volunteers/Teams      Ac-on/Feedback     Other,  please  describe______  Which  parts  of  the  project  need  improvement  and  may  be  preven-ng  the  project  from  achieving  its  objec-ves     Partnerships      Communica-on     Technology      Maps/Visualiza-on     Campaign/Messaging     Informa-on  Security       Informa-on  Flows      Volunteers/Teams      Ac-on/Feedback     Other,  please  describe______   User  ac5ons:    A)  select  2  from  “working  well”  and  2  from  “needs  improvement.    B)  select  all  that  apply  
  6. 6. So  now  you’ve  learned  from  &   Video   Audio   evaluated  your  project  …What   Image   are  your  ac-on  plans  for  change?   Text   Some  way  to  auto-­‐import  the  selected  categories  on  slide  6.  Here  is  a  RTE  example    Findings   Ac<on  Plan   Check-­‐in  (EXAMPLE)  Partnerships-­‐  We  have  a  strong   (write  your  ac5on  plan  here  )  We  will  con5nue  to  have  monthly   (when  will  you  rela-onships  with  our  community  partners,   mee5ngs,  and  will  look  for  people  to  support  the  workshop  space  to   check  in  to  see  we  trust  each  other.  We  have  monthly   help  con5nue  to  bring  people  together.     how  you’re  mee-ngs  to  keep  our  rela-onship  strong.   doing  with  your  (SUCCESS!)   ac-on  plan?)  (EXAMPLE)  Informa-on  Security-­‐  our  informa-on  security  plan  is  working,  Our  servers  are  secure  and  there  is  a  con-ngency  plan.  Our  partners  provide  monthly  feedback  about  their  concerns.  There  has  been  no  security  events  yet.  (SUCCESS)  (EXAMPLE)  Volunteers-­‐  It  has  been  hard  to    keep  our  volunteers.  They  say  the  long  hours  and  travel  are  difficult.  It  is  difficult  for     Print  your   ac-on  plan  the  project  manager  to  train  each  new   now!….  volunteer  every  day.  (CHALLENGE)  
  7. 7. Print  out  your  Evalua-on  Here  are  the  different  parts  of  your  evalua-on    (you  can  click  on  each  one  to  print  specific  sec5ons)     –   (autofill  in  base  on  what  is  check  on  slide  5)   –     (autofill  in  base  on  what  is  check  on  slide  5)   –     (autofill  in  base  on  what  is  check  on  slide  5)   –   (autofill  in  base  on  what  is  check  on  slide  5)   –   (autofill  in  base  on  what  is  check  on  slide  5)   –   (autofill  in  base  on  what  is  check  on  slide  5)     Print  your  self-­‐•  Print  all  of  your  checklists  and  ac-on  plan   evalua-on   report  here….  
  8. 8. Video   Audio   Partnerships  Checklist   Image   Text      1.    Which  partnerships  in  your  project  are  working  well?  Why?          2.  Which  partnerships  in  your  projects  are  difficult?  Why?             3.  Is  there  trust  between  your  partnerships?  Why  or  why  not?       4.  If  your  audience  include  local  communi-es  are  they  also  your  partners?  How  is  it  going?      
  9. 9. Video   Audio   Technology  Checklist   Image   Text       1.    How  is  the  technology  support  (  example,  server  support,  PHP  support,  ***)?  What  are  the  challenges?         2.  Do  you  feel  your  partners,  volunteers,  and  audience  are  comfortable  with  the  technology?  (SMS,  ushahidi  instance/crowdmap,  etc,  viewing  the  map)         3.  Are  your  volunteer  or  teams  who  are  responsible  to  processing  messages,  reports,  comfortable  with  the  technology?  What  are  the  challenges?        
  10. 10. Video   Campaign/Messaging  Checklist   Audio   Image   Text       1.    Do  you  believe  that  your  audience  (or  crowd)  understand  the  purpose  of  sending  in  informa-on?    Why  or  Why  not.         2.  Do  you  believe  that  your  audience  understands  what  you  are  doing  with  the  informa-on?  And  how  they  will  be  able  to  see  the  results?       3.  Are  the  expecta-ons  of  your  audience  the  same  as  your  expecta-ons?  Why  or  Why  not?       4.  What  needs  to  con-nue…..      to  keep  your  messaging/campaign  successful  and  sustainable?        to  improve  your  campaign  strategy.        
  11. 11. Video   Audio   Informa-on  Flows  Checklist   Image   Text       1.  Are  you  collec-ng  the  type  of  informa-on  you  planned  on?  Is  it  helping  you  achieve  your  goals  and  objec-ves?  How?  How  not?             2.  How  are  you  analyzing  informa-on  (  verifying,  confirming,  mapping)  What  is  working  well  and  what  can  be  improved?       3.    Do  you  feel  now  that  your  categories  are  appropriate,  why  or  why  not?             3.  Click  here  to  evaluate  and  revise  (  or  re-­‐evaluate)  your  informa-on  flow  diagram      
  12. 12. Video   Ac-on/Response/Feedback   Audio   Image   Checklist   Text       1.    Is  the  informa-on  collected  being  used  in  the  way  you  planned?         2.  Have  you  been  communica-ng  back  to  your  target  audience/or  the  public?         3.  How  well  do  you  and  your  partners  feel  the  report  and  maps  are  returning  to  decision  makers  for  program  change?       4.  If  it  has  been  more  difficult  than  you  thought  to  share  informa-on  and  the  maps  with  your  par-cipants  and  audience,  why  do  you  think  this  is  happening?        5.  Click  here  to  see  your  informa-on  flow  and  feedback  diagram,  what  would  you  keep  the  same  and  what  would  you  do  differently?        
  13. 13. Video   Audio   Communica-ons  Checklist   Image   Text     1.  Are  your  par-cipants/crowd  communica-ng/sending  you  informa-on  in  ways  that  you  thought?    Why  or  why  not?         2.  What  part  of  these  communica-on  successes  are  due  to  technology?           3.  What  part  of  these  communica-on  successes  are  due  to  trust,  other  reasons?        
  14. 14. Video   Audio   Maps/Visualiza-on  Checklist   Image   Text      1.  What  do  the  dots  on  the  map  mean  to  your  project?  To  your  par-cipants?  To  Others?                 2.    Are  people  specifically  using  the  map  for  decision  making  and  ac-on?  How  are  they  using  it?  If  they  are  not,  what  are  they  telling  you  as  the  reasons  why  they  are  not  using  it?         3.  How  do  all  of  your  audiences  have  access  to  the  map?  Do  they  know  the  way  to  find  needed  informa-on  on  the  map?  If  they  can’t  or  have  difficulty,  why?  How  have  they  told  you  ways  to  improved  the  access  and    find  informa-on  on  the  map?      
  15. 15. Video   Informa-on,  Privacy    and     Audio   Security  Checklist   Image   Text       1.  Do  you  have  a  current  program  strategy    for  privacy  and  security?  If  so  what  is  working  well,  what  will  need  improvements?    If  not  write  here  what  your  strategy  will  be.       2.  How  is  your  project  managing  the  risks  of  sharing  informa-on  (including  the  maps)?       3.    Ask  all  of  your  partners  and  contact  your  par-cipants  to  ask  them  if  they  feel  secure  providing  informa-on.  List  the  answers  here.           3.  If  your  partners  and  par-cipants  tell  you  they  are  worried  about  privacy  what  are  their  recommenda-ons  on  improving?    
  16. 16. Video   Audio   Teams/Volunteer  Checklist   Image   Text       1.                   2.               3.        
  17. 17. Video   Audio   (Create  your  own)  Checklist   Image   Text       1.             2.           3.            4.            5.            6.        
  18. 18. THANK YOU FOR COMPLETING TOOLBOX #3! Please save this file and add your name to the title, then return it to or You will receive a report from Toolbox #3 to use in your project. In your email please send comments on how you would like this tool improved or changed! Thank you for being a very important part of Ushahidi’s User community!