Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Green Lab - [02 A] The experimental process


Published on

This presentation is about a lecture I gave within the "Green Lab" course of the Computer Science master, Software Engineering and Green IT track of the Vrije Universiteit Amsterdam:

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The Green Lab - [02 A] The experimental process

  1. 1. Vrije Universiteit Amsterdam
  2. 2. Vrije Universiteit Amsterdam Experiment principles
  3. 3. Vrije Universiteit Amsterdam Image encoding: which algorithm should I use to optimize battery usage? ○ PNG ○ JPEG ○ …
  4. 4. Vrije Universiteit Amsterdam MAIN DRIVERS of experimentation: ● to be able to scientifically assess an idea ● generalize
  5. 5. Vrije Universiteit Amsterdam Theory Observation Cause EffectCausation e.g. green encoding algorithm e.g. energy efficiency Treatment Experiment Outcome e.g. JPEG, PNG e.g. energy consumption per image
  6. 6. Vrije Universiteit Amsterdam Terminology
  7. 7. Vrije Universiteit Amsterdam ● Dependent variables: quantities observed in the study (a.k.a. response, output variables) ○ e.g. energy consumption, image quality ● Independent variables: quantities that we are able to manipulate/control (a.k.a. input variables) ○ e.g. encoding algorithm, size of image, operating system
  8. 8. Vrije Universiteit Amsterdam ● Factor: an independent variable that we deliberately manipulate/control ○ e.g. image encoding algorithm ● Treatment: a specific value of a factor ○ e.g. JPEG, PNG
  9. 9. Vrije Universiteit Amsterdam ● Subject: who applies the treatment ○ e.g. software developer, software architect, user ● Object: the receiver of the treatment ○ e.g. software application
  10. 10. Vrije Universiteit Amsterdam ● Trial: combination of subject, object and treatment ○ e.g. a software developer develops an app that encodes images using the JPEG algorithm ● Experiment: a combination of several trials to observe the effects of the treatments ○ e.g. ???
  11. 11. Vrije Universiteit Amsterdam
  12. 12. Vrije Universiteit Amsterdam The experimental process
  13. 13. Vrije Universiteit Amsterdam ● Main idea behind the experiment ● The hypotheses must be stated clearly ○ Not formally, just clearly Idea
  14. 14. Vrije Universiteit Amsterdam Experiment scoping ● We scope the experiment by stating the problem, objectives, and goals ● We will use the Goal-Question-Metric (GQM) framework Idea
  15. 15. Vrije Universiteit Amsterdam ● Define context ● Formulate hypotheses ● Identify input and output variables ● Design the study ● Instrumentation ● Analyze validity threats Experiment scoping Experiment planning Idea
  16. 16. Vrije Universiteit Amsterdam ● Preparation ○ Guidelines, subjects training, code instrumentation, ... ● Execution ○ aka measurements collection ● Data Validation Experiment scoping Experiment planning Idea Experiment operation
  17. 17. Vrije Universiteit Amsterdam ● Understanding the data ○ descriptive statistics ○ exploratory data analysis (e.g. box plots, scatter plots, … ) ● Hypothesis testing ● Results interpretation Experiment scoping Experiment planning Idea Experiment operation Analysis & interpretation
  18. 18. Vrije Universiteit Amsterdam ● Document the results ● Prepare replication package ● Sum up lessons learned ● Write down reflections Experiment scoping Experiment planning Idea Experiment operation Analysis & interpretation Presentation & package
  19. 19. Vrije Universiteit Amsterdam ● ○ ○ … ● ○ ■
  20. 20. Vrije Universiteit Amsterdam Chapter 6
  21. 21. Vrije Universiteit Amsterdam Some contents of lecture extracted from: ● Giuseppe Procaccianti’s lectures at VU