0
Sim, no sim; sim, sim plus
(1) Simimlation vs no simulation and (2) simulation vs simulation
plus
Two research questions:
Forty-eight articles were coded.
Random control experiments and quasi-experimental studies were included.
In addition, it ...
Rejected articles
Liu (2006) Used a counterbalanced repeated measures design
and did not report results after each interve...
|QUESTION 1: Sim vs. no
sim|
How do simulation instructional treatments compare to
non-simulation instructional treatments...
Two kinds of variance
Within-study variance derives mainly from the sample size of the
study as reflected in the standard ...
Fixed effect model and random effects model
Fixed effect model assumes that there is one
fixed average effect size in the ...
Question 1: Forest plot & effect sizes of individual studies
Large studies with negative results
are skewing the fixed eff...
Question 1: Funnel plot of standard error by Hedge's g
The funnel plot looks at publication
bias.
The scale on the bottom ...
Tau -squared value
0.69 Tau squared value
Truncated values
We truncated three g-values to 2.5 because they were too
high (one was 2.7, one was over 3, and one was o...
Random effect size:
Random effect size = .89, k = df-1 = (26), z-value
5.19, p = .00 (results are significant)
*The unweig...
Moderator variables:
Grade level and research design
Moderating grade
level
K-5, 2: 6-8, 3: 9-12, 4: (multiple ranges) *
A mixed effects analysis q-value= 7.469, df =(2), p-va...
Moderating research design
RCT or Quasi-Experimental
A mixed effects analysis, revealed a Q-between
value = 0.01, df=1, p=...
Results and implications:
The meta-analysis of research studies comparing simulation
instructional treatments to non-simul...
QUESTION 2: Sim+modification vs.
sim
How do simulation alone compare with simulations
with instructional enhancements.
Fix...
Question 2: Forest plot and effect sizes of individual
studies
Large studies are not skewing the
results since they fall i...
Question 2: Funnel plot of standard error by Hedge's g
The funnel plot looks at publication
bias.
The scale on the bottom
...
Tau -squared value
1.09 Tau squared value
No truncation
Some effect sizes were found with large weights but they were in the
middle range.
Random effect size
Random effect size = 0.638, df = k-1 = (16), z-value 5.19,
p =.00 (results are significant)
Fixed effec...
Moderator variables:
Grade level and research design
Moderating grade level
K-5, 2: 6-8, 3: 9-12, (multiple ranges)
A mixed effects analysis reveals a q-value = 5.13, df=2, p=...
Moderating research design
RCT or Quasi-Experimental
A mixed effects analysis reveals a Q-between value =
0.02, df=1, p=0....
Results and implications:
Our meta-analysis of research studies comparing modified
simulation, simulation combined with ad...
A extremidade, the end, FIN
Upcoming SlideShare
Loading in...5
×

Meta analysis presentation-sim vs. no sim

222

Published on

meta-analysis statistics tutorial training

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
222
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Meta analysis presentation-sim vs. no sim"

  1. 1. Sim, no sim; sim, sim plus
  2. 2. (1) Simimlation vs no simulation and (2) simulation vs simulation plus Two research questions:
  3. 3. Forty-eight articles were coded. Random control experiments and quasi-experimental studies were included. In addition, it was necessary for the articles to measure achievement gains and compare means under one of two possible control conditions: Simulation vs traditional instructional treatment Simulation vs simulation plus modification The coding of articles
  4. 4. Rejected articles Liu (2006) Used a counterbalanced repeated measures design and did not report results after each intervention, therefore, there is no clear sim vs. no sim that can be included in the meta-analyis Son, Robert, Goldstone (2009) The experimental and control groups receive the same sim- what is being tested is the language used in the sim (content vs intuitive descriptors) Sheehy, Wylie & Orchard (2000) No control group-pre- experimental design LambAnnetta (2010) Sierra-Fernandez & Perales-Palacios (2003) no achievement gains were measured quantitatively Jimoyiannis & Komis (2003) Did not compare group means
  5. 5. |QUESTION 1: Sim vs. no sim| How do simulation instructional treatments compare to non-simulation instructional treatments? Fixed effect size = .59 (truncated), df= k-1= (26), z-value 12.43, p = .00 (results are significant) q-value = 311.29, df = k-1 = (26), p = .00 (sum of squares of within study variance) i-squared = 91.65 (measure of heterogeneity of studies) *Using a random effect size is necessary because the percentage of between studies heterogenity is quite large: 92%; for example, in terms of technology choices, design and sample size.
  6. 6. Two kinds of variance Within-study variance derives mainly from the sample size of the study as reflected in the standard deviation (SD). The within-study variance (V) is the SD squared. Large studies will have smaller within-study variances and small studies will have larger within-study variances. Between-study variance can only be calculated when the studies are synthesized. It is expressed as a sum of squares (SS of within study variance). When the V for each study is summed, the result is Q. Slide authored by Bob
  7. 7. Fixed effect model and random effects model Fixed effect model assumes that there is one fixed average effect size in the population and that between-study variance is sampling error. A Q-value that exceeds chance (using the Chi- squared distribution with df = k -1) indicates that the distribution is heterogeneous. Random effects model does not assume homogeneity and applies when effect sizes are assumed to estimate different populations that may vary one from another. Between-study variance represents differences among populations, not between-study variation within populations. Slide authored by Bob
  8. 8. Question 1: Forest plot & effect sizes of individual studies Large studies with negative results are skewing the fixed effect model.
  9. 9. Question 1: Funnel plot of standard error by Hedge's g The funnel plot looks at publication bias. The scale on the bottom demonstrates the distribution of effect sizes and the scale on the side shows standard error (sample size). When you read this chart, you want to determine if the distribution is biased across effect size and sample size. In this instance, while there are too many positive studies, the balance for sample sizes is reasonable.
  10. 10. Tau -squared value 0.69 Tau squared value
  11. 11. Truncated values We truncated three g-values to 2.5 because they were too high (one was 2.7, one was over 3, and one was over 4) fixed: .591 (.65 original) random: .859 (.99 original)
  12. 12. Random effect size: Random effect size = .89, k = df-1 = (26), z-value 5.19, p = .00 (results are significant) *The unweighted average effect size is close to the random effect size (approx. 1.02) Note: Fixed effect size = .59, k= df-1= (26), z-value 12.43, p = .00 (results are significant)
  13. 13. Moderator variables: Grade level and research design
  14. 14. Moderating grade level K-5, 2: 6-8, 3: 9-12, 4: (multiple ranges) * A mixed effects analysis q-value= 7.469, df =(2), p-value = 0.024 demonstrates that grade level is a significant moderator From the data, one can conclude that simulations work better for younger learners than for older learners ** *The "4" multiple range was removed because there was only one study or effect and it wouldn't mean anything *NOTE: the younger learner category has only 2 main effects compared to 6 and 18 effect sizes for the other 2 ranges.
  15. 15. Moderating research design RCT or Quasi-Experimental A mixed effects analysis, revealed a Q-between value = 0.01, df=1, p=.91 (results not significant) Research design was not a significant moderator These data are collapsed
  16. 16. Results and implications: The meta-analysis of research studies comparing simulation instructional treatments to non-simulation instructional treatments rejects the null hypothesis that there is no difference in knowledge acquisition between the two treatments. The students in the bulk of the studies who received simulation training received significantly higher scores when tested for learning or mastery of the material. Instructors who provide training through simulation will most likely improve learning outcomes over those who follow traditional instructional methods. Furthermore, grade level was found to be a moderating variable, however, the sample size in the elementary age group makes reaching any definitive conclusion unwarranted. Research design was found not to be a significant moderating variable.
  17. 17. QUESTION 2: Sim+modification vs. sim How do simulation alone compare with simulations with instructional enhancements. Fixed effect size = .766, df= k-1= (16), z-value 14.63, p = .00 (results are significant) q-value (sum of squares of within study variance) = 380.87, df= k-1= (16), p =.00 (sum of squares of variance) i-squared = 95.80 (measure of heterogeneity of studies)* *Using a random effect size is necessary because the percentage of between studies heterogenity is quite large: 96%; for example, in terms of technology choices, design and sample size.
  18. 18. Question 2: Forest plot and effect sizes of individual studies Large studies are not skewing the results since they fall into the middle range of effect sizes.
  19. 19. Question 2: Funnel plot of standard error by Hedge's g The funnel plot looks at publication bias. The scale on the bottom demonstrates the distribution of effect sizes and the scale on the side shows standard error (sample size). When you read this chart, you want to determine if the distribution is biased across effect size and sample size. In this instance, there is a balanced distribution of effect size and
  20. 20. Tau -squared value 1.09 Tau squared value
  21. 21. No truncation Some effect sizes were found with large weights but they were in the middle range.
  22. 22. Random effect size Random effect size = 0.638, df = k-1 = (16), z-value 5.19, p =.00 (results are significant) Fixed effect size = .766, df= k-1= (16), z-value 14.63, p =.00 (results are significant) The fixed and random effect sizes are closer together than for Q1, with no large effect sizes skewing the fixed effect size Some effect sizes were found with large weights, but they were in the middle range of the forest plot and, therefore, did not dramatically skew the fixed effect size.
  23. 23. Moderator variables: Grade level and research design
  24. 24. Moderating grade level K-5, 2: 6-8, 3: 9-12, (multiple ranges) A mixed effects analysis reveals a q-value = 5.13, df=2, p=.08 This results demonstrates that grade level is not a significant moderator From the data, simulations plus modification does not work better for younger learners These data are collapsed
  25. 25. Moderating research design RCT or Quasi-Experimental A mixed effects analysis reveals a Q-between value = 0.02, df=1, p=0.87 Research design was not a significant moderator These data are collapsed
  26. 26. Results and implications: Our meta-analysis of research studies comparing modified simulation, simulation combined with additional instructional treatment, to simulation only instruction rejected the null hypothesis that there is no difference in knowledge acquisition between the two treatment methods. The students in the bulk of the studies who received simulation plus supplemental training groups received significantly higher scores when tested for learning or mastery of the material over the simulation only groups. Instructors who provide training through simulation and with additional modifications will most likely improve learning outcomes over those who follow simulation only instructional methods. Furthermore, no moderating variables were identified in terms of grade level or research design.
  27. 27. A extremidade, the end, FIN
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×