Whether you work at a non-profit or for-profit, creating games/sims with positive learning outcomes is your mission. It’s also critical to your success. This session will share perspectives from a principal investigator and edtech co-founder who has led studies for the US Department of Education, NSF, school systems, and media companies. It will include a robust, audience-driven discussion on these topics: – What “effective” means to K-12 stakeholders – Study designs, partners, and populations – Fidelity of implementation – Funding and budgeting – Risks

1. Proving Game Efficacy:
Your Most Important Next Step
© Speak Agent, Inc.
Ben Grimley, CEO
ben@speakagent.com
(301) 956-9229

2. Gee, J.P. (2008). Getting Over the Slump: Innovation Strategies to Promote Children’s Learning. New York: The Joan Ganz Cooney Center at Sesame Workshop.
Speak Agent = 1st research-based academic language learning platform
Academic language is the #1 predictor of K-12 success.

3. Why are you doing research?
Lesson #1: Research is a strategy, not a tactic.

4. Research impacts everything!
Product Design
Funding Your MissionPartnerships
Marketing
Research is a continuous, iterative cycle.

5. Research begets more research.
And more funding, if you do it right!

6. What is efficacy?
Lesson #2: The answer is based on your perspective.

7. Feasibility
+ Outcomes
Playability
+ Outcomes
Marketability
+ Outcomes
Measurability
+ Outcomes
Who are your stakeholders?
Educators
AdministratorsTechnologists
Students
Each may have a different threshold for evidence.

8. Yay, it works! We’re done here, right?
10.3%
3.3%
0%
3%
6%
8%
11%
Li, Julia (2018). Speak Agent in the Classroom Summary Report. Bloomington: Rockman et al, June 2018. Funded by National Science Foundation award no. 1632488.
Treatment Control
Gains in science concepts:

9. Sorry to rain on your parade, but...

10. There’s levels to your research.
Lesson #3: There is always another level to beat.

11. Choose a level, Knight.
Institute of Education Services, U.S. Department of Education, and the National Science Foundation. (2013). Common guidelines for education research and development: A joint report. Washington, DC.

12. Did you prove cause and effect?
31%
15%
8%
0%
10%
20%
30%
40%
Reading Science Social Studies
Pre-post gains reproduced the NSF study:
Watch out for confounds and avoid selection bias.

13. Research requires partners.

14. Research requires funding.
Research design
PD and support
Data collection &
analysis
Send honoraria

15. Choose good sidekicks.
Lesson #4: So much can go wrong if you pick the wrong partners.

16. Risk: Do they have the right stuff?
Sample Population
Staff TimeInstruction Time
Tech
Commitment
Data
Line up partners before you shop for funding.
Size +
composition
Type +
accessibility
Collection +
availability
Content +
tech time
Planning +
training time
Incentive +
leadership

17. Risk: Usability
If users can’t figure it out, you won’t get efficacy.

18. Risk: Feasibility
Yeah, but can that work in a real environment?

19. Risk: Fidelity
Are users using the usable, feasible intervention as you intended?
Lesson #5: Do not underestimate this risk.

20. A commonly missed ingredient...
I need training. Seriously.

21. Risk: Scalability
Lesson #6: Your intervention must be able to succeed without you.

22. Let’s discuss!