Your SlideShare is downloading. ×
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Handbook of psychology vol 02 research methods in psychology (2003) ww
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Handbook of psychology vol 02 research methods in psychology (2003) ww

2,947

Published on

Published in: Technology, Education
1 Comment
3 Likes
Statistics
Notes
  • amazing material and contribution!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,947
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
309
Comments
1
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. HANDBOOKofPSYCHOLOGYVOLUME 2RESEARCH METHODS IN PSYCHOLOGYJohn A. SchinkaWayne F. VelicerVolume EditorsIrving B. WeinerEditor-in-ChiefJohn Wiley & Sons, Inc.
  • 2. HANDBOOKofPSYCHOLOGY
  • 3. HANDBOOKofPSYCHOLOGYVOLUME 2RESEARCH METHODS IN PSYCHOLOGYJohn A. SchinkaWayne F. VelicerVolume EditorsIrving B. WeinerEditor-in-ChiefJohn Wiley & Sons, Inc.
  • 4. This book is printed on acid-free paper.Copyright © 2003 by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.Published simultaneously in Canada.No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic,mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 UnitedStates Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of theappropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400,fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to thePermissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008,e-mail: permcoordinator@wiley.com.Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, theymake no representations or warranties with respect to the accuracy or completeness of the contents of this book and specificallydisclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended bysales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation.You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit orany other commercial damages, including but not limited to special, incidental, consequential, or other damages.This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is soldwith the understanding that the publisher is not engaged in rendering professional services. If legal, accounting, medical,psychological or any other expert assistance is required, the services of a competent professional person should be sought.Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where John Wiley &Sons, Inc. is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact theappropriate companies for more complete information regarding trademarks and registration.For general information on our other products and services please contact our Customer Care Department within the U.S. at(800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available inelectronic books.Library of Congress Cataloging-in-Publication Data:Handbook of psychology / Irving B. Weiner, editor-in-chief.p. cm.Includes bibliographical references and indexes.Contents: v. 1. History of psychology / edited by Donald K. Freedheim — v. 2. Researchmethods in psychology / edited by John A. Schinka, Wayne F. Velicer — v. 3. Biologicalpsychology / edited by Michela Gallagher, Randy J. Nelson — v. 4. Experimentalpsychology / edited by Alice F. Healy, Robert W. Proctor — v. 5. Personality and socialpsychology / edited by Theodore Millon, Melvin J. Lerner — v. 6. Developmentalpsychology / edited by Richard M. Lerner, M. Ann Easterbrooks, Jayanthi Mistry — v. 7.Educational psychology / edited by William M. Reynolds, Gloria E. Miller — v. 8.Clinical psychology / edited by George Stricker, Thomas A. Widiger — v. 9. Health psychology /edited by Arthur M. Nezu, Christine Maguth Nezu, Pamela A. Geller — v. 10. Assessmentpsychology / edited by John R. Graham, Jack A. Naglieri — v. 11. Forensic psychology /edited by Alan M. Goldstein — v. 12. Industrial and organizational psychology / editedby Walter C. Borman, Daniel R. Ilgen, Richard J. Klimoski.ISBN 0-471-17669-9 (set) — ISBN 0-471-38320-1 (cloth : alk. paper : v. 1)— ISBN 0-471-38513-1 (cloth : alk. paper : v. 2) — ISBN 0-471-38403-8 (cloth : alk. paper : v. 3)— ISBN 0-471-39262-6 (cloth : alk. paper : v. 4) — ISBN 0-471-38404-6 (cloth : alk. paper : v. 5)— ISBN 0-471-38405-4 (cloth : alk. paper : v. 6) — ISBN 0-471-38406-2 (cloth : alk. paper : v. 7)— ISBN 0-471-39263-4 (cloth : alk. paper : v. 8) — ISBN 0-471-38514-X (cloth : alk. paper : v. 9)— ISBN 0-471-38407-0 (cloth : alk. paper : v. 10) — ISBN 0-471-38321-X (cloth : alk. paper : v. 11)— ISBN 0-471-38408-9 (cloth : alk. paper : v. 12)1. Psychology. I. Weiner, Irving B.BF121.H1955 2003150—dc212002066380Printed in the United States of America.10 9 8 7 6 5 4 3 2 1➇
  • 5. Editorial BoardVolume 1History of PsychologyDonald K. Freedheim, PhDCase Western Reserve UniversityCleveland, OhioVolume 2Research Methods in PsychologyJohn A. Schinka, PhDUniversity of South FloridaTampa, FloridaWayne F. Velicer, PhDUniversity of Rhode IslandKingston, Rhode IslandVolume 3Biological PsychologyMichela Gallagher, PhDJohns Hopkins UniversityBaltimore, MarylandRandy J. Nelson, PhDOhio State UniversityColumbus, OhioVolume 4Experimental PsychologyAlice F. Healy, PhDUniversity of ColoradoBoulder, ColoradoRobert W. Proctor, PhDPurdue UniversityWest Lafayette, IndianaVolume 5Personality and Social PsychologyTheodore Millon, PhDInstitute for Advanced Studies inPersonology and PsychopathologyCoral Gables, FloridaMelvin J. Lerner, PhDFlorida Atlantic UniversityBoca Raton, FloridaVolume 6Developmental PsychologyRichard M. Lerner, PhDM. Ann Easterbrooks, PhDJayanthi Mistry, PhDTufts UniversityMedford, MassachusettsVolume 7Educational PsychologyWilliam M. Reynolds, PhDHumboldt State UniversityArcata, CaliforniaGloria E. Miller, PhDUniversity of DenverDenver, ColoradoVolume 8Clinical PsychologyGeorge Stricker, PhDAdelphi UniversityGarden City, New YorkThomas A. Widiger, PhDUniversity of KentuckyLexington, KentuckyVolume 9Health PsychologyArthur M. Nezu, PhDChristine Maguth Nezu, PhDPamela A. Geller, PhDDrexel UniversityPhiladelphia, PennsylvaniaVolume 10Assessment PsychologyJohn R. Graham, PhDKent State UniversityKent, OhioJack A. Naglieri, PhDGeorge Mason UniversityFairfax, VirginiaVolume 11Forensic PsychologyAlan M. Goldstein, PhDJohn Jay College of CriminalJustice–CUNYNew York, New YorkVolume 12Industrial and OrganizationalPsychologyWalter C. Borman, PhDUniversity of South FloridaTampa, FloridaDaniel R. Ilgen, PhDMichigan State UniversityEast Lansing, MichiganRichard J. Klimoski, PhDGeorge Mason UniversityFairfax, Virginiav
  • 6. My efforts in this work are proudly dedicated to Katherine, Christy, and John C. Schinka.J. A. S.This work is dedicated to Sue, the perfect companion for life’s many journeysand the center of my personal universe.W. F. V.
  • 7. Handbook of Psychology PrefacePsychology at the beginning of the twenty-first century hasbecome a highly diverse field of scientific study and appliedtechnology. Psychologists commonly regard their disciplineas the science of behavior, and the American PsychologicalAssociation has formally designated 2000 to 2010 as the“Decade of Behavior.” The pursuits of behavioral scientistsrange from the natural sciences to the social sciences and em-brace a wide variety of objects of investigation. Some psy-chologists have more in common with biologists than withmost other psychologists, and some have more in commonwith sociologists than with most of their psychological col-leagues. Some psychologists are interested primarily in the be-havior of animals, some in the behavior of people, and othersin the behavior of organizations. These and other dimensionsof difference among psychological scientists are matched byequal if not greater heterogeneity among psychological practi-tioners, who currently apply a vast array of methods in manydifferent settings to achieve highly varied purposes.Psychology has been rich in comprehensive encyclope-dias and in handbooks devoted to specific topics in the field.However, there has not previously been any single handbookdesigned to cover the broad scope of psychological scienceand practice. The present 12-volume Handbook of Psychol-ogy was conceived to occupy this place in the literature.Leading national and international scholars and practitionershave collaborated to produce 297 authoritative and detailedchapters covering all fundamental facets of the discipline,and the Handbook has been organized to capture the breadthand diversity of psychology and to encompass interests andconcerns shared by psychologists in all branches of the field.Two unifying threads run through the science of behavior.The first is a common history rooted in conceptual and em-pirical approaches to understanding the nature of behavior.The specific histories of all specialty areas in psychologytrace their origins to the formulations of the classical philoso-phers and the methodology of the early experimentalists, andappreciation for the historical evolution of psychology in allof its variations transcends individual identities as being onekind of psychologist or another. Accordingly, Volume 1 inthe Handbook is devoted to the history of psychology asit emerged in many areas of scientific study and appliedtechnology.A second unifying thread in psychology is a commitmentto the development and utilization of research methodssuitable for collecting and analyzing behavioral data. Withattention both to specific procedures and their applicationin particular settings, Volume 2 addresses research methodsin psychology.Volumes 3 through 7 of the Handbook present the sub-stantive content of psychological knowledge in five broadareas of study: biological psychology (Volume 3), experi-mental psychology (Volume 4), personality and social psy-chology (Volume 5), developmental psychology (Volume 6),and educational psychology (Volume 7). Volumes 8 through12 address the application of psychological knowledge infive broad areas of professional practice: clinical psychology(Volume 8), health psychology (Volume 9), assessment psy-chology (Volume 10), forensic psychology (Volume 11), andindustrial and organizational psychology (Volume 12). Eachof these volumes reviews what is currently known in theseareas of study and application and identifies pertinent sourcesof information in the literature. Each discusses unresolved is-sues and unanswered questions and proposes future direc-tions in conceptualization, research, and practice. Each of thevolumes also reflects the investment of scientific psycholo-gists in practical applications of their findings and the atten-tion of applied psychologists to the scientific basis of theirmethods.The Handbook of Psychology was prepared for the pur-pose of educating and informing readers about the presentstate of psychological knowledge and about anticipated ad-vances in behavioral science research and practice. With thispurpose in mind, the individual Handbook volumes addressthe needs and interests of three groups. First, for graduate stu-dents in behavioral science, the volumes provide advancedinstruction in the basic concepts and methods that define thefields they cover, together with a review of current knowl-edge, core literature, and likely future developments. Second,in addition to serving as graduate textbooks, the volumesoffer professional psychologists an opportunity to read andcontemplate the views of distinguished colleagues concern-ing the central thrusts of research and leading edges of prac-tice in their respective fields. Third, for psychologists seekingto become conversant with fields outside their own specialtyix
  • 8. x Handbook of Psychology Prefaceand for persons outside of psychology seeking informa-tion about psychological matters, the Handbook volumesserve as a reference source for expanding their knowledgeand directing them to additional sources in the literature.The preparation of this Handbook was made possible bythe diligence and scholarly sophistication of the 25 volumeeditors and co-editors who constituted the Editorial Board.As Editor-in-Chief, I want to thank each of them for the plea-sure of their collaboration in this project. I compliment themfor having recruited an outstanding cast of contributors totheir volumes and then working closely with these authors toachieve chapters that will stand each in their own right asvaluable contributions to the literature. I would like finally toexpress my appreciation to the editorial staff of John Wileyand Sons for the opportunity to share in the development ofthis project and its pursuit to fruition, most particularly toJennifer Simon, Senior Editor, and her two assistants, MaryPorterfield and Isabel Pratt. Without Jennifer’s vision of theHandbook and her keen judgment and unflagging support inproducing it, the occasion to write this preface would nothave arrived.IRVING B. WEINERTampa, Florida
  • 9. Volume PrefacexiA scientific discipline is defined in many ways by the re-search methods it employs. These methods can be said to rep-resent the common language of the discipline’s researchers.Consistent with the evolution of a lexicon, new researchmethods frequently arise from the development of newcontent areas. By every available measure—number of re-searchers, number of publications, number of journals, num-ber of new subdisciplines—psychology has undergone atremendous growth over the last half-century. This growth isreflected in a parallel increase in the number of new researchmethods available.As we were planning and editing this volume, we dis-cussed on many occasions the extent to which psychologyand the available research methods have become increasingcomplex over the course of our careers. When our generationof researchers began their careers in the late 1960s and early1970s, experimental design was largely limited to simplebetween-group designs, and data analysis was dominated bya single method, the analysis of variance. A few other ap-proaches were employed, but by a limited number of re-searchers. Multivariate statistics had been developed, butmultiple regression analysis was the only method that wasapplied with any frequency. Factor analysis was used almostexclusively as a method in scale development. Classical testtheory was the basis of most psychological and educationalmeasures. Analysis of data from studies that did not meeteither the design or measurement assumptions required for ananalysis of variance was covered for most researchers by asingle book on nonparametric statistics by Siegel (1956). Asa review of the contents of this volume illustrates, the choiceof experimental and analytic methods available to thepresent-day researcher is much broader. It would be fair tosay that the researcher in the 1960s had to formulate researchquestions to fit the available methods. Currently, there are re-search methods available to address most research questions.In the history of science, an explosion of knowledge isusually the result of an advance in technology, new theoreti-cal models, or unexpected empirical findings. Advances inresearch methods have occurred as the result of all three fac-tors, typically in an interactive manner. Some of the specificfactors include advances in instrumentation and measure-ment technology, the availability of inexpensive desktopcomputers to perform complex methods of data analysis, in-creased computer capacity allowing for more intense analysisof larger datasets, computer simulations that permit the eval-uation of procedures across a wide variety of situations, newapproaches to data analysis and statistical control, and ad-vances in companion sciences that opened pathways to theexploration of behavior and created new areas of researchspecialization and collaboration.Consider the advances since the publication of thefirst edition of Kirk’s (1968) text on experimental design.At that time most studies were relatively small N experimentsthat were conducted in psychology laboratories. Research ac-tivity has subsequently exploded in applied and clinicalareas, with a proliferation of new journals largely dedicatedto quasi-experimental studies and studies in the natural envi-ronment (e.g., in neuropsychology and health psychology).Techniques such as polymerase chain reaction allow psychol-ogists to test specific genes as risk candidates for behavioraldisorders. These studies rely on statistical procedures that arestill largely ignored by many researchers (e.g., logistic re-gression, structural equation modeling). Brain imagingprocedures such as magnetic resonance imaging, magnetoen-cephalography, and positron-emission tomography providecognitive psychologists and neuropsychologists the opportu-nity to study cortical activity on-line. Clinical trials involvingbehavioral interventions applied to large, representative sam-ples are commonplace in health psychology. Research em-ploying each of these procedures requires not only highlyspecific and rigorous research methods, but also specialmethods for handling and analyzing extremely large volumesof data. Even in more traditional areas of research that con-tinue to rely on group experimental designs, issues of mea-suring practical significance, determination of sample sizeand power, and procedures for handling nuisance variablesare now important concerns. Not surprisingly, the third edi-tion of Kirk’s (1995) text has grown in page length by 60%.Our review of these trends leads to several conclusions,which are reflected in the selection of topics covered by thechapters in this volume. Six features appear to characterizethe evolution in research methodology in psychology.First, there has been a focus on the development of proce-dures that employ statistical control rather than experimental
  • 10. xii Volume Prefacecontrol. Because most of the recent growth involves researchin areas that preclude direct control of independent variables,multivariate statistics and the development of methods suchas path analysis and structural equation modeling have beencritical developments. The use of statistical control has al-lowed psychology to move from the carefully controlled con-fines of the laboratory to the natural environment.Second, there has been an increasing focus on construct-driven, or latent-variable, research. A construct is defined bymultiple observed variables. Constructs can be viewed asmore reliable and more generalizable than a single observedvariable. Constructs serve to organize a large set of observedvariables, resulting in parsimony. Constructs are also theoret-ically based. This theory-based approach serves to guidestudy design, the choice of variables, the data analysis, andthe data interpretation.Third, there has been an increasing emphasis on the de-velopment of new measures and new measurement models.This is not a new trend but an acceleration of an old trend.The behavioral sciences have always placed the most empha-sis on the issue of measurement. With the movement of thefield out of the laboratory combined with advances in tech-nology, the repertoire of measures, the quality of the mea-sures, and the sophistication of the measurement models haveall increased dramatically.Fourth, there is increasing recognition of the importance ofthe temporal dimension in understanding a broad range of psy-chological phenomena. We have become a more intervention-oriented science, recognizing not only the complexity oftreatment effects but also the importance of the change in pat-terns of the effects over time. The effects of an interventionmay be very different at different points in time. New statisti-cal models for modeling temporal data have resulted.Fifth, new methods of analysis have been developedthat no longer require the assumption of a continuous, equal-interval, normally distributed variable. Previously, re-searchers had the choice between very simple but limitedmethods of data analysis that corresponded to the propertiesof the measure or more complex sophisticated methods ofanalysis that assumed, often inappropriately, that the measuremet very rigid assumptions. New methods have been devel-oped for categorical, ordinal, or simply nonnormal variablesthat can perform an equally sophisticated analysis.Sixth, the importance of individual differences is increas-ingly emphasized in intervention studies. Psychology hasalways been interested in individual differences, but meth-ods of data analysis have focused almost entirely on the rela-tionships between variables. Individuals were studied asmembers of groups, and individual differences served only toinflate the error variance. New techniques permit researchersto focus on the individual and model individual differences.This becomes increasingly important as we recognize that in-terventions do not affect everyone in exactly the same waysand that interventions become more and more tailored to theindividual.The text is organized into four parts. The first part, titled“Foundations of Research,” addresses issues that are funda-mental to all behavioral science research. The focus is onstudy design, data management, data reduction, and data syn-thesis. The first chapter, “Experimental Design” by Roger E.Kirk, provides an overview of the basic considerations thatgo into the design of a study. Once, a chapter on this topicwould have had to devote a great deal of attention to compu-tational procedures. The availability of computers permits ashift in focus to the conceptual rather than the computationalissues. The second chapter, “Exploratory Data Analysis” byJohn T. Behrens and Chong-ho Yu, reminds us of the funda-mental importance of looking at data in the most basic waysas a first step in any data analysis. In some ways this repre-sents a “back to the future” chapter. Advances in computer-based graphical methods have brought a great deal of sophis-tication to this very basic first step.The third chapter, “Power: Basics, Practical Problems,and Possible Solutions” by Rand R. Wilcox, reflects the crit-ical change in focus for psychological research. Originally,the central focus of a test of significance was on controllingType I error rates. The late Jacob Cohen emphasized that re-searchers should be equally concerned by Type II errors.This resulted in an emphasis on the careful planning of astudy and a concern with effect size and selecting the appro-priate sample size. Wilcox updates and extends these con-cepts. Chapter 4, “Methods for Handling Missing Data” byJohn W. Graham, Patricio E. Cumsille, and Elvira Elek-Fisk,describes the impressive statistical advances in addressingthe common practical problem of missing observations.Previously, researchers had relied on a series of ad hoc pro-cedures, often resulting in very inaccurate estimates. The newstatistical procedures allow the researcher to articulate theassumptions about the reason the data is missing and makevery sophisticated estimates of the missing value based on allthe available information. This topic has taken on even moreimportance with the increasing emphasis on longitudinalstudies and the inevitable problem of attrition.The fifth chapter, “Preparatory Data Analysis” by Linda S.Fidell and Barbara G. Tabachnick, describes methods of pre-processing data before the application of other methods ofstatistical analysis. Extreme values can distort the results ofthe data analysis if not addressed. Diagnostic methods canpreprocess the data so that complex procedures are not un-duly affected by a limited number of cases that often are the
  • 11. Volume Preface xiiiresult of some type of error. The last two chapters in this part,“Factor Analysis” by Richard L. Gorsuch and “Clusteringand Classification Methods” by Glenn W. Milligan andStephen C. Hirtle, describe two widely employed parsimonymethods. Factor analysis operates in the variable domain andattempts to reduce a set of p observed variables to a smallerset of m factors. These factors, or latent variables, are moreeasily interpreted and thus facilitate interpretation. Clusteranalysis operates in the person domain and attempts to reducea set of N individuals to a set of k clusters. Cluster analysisserves to explore the relationships among individuals and or-ganize the set of individuals into a limited number of sub-types that share essential features. These methods are basic tothe development of construct-driven methods and the focuson individual differences.The second part, “Research Methods in Specific ContentAreas,” addresses research methods and issues as they applyto specific content areas. Content areas were chosen in part toparallel the other volumes of the Handbook. More important,however, we attempted to sample content areas from a broadspectrum of specialization with the hope that these chapterswould provide insights into methodological concerns andsolutions that would generalize to other areas. Chapter 8,“Clinical Forensic Psychology” by Kevin S. Douglas, RandyK. Otto, and Randy Borum, addresses research methods andissues that occur in assessment and treatment contexts. Foreach task that is unique to clinical forensic psychologyresearch, they provide examples of the clinical challengesconfronting the psychologist, identify problems faced whenresearching the issues or constructs, and describe not only re-search strategies that have been employed but also theirstrengths and limitations. In Chapter 9, “Psychotherapy Out-come Research,” Evelyn S. Behar and Thomas D. Borkovecaddress the methodological issues that need to be consideredfor investigators to draw the strongest and most specificcause-and-effect conclusions about the active components oftreatments, human behavior, and the effectiveness of thera-peutic interventions.The field of health psychology is largely defined by threetopics: the role of behavior (e.g., smoking) in the develop-ment and prevention of disease, the role of stress and emotionas psychobiological influences on disease, and psychologicalaspects of acute and chronic illness and medical care. Insightinto the methodological issues and solutions for research ineach of these topical areas is provided by Timothy W. Smithin Chapter 10, “Health Psychology.”At one time, most behavioral experimentation was con-ducted by individuals whose training focused heavily on ani-mal research. Now many neuroscientists, trained in variousfields, conduct research in animal learning and publishfindings that are of interest to psychologists in many fields.The major goal of Chapter 11, “Animal Learning” by RussellM. Church, is to transfer what is fairly common knowledge inexperimental animal psychology to investigators with limitedexposure to this area of research. In Chapter 12, “Neuropsy-chology,” Russell M. Bauer, Elizabeth C. Leritz, and DawnBowers provide a discussion of neuropsychological inference,an overview of major approaches to neuropsychological re-search, and a review of newer techniques, including functionalneuroimaging, electrophysiology, magnetoencephalography,and reversible lesion methods. In each section, they describethe conceptual basis of the technique, outline its strengths andweaknesses, and cite examples of how it has been used inaddressing conceptual problems in neuropsychology.Whatever their specialty area, when psychologists evalu-ate a program or policy, the question of impact is often at cen-ter stage. The last chapter in this part, “Program Evaluation”by Melvin M. Mark, focuses on key methods for estimatingthe effects of policies and programs in the context of evalua-tion. Additionally, Mark addresses several noncausal formsof program evaluation research that are infrequently ad-dressed in methodological treatises.The third part is titled “Measurement Issues.” Advances inmeasurement typically combine innovation in technologyand progress in theory. As our measures become more so-phisticated, the areas of application also increase.Mood emerged as a seminal concept within psychologyduring the 1980s, and its prominence has continued unabatedever since. In Chapter 14, “Mood Measurement: CurrentStatus and Future Directions,” David Watson and Jatin Vaidyaexamine current research regarding the underlying structureof mood, describe and evaluate many of the most importantmood measures, and discuss several issues related to thereliability and construct validity of mood measurement. InChapter 15, “Measuring Personality and Psychopathology,”Leslie C. Morey uses objective self-report methods of mea-surement to illustrate contemporary procedures for scaledevelopment and validation, addressing issues critical to allmeasurement methods such as theoretical articulation, situa-tional context, and the need for discriminant validity.The appeal of circular models lies in the combination of acircle’s aesthetic (organizational) simplicity and its powerfulpotential to describe data in uniquely compelling substantiveand geometric ways, as has been demonstrated in describ-ing interpersonal behavior and occupational interests. InChapter 16, “The Circumplex Model: Methods and ResearchApplications,” Michael B. Gurtman and Aaron L. Pincus dis-cuss the application of the circumplex model to the descrip-tions of individuals, comparisons of groups, and evaluationsof constructs and their measures.
  • 12. xiv Volume PrefaceChapter 17, “Item Response Theory and Measuring Abili-ties” by Karen M. Schmidt and Susan E. Embretson, de-scribes the types of formal models that have been designed toguide measure development. For many years, most tests ofability and achievement have relied on classical test theory asa framework to guide both measure development and mea-sure evaluation. Item response theory updates this model inmany important ways, permitting the development of a newgeneration of measures of abilities and achievement that areparticularly appropriate for a more interactive model of as-sessment. The last chapter of this part, “Growth CurveAnaly-sis in Contemporary Psychological Research” by John J.McArdle and John R. Nesselroade, describes new quantita-tive methods for the study of change in development psy-chology. The methods permit the researcher to model a widevariety of different patterns of developmental change overtime.The final part, “Data Analysis Methods,” addresses statis-tical procedures that have been developed recently and arestill not widely employed by many researchers. They are typ-ically dependent on the availability of high-speed computersand permit researchers to investigate novel and complex re-search questions. Chapter 19, “Multiple Linear Regression”by Leona Aiken, Stephen G. West, and Steven C. Pitts, de-scribes the advances in multiple linear regression that permitapplications of this very basic method to the analysis of com-plex data sets and the incorporation of conceptual models toguide the analysis. The testing of theoretical predictions andthe identification of implementation problems are the twomajor foci of this chapter. Chapter 20, “Logistic Regression”by Alfred DeMaris, describes a parallel method to multipleregression analysis for categorical variables. The procedurehas been developed primarily outside of psychology and isnow being used much more frequently to address psycholog-ical questions. Chapter 21, “Meta-Analysis” by Frank L.Schmidt and John E. Hunter, describes procedures that havebeen developed for the quantitative integration of researchfindings across multiple studies. Previously, research findingswere integrated in narrative form and were subject to the bi-ases of the reviewer. The method also focuses attention on theimportance of effect size estimation.Chapter 22, “Survival Analysis” by Judith D. Singer andJohn B. Willett, describes a recently developed method foranalyzing longitudinal data. One approach is to code whetheran event has occurred at a given occasion. By switching thefocus on the time to the occurrence of the event, a much morepowerful and sophisticated analysis can be performed.Again,the development of this procedure has occurred largely out-side psychology but is being employed much more fre-quently. In Chapter 23, “Time Series Analysis,” WayneVelicer and Joseph L. Fava describe a method for studyingthe change in a single individual over time. Instead of a sin-gle observation on many subjects, this method relies on manyobservations on a single subject. In many ways, this methodis the prime exemplar of longitudinal research methods.Chapter 24, “Structural Equation Modeling” by Jodie B.Ullman and Peter M. Bentler, describes a very generalmethod that combines three key themes: constructs or latentvariables, statistical control, and theory to guide data analy-sis. First employed as an analytic method little more than20 years ago, the method is now widely disseminated in thebehavioral sciences. Chapter 25, “OrdinalAnalysis of Behav-ioral Data” by Jeffrey D. Long, Du Feng, and Norman Cliff,discusses the assumptions that underlie many of the widelyused statistical methods and describes a parallel series ofmethods of analysis that only assume that the measure pro-vides ordinal information. The last chapter, “Latent Class andLatent Transition Analysis” by Stephanie L. Lanza, Brian P.Flaherty, and Linda M. Collins, describes a new method foranalyzing change over time. It is particularly appropriatewhen the change process can be conceptualized as a series ofdiscrete states.In completing this project, we realized that we were veryfortunate in several ways. Irving Weiner’s performance aseditor-in-chief was simply wonderful. He applied just the rightmixofobsessiveconcernandresponsivesupporttokeepthingson schedule. His comments on issues of emphasis, perspective,and quality were insightful and inevitably on target.We continue to be impressed with the professionalism ofthe authors that we were able to recruit into this effort.Consistent with their reputations, these individuals deliv-ered chapters of exceptional quality, making our burdenpale in comparison to other editorial experiences. Because ofthe length of the project, we shared many contributors’experiences-marriages, births, illnesses, family crises. A def-inite plus for us has been the formation of new friendshipsand professional liaisons.Our editorial tasks were also aided greatly by the generousassistance of our reviewers, most of whom will be quicklyrecognized by our readers for their own expertise in researchmethodology. We are pleased to thank James Algina, PhippsArabie, Patti Barrows, Betsy Jane Becker, Lisa M. Brown,Barbara M. Byrne, William F. Chaplin, Pat Cohen, Patrick J.Curren, Glenn Curtiss, Richard B. Darlington, SusanDuncan, Brian Everitt, Kerry Evers, Ron Gironda, LisaHarlow, Michael R. Harwell, Don Hedeker, David CharlesHowell, Lawrence J. Hubert, Bradley E. Huitema, BethJenkins, Herbert W. Marsh, Rosemarie A. Martin, Scott E.Maxwell, Kevin R. Murphy, Gregory Norman, Daniel J.Ozer, Melanie Page, Mark D. Reckase, Charles S. Reichardt,
  • 13. Volume Preface xvSteven Reise, Joseph L. Rogers, Joseph Rossi, JamesRounds, Shlomo S. Sawilowsky, Ian Spence, James H.Steiger, Xiaowu Sun, Randall C. Swaim, David Thissen,Bruce Thompson, Terence J. G. Tracey, Rod Vanderploeg,Paul F. Velleman, Howard Wainer, Douglas Williams, andseveral anonymous reviewers for their thorough work andgood counsel.We finish this preface with a caveat. Readers will in-evitably discover several contradictions or disagreementsacross the chapter offerings. Inevitably, researchers in differ-ent areas solve similar methodological problems in differentways. These differences are reflected in the offerings of thistext, and we have not attempted to mediate these differingviewpoints. Rather, we believe that the serious researcherwill welcome the opportunity to review solutions suggestedor supported by differing approaches. For flaws in the text,however, the usual rule applies: We assume all responsibility.JOHN A. SCHINKAWAYNE F. VELICERREFERENCESKirk, Roger E. (1968). Experimental design: Procedures for thebehavioral sciences. Pacific Grove, CA: Brooks/Cole.Kirk, Roger E. (1995). Experimental design: Procedures for thebehavioral sciences (3rd ed.). Pacific Grove, CA: Brooks/Cole.Siegel, S. (1956). Nonparametric statistics for the behavioralsciences. New York: McGraw-Hill.
  • 14. Handbook of Psychology Preface ixIrving B. WeinerVolume Preface xiJohn A. Schinka and Wayne F. VelicerContributors xxiPART ONEFOUNDATIONS OF RESEARCH ISSUES: STUDY DESIGN,DATA MANAGEMENT, DATA REDUCTION, AND DATA SYNTHESIS1 EXPERIMENTAL DESIGN 3Roger E. Kirk2 EXPLORATORY DATAANALYSIS 33John T. Behrens and Chong-ho Yu3 POWER: BASICS, PRACTICAL PROBLEMS, AND POSSIBLE SOLUTIONS 65Rand R. Wilcox4 METHODS FOR HANDLING MISSING DATA 87John W. Graham, Patricio E. Cumsille, and Elvira Elek-Fisk5 PREPARATORY DATAANALYSIS 115Linda S. Fidell and Barbara G. Tabachnick6 FACTOR ANALYSIS 143Richard L. Gorsuch7 CLUSTERING AND CLASSIFICATION METHODS 165Glenn W. Milligan and Stephen C. HirtlePART TWORESEARCH METHODS IN SPECIFIC CONTENT AREAS8 CLINICAL FORENSIC PSYCHOLOGY 189Kevin S. Douglas, Randy K. Otto, and Randy Borum9 PSYCHOTHERAPY OUTCOME RESEARCH 213Evelyn S. Behar and Thomas D. BorkovecContentsxvii
  • 15. xviii Contents10 HEALTH PSYCHOLOGY 241Timothy W. Smith11 ANIMAL LEARNING 271Russell M. Church12 NEUROPSYCHOLOGY 289Russell M. Bauer, Elizabeth C. Leritz, and Dawn Bowers13 PROGRAM EVALUATION 323Melvin M. MarkPART THREEMEASUREMENT ISSUES14 MOOD MEASUREMENT: CURRENT STATUS AND FUTURE DIRECTIONS 351David Watson and Jatin Vaidya15 MEASURING PERSONALITY AND PSYCHOPATHOLOGY 377Leslie C. Morey16 THE CIRCUMPLEX MODEL: METHODS AND RESEARCH APPLICATIONS 407Michael B. Gurtman and Aaron L. Pincus17 ITEM RESPONSE THEORY AND MEASURING ABILITIES 429Karen M. Schmidt and Susan E. Embretson18 GROWTH CURVE ANALYSIS IN CONTEMPORARY PSYCHOLOGICAL RESEARCH 447John J. McArdle and John R. NesselroadePART FOURDATAANALYSIS METHODS19 MULTIPLE LINEAR REGRESSION 483Leona S. Aiken, Stephen G. West, and Steven C. Pitts20 LOGISTIC REGRESSION 509Alfred DeMaris21 META-ANALYSIS 533Frank L. Schmidt and John E. Hunter22 SURVIVALANALYSIS 555Judith D. Singer and John B. Willett23 TIME SERIES ANALYSIS 581Wayne F. Velicer and Joseph L. Fava
  • 16. Contents xix24 STRUCTURAL EQUATION MODELING 607Jodie B. Ullman and Peter M. Bentler25 ORDINALANALYSIS OF BEHAVIORAL DATA 635Jeffrey D. Long, Du Feng, and Norman Cliff26 LATENT CLASS AND LATENT TRANSITION ANALYSIS 663Stephanie T. Lanza, Brian P. Flaherty, and Linda M. CollinsAuthor Index 687Subject Index 703
  • 17. Leona S. Aiken, PhDDepartment of PsychologyArizona State UniversityTempe, ArizonaRussell M. Bauer, PhDDepartment of Clinical and Health PsychologyUniversity of FloridaGainesville, FloridaEvelyn S. Behar, MSDepartment of PsychologyPennsylvania State UniversityUniversity Park, PennsylvaniaJohn T. Behrens, PhDCisco Networking Academy ProgramCisco Systems, Inc.Phoenix, ArizonaPeter M. Bentler, PhDDepartment of PsychologyUniversity of CaliforniaLos Angeles, CaliforniaThomas D. Borkovec, PhDDepartment of PsychologyPennsylvania State UniversityUniversity Park, PennsylvaniaRandy Borum, PsyDDepartment of Mental Health Law & PolicyFlorida Mental Health InstituteUniversity of South FloridaTampa, FloridaDawn Bowers, PhDDepartment of Clinical and Health PsychologyUniversity of FloridaGainesville, FloridaRussell M. Church, PhDDepartment of PsychologyBrown UniversityProvidence, Rhode IslandNorman Cliff, PhDProfessor of Psychology EmeritusUniversity of Southern CaliforniaLos Angeles, CaliforniaLinda M. Collins, PhDThe Methodology CenterPennsylvania State UniversityUniversity Park, PennsylvaniaPatricio E. Cumsille, PhDEscuela de PsicologiaUniversidad Católica de ChileSantiago, ChileAlfred DeMaris, PhDDepartment of SociologyBowling Green State UniversityBowling Green, OhioKevin S. Douglas, PhD, LLBDepartment of Mental Health Law & PolicyFlorida Mental Health InstituteUniversity of South FloridaTampa, FloridaDu Feng, PhDHuman Development and Family StudiesTexas Tech UniversityLubbock, TexasElvira Elek-Fisk, PhDThe Methodology CenterPennsylvania State UniversityUniversity Park, PennsylvaniaSusan E. Embretson, PhDDepartment of PsychologyUniversity of KansasLawrence, KansasJoseph L. Fava, PhDCancer Prevention Research CenterUniversity of Rhode IslandKingston, Rhode IslandContributorsxxi
  • 18. xxii ContributorsLinda S. Fidell, PhDDepartment of PsychologyCalifornia State UniversityNorthridge, CaliforniaBrian P. Flaherty, MSThe Methodology CenterPennsylvania State UniversityUniversity Park, PennsylvaniaRichard L. Gorsuch, PhDGraduate School of PsychologyFuller Theological SeminaryPasadena, CaliforniaJohn W. Graham, PhDDepartment of Biobehavioral HealthPennsylvania State UniversityUniversity Park, PennsylvaniaMichael B. Gurtman, PhDDepartment of PsychologyUniversity of Wisconsin-ParksideKenosha, WisconsinStephen C. Hirtle, PhDSchool of Information SciencesUniversity of PittsburghPittsburgh, PennsylvaniaJohn E. Hunter, PhDDepartment of PsychologyMichigan State UniversityEast Lansing, MichiganRoger E. Kirk, PhDDepartment of Psychology and NeuroscienceBaylor UniversityWaco, TexasStephanie T. Lanza, MSThe Methodology CenterPennsylvania State UniversityUniversity Park, PennsylvaniaElizabeth C. Leritz, MSDepartment of Clinical and Health PsychologyUniversity of FloridaGainesville, FloridaJeffrey D. Long, PhDDepartment of Educational PsychologyUniversity of MinnesotaMinneapolis, MinnesotaMelvin M. Mark, PhDDepartment of PsychologyPennsylvania State UniversityUniversity Park, PennsylvaniaJohn J. McArdle, PhDDepartment of PsychologyUniversity of VirginiaCharlottesville, VirginiaGlenn W. Milligan, PhDDepartment of Management SciencesOhio State UniversityColumbus, OhioLeslie C. Morey, PhDDepartment of PsychologyTexas A&M UniversityCollege Station, TexasJohn R. Nesselroade, PhDDepartment of PsychologyUniversity of VirginiaCharlottesville, VirginiaRandy K. Otto, PhDDepartment of Mental Health Law & PolicyFlorida Mental Health InstituteUniversity of South FloridaTampa, FloridaAaron L. Pincus, PhDDepartment of PsychologyPennsylvania State UniversityUniversity Park, PennsylvaniaSteven C. Pitts, PhDDepartment of PsychologyUniversity of Maryland, Baltimore CountyBaltimore, MarylandKaren M. Schmidt, PhDDepartment of PsychologyUniversity of VirginiaCharlottesville, VirginiaFrank L. Schmidt, PhDDepartment of Management and OrganizationUniversity of IowaIowa City, IowaJudith D. Singer, PhDGraduate School of EducationHarvard UniversityCambridge, Massachusetts
  • 19. Contributors xxiiiTimothy W. Smith, PhDDepartment of PsychologyUniversity of UtahSalt Lake City, UtahBarbara G. Tabachnick, PhDDepartment of PsychologyCalifornia State UniversityNorthridge, CaliforniaJodie B. Ullman, PhDDepartment of PsychologyCalifornia State UniversitySan Bernadino, CaliforniaJatin VaidyaDepartment of PsychologyUniversity of IowaIowa City, IowaWayne F. Velicer, PhDCancer Prevention Research CenterUniversity of Rhode IslandKingston, Rhode IslandDavid Watson, PhDDepartment of PsychologyUniversity of IowaIowa City, IowaStephen G. West, PhDDepartment of PsychologyArizona State UniversityTempe, ArizonaRand R. Wilcox, PhDDepartment of PsychologyUniversity of Southern CaliforniaLos Angeles, CaliforniaJohn B. Willett, PhDGraduate School of EducationHarvard UniversityCambridge, MassachusettsChong-ho Yu, PhDCisco Networking Academy ProgramCisco Systems, Inc.Chandler, Arizona
  • 20. PART ONEFOUNDATIONS OF RESEARCH ISSUES:STUDY DESIGN, DATA MANAGEMENT,DATA REDUCTION, AND DATA SYNTHESIS
  • 21. CHAPTER 1Experimental DesignROGER E. KIRK3SOME BASIC EXPERIMENTAL DESIGN CONCEPTS 3THREE BUILDING BLOCK DESIGNS 4Completely Randomized Design 4Randomized Block Design 6Latin Square Design 9CLASSIFICATION OF EXPERIMENTAL DESIGNS 10FACTORIAL DESIGNS 11Completely Randomized Factorial Design 11Alternative Models 14Randomized Block Factorial Design 19FACTORIAL DESIGNS WITH CONFOUNDING 21Split-Plot Factorial Design 21Confounded Factorial Designs 24Fractional Factorial Designs 25HIERARCHICAL DESIGNS 27Hierarchical Designs With One orTwo Nested Treatments 27Hierarchical Design With Crossedand Nested Treatments 28EXPERIMENTAL DESIGNS WITH A COVARIATE 29REFERENCES 31SOME BASIC EXPERIMENTALDESIGN CONCEPTSExperimental design is concerned with the skillful interroga-tion of nature. Unfortunately, nature is reluctant to revealher secrets. Joan Fisher Box (1978) observed in her autobiog-raphy of her father, Ronald A. Fisher, “Far from behavingconsistently, however, Nature appears vacillating, coy, andambiguous in her answers” (p. 140). Her most effectivetool for confusing researchers is variability—in particular,variability among participants or experimental units. Buttwo can play the variability game. By comparing the variabil-ity among participants treated differently to the variabilityamong participants treated alike, researchers can make in-formed choices between competing hypotheses in scienceand technology.We must never underestimate nature—she is a formidablefoe. Carefully designed and executed experiments are re-quired to learn her secrets. An experimental design is a planfor assigning participants to experimental conditions and thestatistical analysis associated with the plan (Kirk, 1995, p. 1).The design of an experiment involves a number of inter-related activities:1. Formulationofstatisticalhypothesesthataregermanetothescientific hypothesis. A statistical hypothesis is a statementabout (a) one or more parameters of a population or (b) thefunctional form of a population. Statistical hypothesesare rarely identical to scientific hypotheses—they aretestable formulations of scientific hypotheses.2. Determination of the experimental conditions (independentvariable) to be manipulated, the measurement (dependentvariable) to be recorded, and the extraneous conditions(nuisance variables) that must be controlled.3. Specification of the number of participants required andthe population from which they will be sampled.4. Specification of the procedure for assigning the partici-pants to the experimental conditions.5. Determination of the statistical analysis that will beperformed.In short, an experimental design identifies the independent,dependent, and nuisance variables and indicates the way inwhich the randomization and statistical aspects of an experi-ment are to be carried out.Analysis of VarianceAnalysis of variance (ANOVA) is a useful tool for under-standing the variability in designed experiments. The seminalideas for both ANOVA and experimental design can be traced
  • 22. 4 Experimental Designto Ronald A. Fisher, a statistician who worked at the Rotham-sted Experimental Station. According to Box (1978, p. 100),Fisher developed the basic ideas ofANOVAbetween 1919 and1925. The first hint of what was to come appeared in a 1918paper in which Fisher partitioned the total variance of a humanattribute into portions attributed to heredity, environment, andother factors. The analysis of variance table for a two-treat-ment factorial design appeared in a 1923 paper published withM.A. Mackenzie (Fisher & Mackenzie, 1923). Fisher referredto the table as a convenient way of arranging the arithmetic. In1924 Fisher (1925) introduced the Latin square design in con-nection with a forest nursery experiment. The publication in1925 of his classic textbook Statistical Methods for ResearchWorkers and a short paper the following year (Fisher, 1926)presented all the essential ideas of analysis of variance. Thetextbook (Fisher, 1925, pp. 244–249) included a table of thecritical values of the ANOVA test statistic in terms of a func-tion called z, where z = 12(ln ˆ␴2Treatment − ln ˆ␴2Error). The statis-tics ˆ␴2Treatment and ˆ␴2Error denote, respectively, treatment anderror variance.Amore convenient form of Fisher’s z table thatdid not require looking up log values was developed byGeorge Snedecor (1934). His critical values are expressed interms of the function F = ˆ␴2Treatment/ˆ␴2Error that is obtaineddirectly from theANOVAcalculations. He named it F in honorof Fisher. Fisher’s field of experimentation—agriculture—was a fortunate choice because results had immediate applica-tion with assessable economic value, because simplifyingassumptions such as normality and independence of errorswere usually tenable, and because the cost of conductingexperiments was modest.Three Principles of Good Experimental DesignThe publication of Fisher’s Statistical Methods for ResearchWorkers and his 1935 The Design of Experiments graduallyled to the acceptance of what today is considered to be thecornerstone of good experimental design: randomization.It is hard to imagine the hostility that greeted the suggestionthat participants or experimental units should be randomlyassigned to treatment levels. Before Fisher’s work, mostresearchers used systematic schemes, not subject to the lawsof chance, to assign participants. According to Fisher, ran-dom assignment has several purposes. It helps to distributethe idiosyncratic characteristics of participants over the treat-ment levels so that they do not selectively bias the outcome ofthe experiment. Also, random assignment permits the com-putation of an unbiased estimate of error effects—thoseeffects not attributable to the manipulation of the independentvariable—and it helps to ensure that the error effects arestatistically independent.Fisher popularized two other principles of good experi-mentation: replication and local control or blocking. Replica-tion is the observation of two or more participants underidentical experimental conditions. Fisher observed that repli-cation enables a researcher to estimate error effects andobtain a more precise estimate of treatment effects. Blocking,on the other hand, is an experimental procedure for isolatingvariation attributable to a nuisance variable. As the namesuggests, nuisance variables are undesired sources of varia-tion that can affect the dependent variable. There are manysources of nuisance variation. Differences among partici-pants comprise one source. Other sources include variationin the presentation of instructions to participants, changes inenvironmental conditions, and the effects of fatigue andlearning when participants are observed several times. Threeexperimental approaches are used to deal with nuisancevariables:1. Holding the variable constant.2. Assigning participants randomly to the treatment levels sothat known and unsuspected sources of variation amongthe participants are distributed over the entire experimentand do not affect just one or a limited number of treatmentlevels.3. Including the nuisance variable as one of the factors in theexperiment.The last experimental approach uses local control or blockingto isolate variation attributable to the nuisance variable sothat it does not appear in estimates of treatment and erroreffects. A statistical approach also can be used to deal withnuisance variables. The approach is called analysis of covari-ance and is described in the last section of this chapter.The three principles that Fisher vigorously championed—randomization, replication, and local control—remain thecornerstones of good experimental design.THREE BUILDING BLOCK DESIGNSCompletely Randomized DesignOne of the simplest experimental designs is the randomizationand analysis plan that is used with a t statistic for independentsamples. Consider an experiment to compare the effectivenessof two diets for obese teenagers. The independent variable isthe two kinds of diets; the dependent variable is the amount ofweight loss two months after going on a diet. For notationalconvenience, the two diets are called treatment A. The levelsof treatment A corresponding to the specific diets are denoted
  • 23. Three Building Block Designs 5by the lowercase letter a and a subscript: a1 denotes one dietand a2 denotes the other. A particular but unspecified level oftreatment A is denoted by aj, where j ranges over the values 1and 2. The amount of weight loss in pounds 2 months afterparticipant i went on diet j is denoted by Yij.The null and alternative hypotheses for the weight-lossexperiment are, respectively,H0: ␮1 − ␮2 = 0H1: ␮1 − ␮2 = 0,where ␮1 and ␮2 denote the mean weight loss of the respec-tive populations. Assume that 30 girls who want to loseweight are available to participate in the experiment. Theresearcher assigns n = 15 girls to each of the p = 2 diets sothat each of the (np)!/(n!)p= 155,117,520 possible assign-ments has the same probability. This is accomplished bynumbering the girls from 1 to 30 and drawing numbers froma random numbers table. The first 15 numbers drawn between1 and 30 are assigned to treatment level a1; the remaining 15numbers are assigned to a2. The layout for this experiment isshown in Figure 1.1. The girls who were assigned to treat-ment level a1 are called Group1; those assigned to treatmentlevel a2 are called Group2. The mean weight losses of the twogroups of girls are denoted by Y·1 and Y·2.The t independent-samples design involves randomlyassigning participants to two levels of a treatment. A com-pletely randomized design, which is described next, extendsthis design strategy to two or more treatment levels. The com-pletely randomized design is denoted by the letters CR-p,where CR stands for “completely randomized” and p is thenumber of levels of the treatment.Again, consider the weight-loss experiment and supposethat the researcher wants to evaluate the effectiveness ofthree diets. The null and alternative hypotheses for theexperiment are, respectively,H0: ␮1 = ␮2 = ␮3H1: ␮j = ␮j for some j and j .Assume that 45 girls who want to lose weight are available toparticipate in the experiment. The girls are randomly as-signed to the three diets with the restriction that 15 girls areassigned to each diet. The layout for the experiment is shownin Figure 1.2. A comparison of the layout in this figure withthat in Figure 1.1 for a t independent-samples design revealsthat they are the same except that the completely randomizeddesign has three treatment levels. The t independent-samplesdesign can be thought of as a special case of a completelyrandomized design. When p is equal to two, the layouts andrandomization plans for the designs are identical.Thus far I have identified the null hypothesis that theresearcher wants to test, ␮1 = ␮2 = ␮3, and described themanner in which the participants are assigned to the threetreatmentlevels.InthefollowingparagraphsIdiscussthecom-posite nature of an observation, describe the classical modelequation for a CR-p design, and examine the meaning of theterms treatment effect and error effect.An observation, which is a measure of the dependent vari-able, can be thought of as a composite that reflects theeffects of the (a) independent variable, (b) individual charac-teristics of the participant or experimental unit, (c) chancefluctuations in the participant’s performance, (d) measure-ment and recording errors that occur during data collection,Figure 1.1 Layout for a t independent-samples design. Thirty girls are ran-domly assigned to two levels of treatment A with the restriction that 15 girlsare assigned to each level. The mean weight loss in pounds for the girls intreatment levels a1 and a2 is denoted by Y·1 and Y·2, respectively.Figure 1.2 Layout for a completely randomized design (CR-3 design).Forty-five girls are randomly assigned to three levels of treatment A with therestriction that 15 girls are assigned to each level. The mean weight loss inpounds for the girls in treatment levels a1, a2, and a3 is denoted by Y·1, Y·2,and Y·3, respectively.
  • 24. 6 Experimental Designand (e) any other nuisance variables such as environmentalconditions that have not been controlled. Consider the weightloss of the fifth participant in treatment level a2. Suppose thattwo months after beginning the diet this participant has lost13 pounds (Y52 = 13). What factors have affected the value ofY52? One factor is the effectiveness of the diet. Other factorsare her weight prior to starting the diet, the degree to whichshe stayed on the diet, and the amount she exercised duringthe two-month trial, to mention only a few. In summary, Y52 isa composite that reflects (a) the effects of treatment level a2,(b) effects unique to the participant, (c) effects attributable tochance fluctuations in the participant’s behavior, (d) errors inmeasuring and recording the participant’s weight loss, and(e) any other effects that have not been controlled. Our con-jectures about Y52 or any of the other 44 observations can beexpressed more formally by a model equation. The classicalmodel equation for the weight-loss experiment isYi j = ␮ + ␣j + ␧i( j) (i = 1, . . . , n; j = 1, . . . , p),whereYi j is the weight loss for participant i in treatmentlevel aj.␮ is the grand mean of the three weight-loss popula-tion means.␣j is the treatment effect for population j and is equal to␮j − ␮. It reflects the effects of diet aj.␧i( j) is the within-groups error effect associated with Yi jand is equal to Yi j − ␮ − ␣j . It reflects all effectsnot attributable to treatment level aj. The notationi(j) indicates that the ith participant appears only intreatment level j. Participant i is said to be nestedwithin the jth treatment level. Nesting is discussedin the section titled “Hierarchical Designs.”According to the equation for this completely randomizeddesign, each observation is the sum of three parameters␮, ␣j , and ␧i( j). The values of the parameters in the equationare unknown but can be estimated from sample data.The meanings of the terms grand mean, ␮, and treatmenteffect, ␣j , in the model equation seem fairly clear; the mean-ing of error effect, ␧i( j), requires a bit more explanation. Whydo observations, Yi js, in the same treatment level vary fromone participant to the next? This variation must be due to dif-ferences among the participants and to other uncontrolledvariables because the parameters ␮ and ␣j in the model equa-tion are constants for all participants in the same treatmentlevel.To put it another way, observations in the same treatmentlevel are different because the error effects, ␧i( j)s, for theobservations are different. Recall that error effects reflect idio-syncratic characteristics of the participants—those character-istics that differ from one participant to another—and anyother variables that have not been controlled. Researchers at-tempt to minimize the size of error effects by holding sourcesof variation that might contribute to the error effects constantand by the judicial choice of an experimental design. Designsthat are described next permit a researcher to isolate and re-move some sources of variation that would ordinarily be in-cluded in the error effects.Randomized Block DesignThe two designs just described use independent samples. Twosamples are independent if, for example, a researcher ran-domly samples from two populations or randomly assigns par-ticipants to p groups. Dependent samples, on the other hand,can be obtained by any of the following procedures.1. Observe each participant under each treatment level inthe experiment—that is, obtain repeated measures on theparticipants.2. Form sets of participants who are similar with respect toa variable that is correlated with the dependent variable.This procedure is called participant matching.3. Obtain sets of identical twins or littermates in which casethe participants have similar genetic characteristics.4. Obtain participants who are matched by mutual selection,for example, husband and wife pairs or business partners.In the behavioral and social sciences, the participants areoften people whose aptitudes and experiences differ markedly.Individual differences are inevitable, but it is often possibleto isolate or partition out a portion of these effects so thatthey do not appear in estimates of the error effects. One designfor accomplishing this is the design used with a t statistic fordependent samples. As the name suggests, the design usesdependent samples. A t dependent-samples design also uses amore complex randomization and analysis plan than does a tindependent-samples design. However, the added complexityis often accompanied by greater power—a point that I will de-velop later in connection with a randomized block design.Let’s reconsider the weight-loss experiment. It is reason-able to assume that ease of losing weight is related to theamount by which a girl is overweight. The design of the exper-iment can be improved by isolating this nuisance variable.Suppose that instead of randomly assigning 30 participants tothetreatmentlevels,theresearcherformedpairsofparticipants
  • 25. Three Building Block Designs 7Figure 1.3 Layout for a t dependent-samples design. Each block containstwo girls who are overweight by about the same amount. The two girls in ablock are randomly assigned to the treatment levels. The mean weight loss inpounds for the girls in treatment levels a1 and a2 is denoted by Y·1 and Y·2,respectively.so that prior to going on a diet the participants in each pair areoverweightbyaboutthesameamount.Theparticipantsineachpair constitute a block or set of matched participants.Asimpleway to form blocks of matched participants is to rank themfrom least to most overweight. The participants ranked 1 and 2are assigned to block one, those ranked 3 and 4 are assigned toblock two, and so on. In this example, 15 blocks of dependentsamples can be formed from the 30 participants.After all of theblocks have been formed, the two participants in each blockare randomly assigned to the two diets. The layout for this ex-periment is shown in Figure 1.3. If the researcher’s hunch iscorrect that ease in losing weight is related to the amount bywhich a girl is overweight, this design should result in a morepowerfultestofthenullhypothesis,␮·1 − ␮·2 = 0,thanwoulda t test for independent samples. As we will see, the increasedpower results from isolating the nuisance variable (the amountby which the girls are overweight) so that it does not appear inthe estimate of the error effects.Earlier we saw that the layout and randomization proce-dures for a t independent-samples design and a completelyrandomized design are the same except that a completely ran-domized design can have more than two treatment levels.The same comparison can be drawn between a t dependent-samples design and a randomized block design. A random-ized block design is denoted by the letters RB-p, where RBstands for “randomized block” and p is the number of levelsof the treatment. The four procedures for obtaining depen-dent samples that were described earlier can be used to formthe blocks in a randomized block design. The procedure thatis used does not affect the computation of significance tests,but the procedure does affect the interpretation of the results.The results of an experiment with repeated measures general-ize to a population of participants who have been exposed toall of the treatment levels. However, the results of an experi-ment with matched participants generalize to a population ofparticipants who have been exposed to only one treatmentlevel. Some writers reserve the designation randomizedblock design for this latter case. They refer to a design withrepeated measurements in which the order of administrationof the treatment levels is randomized independently for eachparticipant as a subjects-by-treatments design. A design withrepeated measurements in which the order of administrationof the treatment levels is the same for all participants isreferred to as a subject-by-trials design. I use the designationrandomized block design for all three cases.Of the four ways of obtaining dependent samples, the useof repeated measures on the participants typically results inthe greatest homogeneity within the blocks. However, if re-peated measures are used, the effects of one treatment levelshould dissipate before the participant is observed under an-other treatment level. Otherwise the subsequent observationswill reflect the cumulative effects of the preceding treatmentlevels. There is no such restriction, of course, if carryover ef-fects such as learning or fatigue are the researcher’s principalinterest. If blocks are composed of identical twins or litter-mates, it is assumed that the performance of participants hav-ing identical or similar heredities will be more homogeneousthan the performance of participants having dissimilar hered-ities. If blocks are composed of participants who are matchedby mutual selection (e.g., husband and wife pairs or businesspartners), a researcher should ascertain that the participantsin a block are in fact more homogeneous with respect to thedependent variable than are unmatched participants. A hus-band and wife often have similar political attitudes; the cou-ple is less likely to have similar mechanical aptitudes.Suppose that in the weight-loss experiment the researcherwants to evaluate the effectiveness of three diets, denotedby a1, a2, and a3. The researcher suspects that ease of losingweight is related to the amount by which a girl is overweight.If a sample of 45 girls is available, the blocking proceduredescribed in connection with a t dependent-samples designcan be used to form 15 blocks of participants. The three par-ticipants in a block are matched with respect to the nuisancevariable, the amount by which a girl is overweight. The lay-out for this experiment is shown in Figure 1.4. A comparisonof the layout in this figure with that in Figure 1.3 for a tdependent-samples design reveals that they are the same ex-cept that the randomized block design has p = 3 treatmentlevels. When p = 2, the layouts and randomization plans forthe designs are identical. In this and later examples, I assumethat all of the treatment levels and blocks of interest are rep-resented in the experiment. In other words, the treatment lev-els and blocks represent fixed effects.Adiscussion of the casein which either the treatment levels or blocks or both are ran-domly sampled from a population of levels, the mixed and
  • 26. 8 Experimental Designrandom effects cases, is beyond the scope of this chapter. Thereader is referred to Kirk (1995, pp. 256–257, 265–268).A randomized block design enables a researcher to testtwo null hypotheses.H0: ␮·1 = ␮·2 = ␮·3(Treatment population means are equal.)H0: ␮1· = ␮2· = · · · = ␮15·(Block population means are equal.)The second hypothesis, which is usually of little interest,states that the population weight-loss means for the 15 levelsof the nuisance variable are equal. The researcher expects atest of this null hypothesis to be significant. If the nuisancevariable represented by the blocks does not account for an ap-preciable proportion of the total variation in the experiment,little has been gained by isolating the effects of the variable.Before exploring this point, I describe the model equation foran RB-p design.The classical model equation for the weight-loss experi-ment isYi j = ␮ + ␣j + ␲i + ␧i j (i = 1, . . . , n; j = 1, . . . , p),whereYi j is the weight loss for the participant in Blocki andtreatment level aj.␮ is the grand mean of the three weight-loss popula-tion means.␣j is the treatment effect for population j and is equal to␮· j − ␮. It reflects the effect of diet aj.␲i is the block effect for population i and is equal to␮i· − ␮. It reflects the effect of the nuisance variablein Blocki.␧i j is the residual error effect associated with Yi j and isequal to Yi j − ␮ − ␣j − ␲i . It reflects all effects notattributable to treatment level aj and Blocki.According to the model equation for this randomized blockdesign, each observation is the sum of four parameters:␮,␣j , ␲i , and ␧i j . A residual error effect is that portion of anobservation that remains after the grand mean, treatmenteffect, and block effect have been subtracted from it; thatis, ␧i j = Yi j − ␮ − ␣j − ␲i . The sum of the squared erroreffects for this randomized block design,␧2i j = (Yi j − ␮ − ␣j − ␲i )2,will be smaller than the sum for the completely randomizeddesign,␧2i( j) = (Yi j − ␮ − ␣j )2,if ␲2i is not equal to zero for one or more blocks. This idea isillustrated in Figure 1.5, where the total sum of squares anddegrees of freedom for the two designs are partitioned. The Fstatistic that is used to test the null hypothesis can be thoughtof as a ratio of error and treatment effects,F =f (error effects) + f (treatment effects)f (error effects)where f( ) denotes a function of the effects in parentheses. Itis apparent from an examination of this ratio that the smallerthe sum of the squared error effects, the larger the F statisticand, hence, the greater the probability of rejecting a false nullSSRES(n Ϫ 1)(p Ϫ 1) ϭ 28SSWGp(n Ϫ 1) ϭ 42Figure 1.5 Partition of the total sum of squares (SSTOTAL) and degrees offreedom (np − 1 = 44) for CR-3 and RB-3 designs. The treatment andwithin-groups sums of squares are denoted by, respectively, SSA and SSWG.The block and residual sums of squares are denoted by, respectively, SSBLand SSRES. The shaded rectangles indicate the sums of squares that are usedto compute the error variance for each design: MSWG = SSWG/p(n − 1)and MSRES = SSRES/(n − 1)(p − 1). If the nuisance variable (SSBL) in therandomized block design accounts for an appreciable portion of the total sumof squares, the design will have a smaller error variance and, hence, greaterpower than the completely randomized design.Figure 1.4 Layout for a randomized block design (RB-3 design). Eachblock contains three girls who are overweight by about the same amount.The three girls in a block are randomly assigned to the treatment levels. Themean weight loss in pounds for the girls in treatment levels a1, a2, and a3 isdenoted by Y·1, Y·2, and Y·3, respectively. The mean weight loss for thegirls in Block1, Block2, . . . , Block15 is denoted by Y1·, Y2·, . . . , Y15·,respectively.
  • 27. Three Building Block Designs 9hypothesis. Thus, by isolating a nuisance variable that ac-counts for an appreciable portion of the total variation in arandomized block design, a researcher is rewarded with amore powerful test of a false null hypothesis.As we have seen, blocking with respect to the nuisancevariable (the amount by which the girls are overweight)enables the researcher to isolate this variable and remove itfrom the error effects. But what if the nuisance variabledoesn’t account for any of the variation in the experiment? Inother words, what if all of the block effects in the experimentare equal to zero? In this unlikely case, the sum of the squarederror effects for the randomized block and completely ran-domized designs will be equal. In this case, the randomizedblock design will be less powerful than the completely ran-domized design because its error variance, the denominatorof the F statistic, has n − 1 fewer degrees of freedom thanthe error variance for the completely randomized design. Itshould be obvious that the nuisance variable should be se-lected with care. The larger the correlation between the nui-sance variable and the dependent variable, the more likely itis that the block effects will account for an appreciableproportion of the total variation in the experiment.Latin Square DesignThe Latin square design described in this section derives itsname from an ancient puzzle that was concerned with thenumber of different ways that Latin letters can be arranged ina square matrix so that each letter appears once in each rowand once in each column. An example of a 3 × 3 Latin squareis shown in Figure 1.6. In this figure I have used the letter awith subscripts in place of Latin letters. The Latin square de-sign is denoted by the letters LS-p, where LS stands for“Latin square” and p is the number of levels of the treatment.A Latin square design enables a researcher to isolate the ef-fects of not one but two nuisance variables. The levels of onenuisance variable are assigned to the rows of the square; thelevels of the other nuisance variable are assigned to thecolumns. The levels of the treatment are assigned to the cellsof the square.Let’s return to the weight-loss experiment. With a Latinsquare design the researcher can isolate the effects of theamount by which girls are overweight and the effects of a sec-ond nuisance variable, for example, genetic predisposition tobe overweight. A rough measure of the second nuisance vari-able can be obtained by asking a girl’s parents whether theywere overweight as teenagers: c1 denotes neither parent over-weight, c2 denotes one parent overweight, and c3 denotes bothparents overweight. This nuisance variable can be assigned tothe columns of the Latin square. Three levels of the amount bywhich girls are overweight can be assigned to the rows of theLatin square: b1 is less than 15 pounds, b2 is 15 to 25 pounds,and b3 is more than 25 pounds. The advantage of being able toisolate two nuisance variables comes at a price. The ran-domization procedures for a Latin square design are morecomplex than those for a randomized block design. Also, thenumber of rows and columns of a Latin square must eachequal the number of treatment levels, which is three in the ex-ample. This requirement can be very restrictive. For example,it was necessary to restrict the continuous variable of theamount by which girls are overweight to only three levels.The layout of the LS-3 design is shown in Figure 1.7.Figure 1.6 Three-by-three Latin square, where aj denotes one of thej = 1, . . . , p levels of treatment A; bk denotes one of the k = 1, . . . , p levelsof nuisance variable B; and cl denotes one of the l = 1, . . . , p levels of nui-sance variable C. Each level of treatment A appears once in each row andonce in each column as required for a Latin square.Figure 1.7 Layout for a Latin square design (LS-3 design) that is based onthe Latin square in Figure 1.6. Treatment A represents three kinds of diets;nuisance variable B represents amount by which the girls are overweight;and nuisance variable C represents genetic predisposition to be overweight.The girls in Group1, for example, received diet a1, were less than fifteenpounds overweight (b1), and neither parent had been overweight as ateenager (c1). The mean weight loss in pounds for the girls in the nine groupsis denoted by Y·111, Y·123, . . . , Y·331.
  • 28. 10 Experimental DesignThe design in Figure 1.7 enables the researcher to testthree null hypotheses:H0: ␮1·· = ␮2·· = ␮3··(Treatment population means are equal.)H0: ␮·1· = ␮·2· = ␮·3·(Row population means are equal.)H0: ␮··1 = ␮··2 = ␮··3(Column population means are equal.)The first hypothesis states that the population means for thethree diets are equal. The second and third hypotheses makesimilar assertions about the population means for the twonuisance variables. Tests of these nuisance variables are ex-pected to be significant. As discussed earlier, if the nuisancevariables do not account for an appreciable proportion of thetotal variation in the experiment, little has been gained by iso-lating the effects of the variables.The classical model equation for this version of theweight-loss experiment isYi jkl = ␮ + ␣j + ␤k + ␥l + ␧jkl + ␧i( jkl)(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , p;l = 1, . . . , p),whereYi jklis the weight loss for the ith participant in treat-ment level aj, row bk, and column cl.␣j is the treatment effect for population j and is equalto ␮j·· − ␮. It reflects the effect of diet aj.␤k is the row effect for population k and is equalto ␮·k· − ␮. It reflects the effect of nuisance vari-able bk.␥l is the column effect for population l and is equalto ␮··l − ␮. It reflects the effects of nuisance vari-able cl.␧jkl is the residual effect that is equal to ␮jkl − ␮j··−␮·k· − ␮··l + 2␮.␧i( jkl) is the within-cell error effect associated with Yijkland is equal to Yi jkl − ␮ − ␣j − ␤k − ␥l − ␧jkl.According to the model equation for this Latin square design,each observation is the sum of six parameters: ␮, ␣j , ␤k,␥l, ␧jkl, and ␧i( jkl). The sum of the squared within-cell erroreffects for the Latin square design,␧2i( jkl) = (Yi jkl − ␮ − ␣j −␤k − ␥l − ␧jkl)2,will be smaller than the sum for the randomized block design,␧2i j = (Yi j − ␮ − ␣j − ␲i )2,if the combined effects of ␤2k, ␥2l , and ␧2jkl aregreater than ␲2i . The benefits of isolating two nuisancevariables are a smaller error variance and increased power.Thus far I have described three of the simplest experimen-tal designs: the completely randomized design, randomizedblock design, and Latin square design. The three designs arecalled building block designs because complex experimentaldesigns can be constructed by combining two or more of thesesimple designs (Kirk, 1995, p. 40). Furthermore, the random-ization procedures, data analysis, and model assumptions forcomplex designs represent extensions of those for the threebuilding block designs. The three designs provide the organi-zational structure for the design nomenclature and classifica-tion scheme that is described next.CLASSIFICATION OF EXPERIMENTAL DESIGNSA classification scheme for experimental designs is given inTable 1.1. The designs in the category systematic designs donot use random assignment of participants or experimentalunits and are of historical interest only. According to Leonardand Clark (1939), agricultural field research employing sys-tematic designs on a practical scale dates back to 1834. Overthe last 80 years systematic designs have fallen into disuse be-cause designs employing random assignment are more likelyto provide valid estimates of treatment and error effects andcan be analyzed using the powerful tools of statistical infer-ence such as analysis of variance. Experimental designs usingrandom assignment are called randomized designs. The ran-domized designs in Table 1.1 are subdivided into categoriesbased on (a) the number of treatments, (b) whether participantsare assigned to relatively homogeneous blocks prior to randomassignment, (c) presence or absence of confounding, (d) use ofcrossed or nested treatments, and (e) use of a covariate.The letters p and q in the abbreviated designations denotethe number of levels of treatments A and B, respectively. If adesign includes a third and fourth treatment, say treatments Cand D, the number of their levels is denoted by r and t,respectively. In general, the designation for designs with twoor more treatments includes the letters CR, RB, or LS toindicate the building block design. The letter F or H is addedto the designation to indicate that the design is, respectively, afactorial design or a hierarchical design. For example, the F inthe designation CRF-pq indicates that it is a factorial design;the CR and pq indicate that the design was constructed bycombining two completely randomized designs with p and qtreatment levels. The letters CF, PF, FF, and AC are added tothe designation if the design is, respectively, a confoundedfactorial design, partially confounded factorial design, frac-tional factorial design, or analysis of covariance design.
  • 29. Factorial Designs 11TABLE 1.1 Classification of Experimental DesignsAbbreviatedExperimental Design DesignationaI. Systematic Designs (selected examples).1. Beavan’s chessboard design.2. Beavan’s half-drill strip design.3. Diagonal square design.4. Knut Vik square design.II. Randomized Designs With One Treatment.A. Experimental units randomly assigned totreatment levels.1. Completely randomized design. CR-pB. Experimental units assigned to relativelyhomogeneous blocks or groups prior torandom assignment.1. Balanced incomplete block design. BIB-p2. Cross-over design. CO-p3. Generalized randomized block design. GRB-p4. Graeco-Latin square design. GLS-p5. Hyper-Graeco-Latin square design. HGLS-p6. Latin square design. LS-p7. Lattice balanced incomplete block design. LBIB-p8. Lattice partially balanced incomplete LPBIB-pblock design.9. Lattice unbalanced incomplete block design. LUBIB-p10. Partially balanced incomplete block design. PBIB-p11. Randomized block design. RB-p12. Youden square design. YBIB-pIII. Randomized Designs With Two or More Treatments.A. Factorial designs: designs in which all treatmentsare crossed.1. Designs without confounding.a. Completely randomized factorial design. CRF-pqb. Generalized randomized block factorial design. GRBF-pqc. Randomized block factorial design. RBF-pq2. Design with group-treatment confounding.a. Split-plot factorial design. SPF-p·q3. Designs with group-interaction confounding.a. Latin square confounded factorial design. LSCF-pkaThe abbreviated designations are discussed later.AbbreviatedExperimental Design Designationab. Randomized block completely confounded RBCF-pkfactorial design.c. Randomized block partially confounded RBPF-pkfactorial design.4. Designs with treatment-interaction confounding.a. Completely randomized fractional CRFF-pk−ifactorial design.b. Graeco-Latin square fractional factorial design. GLSFF-pkc. Latin square fractional factorial design. LSFF-pkd. Randomized block fractional factorial design. RBFF-pk−iB. Hierarchical designs: designs in which one ormore treatments are nested.1. Designs with complete nesting.a. Completely randomized hierarchical design. CRH-pq(A)b. Randomized block hierarchical design. RBH-pq(A)2. Designs with partial nesting.a. Completely randomized partial CRPH-pq(A)rhierarchical design.b. Randomized block partial hierarchical design. RBPH-pq(A)rc. Split-plot partial hierarchical design. SPH-p·qr(B)IV. Randomized Designs With One or More Covariates.A. Designs that include a covariate havethe letters AC added to the abbreviateddesignation as in the following examples.1. Completely randomized analysis of covariance CRAC-pdesign.2. Completely randomized factorial analysis CRFAC-pqof covariance design.3. Latin square analysis of covariance design. LSAC-p4. Randomized block analysis of covariance design. RBAC-p5. Split-plot factorial analysis of covariance design. SPFAC-p·qV. Miscellaneous Designs (select examples).1. Solomon four-group design.2. Interrupted time-series design.choose. Because of the wide variety of designs available, it isimportant to identify them clearly in research reports. Oneoften sees statements such as “a two-treatment factorial de-sign was used.” It should be evident that a more precisedescription is required. This description could refer to 10 ofthe 11 factorial designs in Table 1.1.Thus far, the discussion has been limited to designs withone treatment and one or two nuisance variables. In the fol-lowing sections I describe designs with two or more treat-ments that are constructed by combining several buildingblock designs.FACTORIAL DESIGNSCompletely Randomized Factorial DesignFactorial designs differ from those described previously inthat two or more treatments can be evaluated simultaneouslyThree of these designs are described later. Because of spacelimitations, I cannot describe all of the designs in Table 1.1.I will focus on those designs that are potentially the mostuseful in the behavioral and social sciences.It is apparent from Table 1.1 that a wide array of designsis available to researchers. Unfortunately, there is no univer-sally accepted designation for the various designs—somedesigns have as many as five different names. For example,the completely randomized design has been called a one-wayclassification design, single-factor design, randomized groupdesign, simple randomized design, and single variable exper-iment. Also, a variety of design classification schemes havebeen proposed. The classification scheme in Table 1.1 owesmuch to Cochran and Cox (1957, chaps. 4–13) and Federer(1955, pp. 11–12).A quick perusal of Table 1.1 reveals why researcherssometimes have difficulty selecting an appropriate experi-mental design—there are a lot of designs from which to
  • 30. 12 Experimental DesignFigure 1.8 Layout for a two-treatment completely randomized factorialdesign (CRF-32 design). Thirty girls are randomly assigned to six combina-tions of treatments A and B with the restriction that five girls are assigned toeach combination. The mean weight loss in pounds for girls in the six groupsis denoted by Y·11, Y·12, . . . , Y·32.in an experiment. The simplest factorial design from thestandpoint of randomization, data analysis, and model as-sumptions is based on a completely randomized design and,hence, is called a completely randomized factorial design. Atwo-treatment completely randomized factorial design is de-noted by the letters CRF-pq, where p and q denote the num-ber of levels, respectively, of treatments A and B.In the weight-loss experiment, a researcher might be inter-ested in knowing also whether walking on a treadmill for20 minutes a day would contribute to losing weight, as wellas whether the difference between the effects of walking ornot walking on the treadmill would be the same for each ofthe three diets. To answer these additional questions, a re-searcher can use a two-treatment completely randomized fac-torial design. Let treatment A consist of the three diets (a1, a2,and a3) and treatment B consist of no exercise on the tread-mill (b1) and exercise for 20 minutes a day on the treadmill(b2). This design is a CRF-32 design, where 3 is the numberof levels of treatment A and 2 is the number of levels of treat-ment B. The layout for the design is obtained by combiningthe treatment levels of a CR-3 design with those of a CR-2design so that each treatment level of the CR-3 design ap-pears once with each level of the CR-2 design and vice versa.The resulting design has 3 × 2 = 6 treatment combinationsas follows: a1b1, a1b2, a2b1, a2b2, a3b1, a3b2. When treatmentlevels are combined in this way, the treatments are said to becrossed. The use of crossed treatments is a characteristic ofall factorial designs. The layout of the design with 30 girlsrandomly assigned to the six treatment combinations isshown in Figure 1.8.The classical model equation for the weight-loss experi-ment isYi jk = ␮ + ␣j + ␤k + (␣␤)jk + ␧i( jk)(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , q),whereYi jk is the weight loss for participant i in treatmentcombination ajbk.␮ is the grand mean of the six weight-loss popula-tion means.␣j is the treatment effect for population aj and isequal to ␮j· − ␮. It reflects the effect of diet aj.␤k is the treatment effect for population bk and isequal to ␮·k − ␮. It reflects the effects of exercisecondition bk.(␣␤)jk is the interaction effect for populations aj and bkand is equal to ␮jk − ␮j· − ␮·k − ␮. Interactioneffects are discussed later.␧i( jk) is the within-cell error effect associated with Yi jkand is equal to Yi jk − ␮ − ␣j − ␤k − (␣␤)jk. Itreflects all effects not attributable to treatmentlevel aj, treatment level bk, and the interaction of ajand bk.The CRF-32 design enables a researcher to test three nullhypotheses:H0: ␮1· = ␮2· = ␮3·(Treatment A population means are equal.)H0: ␮·1 = ␮·2(Treatment B population means are equal.)H0: ␮jk − ␮jk − ␮j k + ␮j k = 0 for all j and k(All A × B interaction effects equal zero.)The last hypothesis is unique to factorial designs. It states thatthe joint effects (interaction) of treatments A and B are equalto zero for all combinations of the two treatments. Two treat-ments are said to interact if any difference in the dependentvariable for one treatment is different at two or more levels ofthe other treatment.Thirtygirlsareavailabletoparticipateintheweight-lossex-periment and have been randomly assigned to the six treatmentcombinations with the restriction that five girls are assigned to
  • 31. Factorial Designs 13TABLE 1.2 Weight-Loss Data for the Diet (aj) and ExerciseConditions (bk)a1b1 a1b2 a2b1 a2b2 a3b1 a3b27 7 9 10 15 1313 14 4 5 10 169 11 7 7 12 205 4 14 15 5 191 9 11 13 8 12TABLE 1.3 Descriptive Summary of the Weight-Loss Data: Means(Y) and Standard Deviations (S)MeanStandardDiet a1 Diet a2 Diet a3 DeviationNo treadmill Y·11 = 7.0 Y·21 = 9.0 Y·31 = 10.0 Y··1 = 8.7exercise (b1) S·11 = 4.0 S·21 = 3.4 S·31 = 3.4 S··1 = 3.8Treadmill Y·12 = 9.0 Y·22 = 10.0 Y·32 = 16.0 Y··2 = 11.7exercise (b2) S·12 = 3.4 S·22 = 3.7 S·32 = 3.2 S··2 = 4.6Y·1· = 8.0 Y·2· = 9.5 Y·3· = 13.0S·1· = 3.8 S·2· = 3.6 S·3· = 4.4TABLE 1.4 Analysis of Variance for the Weight-Loss DataSource SS df MS F pTreatment A (Diet) 131.6667 2 65.8334 4.25 .026Treatment B (Exercise) 67.5000 1 67.5000 4.35 .048A × B 35.0000 2 17.5000 1.13 .340Within cell 372.0000 24 15.5000Total 606.1667 29each combination.The data, weight loss for each girl, are givenin Table 1.2. A descriptive summary of the data—samplemeans and standard deviations—is given in Table 1.3.An examination of Table 1.3 suggests that diet a3 resultedin more weight loss than did the other diets and 20 minutes aday on the treadmill was beneficial. The analysis of variancefor the weight-loss data is summarized in Table 1.4, whichshows that the null hypotheses for treatments A and B can berejected. We know that at least one contrast or differenceamong the diet population means is not equal to zero. Also,from Tables 1.3 and 1.4 we know that 20 minutes a day onthe treadmill resulted in greater weight loss than did theno-exercise condition. The A × B interaction test is notsignificant. When two treatments interact, a graph in whichtreatment-combination population means are connected bylines will always reveal at least two nonparallel lines for oneor more segments of the lines. The nonsignificant interac-tion test in Table 1.4 tells us that there is no reason for be-lieving that the population difference in weight loss betweenthe treadmill and no-treadmill conditions is different for thethree diets. If the interaction had been significant, our interestwould have shifted from interpreting the tests of treatments Aand B to understanding the nature of the interaction. Proce-dures for interpreting interactions are described by Kirk(1995, pp. 370–372, 377–389).Statistical Significance Versus Practical SignificanceThe rejection of the null hypotheses for the diet and exercisetreatments is not very informative. We know in advance thatthe hypotheses are false. As John Tukey (1991) wrote, “theeffects of A and B are always different—in some decimalplace—for any A and B. Thus asking ‘Are the effects differ-ent?’ is foolish” (p. 100). Furthermore, rejection of a nullhypothesis tells us nothing about the size of the treatmenteffects or whether they are important or large enough to beuseful—that is, their practical significance. In spite of numer-ous criticisms of null hypothesis significance testing, re-searchers continue to focus on null hypotheses and p values.The focus should be on the data and on what the data tell theresearcher about the scientific hypothesis. This is not a newidea. It was originally touched on by Karl Pearson in 1901and more explicitly by Fisher in 1925. Fisher (1925) pro-posed that researchers supplement null hypothesis signifi-cance tests with measures of strength of association. Sincethen over 40 supplementary measures of effect magnitudehave been proposed (Kirk, 1996). The majority of the mea-sures fall into one of two categories: measures of strength ofassociation and measures of effect size (typically, standard-ized mean differences). Hays (1963) introduced a measureof strength of association that can assist a researcher in as-sessing the importance or usefulness of a treatment: omegasquared, ˆ␻2. Omega squared estimates the proportion of thepopulation variance in the dependent variable accountedfor by a treatment. For experiments with several treatments,as in the weight-loss experiment, partial omega squared iscomputed. For example, the proportion of variance in thedependent variable, Y, accounted for by treatment A eliminat-ing treatment B and the A × B interaction is denoted byˆ␻2Y|A·B, AB. Similarly, ˆ␻2Y|B·A, AB denotes the proportion of thevariance accounted for by treatment B eliminating treatmentA and the A × B interaction. For the weight-loss experiment,the partial omega squareds for treatments A and B are,respectively,ˆ␻2Y|A·B, AB =(p − 1)(FA − 1)(p − 1)(FA − 1) + npq=(3 − 1)(4.247 − 1)(3 − 1)(4.247 − 1) + (5)(3)(2)= 0.18
  • 32. 14 Experimental Designˆ␻2Y|B·A, AB =(q − 1)(FB − 1)(q − 1)(FB − 1) + npq=(2 − 1)(4.376 − 1)(2 − 1)(4.376 − 1) + (5)(3)(2)= 0.10.Following Cohen’s (1988, pp. 284–288) guidelines for inter-preting omega squared,.010 is a small association.059 is a medium association.138 is a large association,we conclude that the diets accounted for a large proportionof the population variance in weight loss. This is consistentwith our perception of the differences between the weight-loss means for the three diets: girls on diet a3 lost five morepounds than did those on a1. Certainly, any girl who is anx-ious to lose weight would want to be on diet a3. Likewise, themedium association between the exercise conditions andweight loss is practically significant: Walking on the tread-mill resulted in a mean weight loss of 3 pounds. Based onTukey’s HSD statistic, 95% confidence intervals for the threepairwise contrasts among the diet means are−5.9 < ␮·1 − ␮·2 < 2.9−9.4 < ␮·1 − ␮·3 < −0.6−7.9 < ␮·2 − ␮·3 < 0.9.Because the confidence interval for ␮·1 − ␮·3 does not con-tain 0, we can be confident that diet a3 is superior to diet a1.Hedges’s (1981) effect size for the difference between dietsa1 and a3 isg =Y··1 − Y··2ˆσPooled=|8.0 − 13.0|3.937= 1.27,a large effect.Unfortunately, there is no statistic that measures practicalsignificance. The determination of whether results are impor-tant or useful must be made by the researcher. However, con-fidence intervals and measures of effect magnitude can helpthe researcher make this decision. If our discipline is toprogress as it should, researchers must look beyond signifi-cance tests and p values and focus on what their data tell themabout the phenomenon under investigation. For a fullerdiscussion of this point, see Kirk (2001).Alternative ModelsThus far, I have described the classical model equation forseveral experimental designs. This model and associatedprocedures for computing sums of squares assume that allcell ns in multitreatment experiments are equal. If the cell nsare not equal, some researchers use one of the following pro-cedures to obtain approximate tests of null hypotheses: (a) es-timate the missing observations under the assumption that thetreatments do not interact, (b) randomly set aside data to re-duce all cell ns to the same size, and (c) use an unweighted-means analysis. The latter approach consists of performingan ANOVA on the cell means and then multiplying the sumsof squares by the harmonic mean of the cell ns. None of theseprocedures is entirely satisfactory. Fortunately, exact solu-tions to the unequal cell n problem exist. Two solutions thatare described next are based on a regression model and a cellmeans model. Unlike the classical model approach, theregression and cell means model approaches require a com-puter and software for manipulating matrices.Suppose that halfway through the weight-loss experimentthe third participant in treatment combination a2b2 (Y322 = 7)moved to another area of the country and dropped out of theexperiment. The loss of this participant resulted in unequalcell ns. Cell a2b2 has four participants; the other cells have fiveparticipants. The analysis of the weight-loss data using theregression model is described next.Regression ModelA qualitative regression model equation with h − 1 =(p − 1) + (q − 1) + (p − 1)(q − 1) = 5 independent vari-ables (Xi1, Xi2,. . ., Xi2 Xi3) and h = 6 parameters (␤0, ␤1, . . . ,␤5),Yi = ␤0 +A effects␤1 Xi1 + ␤2 Xi2 +B effects␤3 Xi3 +A×B effects␤4 Xi1 Xi3 + ␤5 Xi2 Xi3 + ei ,can be formulated so that tests of selected parameters of theregression model provide tests of null hypotheses for A, B,and A × B in the weight-loss experiment. Tests of the fol-lowing null hypotheses for this regression model are of par-ticular interest:H0: ␤1 = ␤2 = 0H0: ␤3 = 0H0: ␤4 = ␤5 = 0In order for tests of these null hypotheses to provide testsof ANOVA null hypotheses, it is necessary to establish acorrespondence between the five independent variables ofthe regression model equation and (p − 1) + (q − 1) +(p − 1)(q − 1) = 5 treatment and interaction effects of theCRF-32 design. One way to establish this correspondence isto code the independent variables of the regression model as
  • 33. Factorial Designs 15TABLE 1.5 Data Vector, y, and X Matrix for the Regression Modely29×1X29×6A B A×Bx0 x1 x2 x3 x1x3 x2x37 1 1 0 1 1 0a1b1.....................1 1 1 0 1 1 07 1 1 0 −1 −1 0a1b2.....................9 1 1 0 −1 −1 09 1 0 1 1 0 1a2b1.....................11 1 0 1 1 0 110 1 0 1 −1 0 −1a2b2.....................13 1 0 1 −1 0 −115 1 −1 −1 1 −1 −1a3b1.....................8 1 −1 −1 1 −1 −113 1 −1 −1 −1 1 1a3b2.....................12 1 −1 −1 −1 1 1ΆΆΆΆΆΆΆΆΆfollows:Xi1 =1, if an observation is in a1−1, if an observation is in a30, otherwiseXi2 =1, if an observation is in a2−1, if an observation is in a30, otherwiseXi3 =1, if an observation is in b1−1, if an observation is in b2Xi1 Xi3 =product of coded valuesassociated with a1 and b1Xi2 Xi3 =product of coded valuesassociated with a2 and b1This coding scheme, which is called effect coding, producedthe X matrix in Table 1.5. The y vector in Table 1.5 containsweight-loss observations for the six treatment combinations.The first column vector, x0, in the X matrix contains ones; thesecond through the sixth column vectors contain codedvalues for Xi1, Xi2, . . . , Xi2Xi3. To save space, only a portionof the 29 rows of X and y are shown. As mentioned earlier,observation Y322 is missing. Hence, each of the treatmentcombinations contains five observations except for a2b2,which contains four.F statistics for testing hypotheses for selected regressionparameters are obtained by dividing a regression mean square,MSR, by an error mean square, MSE, where MSR = SSR/dfregand MSE = SSE/dferror. The regression sum of squares, SSR,that reflects the contribution of independent variables X1 andX2 over and above the contribution of X3, X1X3, and X2X3 isgiven by the difference between two error sums of squares,SSE, as follows:SSR(AX1 X2 |BX3A×BX1 X3 X2 X3)= SSE(BX3A×BX1 X3 X2 X3) – SSE(AX1 X2BX3A×BX1 X3 X2 X3)An error sum of squares is given bySSE( ) = y y − [(Xi Xi )−1(Xi y)] (Xi y),where the Xi matrix contains the first column, x0, of X andthe columns corresponding the independent variables con-tained in SSE( ). For example, the X matrix used in comput-ing SSE(X3 X1X3 X2X3) contains four columns: x0, x3, x1x3,and x2x3. The regression sum of squares corresponding toSSA in ANOVA isSSR(AX1 X2 |BX3A×BX1 X3 X2 X3)= SSE(BX3A×BX1 X3 X2 X3) – SSE(AX1 X2BX3A×BX1 X3 X2 X3)= 488.1538 − 360.7500 = 127.4038with p − 1 = 2 degrees of freedom. This sum of squaresis used in testing the regression null hypothesis H0: ␤1 =␤2 = 0. Because of the correspondence between the regres-sion and ANOVA parameters, a test of this regression nullhypothesis is equivalent to testing the ANOVA null hypothe-sis for treatment A.The regression sum of squares corresponding to SSB inANOVA isSSR(BX3 |AX1 X2A×BX1 X3 X2 X3)= SSE(AX1 X2A×BX1 X3 X2 X3) – SSE(AX1 X2BX3A×BX1 X3 X2 X3)= 436.8000 − 360.7500 = 76.0500with q − 1 = 1 degree of freedom.
  • 34. 16 Experimental DesignThe regression sum of squares corresponding to SSA × Bin ANOVA isSSR(A×BX1 X3 X2 X3 |AX1 X2BX3)= SSE(AX1 X2BX3) − SSE(AX1 X2BX3A×BX1 X3 X2 X3)= 388.5385 − 360.7500 = 27.7885with (p − 1)(q − 1) = 2 degrees of freedom.The regression error sum of squares corresponding toSSWCELL in ANOVA isSSE(AX1 X2BX3A×BX1 X3 X2 X3) = 360.7500with N − h = 29 − 6 = 23 degrees of freedom.The total sum of squares isSSTO = y y − y JyN−1= 595.7931,where J is a 29 × 29 matrix of ones and N = 29, the numberof weight-loss observations. The total sum of squares hasN − 1 = 28 degrees of freedom. The analysis of the weight-loss data is summarized in Table 1.6. The null hypotheses␤1 = ␤2 = 0 and ␤3 = 0 can be rejected. Hence, indepen-dent variables X1 or X2 as well as X3 contribute to predictingthe dependent variable. As we see in the next section, theF statistics in Table 1.6 are identical to the ANOVA F statis-tics for the cell means model.Cell Means ModelThe classical model equation for a CRF-pq design,Yi jk = ␮ + ␣j + ␤k + (␣␤)jk + ␧i( jk)(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , q),focuses on the grand mean, treatment effects, and interactioneffects.The cell means model equation for the CRF-pq design,Yi jk = ␮jk + ␧i( jk)(i = 1, . . . , n; j = 1, . . . , p; k = 1, . . . , q),focuses on cell means, where ␮jk denotes the mean in cell ajand bk. Although I described the classical model first, this isnot the order in which the models evolved historically.According to Urquhart, Weeks, and Henderson (1973),Fisher’s early development of ANOVA was conceptualizedby his colleagues in terms of cell means. It was not until laterthat cell means were given a linear structure in terms of thegrand mean and model effects, that is, ␮jk = ␮ + ␣j + ␤k +(␣␤)jk . The classical model equation for a CRF-pq designuses four parameters, ␮ + ␣j + ␤k + (␣␤)jk, to representone parameter, ␮jk. Because of this structure, the classicalmodel is overparameterized. For example, the expectation ofthe classical model equation for the weight-loss experimentcontains 12 parameters: ␮, ␣1, ␣2, ␣3, ␤1, ␤2, (␣␤)11, (␣␤)12,(␣␤)21, (␣␤)22, (␣␤)31, (␣␤)32. However, there are only sixcells means from which to estimate the 12 parameters. Whenthere are missing cells in multitreatment designs, a researcheris faced with the question of which parameters or parametricfunctions are estimable. For a discussion of this and otherproblems, see Hocking (1985), Hocking and Speed (1975),Searle (1987), and Timm (1975).The cell means model avoids the problems associated withoverparameterization. A population mean can be estimatedfor each cell that contains one or more observations. Thus,the model is fully parameterized. Unlike the classical model,the cell means model does not impose a structure on theanalysis of data. Consequently, the model can be used to testhypotheses about any linear combination of population cellmeans. It is up to the researcher to decide which tests aremeaningful or useful based on the original research hypothe-ses, the way the experiment was conducted, and the data thatare available.I will use the weight-loss data in Table 1.2 to illustrate thecomputational procedures for the cell means model. Again,TABLE 1.6 Analysis of Variance for the Weight-Loss Data (Observation Y322 is missing)Source SS df MS F pX1 X2 | X3 X1X3 X2X3 127.4038 p − 1 = 2 63.7019 4.06 .031X3 | X1 X2 X1X3 X2X3 76.0500 q − 1 = 1 76.0500 4.85 .038X1X3 X2X3 | X1 X2 X3 27.7885 (p − 1)(q − 1) = 2 13.8943 0.89 .426Error 360.7500 N − h = 23 15.6848Total 595.7931 N − 1 = 28
  • 35. Factorial Designs 17we will assume that observation Y322 is missing. The nullhypothesis for treatment A isH0: ␮1· = ␮2· = ␮3·.An equivalent null hypothesis that is used with the cell meansmodel isH0: ␮1· − ␮2· = 0(1.1)␮2· − ␮3· = 0.In terms of cell means, this hypothesis can be expressed asH0:␮11 + ␮122−␮21 + ␮222= 0(1.2)␮21 + ␮222−␮31 + ␮322= 0,where ␮1· = (␮11 + ␮12)/2, ␮2· = (␮21 + ␮22)/2, and soon. In matrix notation, the null hypothesis isCA(p−1) × h␮h×10(p−1)×1H0:121 1 −1 −1 0 00 0 1 1 −1 −1␮11␮12␮21␮22␮31␮32=00,where p is the number of levels of treatment A and h is thenumber of cell means. In order for the null hypothesisCA␮ = 0 to be testable, the CA matrix must be of full rowrank. This means that each row of CA must be linearlyindependent of every other row. The maximum number ofsuch rows is p − 1, which is why it is necessary to express thenull hypothesis as Equation 1.1 or 1.2.An estimator of the nullhypothesis, CA␮ − 0, is incorporated in the formula for com-puting a sum of squares. For example, the estimator appearsas CA ˆ␮ − 0 in the formula for the treatment A sum of squaresSSA = (CA ˆ␮ − 0) [CA(X X)−1CA]−1(CA ˆ␮ − 0), (1.3)where ˆ␮ is a vector of sample cell means. Equation 1.3 sim-plifies toSSA = (CA ˆ␮) [CA(X X)−1CA]−1(CA ˆ␮)because 0 is a vector of zeros. In the formula, CA is acoefficient matrix that defines the null hypothesis, ˆ␮ =[(X X)−1(X y)] = [Y ·11, Y ·12 · · · Y ·23] , and X is a struc-tural matrix. The structural matrix for the weight-lossexperiment is given in Table 1.7. The structural matrix iscoded as follows:x1 =1, if an observation is in a1b10, otherwisex2 =1, if an observation is in a1b20, otherwisex3 =1, if an observation is in a2b10, otherwise...x6 =1, if an observation is in a3b20, otherwiseFor the weight-loss data, the sum of squares for treatmentA isSSA = (CA ˆ␮) [CA(X X)−1CA]−1(CA ˆ␮) = 127.4038with p − 1 = 2 degrees of freedom.The null hypothesis for treatment B isH0: ␮·1 = ␮·2.TABLE 1.7 Data Vector, y, and X Matrix for the Cell Means Modely29×1X29×6x1 x2 x3 x4 x5 x67 1 0 0 0 0 0a1b1.....................1 1 0 0 0 0 07 0 1 0 0 0 0a1b2.....................9 0 1 0 0 0 09 0 0 1 0 0 0a2b1.....................11 0 0 1 0 0 010 0 0 0 1 0 0a2b2.....................13 0 0 0 1 0 015 0 0 0 0 1 0a3b1.....................8 0 0 0 0 1 013 0 0 0 0 0 1a3b2.....................12 0 0 0 0 0 1ΆΆΆΆΆΆ
  • 36. 18 Experimental DesignFigure 1.9 Two interaction terms of the form␮jk − ␮jk − ␮j k + ␮j k areobtained from the crossed lines by subtracting the two ␮ijs connected by adashed line from the two ␮ijs connected by a solid line.An equivalent null hypothesis that is used with the cell meansmodel isH0: ␮·1 − ␮·2 = 0.In terms of cell means, this hypothesis is expressed asH0:␮11 + ␮21 + ␮313−␮12 + ␮22 + ␮323= 0.In matrix notation, the null hypothesis isCB(q−1)×h␮h×10(q−1)×1H0:13[ 1 −1 1 −1 1 −1 ]␮11␮12␮21␮22␮31␮32= [0],where q is the number of levels of treatment B and h is thenumber of cell means. The sum of squares for treatmentB isSSB = (CB ˆ␮) [CB(X X)−1CB]−1(CB ˆ␮) = 76.0500with q − 1 = 1 degree of freedom.The null hypothesis for the A × B interaction isH0: ␮jk − ␮jk − ␮j k + ␮j k = 0 for all j and k.For the weight-loss data, the interaction null hypothesis isH0: ␮11 − ␮12 − ␮21 + ␮22 = 0␮21 − ␮22 − ␮31 + ␮32 = 0The two rows of the null hypothesis correspond to the twosets of means connected by crossed lines in Figure 1.9. Inmatrix notation, the null hypothesis isCA×B(p−1)(q−1)×h␮h×10(p−1)(q−1)×1H0 :1 −1 −1 1 0 00 0 1 −1 −1 1␮11␮12␮21␮22␮31␮32=00.The sum of squares for the A × B interaction isSSA × B = (CA×B ˆ␮) [CA ×B(X X)−1CA×B]−1(CA×B ˆ␮)= 27.7885with (p − 1)(q − 1) = 2 degrees of freedom.The within-cell sum of squares isSSWCELL = y y − ˆ␮ (X y) = 360.7500,where y is the vector of weight-loss observations:[7 13 9 . . . 12]. The within-cell sum of squares has N − h =29 − 6 = 23 degrees of freedom.The total sum of squares isSSTO = y y − y JyN−1= 595.7931,where J is a 29 × 29 matrix of ones and N = 29, the numberof weight-loss observations. The total sum of squares hasN − 1 = 28 degrees of freedom.The analysis of the weight-loss data is summarized inTable 1.8. The F statistics in Table 1.8 are identical to thosein Table 1.6, where the regression model was used.The cell means model is extremely versatile. It can beused when observations are missing and when entire cellsare missing. It allows a researcher to test hypotheses aboutany linear combination of population cell means. It has animportant advantage over the regression model. With the cellmeans model, there is never any ambiguity about the hypoth-esis that is tested because an estimator of the null hypothesis,C ˆ␮ − 0, appears in the formula for a sum of squares. Lack ofspace prevents a discussion of the many other advantages ofthe model; the reader is referred to Kirk (1995, pp. 289–301,413–431). However, before leaving the subject, the modelwill be used to test a null hypothesis for weighted means.Occasionally, researchers collect data in which the samplesizes are proportional to the population sizes. This might
  • 37. Factorial Designs 19TABLE 1.8 Analysis of Variance for the Weight-Loss Data (Observation Y322 is missing)Source SS df MS F pTreatment A (Diet) 127.4038 p − 1 = 2 63.7019 4.06 .031Treatment B (Exercise) 76.0500 q − 1 = 1 76.0500 4.85 .038A × B 27.7885 (p − 1)(q − 1) = 2 13.8943 0.89 .426Within cell 360.7500 N − h = 23 15.6848Total 595.7931 N − 1 = 28occur, for example, in survey research. When cell ns areunequal, a researcher has a choice between computing un-weighted means or weighted means. Unweighted means aresimple averages of cell means. These are the means that wereused in the previous analyses. Weighted means are weightedaverages of cell means in which the weights are the samplecell sizes, njk. Consider again the weight-loss data in whichobservation Y322 is missing. Unweighted and weighted sam-ple means for treatment level a2 where observation Y322 ismissing are, respectively,ˆ␮2· =ˆ␮21 + ˆ␮22q=9.00 + 10.752= 9.88ˆ␮2· =n21 ˆ␮21 + n22 ˆ␮22nj·=5(9.00) + 4(10.75)9= 9.78;nj. is the number of observations in the jth level of treatmentA. The null hypothesis using weighted cell means for treat-ment A isH0:n11␮11 + n12␮12n1·−n21␮21 + n22␮22n2·= 0n21␮21 + n22␮22n2·−n31␮31 + n32␮32n3·= 0.The coefficient matrix for computing SSA isCA =510510−59−490 00 0 5949− 510− 510,where the entries in CA are ±njk/nj. and zero. The sum ofsquares and mean square for treatment A are, respectively,SSA = (CA ˆ␮) [CA(X X)−1CA]−1(CA ˆ␮) = 128.2375MSA = SSA/(p − 1) = 146.3556/(3 − 1) = 64.1188.The F statistic and p value for treatment A areF =MSAMSWCELL=64.118815.6848= 4.09 p = .030,where MSWCELL is obtained from Table 1.8. The null hy-pothesis is rejected. This is another example of the versatilityof the cell means model. A researcher can test hypothesesabout any linear combination of population cell means.In most research situations, sample sizes are not propor-tional to population sizes. Unless a researcher has a com-pelling reason to weight the sample means proportional to thesample sizes, unweighted means should be used.Randomized Block Factorial DesignNext I describe a factorial design that is constructed from tworandomized block designs. The design is called a randomizedblock factorial design and is denoted by RBF-pq. The RBF-pqdesign is obtained by combining the levels of an RB-p designwith those of an RB-q design so that each level of the RB-pdesign appears once with each level of the RB-q design andvice versa. The design uses the blocking technique describedin connection with an RB-p design to isolate variation attrib-utable to a nuisance variable while simultaneously evaluatingtwo or more treatments and associated interactions.In discussing the weight-loss experiment, I hypothesizedthat ease of losing weight is related to the amount by which agirl is overweight. If the hypothesis is correct, a researchercan improve on the CRF-32 design by isolating this nuisancevariable. Suppose that instead of randomly assigning 30 girlsto the six treatment combinations in the diet experiment, theresearcher formed blocks of six girls such that the girls in ablock are overweight by about the same amount. One way toform the blocks is to rank the girls from the least to the mostoverweight. The six least overweight girls are assigned toblock 1. The next six girls