Carma internet research module   preparing for manuscript submission
Upcoming SlideShare
Loading in...5
×
 

Carma internet research module preparing for manuscript submission

on

  • 651 views

 

Statistics

Views

Total Views
651
Views on SlideShare
650
Embed Views
1

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Carma internet research module   preparing for manuscript submission Carma internet research module preparing for manuscript submission Presentation Transcript

    • Preparing for Manuscript Submission
      CARMA Internet Research Module
      Jeff Stanton
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-2)
      Standard Preparation for MS Submission
      Report analysis of a cross validation sample
      Assess non-response bias; analyze and report
      Do duplicate detection; report
      Do malicious data detection; report
      Read and cite Internet research method reviews
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-3)
      Common Objections that Reviewers or Editors may Mount in R&R and Your Rebuttals
      Lack of access control leads to junk data
      You used a password protected consent form
      You filtered responses using timestamp and IP address
      You detected similarities in (or identical) response patterns:
      Flip data, run correlations, look for high values
      StudyResponse studies ranged from 0% to 6.9% duplicates using this screening method
      Simulations showed that repeats would be needed on more than 20% of cases to substantially disturb means/correlations
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-4)
      Common Objections that Reviewers or Editors may Mount in R&R and Your Rebuttals II
      Your sample is bogus because of coverage errors
      You argue that sample representativeness is a challenge in all research and that purposive sampling is a better goal anyway
      Make the sample fit the question: An Internet survey of migrant workers? Coal miners?
      You show the consistency of results between web and cross-validation samples
      You argue that a typical group of Internet respondents has to be an improvement over pure undergrad samples
      You cite demographic studies of Internet: increasing normalization to the general population over time
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-5)
      Common Objections that Reviewers or Editors may Mount in R&R and Your Rebuttals III
      No one has ever demonstrated the equivalence of the measures you used when administered over the web
      You cite research that factor structures replicate, substantive conclusions replicate, correlations generally replicate within the limits of sampling error, be wary of mean comparisons
      You argue this is a higher standard than many other published studies in which:
      Researchers routinely make up their own items
      Modify items or response options of existing scales
      Trim scale lengths and field abridged versions
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-6)
      Common Objections that Reviewers or Editors may Mount in R&R and Your Rebuttals IV
      All Internet research participants are volunteers by definition and therefore volunteer bias makes your sample unusable
      Belmont report and federal legislation require all research to be conducted on volunteers, so volunteer bias is endemic to the whole social research enterprise
      Volunteer bias can substantially limit projectability of means, but my study doesn’t care about means
      Studies and simulations of the effect of volunteer bias generally show that correlations are reduced in magnitude because of restriction of range effects that volunteer bias causes
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-7)
      Strengthening Research Plans for Web Studies
      A cross-validation sample using traditional RMs is never a bad thing
      Use your web sample only to make tests of correlative structures and self-referential comparisons of means (e.g., within subjects)
      Don’t compare means from web study to means from prior paper and pencil study without formal equating
      Speeded and objective tests need careful testing and cross-validation
      Assess correlations between substantive variables to demographics: If they don’t correlate, then the non-response bias may carry less weight
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-8)
      Useful References I
      Birnbaum, M. H. (1999). Testing critical properties of decision making on the Internet. Psychological Science, 10, 399-407.
      Buchanan, T., & Smith, J. L. (1999). Using the Internet for psychological research: Personality testing on the World Wide Web. British Journal of Psychology, 90, 125-144.
      Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and Word-Wide Web samples on determinants of female attractiveness. Behavior Research Methods, Instruments, and Computers, 29, 264-269.
      Pasveer, K. A., & Ellard, J. H. (1998). The making of a personality inventory: Help from the WWW. Behavior Research Methods, Instruments, and Computers, 30, 309-313.
    • May 18-20, 2006
      Internet Data Collection Methods (Day 2-9)
      Useful References II
      Smith, M. A., & Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment. Behavior Research Methods, Instruments, & Computers, 29, 496-505.
      Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725.
      Yost, P.R. & Homer, L.E. (1998, April). Electronic versus Paper Surveys: Does the Medium Affect the Response? Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology. Dallas, TX.