Main description:

Insightful observations on common question evaluation methodsand best practices for data collection in survey research

Featuring contributions from leading researchers andacademicians in the field of survey research, QuestionEvaluation Methods: Contributing to the Science of Data Qualitysheds light on question response error and introduces aninterdisciplinary, cross–method approach that is essential foradvancing knowledge about data quality and ensuring the credibilityof conclusions drawn from surveys and censuses. Offering a varietyof expert analyses of question evaluation methods, the bookprovides recommendations and best practices for researchers workingwith data in the health and social sciences.

Based on a workshop held at the National Center for HealthStatistics (NCHS), this book presents and compares various questionevaluation methods that are used in modern–day data collection andanalysis. Each section includes an introduction to a method by aleading authority in the field, followed by responses from otherexperts that outline related strengths, weaknesses, and underlyingassumptions. Topics covered include:

• 
  
  Behavior coding
  



• Cognitive interviewing




• Item response theory




• Latent class analysis




• Split–sample experiments




• Multitrait–multimethod experiments




• Field–based data methods

A concluding discussion identifies common themes across thepresented material and their relevance to the future of surveymethods, data analysis, and the production of Federal statistics.Together, the methods presented in this book offer researchersvarious scientific approaches to evaluating survey quality toensure that the responses to these questions result in reliable,high–quality data.

Question Evaluation Methods is a valuable supplement forcourses on questionnaire design, survey methods, and evaluationmethods at the upper–undergraduate and graduate levels. It alsoserves as a reference for government statisticians, surveymethodologists, and researchers and practitioners who carry outsurvey research in the areas of the social and health sciences.

Back cover:

Insightful observations on common question evaluation methodsand best practices for data collection in survey research

Featuring contributions from leading researchers andacademicians in the field of survey research, QuestionEvaluation Methods: Contributing to the Science of Data Qualitysheds light on question response error and introduces aninterdisciplinary, cross–method approach that is essential foradvancing knowledge about data quality and ensuring the credibilityof conclusions drawn from surveys and censuses. Offering a varietyof expert analyses of question evaluation methods, the bookprovides recommendations and best practices for researchers workingwith data in the health and social sciences.

Based on a workshop held at the National Center for HealthStatistics (NCHS), this book presents and compares various questionevaluation methods that are used in modern–day data collection andanalysis. Each section includes an introduction to a method by aleading authority in the field, followed by responses from otherexperts that outline related strengths, weaknesses, and underlyingassumptions. Topics covered include:

• 
  
  Behavior coding
  



• Cognitive interviewing




• Item response theory




• Latent class analysis




• Split–sample experiments




• Multitrait–multimethod experiments




• Field–based data methods

A concluding discussion identifies common themes across thepresented material and their relevance to the future of surveymethods, data analysis, and the production of Federal statistics.Together, the methods presented in this book offer researchersvarious scientific approaches to evaluating survey quality toensure that the responses to these questions result in reliable,high–quality data.

Question Evaluation Methods is a valuable supplement forcourses on questionnaire design, survey methods, and evaluationmethods at the upper–undergraduate and graduate levels. It alsoserves as a reference for government statisticians, surveymethodologists, and researchers and practitioners who carry outsurvey research in the areas of the social and health sciences.

Contents:

Contributors ix

Preface xi

1 Introduction 1
Jennifer Madans, Kristen Miller, Aaron Maitland, and GordonWillis

I BEHAVIOR CODING 5

2 Coding the Behavior of Interviewers and Respondents toEvaluate Survey Questions 7
Floyd J. Fowler, Jr.

3 Response 1 to Fowler s Chapter: Coding the Behaviorof Interviewers and Respondents to Evaluate Survey Questions23
Nora Cate Schaeffer and Jennifer Dykema

4 Response 2 to Fowler s Chapter: Coding the Behaviorof Interviewers and Respondents to Evaluate Survey Questions41
Alisú Schoua–Glusberg

II COGNITIVE INTERVIEWING 49

5 Cognitive Interviewing 51
Kristen Miller

6 Response 1 to Miller s Chapter: CognitiveInterviewing 77
Gordon Willis

7 Response 2 to Miller s Chapter: CognitiveInterviewing 93
Frederick G. Conrad

III ITEM RESPONSE THEORY 103

8 Applying Item Response Theory for Questionnaire Evaluation105
Bryce B. Reeve

9 Response 1 to Reeve s Chapter: Applying Item ResponseTheory for Questionnaire Evaluation 125
Ron D. Hays

10 Response 2 to Reeve s Chapter: Applying ItemResponse Theory for Questionnaire Evaluation 137
Clyde Tucker, Brian Meekins, Jennifer Edgar, and Paul P.Biemer

IV LATENT CLASS ANALYSIS 151

11 Some Issues in the Application of Latent Class Models forQuestionnaire Design 153
Paul P. Biemer and Marcus Berzofsky

12 Response 1 to Biemer and Berzofsky s Chapter: SomeIssues in the Application of Latent Class Models for QuestionnaireDesign 187
Frauke Kreuter

13 Response 2 to Biemer and Berzofsky s Chapter: SomeIssues in the Application of Latent Class Models for QuestionnaireDesign 199
Janet A. Harkness and Timothy P. Johnson

V SPLIT–SAMPLE EXPERIMENTS 213

14 Experiments for Evaluating Survey Questions 215
Jon A. Krosnick

15 Response 1 to Krosnick s Chapter: Experiments forEvaluating Survey Questions 239
Johnny Blair

16 Response 2 to Krosnick s Chapter: Experiments forEvaluating Survey Questions 253
Theresa DeMaio and Stephanie Willson

VI MULTITRAIT–MULTIMETHOD EXPERIMENTS 263

17 Evaluating the Reliability and Validity of SurveyInterview Data Using the MTMM Approach 265
Duane F. Alwin

18 Response to Alwin s Chapter: Evaluating theReliability and Validity of Survey Interview Data Using the MTMMApproach 295
Peter Ph. Mohler

VII FIELD–BASED DATA METHODS 319

19 Using Field Tests to Evaluate Federal Statistical SurveyQuestionnaires 321
Brian A. Harris–Kojetin and James M. Dahlhamer

Index 345