Survey Design

Survey research methodology contains tried-and-true principles to guide a chain of critical decisions, starting with sample design, question testing, mode options, sample weighting, and quality control. A weak link anywhere along the process can greatly undermine efforts downstream. My work covers the complete protocol of survey research principles, and I have studied methodological challenges underlying every step of survey responding, starting from sample design and weighting, coverage and nonresponse errors, psychometric tool development, cognitive biases in recall and response across multiple modes of data collection, and techniques to assess data quality and veracity.

I have conducted survey research across diverse contexts and languages, spanning the globe from Africa (Burkina Faso, Côte d'Ivoire / Ivory Coast, the Democratic Republic of the Congo, Ethiopia, Kenya, Nigeria, South Africa, Zambia, and Zimbabwe), the Indian subcontinent (Bangladesh, India, and Pakistan), East and Southeast Asia (Japan, the Philippines, Singapore, Vietnam), Europe (Germany, France, Italy, Poland, Spain, Sweden, Turkey, and the UK), and North America - Canada, Costa Rica, Dominican Republic, Mexico and the USA.

Different contexts require different sampling designs and modes of data collection. A good design should be based on a high coverage sampling frame that maximizes representativeness of regional/population parameters, probability-based sampling protocols to ensure known chances of selection, and paradata to support unit nonresponse bias analysis. I have executed in person (CAPI), telephone (CATI), and online surveys on wide ranging topics from commercial market research to sensitive social issues in sexual and reproductive health. Probability-based national samples of households to support valid population estimates, and more innovative approaches to reach key populations with no viable sampling frame. Other target populations I have surveyed include physicians, clinicians, patients, online mobile app users, bank customers, restaurant customers, health insurance beneficiaries, and employees of multi-national companies, universities, and the U.S. Air Force.

I also provide expertise to support the expanding of existing research designs to accommodate a mix of traditional and new data collection modes, and in migrating recurring studies from legacy modes to new technologies — transitions that require real thought and careful testing and calibration to ensure the estimates and trends remain valid and comparable before and after transition.

Below is an overview of my core service areas. Every engagement is scoped to the specific questions you aim to answer. Get in touch to talk through your project!

Sample Design & Weighting

Every projection is only as good as the sample it's built on. This covers probability-based sample design, stratification and clustering strategy, and the post-stratification and propensity weighting schemes needed to make a sample defensibly represent the population it's meant to speak for.

Question Design & Testing

Question wording, order, and response format quietly determine data quality more than almost anything else in a survey. This includes drafting and refining instruments, pretesting items, and running formal experiments on question wording and format to identify which versions produce the most valid and reliable answers.

Nonresponse Bias Analysis

A completed survey's response rate doesn't tell you whether it's representative — who didn't respond matters as much as who did. This means formally assessing whether nonrespondents differ systematically from respondents, and what that implies for the validity of your estimates.

Psychometrics

Measuring latent constructs — attitudes, satisfaction, needs, beliefs — well requires more than writing a plausible question. This covers item analysis, factor analysis, and tests of reliability and validity (internal consistency, concurrent validity, predictive validity) to ensure your instrument is actually measuring what you think it's measuring, and specific recommendations for improvements.

Experiments embedded in surveys

Surveys are an efficient vehicle for genuine experiments — randomized question wording or format tests, discrete choice experiments (DCEs) testing product concepts or features, and other embedded designs that let us draw causal conclusions from the same fielding effort as your descriptive data.

Methods to detect mindless responding

Straight-lining, speeding, and other forms of low-effort responding quietly degrade data quality, especially on longer or panel-based surveys. This includes building in the attention checks, response-time analysis, and pattern-detection methods needed to identify and, where appropriate, exclude low-quality response patterns.

Longitudinal Tracking

Tracking studies carry their own design challenges — maintaining comparability across waves while still allowing the instrument to evolve, managing panel attrition and panel conditioning (where repeated participation itself changes how people respond), and distinguishing genuine trend from measurement artifact.

Small Area Estimation

When you need reliable estimates for geographies or subgroups too small to support a direct sample-based estimate, small area estimation techniques borrow strength from related areas and auxiliary data to produce defensible local-level figures.

Cognitive Interviews

Before a question goes into the field at scale, cognitive interviewing — walking a small number of respondents through how they interpret and answer a question — surfaces ambiguities, misreadings, and unintended interpretations that no amount of expert review will catch on its own.

Multi-lingual Survey Design

Translating a survey is not the same as adapting it. My services cover instrument design for multi-lingual and multi-country fielding, including translation and back-translation protocols, cultural and linguistic adaptation of constructs, and checking for measurement equivalence across language versions so that scores are actually comparable across countries.

Methodology R&D

Beyond above offerings, I maintained a personal interest in dedicated methodological research: comparing competing data collection modes, alternative sample recruitment sources, individual interviews or field teams, alternative question formulations head-to-head, and more.

1) Gridded Area Sampling Frames

Sample designs for probability-based national CAPI surveys require valid sampling frames that provide complete and unbiased coverage of the entire country. Such national frames are often lacking in countries with outdated, incomplete or unavailable census data.

In collaboration with researchers in Africa, we drew on a globally consistent public domain gridded area sampling frame to support multi-stage random cluster sampling with probability proportional to size (PPS). Leveraging a high-resolution micro area dataset with 1 km² granularity, we created a mega sampling frame per country that integrates geospatial and demographic attributes including population density, urbanicity, administrative markers, proximity to key infrastructure such as health facilities, schools, markets, and essential utilities such as water and power plants. Further, precipitation and temperature data is incorporated to enable stratification by climate elements, or identify areas where human settlements have been especially hard hit and possibly displaced by floods or drought.

We demonstrate the validity and limitations of this approach using a few specific case studies in this paper to underscore the transformative potential of geo-spatially integrated sampling frames - with proven benefits for survey sampling, logistics, and quality assurance.

2) Online Survey Panels

The vast majority of online surveys today draw from web panels, and despite how standard that's become, self-selection into a panel remains a real threat to representativeness — panel members can differ systematically in attitudes, needs, and behavior from the population they're meant to represent. My work in this area has included evaluating samples from general consumer panels as well as specialty panels of physicians, IT executives, and patients with specific chronic conditions, and comparing sample quality across data collection modes (e.g., web vs. telephone) and across competing panel providers with different sampling and adjustment approaches.

Two related threats deserve particular attention on any recurring or panel-based study:

• Panel conditioning: repeated participation can itself change how people respond, whether by raising awareness of a topic, hardening attitudes through repeated reflection, or simply making frequent panel members more efficient at finishing quickly to collect their incentive.

• Panel attrition: attrition only threatens data quality when it's non-random. A panel that starts out representative can drift over time if the people most likely to drop out share a particular characteristic — for instance, physicians with heavier patient loads, or panelists dissatisfied with incentive levels.

My evaluations of web panel feasibility are delivered with a balanced view of both strengths and limitations, so you can make informed decisions, with customized weighting schemes built for population-level projections where needed.

3) Global Call Centers

I was asked to evaluate data from the same telephone survey fielded concurrently through a long-standing USA research house and four offshore call centers (located in Dominican Republic, Costa Rica, India, and the Philippines), comparing sample and estimate quality across vendors — covering health status, insurance, vehicle ownership, transportation, spending plans, prior survey participation, demographics, and a set of embedded experiments testing response quality — with corresponding cost ratios to directly inform vendor choice.

4) Longitudinal Tracker Calibration

On recurring tracking studies, my work also extends to structured recommendations for improving survey instrument between waves — e.g, executing a factorial design on a multi-wave survey of IT executives to test the impact of specific sample and question changes, so the questionnaire could be improved while preserving comparability with prior waves.

5) Field Team Monitoring

Any survey design is only as good as the extent to which it is executed correctly. Even though initial training is critical, retraining and feedback to field teams throughout the fieldwork period is even more important in sustaining a high level of data quality. My team always host the data platform, so all survey data is collected and uploaded to us in real time, allowing for us to check on data issues quickly. We have developed an approach of computing metrics for each individual enumerator and supervisor/team that quickly reveal patterns of behavior that require feedback and correction. Besides standard metrics, we routinely add specific critical study variables as well as things to watch out for in local contexts.

Beyond on-the-ground validation (i.e. spot checks), our in-office team monitor incoming data in real time, and audio recordings are accessed systematically for further verification. All validations are completed within 24 hours after the interviewing has taken place, so that team supervisors and enumerators are given prompt and accurate feedback constantly. We have found that when enumerators have a sense that they could be monitored anytime, that awareness is in and of itself a big deterrent to anyone cutting corners.

This approach has been executed by fieldwork agencies in multiple countries across Africa and the Indian subcontinent to improve overall performance of field teams. Because not all field team members have the same work ethic, a fair and consistent reward system was needed to counteract an industry norm of cutting corners, to reward honest work and delegate more work to better workers. I have presented the value of this approach at multiple conferences with my local colleagues, example of a paper here.

Let's Talk About Your Project

Whether you need a single study designed from scratch, a second opinion on an existing instrument or sample plan, or a partner to migrate a long-running tracker onto a new mode, get in touch to discuss your project needs.

Survey Methodology Research Papers

Arshad Aminu Yakasai & LinChiat Chang. 2025. Gridded Sampling Frames for Global Surveys: Methodology, Validation, and Real World Applications. Presented at the Applied Statistics International Conference in Koper/Capodistria, Slovenia.

Yacoob, Shameen, Nomathemba Dhladhla & LinChiat Chang. 2023. Multifaceted Approach to Monitoring CAPI Survey Data Quality: Applications in 6 Countries. Presented at the annual meeting of the World Association for Public Opinion Research (WAPOR). Salzburg, Austria.

Chang, LinChiat, Mpumi Mbethe & Shameen Yacoob. 2019. Cutting Corners: Detecting Gaps between Household Contact Protocol vs. Ingrained Practices in the Field. Paper presented at the annual meeting of the European Survey Research Association (ESRA). Zagreb, Croatia.

Krosnick, Jon A., Neil Malhotra, Cecilia Hyunjung Mo, Eduardo F. Bruera, LinChiat Chang, Josh Pasek, Randall K. Thomas. 2017. Perceptions of health risks of cigarette smoking: A new measure reveals widespread misunderstanding. PLOS ONE 14(2): e0182063. <PDF>

Chang, LinChiat and Chung-Tung Jordan Lin. 2015. Comparing FDA Food Label Experiments Using Samples from Web Panels vs. Mall-Intercepts.Field Methods 27:182-198. <PDF>  

Chang, LinChiat. 2015. Impact of Nonresponse on Survey Estimates of Physical Fitness and Sleep Quality. Paper presented at the 2015 annual meeting of the European Survey Research Association in Reykjavik, Iceland. 

Chang, LinChiat and Karan Shah. 2015. Comparing Response Quality across Multiple Web Sample Sources. Paper presented at the 68th Annual Conference of the World Association for Public Opinion Research in Buenos Aires, Argentina.

Chang, LinChiat. 2014. Estimating Population Health in Selected Geographic Areas: Applying Machine Learning Algorithms on Large-scale Survey Data. Paper presented at the annual meeting of the American Association for Public Opinion Research in Anaheim, California.

Chang, LinChiat and Kavita Jayaraman. 2013. Comparing Outbound vs. Inbound Census-balanced Web Panel Samples. Paper presented at the 2013 annual meeting of the European Survey Research Association in Ljubljana, Slovenia.

Yeager, David, Jon A. Krosnick, LinChiat Chang, Harold S. Javitz, Matthew S. Levendusky, Alberto Simpser and Rui Wang. 2011. Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples. Public Opinion Quarterly 75: 709-747. <PDF>

Chang, LinChiat, Jon Krosnick, Elaine Albertson. 2011. How Accurate Are Survey Measurements of Objective Phenomena? Paper presented at the annual meeting of the American Association for Public Opinion Research. 

Chang, LinChiat and Jon A. Krosnick. 2010. Comparing Oral Interviewing with Self-administered Computerized Questionnaires: An Experiment. Public Opinion Quarterly 74: 154-167. <PDF>

Chang, LinChiat, Lucinda Z. Frost, Susan Chao, and Malcolm Ree. 2010. Instrument Development with Web Surveys and Multiple Imputations.Military Psychology 22: 7-23. <PDF>

Chang, LinChiat and Jeremy Brody. 2010. Comparing Web Panel Samples vs. Non-Panel Samples of Medical Doctors. Paper presented at the 2010 Joint Statistical Meetings. <PDF>

Chang, LinChiat and Jon A. Krosnick. 2009. National Surveys via RDD Telephone Interviewing vs. the Internet: Comparing Sample Representativeness and Response Quality.Public Opinion Quarterly 73: 641-678. <PDF>

Chang, LinChiat, Linda Shea, Eric Wendler, and Lawrence Luskin. 2007. An Experiment Comparing 5-point vs. 10-point Scales. Paper presented at the annual meeting of the American Association for Public Opinion Research. 

Chang, LinChiat, Channing Stave, Corrine O’Brien, Fred Rappard, Jill Glathar, and Joseph Cronin. 2007. Comparing Web Survey Samples of Schizophrenic and Bipolar Patients with Concurrent RDD and In-Person Samples. Paper presented at the 2007 Joint Statistical Meetings.

Chang, LinChiat and Keating Holland. 2007. Evaluating Follow-up Probes to “Don’t Know” Responses in Political Polls. Paper presented at the annual meeting of the American Association for Public Opinion Research.

Chang, LinChiat and Todd Myers. 2006. Evaluating RDD Telephone Surveys Conducted by Offshore Firms. Paper presented at the annual meeting of the American Association for Public Opinion Research.

Chang, LinChiat and Lucinda Z. Frost. 2005. Exploring the Impact of Field Period on Web Survey Results. Paper presented at the annual meeting of the American Psychological Association.

Chang, LinChiat and Jon A. Krosnick. 2004. Assessing the Accuracy of Event Rate Estimates from National Surveys. Paper presented at the annual meeting of the American Association for Public Opinion Research. 

Chang, LinChiat and Jon A. Krosnick. 2003. Measuring the Frequency of Regular Behaviors: Comparing the Typical Week to the Past Week.Sociological Methodology 33: 55-80. <PDF>