Pick the wrong survey method and you don’t just waste a budget line — you get data that’s confidently wrong. A phone poll and a web panel asked Pew Research Center’s American Trends Panel the same question about discrimination against gay and lesbian Americans. On the phone, 62% said “a lot.” Online, 48% said the same. Same population, same question, a 14-point gap, entirely because of how the question got asked. That’s not noise. That’s mode effect, and it’s the reason “just send a survey” isn’t a strategy.
There are more ways to collect survey data now than at any point in the field’s history — mail, phone, in-person, web, SMS, QR intercepts, in-app prompts, and a fast-growing set of AI-moderated formats. Each one trades cost against speed against the kind of respondent it attracts, and most guides to this topic stop at a pros-and-cons list without telling you what any of it actually costs or how the sampling math underneath it works. This one doesn’t.
Table of Contents
- What “Survey Method” Actually Means
- Probability vs. Non-Probability Sampling
- Survey Methods Compared
- In-Person and Face-to-Face Interviews
- Telephone Surveys
- Mail Surveys
- Online and Web Surveys
- SMS and Mobile Surveys
- QR Code Intercept Surveys
- Focus Groups
- AI-Assisted Surveys
- Quantitative vs. Qualitative Surveys
- How to Choose a Survey Method
What “Survey Method” Actually Means
People use “survey method” to mean two different things, and mixing them up is where most of the confusion starts.
One is the mode — how you deliver the questions and collect answers. Phone, mail, web, in-person, text message. The other is the sampling design — how you decide who gets asked in the first place. A web survey can be built on a rigorous probability sample or a convenience sample of whoever clicks a link on social media. Same mode, wildly different data quality. Most “types of survey methods” listicles conflate the two, which is how you end up with a comparison of “online surveys” against “random sampling” as if they’re the same category of thing.
Get the distinction straight and the rest of this gets easier: mode is about logistics and cost, sampling is about whether your results mean anything beyond the people who happened to answer.
Probability vs. Non-Probability Sampling
This is the part vendor guides skip, and it’s the part that determines whether your results generalize to anyone beyond your respondent list.
Probability sampling means every person in your target population has a known, nonzero chance of being selected, and you pick them using a random mechanism — not “whoever we could reach.” Simple random sampling, stratified sampling (splitting the population into subgroups and sampling within each), systematic sampling (every nth person on a list), and cluster sampling all fall under this umbrella. It’s the standard AAPOR treats as the historical gold standard for research meant to represent a broader population — election polling, public health surveillance, market sizing.
Non-probability sampling covers everything else: convenience sampling (survey your customer email list), quota sampling (fill fixed demographic buckets), snowball sampling (ask respondents to refer others), and purposive sampling (hand-pick people who fit a profile). It’s faster and cheaper, and for a lot of business questions — “do our beta users like this feature” — it’s entirely adequate. What it can’t do is support a statistical claim like “68% of adults nationally believe X,” because there’s no mathematical basis for projecting from a self-selected sample to the general population.
The catch: probability sampling has gotten expensive as landline penetration collapsed and response rates fell across the board, which is exactly why non-probability online panels have taken over so much of the market even though they carry more bias risk. Know which one you’re running before you interpret the results.
Survey Methods Compared
| Method | Typical Cost | Speed | Response Quality | Best-Fit Use Case |
|---|---|---|---|---|
| In-person interview | Highest (labor-intensive) | Slow (days–weeks) | High depth, high rapport, interviewer bias risk | Complex topics, low-literacy or hard-to-reach populations |
| Telephone (live interviewer) | High | Moderate | Good depth, mode effects toward “acceptable” answers | Political polling, populations without reliable internet |
| Mail (paper) | High (printing + postage + data entry) | Very slow (weeks) | Thoughtful but low response volume | Older or offline populations, official/government surveys |
| Online/web (panel or link) | Low–moderate | Fast (hours–days) | Broad reach, self-selection bias if non-probability | Most commercial research, large sample needs |
| Low | Fast | Low response rate, easy to segment | Existing customer/subscriber lists | |
| SMS/text | Low–moderate | Very fast (minutes–hours) | High response rate, must stay short | Transactional feedback, real-time CX |
| In-app/mobile prompt | Low | Fast | Moderate–high, context-relevant | Product feedback tied to a specific action |
| QR code intercept | Low | Fast | Moderate, self-selected | Physical locations — events, retail, packaging |
| Focus group | High per insight | Slow to plan, fast to run | Deep qualitative, small n | Concept testing, exploring “why” behind numbers |
| AI-moderated/chatbot | Low–moderate | Fast | Emerging; good for open-ended follow-up | Conversational feedback at scale, adaptive questioning |
Response rate is quoted separately for each method below because it swings enormously by context — a customer NPS survey and a cold public-opinion poll using the identical mode will land in different ranges entirely.
In-Person and Face-to-Face Interviews
Face-to-face is the oldest survey mode and still the one with the best response rates — in-person efforts commonly clear the 50% mark where phone and mail struggle to hit half that. An interviewer standing in front of someone is hard to ignore, and that’s both the strength and the liability. Respondents tend to give more socially acceptable answers to a live person than they would alone with a screen — the same mode-effect pattern Pew documented between phone and web applies here, often more strongly.
Cost is the ceiling on this method. You’re paying for trained interviewers, travel, and time — a single completed interview can run into the tens of dollars once labor is factored in, versus cents for a self-administered web survey. It earns its keep for populations that don’t respond well to remote contact: rural areas with patchy internet, populations with low literacy, or topics sensitive enough that a trained interviewer catches confusion a paper form would miss.
Telephone Surveys
Random-digit-dial telephone polling used to be the probability-sampling workhorse of the industry, and it’s the method most associated with the “gold standard” label AAPOR historically applied to it. That reputation is aging. Landlines have thinned out, caller ID screening has gutted answer rates, and a growing share of working numbers belong to cell phones with their own legal and cost complications for automated dialing.
What phone still does well: it reaches people who don’t fill out web forms, and a live interviewer can probe a vague answer in real time in a way no online form can. It’s also the mode where Pew’s mode-effect research is clearest — the 8-point gap in “very attached” responses about Israel between phone and web/paper modes isn’t a fluke, it’s a consistent pattern where phone respondents skew toward warmer, more socially agreeable answers when a person is listening. Budget for it when your population skews older, less online, or when you need an interviewer to keep the respondent on a long or complex questionnaire.
Mail Surveys
Paper-and-postage surveys are the slowest and, on a per-response basis, often the most expensive method still in regular use — printing, envelope stuffing, postage both ways, and manual data entry all add fixed costs that don’t shrink no matter how few people respond. Government agencies still lean on it for exactly that reason: the U.S. Census Bureau uses mail as a primary contact mode precisely because it reaches households without phone or internet records, which matters enormously when the mandate is to count everyone.
Response rates from mail can actually be decent when the sender is trusted — a government agency or a known institution — because completing and returning a paper form signals real intent, unlike a one-click web survey. But turnaround measured in weeks makes it a poor fit for anything time-sensitive, and it’s effectively disappeared from commercial market research outside of a few legacy government-adjacent contracts.
Online and Web Surveys
This is where survey volume lives now, for the obvious reason: near-zero marginal cost per response and turnaround measured in hours. A web survey distributed to an opt-in panel or embedded on a website can collect thousands of responses in the time a phone bank collects dozens.
The tradeoff is sampling, not speed. Most web surveys — the ones sent as a link over email or social media, not built from a probability-sampled panel — are non-probability by default, because you can’t guarantee who clicks. Pew’s own American Trends Panel illustrates the gap between individual wave response and cumulative reach well: response to a given web wave typically runs 60–64%, but the cumulative response rate accounting for the original recruitment survey, panel signup, and attrition sits around 3.5%. That’s the honest number for how hard true probability sampling online actually is, even for a well-funded, methodologically rigorous panel.
For most commercial use — customer satisfaction, product feedback, market sizing within a known customer base — non-probability online is entirely fit for purpose. Just don’t present the results as nationally representative unless the panel was actually built that way.
SMS and Mobile Surveys
Text message surveys have quietly become one of the highest-response-rate channels available, largely because SMS open rates are near-universal and immediate — most people read a text within minutes, unlike email, which can sit unopened for days. Short, transactional SMS surveys (“Rate your delivery, 1–5”) routinely pull response rates in the 40–60% range. Stretch the survey past two or three questions, though, and completion drops off fast — SMS is a format for one clean ask, not a questionnaire.
Cost sits in the low-to-moderate range depending on carrier fees and platform pricing, and speed is unmatched — responses often arrive within the hour. The constraint is entirely about length and context: this is the right tool for a single post-transaction pulse check, not a 20-question study.
QR Code Intercept Surveys
QR intercepts bridge a physical moment — a checkout counter, an event booth, a product package — to a digital survey instantly. Cost is low once the code is generated, and response volume depends entirely on foot traffic and how compelling the ask is at that exact moment. The sample is inherently self-selected: only people who scan respond, which makes this a non-probability method by definition, useful for directional feedback rather than population-level claims.
It’s grown fastest in retail, events, and hospitality, where a paper comment card used to sit on a table. Response quality tends to be shallow — people scan mid-errand, not mid-reflection — so keep QR-triggered surveys to two or three questions maximum.
Focus Groups
Focus groups aren’t a survey in the strict sense — they’re a structured group conversation, not a standardized questionnaire — but every major listicle on this topic includes them, and for good reason: they answer the “why” that a numeric survey can’t. A satisfaction score tells you 62% are happy; a focus group tells you what the other 38% actually wanted instead.
Cost per participant is high relative to any of the self-administered methods above, since it requires a moderator, a venue or platform, incentive payments, and hours of qualitative analysis afterward. Sample sizes are small by design — 6 to 10 participants per group is typical — which means results describe themes and hypotheses, not population estimates. Pair a small number of focus groups with a larger quantitative survey and you get both the “how many” and the “why,” which is the combination most rigorous research designs actually use.
AI-Assisted Surveys
This is the category most existing guides mention in a single throwaway line, if at all, and it’s moved fast enough that the gap is worth closing. AI-moderated surveys use a conversational interface — often a chatbot — that adapts its follow-up questions based on what a respondent just said, instead of running a fixed script. Answer a satisfaction question with “meh, the checkout was confusing” and an AI-moderated instrument can immediately ask what specifically was confusing, in the moment, without a human moderator standing by.
The practical upside is depth at scale: you get something closer to a focus-group-style probe on every single respondent, at web-survey cost and speed. The tradeoff is newer and less studied — question phrasing generated dynamically is harder to standardize than a fixed instrument, which complicates comparing results across waves or against historical benchmarks. Treat it as a strong tool for open-ended exploratory feedback right now, and be more cautious using it anywhere you need strict comparability to a study run last year with a fixed script.
Quantitative vs. Qualitative Surveys
This is a separate axis from mode and sampling, and it’s worth untangling because “survey” gets used loosely for both.
Quantitative surveys use closed-ended questions — ratings, multiple choice, yes/no — built to be counted and compared across a large sample. They answer “how many” and “how much,” and they’re what almost every method in the comparison table above is optimized to deliver at scale.
Qualitative surveys — open-ended questions, focus groups, structured interviews — answer “why” and “how,” at the cost of not being reducible to a clean percentage. They need smaller samples and heavier analysis per response.
Neither replaces the other. A quantitative survey tells you the checkout abandonment rate moved; a qualitative follow-up (or an AI-moderated open-ended probe) tells you it’s because a new shipping calculator added a step nobody expected.
How to Choose a Survey Method
Four variables decide this, in roughly this order of importance.
Budget. If cost per response is the binding constraint, online, email, SMS, and QR intercepts all sit well below phone, mail, and in-person on a per-completion basis. In-person research is defensible when the population or the topic demands it, not as a default.
Timeline. Need answers this week? SMS and web surveys turn around in hours to days. Mail and rigorous multi-wave phone studies run weeks. Build the timeline backward from when the decision actually needs to get made, not from how the last study was run.
Sample size and representativeness needs. If you need a number that generalizes to a whole population — a market size estimate, a claim you’ll put in a press release — you need probability sampling, full stop, regardless of mode. If you need directional feedback from people who already engage with you, non-probability online or in-app methods are faster and cheaper, and the rigor of probability sampling would be wasted effort.
Population reachability. Match the mode to where your population actually is. A survey of hospital administrators works over email or phone. A survey of teenagers works over SMS or in-app prompts, not mail. A survey of a rural population with patchy broadband works over phone or in-person, not a web link.
Run those four filters in order and most projects land on an obvious answer within a few minutes — usually a primary quantitative method for the “how many,” paired with a smaller qualitative layer for the “why,” which is how the strongest studies get built in the first place.
