DataAI Nexus

Agents / Skills

Survey Design

Skill

Designs surveys whose results you can actually use: questions that measure one thing each, scales that mean something, the biases engineered out — and the analysis planned before the first response arrives.

Best for

  • Surveys that return 'interesting' data nobody can act on
  • Measuring at scale what interviews found in depth
  • The NPS-and-beyond question of what to ask and how

What you give it

  • What you need to decide, who you will ask, and how you will reach them

What you get back

  • The instrument: each question mapped to a decision, single-barrelled, neutrally phrased, with scales chosen on purpose
  • The bias engineering: order effects, acquiescence, satisficing — handled by design, not hoped away
  • The analysis plan written first: how each question's answers will be used, which crosstabs matter — so dead questions die before shipping

How it works

  1. Starts from the decisions: every question must have a named decision consuming its answer — the analysis plan is written first and kills the dead questions early.
  2. Engineers each question: one thing measured, neutral phrasing, behaviour anchors over frequency adjectives, scales with labelled points and a considered midpoint policy.
  3. Designs against the biases: question order effects, acquiescence (agree-bias), satisficing on long instruments — shortness is a feature, randomisation a tool.
  4. Pilots before shipping: a small read-aloud round catches the misreadings that would have silently corrupted thousands of responses.

Example

You: Survey our 4,000 users about the new dashboard before we decide the next quarter's investment.

Result: The design pass killed six draft questions ('do you find the dashboard useful and intuitive?' — double-barrelled; two 'interesting to know' items with no decision attached — the analysis plan exposed them as dead weight). The instrument: nine questions, each mapped to a named decision, behaviour-anchored where possible ('which of these did you do last week' beats 'how often do you…'), one open text placed last, randomised option order where order biases, and the scale labelled at every point. Pilot with 15 users found two misread questions — rewritten. Result: 640 responses, every question analysed as planned, and the quarter's decision actually made from it.

Limits — please read

  • Surveys measure what people report; where behaviour is loggable, logs beat self-report and the design says so.
  • Response bias (who answers) bounds every claim; the plan includes the response-rate honesty and the non-responder caveat.
  • Causality does not come from crosstabs; the plan words its claims accordingly.