The Evaluation Question Nobody Asks: How to Write Questions That Actually Drive Useful Findings

When I help teams design evaluations, I notice a pattern. They spend a lot of energy debating survey question wording, sample sizes, and data collection tools, and then, almost as an afterthought, they write a few broad evaluation questions at the top of the Terms of Reference and move on. The evaluation questions feel like a formality, a box to check.

That is a mistake that costs the whole evaluation its usefulness.

Evaluation questions are not introductory framing. They are the engine. Everything else, your methods, your tools, your analysis plan, your report structure, flows from them. If those questions are weak, vague, or the wrong questions entirely, the best data collection in the world will still leave your stakeholders without the answers they actually need.

Let me walk through what makes an evaluation question genuinely good, and how to write them that way.

What an Evaluation Question Actually Is

This sounds obvious, but it trips up a lot of teams: an evaluation question is not a survey question. Survey questions are narrow and specific ("How many times did you attend the training?"). Evaluation questions are high-level questions that an entire evaluation is designed to answer [1]. You might need ten survey questions, three focus group probes, and a document review just to answer one evaluation question.

As Eval Academy puts it, if the question you have written feels like something you could answer with a single survey item, you have not written an evaluation question yet [1]. That survey item is an indicator, one data point among several that, together, begin to tell the story.

A practical example of the distinction:

The evaluation question is the judgment you are trying to make. The survey question is one piece of evidence that feeds into it.

Evaluation Questions vs. Research Questions: A Critical Distinction

Here is a distinction I find really useful, drawn from Martina Donkers' work [2]: evaluation questions are inherently evaluative. They position you to give an answer on a scale from bad to good, not just a description.

A research question like "Who did the program engage?" can only be answered with a list or a description. An evaluation question like "How effectively did the program engage its intended beneficiaries?" requires you to make a judgment, to say whether the engagement was sufficient, appropriate, or impactful. You cannot answer it with a description alone [2].

This matters because evaluation is not just applied research. Drawing on Scriven's classic definition, evaluation is a process to determine the merit, worth, or significance of something [2]. You need descriptive research questions to get there, but the evaluation question takes it that final, essential step further.

💡 Tip: When drafting an evaluation question, test it by asking: "Could the answer to this fall somewhere on a scale from 'very poor' to 'very good'?" If yes, you likely have an evaluative question. If the answer is just a list or a number, you have a research or monitoring question.

Start with Purpose, Not Topics

One of the most common mistakes I see is teams jumping straight to topic areas ("we should ask about relevance, effectiveness, and sustainability") without first anchoring those topics to the actual purpose of the evaluation [1].

An evaluation designed to demonstrate accountability to a funder will produce very different questions from one designed to improve program implementation. A formative evaluation asks different things from a summative one. The evaluation purpose is the lens through which every question must pass.

Before writing a single question, your team should be able to complete this sentence clearly: "We are conducting this evaluation because we need to know [X], so that [Y] can happen." If you cannot complete that sentence, you are not ready to write evaluation questions yet.

📝 Note: The OECD-DAC evaluation criteria (relevance, coherence, effectiveness, efficiency, impact, sustainability) are useful starting frameworks, but they are not evaluation questions in themselves. They are categories from which you derive specific questions grounded in your program's context and purpose.

Write Them with Your Stakeholders, Not for Them

This is where a lot of technically solid evaluations still go wrong. Evaluation questions should be developed with your interest holders, not handed down to them [1]. Program leaders, operational staff, donors, and beneficiary representatives all bring different information needs to the table.

Working closely with stakeholders during question development does two things. First, it surfaces the questions that people actually need answered, not just the ones evaluators assume they need. Second, it builds buy-in for the findings. When stakeholders helped frame the questions, they are far more likely to trust and use the answers.

⚠️ Warning: Be careful with purely closed evaluation questions ("Did the program achieve its targets?"). While these are sometimes appropriate, they leave little room for the nuance that makes evaluation findings genuinely useful. Default to open-ended framing ("To what extent did the program achieve its targets, and what factors contributed to or limited achievement?") unless there is a specific reason not to [1].

One Question, Multiple Indicators

A well-written evaluation question will typically be answered through a combination of indicators and data sources, not a single metric [1]. This is actually a strength, not a complication. Triangulation across multiple data points produces more credible and defensible findings.

For example, the evaluation question "To what extent did the program improve household food security?" might draw on:

• Quantitative survey data on dietary diversity scores
• Qualitative FGD findings on perceived changes in food access
• Administrative data on distribution coverage rates
• Key informant interviews with community leaders

No single indicator can answer that question well. The evaluation question is what holds all those data streams together and gives them a shared purpose.

💡 Tip: Once your evaluation questions are settled, use them to drive your indicator selection and method choices. Ask for each question: "What evidence would convince a skeptic that we have answered this?" That grounds your data collection plan in the questions, not the other way around.

A Simple Process to Get It Right

Here is how I would approach writing evaluation questions from scratch:

1. Clarify the evaluation purpose with your team before touching any questions.
2. Identify your key interest holders and schedule a working session with them.
3. Generate candidate questions broadly, using OECD-DAC criteria or your program theory as a prompt.
4. Test each question against the evaluative scale test (bad to good) and the "is this a survey question?" check.
5. Prioritize ruthlessly. A focused evaluation with three to five strong questions beats a sprawling one with fifteen weak ones.
6. Map indicators to each question to confirm each one is actually answerable with available resources and timeframes.

If you are working on a Terms of Reference, baseline, or evaluation design right now and want help turning a rough question list into something evaluation-ready, that is exactly the kind of thing I can help you work through at vera.ignex.io.

The Payoff

Good evaluation questions do not just make for a better report. They make the entire data collection effort more efficient, because your team knows exactly what they are collecting evidence for. They make analysis easier, because there is a clear frame for interpretation. And they make findings more usable, because stakeholders recognize the questions as their own.

The irony is that writing good evaluation questions takes maybe a few hours of careful thinking at the start of an evaluation. But teams routinely skip that investment and spend weeks collecting data that never quite answers what anyone actually needed to know.

Do not be that evaluation. Start with the questions.

Sources are listed below. The Eval Academy articles [1] and Martina Donkers' piece on evaluation vs. research questions [2] are particularly worth reading in full if you are designing an evaluation right now.

Follow Vera for more on MEL & project management: LinkedIn · Instagram · Facebook · X