Questionnaire surveys are a popular research method, as when done well they can provide crucial data for many purposes. The quantitative outputs are usually easy to interpret and share, and the data can often be reused more easily than unstructured data like interview transcripts.

However, surveys represent added risk to the department compared to other research methods, and so we have an assurance process around them. It is important that if you are planning a survey, you use the method appropriately, use the correct survey tool, and get it peer reviewed and approved.

Page contents

Risks when using questionnaire surveys

Surveys represent added risk to the department compared to other research methods for three main reasons:

  • Unmoderated research means any misunderstandings cannot be clarified.
  • A lot of people may be exposed to issues in the survey (e.g. misrepresentation of policy) and surveys can be forwarded to others, which increases reputational risks
  • Respondents may receive multiple survey requests from DfE which impacts research fatigue and burden, particularly key stakeholders with high workloads such as School Leadership Teams.

Running statistically valid surveys can be more expensive and time consuming than qualitative research. When working on agile teams, we might not be afforded the time to do surveys that will allow us to make statistically valid claims.

Consider whether a survey is the best method

If you are considering a survey, you should first ask yourself:

  • Am I confident of getting a response rate to my survey which will provide statistically valid findings?
  • If I do not have a response rate which allows for statistically valid findings, am I comfortable with people making decisions from the data I collect?
  • Have I got a good strategy to promote my survey?
  • Will a survey give me the level of detail I need from respondents to support my research?
  • Is the time I am asking of my respondents proportionate to the benefits of doing the survey? (especially important if you are sampling front-line public-sector workers)
  • Have I explored other options to get the data I need (e.g. service data, existing DfE surveys)? Remember to ask your local social research colleagues and check other DfE data and insight sources
  • Am I and my team clear about the purpose of the survey and the value it will provide?
  • Is my goal to quantify knowledge, attitudes or beliefs from a large number of people?

If your answer to any of these questions is no, think carefully about whether a survey is the right thing to do.

If you are using a survey to triangulate rich qualitative data, you can reasonably accept a less reliable survey. But you must be clear about the limitations of the survey when sharing the results.

Design your survey

Tips and guidance to help you design a great survey and communication strategy (DfE users only), from the DfE Government Social Research team.

Get a peer review from a DfE survey champion

Your must have your survey peer reviewed before launching it.

We have a network of survey champions to peer review surveys (DfE users only) in short timescales.

Survey champions are here to:

  • Support you to design a high-quality survey
  • Facilitate access to survey tools
  • Help resolve any risks linked to your survey

Most portfolios have a survey champion. Survey champions in your own portfolio have the context to provide better support, and are your first port of call for peer review.

If you do not have a Survey Champion in your portfolio ask for a review on the #survey_champions channel in DfE Slack (opens in a new tab).

What happens in a survey peer review

The peer review process is:

As soon as you are considering doing a survey, contact a survey champion to discuss your research goals.

If a survey is the right method, design your questionnaire in a Word document and share it with the survey champion, who will peer review it within 3 working days. They will use this peer review checklist.

After you have made any changes following the peer review, the survey champion will contact the DfE survey tool admin, and a temporary license will be created for you. You will then be able to build you survey using the tool. If you require a longer licence period, speak to a survey champion.

In rare cases the survey may be deemed high risk and submitted to a lead or senior User Researcher for review, which may take longer than 3 days. High risk may be a result of contentious topics, unethical approach, reputational risk, or duplication of existing work.

Become a survey champion

Any user researcher can become a survey champion. Contact Will Finn in DfE Slack (opens in a new tab).

Always use the DfE survey tool

DfE has a specialist survey tool, that allows you to design, distribute, analyse and present survey data, as well as combine your survey data with other data sources.

This tool is available for all user researchers to use. Instructions to access the tool are here (DfE users only), as well as links to training in the tool and support.

This tool is funded centrally, and there is no licence cost to your team, programme or portfolio.

Warning Do not use any other survey tool, unless it has been security and privacy assured, as you may breach DfE or government security rules.

Further support

Contact #survey_champions in DfE Slack (opens in a new tab).