Pre-Event Survey Planning

This week's theme in my workshops (and, by that extension, my posts to you here) is – assessing data collection tools (like surveys) for inclusion and access. Most of my workshops start at the same place – where most have designed at least one survey in the current/past job/education. And then it takes three hours and some meaningful collective learning to realize that planning a survey is much more than just a list of questions. It is an opportunity to connect with your community directly, hear their stories, and understand their experiences and expressions of engagement. In this post, I want to share 5 "red flag" behaviors I often see during a survey design phase: ● When the only questions included are of positive feedback. We all love hearing good things, but only asking for positive feedback disables some real growth opportunities. Example: A question like, "What did you love most about our event?" assumes your respondent only loves the event, and then it offers no room for any different experience. ● When questions are overloaded with complicated words or jargon that only a few will know. You know your mission inside and out, but your community might not understand the same terms you do. Speak in their language. Think of your survey as a conversation. Example: A question like, "How would you rate the efficacy of our donor stewardship activities?" assumes everyone understands the details of "stewardship". ● When every possible question about every possible aspect of the mission is asked – because "why not". Designing surveys – without context – that go on for more than 10-12 minutes - can feel like asking for too much. Be mindful of the respondents and the needs of the data collection. Every question should have a purpose. ● When questions contradict anonymity. Our communities are diverse, and our surveys should hold a neat, safe space for those communities. Ensuring accessibility – balanced with truly useful demographic questions means not harming someone's anonymity – thus making the experience of collecting data easier and meaningful. Example: A survey asking about racial and ethnic diversity in a group of 99% homogenous population (thus making the 1% racially diverse population nervous about the possible breach of anonymity). ● When questions do not offer an 'Opt-Out' option by making everything required. Some questions may feel too personal or uncomfortable for individuals to respond to, and our surveys must create space for that. Give respondents the space to skip a question if they need to. Example: A survey that requires donors to disclose their income range without offering a way to skip the question if they're uncomfortable sharing that information. Stay tuned for a soon-to-be post on what we can do differently then. Have any other such behaviors? Share them here. In the meantime, try some of these resources (all designed to do good with data): https://lnkd.in/gUK-6M_Y #nonprofits #community

Survey design is a critical process for collecting reliable data and informing institutional decision-making. This guide provides a structured approach to planning, designing, and administering surveys, ensuring they effectively capture insights from target populations. It highlights best practices for questionnaire development, sampling strategies, response rate optimization, and data analysis. The document explains essential steps in survey administration, including pre-testing questionnaires, ensuring validity and reliability, and organizing survey questions for clarity and engagement. It also addresses common challenges such as nonresponse bias, survey fatigue, and oversampling, offering strategies to enhance response rates and improve data accuracy. Special focus is placed on survey result interpretation, statistical reporting, and ethical considerations in data collection. For researchers, institutional planners, and survey administrators, this guide is a valuable resource for improving survey effectiveness and data-driven decision-making. It emphasizes the importance of sound methodology, strategic planning, and careful data management to ensure credible, actionable survey results. Whether used for academic assessments, institutional research, or policy evaluations, these insights support better data collection and analysis practices.

✅ Survey Design Cheatsheet (PNG/PDF). With practical techniques to reduce bias, increase completion and get reliable insights ↓ 🚫 Most surveys are biased, misleading and not actionable. 🤔 People often don’t give true answers, or can’t answer truthfully. 🤔 What people answer, think and feel are often very different things. 🤔 Average scores don’t speak to individual differences. ✅ Good questions, scale and sample avoid poor insights at scale. ✅ Industry confidence level: 95%, margin of error 4–5%. ✅ With 10.000 users, you need ≥567 answers to reduce sample bias. ✅ Randomize the order of options to minimize primacy bias. ✅ Allow testers to skip questions, or save and exit to reduce noise. 🚫 Don’t ask multiple questions at once in one single question. 🤔 For long surveys, users regress to neutral or positive answers. 🚫 The more questions, the less time users spend answering them. ✅ Shorter is better: after 7–8 mins completion rates drop by 5–20%. ✅ Pre-test your survey in a pilot run with at least 3 customers. 🚫 Avoid 1–10 scales as there is more variance in larger scales. 🚫 Never ask people about their behavior: observe them. 🚫 Don’t ask what people like/dislike: it rarely matches behavior. 🚫 Asking a question directly is the worst way to get insights. 🚫 Don’t make key decisions based on survey results alone. Surveys aim to uncover what many people think or feel. But often it’s what many people *think* they think or feel. In practice, they aren’t very helpful to learn how users behave, what they actually do, if a product is usable or learn specific user needs. However, they do help to learn where users struggle, what user’s expectations are, if a feature is helpful and to better understand user’s perception or view. But: designing surveys is difficult. The results are often hard to interpret and we always need to verify them by listening to and observing users. Pre-test surveys before sending out. Check if users can answer truthfully. Review the sample size. Define what you want to know first. And, most importantly, what decisions you will and will not make based on the answers you receive. --- ✤ Useful resources: Survey Design Cheatsheet (PNG, PDF), by yours truly https://lnkd.in/ez9XQAk3 A Big Guide To Survey Design, by H Locke https://lnkd.in/eJWRnDRi How to Write (Better) Survey Questions, by Nikki Anderson, MA https://lnkd.in/eHpzr-Q6 Survey Design Guide, by Maze https://lnkd.in/e4cMp5g5 Why Surveys Are Problematic, by Erika Hall https://lnkd.in/eqTd-7xM --- ✤ Books ⦿ Just Enough Research, by Erika Hall ⦿ Designing Surveys That Work, by Caroline Jarrett ⦿ Designing Quality Survey Questions, by Sheila B. Robinson #ux #surveys