Educational Surveys and Data Collection Techniques

Nonprofit friends, planning to collect data soon? Remember: Your questions shape your data—but they don’t always get you what you need. Imagine this: You are filling out a border form, and it asks: "Do you exceed duty-free allowances per person?" The only answers are Yes or No. For someone who didn't bring any goods, selecting No implies they did get something but stayed within the limit. The question doesn't account for people for whom the question is irrelevant, forcing them to provide inaccurate information. Now think about your data collection tools (say, your last survey): ● Are your questions boxing people into answers that don't reflect their reality? ● Are you assuming experiences that don't apply to everyone? ● Are you unintentionally excluding voices by limiting response options? Poorly worded questions = bad data = flawed decisions = a loss of trust. Here are three examples of common pitfalls: ● Assumptions baked into questions Example: “What barriers prevent you from attending our events?” assumes the respondent knows about your events and faces barriers. A better question: “Have you heard of our events?” followed by, “What barriers, if any, prevent you from attending?” ● Excluding relevant options Example: “Which of these programs have you used?” but leaving out “I haven’t used any.” Guess what happens? People pick a random answer or leave it blank, and now your data is a mess. ● Vague questions Example: “On a scale of 1-5, how satisfied are you with our communication?” Without specifying—emails? Social media? In-person?—responses will be all over the place. Your questions are your bridge to listening and understanding. Two things to remember here (and by no means this is the complete list): ● Plan your survey – the why, what, how, when, what-next… before jumping to design ● Use inclusive language, providing options like "Does not apply.", wherever relevant. Ensuring people responding to it can see themselves in the questions and responses is the only way to give them the true choice of what and how much they want to share with us. Please reach out if you want to plan a Survey Kaleidoscope workshop with your team on your upcoming survey (for context, it's a workshop where we solely plan the survey collectively - every single element of how to ensure a successful survey happens) #nonprofits #nonprofitleadership #community

We’ve all been there. You’ve just wrapped a round of surveys, or coded dozens of interviews, and now it’s time to find patterns in the data. But the methods you’ve been taught - like PCA or k-means - assume the data is numerical, clean, and fits neatly into a spreadsheet. That’s not what most UX data looks like. In reality, UX data is messy and mixed. We deal with checkboxes, dropdowns, 5-point Likert scales, open-ended tags, and behavioral categories. Most of it is categorical or ordinal, not truly numerical. And when we force these into methods designed for numbers - treating "Agree" like it’s a 4 and "Strongly Agree" like a 5 - we risk drawing the wrong insights or missing what really matters. The good news? There are clustering methods built specifically for qualitative and mixed data. Latent Class Analysis (LCA) helps you find hidden subgroups in categorical survey data. It’s great for segmenting personas or attitudes - based on real patterns, not assumptions. Multiple Correspondence Analysis (MCA) is like PCA, but for categorical variables. It reduces complexity by turning survey responses into dimensions you can actually visualize and cluster - without treating text like math. Factor Analysis of Mixed Data (FAMD) bridges the gap when your data includes both numeric and categorical responses. It lets you uncover structure across both types without losing context. So if your research involves segmenting users based on qualitative input, or making sense of messy attitudinal patterns - don’t default to methods that weren’t made for your data. These three techniques can help you cluster the right way, without compromising on the richness of your research.

If you’re going to collect primary data, here are 10 things to keep in mind: 1️⃣ Conduct formative research. This doesn’t mean spending thousands of dollars. It means grounding your study. There are two schools of thought in social theory: 👉 Grounded theory → theory flows up from the data. 👉Pre-existing theory → theory guides your data collection. Whichever you lean toward, start by listening. If your survey is about challenges faced by people living with HIV, don’t sit in your room inventing questions. Go talk to them. Also, don’t forget: blogs, forums, and public chats are goldmines of lived experience. 2️⃣ Calculate your sample size. Even for descriptive surveys, you need sample size for precision (for narrow CIs). For analytical studies, you need power (to detect differences). 3️⃣ Create a statistical analysis plan. Most people skip this, but it’s key. A SAP forces you to think about how you’ll analyze data before you collect it. It also reveals gaps: maybe you forgot to include important confounders in your questionnaire. Better to fix that now. Failure to plan is planning to fail. 4️⃣ Build a sampling frame. This is simply a list of the people you want to sample. If you’re doing probabilistic sampling, you need this. Decide upfront: closed survey or open survey? 5️⃣ Perform cognitive testing of your instrument. People talk about “validated questionnaires” as if validation falls from heaven. It doesn’t. Validation = testing how real people interpret your questions. Give your survey to 2-3 people at least. Then sit with them afterward. Ask: “What confused you?” “When you heard this question, what came to mind?” If 10 people interpret a question 10 different ways, you don’t have a valid question. That’s bias. 6️⃣ Publish your protocol. Yes, on ClinicalTrials.gov. It’s not just for clinical trials. Benefits: Forces clarity in your design. Reviewer comments can sharpen your study. 7️⃣ Program survey logic. Never rely on instructions like “skip this question if not applicable.” Nobody reads instructions. If your survey has skip patterns, automate them. Don’t delegate to humans what technology can handle. Platforms like Chisquares™ (www.chisquares.com) make this easy. 8️⃣ Translate into required languages. People always understand best in their mother tongue. Translation isn’t optional in diverse populations—it’s respect and clarity. 9️⃣ Do an early cut test. Don’t wait until the survey closes to discover problems. Run an early check to confirm: 👉The survey is working as intended. 👉Responses make sense. 👉No major errors. Catching issues early saves you. 🔟 Document everything. At minimum, you need three outputs: 👉 A codebook (data dictionary) 👉A clean dataset 👉A methodology report On Chisquares™, all three are generated automatically. 📅 Want to learn more? Join our workshop next week Sep 11-12. We’ll cover study design, questionnaire design, and data collection—end to end. Registration: https://s.chi2.io/afAaa5S