Search
Home Evaluation Toolkit 3.0 Conducting Your Evaluation 3.2 Designing the Tools and Collecting your Data 3.2.3 Building Survey Questions 3.2.3.1 Let’s Talk Data Collection!

3.2.3.1 Let’s Talk Data Collection!

While there are many ways of collecting data, the following methods are the most commonly used:

Methods of Collecting Data
MethodDescriptionAdvantagesDisadvantages
SurveysA set of questions administered to a group of people in person, over the phone or online
  • Can be administered on a large scale if you want to gather a large volume of data
  • Can be cost effective, especially if done online
  • Can be timely
  • Can be anonymous and confidential
  • Data about sensitive subject matter such as addictions, violence or suicide is difficult to collect over a survey
  • Hard to get higher response rates
  • Limited or no opportunity to clarify questions
  • Lots of bad surveys out there – important to make yours one of the good ones!
Focus GroupA focused discussion with key stakeholders i.e. service users or partners led by a facilitator
  • Can lead to rich qualitative discussion that can be used to determine key areas of importance for your key stakeholders
  • Can get diverse views in a short period of time
  • Costly and time consuming depending on how many sessions you wish to host
  • Expertise of a skilled facilitator is needed to gather data and create a comfortable and safe space
Service Data ReviewsA review of existing data on a program or service or initiative captured through service logs and administrative data related to delivery
  • Provides an objective picture of what is happening and is well suited for process related inquiry
  • Does not offer much context to explain why your service is performing in a certain way
In-Depth InterviewsA one-on-one structured or semi-structured interview where an interviewer guides an in-depth discussion on a subject of interest
  • Allows for deep reflection on a service and can yield rich qualitative data, including stories
  • Can be done over the phone for convenience
  • Can follow-up if needed
  • Can be expensive, time consuming and requires a skilled interviewer
  • The information gathered may be very specific to a few people’s experiences and not reflect the experience of a larger group
  • Can be hard to recruit for

As outlined in section 1.2 some evaluation approaches incorporate arts-based or creative methods of data collection using photography, storytelling or sketching. Using art can offer a powerful way to connect to students that allows them to share their thoughts or experiences in different ways.

Which Methods Might Work Best for You

When deciding on which methods might work best for you, consider the following questions below. In Section 3.2, the steps you take to develop the tools listed above that you decide on will be outlined.

  1. How much time do you have to develop tools? Recruit participants? Coordinate
    groups/evaluation activities? Analyze data?
  2. Do you have access to evaluation participants in sufficient numbers depending on the method you are interested in?
  3. Who will pilot your tools?
  4. What kind of technology can you access – e.g., online survey platforms? Analysis software?
Worksheet #4: Data Collection Plan

Download Word File Download Word File with sample content

You can also convert the Word files to Google Doc. Here’s how to do it.

A list of all the toolkit’s worksheets can found in Appendix 3

Case Example 5

Lee is a student leader at Coolwaters University who organizes diverse social awareness events on campus along with the Student Life Centre. He has been learning more about alcohol addiction and mental health among students and has organized a seminar with guest speakers from the local Addictions Treatment Centre. The seminar is planned as an all-day event with two workshops and one keynote speaker. Lee is also developing feedback surveys for all those planning to attend. His learning goals are: (1) to determine how useful the seminar was for students who attended; and (2) if students would like to see more events such as these on campus in future. Lee has developed the following questions:

a.) The following information was most useful to me:

☐ Early signs of mental health and addiction challenges
☐ Information about available services in the area
☐ How to talk about addiction

b.) How satisfied were you with the speakers of workshop 1 and workshop 2:
Unsatisfied     Somewhat Satisfied     Satisfied     Very Satisfied

c.) List three (3) things you learned today that you did not know before:
                                                                        
                                                                        
                                                                        

d.) Has this seminar affected the way you think about alcohol addiction or other forms of addiction?

Consider the following:
Additional Resources
Toolkit: PDF Version What Did You Think of This Toolkit?
Additional Resources

Choosing a data Gathering MethodSurveys versus focus groups versus interviews (PDF)Choosing Appropriate Data CollectionUsing Art to Collect Data: ArtReach Toronto (PDF)