- Introduction to Evaluation
- The 5Qs of this Toolkit
- 1.0 What is Evaluation?
- 2.0 Planning Your Evaluation
- 2.1 Assessing Readiness
- 2.2 Building an Evaluation Plan
- 2.3 Section Summary
- 3.0 Conducting Your Evaluation
- 3.1 Understanding the Ethics of Data Collection
- 3.2 Designing the Tools and Collecting your Data
- 3.3 Inputting, Analyzing and Interpreting Data
- 3.4 Section Summary
- 4.0 Sharing and Learning
- 5.0 Evaluation Projects
- Resource List
- Partner Resources
- Bibliography and References
- Appendix 1: Glossary of Terms
- Appendix 2: Case Study Answers
- Appendix 3: Worksheets & Templates
- 6.0 Apply to the Evaluation Capacity Program for 2023/2024
2.2.6 Developing Data Collection Methods
Outlining your methods means establishing what data will be collected, when and how. Your methods flow directly from your logic model and the evaluation questions you want to answer. To get started some basic information about data is outlined below followed by a list of common data collection methods. A sample data collection plan is also included below as Worksheet #4 to help guide your data collection work.
Data refers to any piece of information and can be described as quantitative or qualitative.
Quantitative data is objective and numerically measured to make sense of information. However, quantitative data can’t always the question of why the numerical value of something is increasing, decreasing or staying the same.
Examples of quantitative data include:
- Standardized test scores
- Answers on a 5-point rating scale
- % of work days on a month a student counsellor is available to provide service
- # of days a student has to wait for services to be offered
- # of referrals a student accesses
- # of users who post comments about a sexual harassment safety app
Qualitative data is subjective, descriptive, contextual and focuses on ‘why’ something has changed as a result of an action or activity. Qualitative data attempts to strengthen a narrative understanding of why students may or may not access a service, or their experience accessing the service, or what happens for them as a result of accessing the service. Qualitative inquiry can also lead to unanticipated insights gained from open-ended questions that allow for reflective response.
Examples of qualitative data include:
- Service improvement suggestions
- Ways that students may have applied new information
- Perceptions of service users and how they change over time
- Experiences of challenges and how they were addressed
Case Example 4
Jonathan is a psychotherapist specializing in youth addiction work and has recently joined the Bold Prairie College Student Counselling Office. In the first six (6) months of his appointment he has only seen an average of two (2) students per week and the office’s efforts to outreach to students about this service has seen little uptake. Students do not tend to stay through for the full course of treatment when they do come for service.
Jonathan’s quantitative data seems to indicate that having a psychotherapist on the team is not what is needed. Jonathan decides to gather some evaluation data and hosts three focus groups with a cross section of students. The decision to host focus groups (i.e., qualitative data gathering) is made so as to be able to learn more deeply about the student experience.
Through the focus groups the Student Counselling Office learns that while many students wish to access Jonathan’s service they are too afraid of being seen making appointments with him and are fearful of the stigma associated with accessing mental health services, especially as a small rural college. Using both qualitative and quantitative data points allow Jonathan and his office to understand not only what is happening but also why it is happening.
Consider the following questions:
Putting it all Together…
After you determine your methods and how and when you will gather your data, you are ready to put it all together in your final evaluation plan. This is exciting! The final plan is a document that includes:
- A title page with the name of your program, and the author of the document
- A description of your program or service
- The evaluation goals and questions you are focusing on
- Your methods and data collection plan
- Information on how the data will be analyzed
- Information on how the results will be shared/disseminated (i.e., with whom, when and how)