It is important to engage your key stakeholders in the process of assessing your program’s readiness for evaluation and in the implementation of your evaluation.
2.1 Assessing Readiness
Undertaking an effective evaluation requires time, effort and resources. Assessing readiness allows you to take stock of what you have in place to conduct an evaluation and in turn decide on a plan that is most appropriate for your work. Taking the time to do this initial work can help ensure that your evaluation will yield useful and actionable results and help set you up for success.
There are some logistical and planning factors to think about. This include your available funds and resources, requirements from funders, maturity of the program, workload, and timing. Consider some of these key questions:
- What kind of evaluation are you required to do, if any?
- What kind of evaluation will your budget support?
- Are there any time constraints you are facing?
- How mature is the program and has it been evaluated before?
Here is a simple checklist to help you determine your capacity to undertake an evaluation. While it may seem daunting to have all these factors in place before you start, not everything on this list is essential. Some factors are more important than others in terms of readiness. The checklist outlines what you ‘need to have’ and what is ‘nice to have.’ Taking stock of your readiness allows you to customize your evaluation plan in a way that works for you.
You can also convert the Word file to a Google Doc. Here’s how to do it.
A list of all the toolkit’s worksheets can be found in Appendix 3
Case Example #1
Karen Tote has been overseeing a 12-hour hotline for students coping with mental health and addictions and/or in crisis at Lakeside College for 2 years. The hotline is staffed by five (5) telephone counsellors who work on rotation. The staff are consistently working at more than full speed, and can’t keep up with the demand. They are turning over quickly because of volume of work. Karen wants to expand to a 24hr service and hire more staff. She decides to conduct an evaluation and gather data to strengthen her case for increased funding.
There is not much room to adjust her workload but she is committed to overseeing the evaluation herself, as no one else is able to lead the work. Her budget is limited as the bulk of her funding is dedicated to running the hotline and is already stretched. However, Karen does have a dedicated group of stakeholders that are willing to support the evaluation.
Karen decides she wants to assess the following:
(a) How well the service delivery model is working;
(b) How useful the hotline has been in offering safe mental health supports and crisis intervention
(c) If the hotline has been useful for diverse students such as self-identified LGBTQ, students of diverse cultural and ethnic backgrounds and students with disabilities.
Consider the Following Questions:
Case Example #2
Erica is a Mental Health Educator at Mapleleaf College and works with the Student Health, and Wellness Centre on campus. Over the last three years she has been organizing training sessions for counsellors, clinical staff, administrators and student body leaders to recognize early signs of addiction among students accessing their services.
The training is conducted twice a year and Erica has decided to evaluate how effective these sessions have been. She spent a considerable amount of time (above her existing workload) developing and delivering a questionnaire to the 54 representatives who had previously attended the training sessions and received an overwhelming response with 48 completed surveys. Once she received the completed surveys she approached her supervisor for the go-ahead to hire someone to help her analyze the surveys. Her supervisor is surprised to hear of this request as she had not been aware of Erica’s evaluation efforts and does not have a budget to assist Erica with her evaluation. Close to 6 months later the completed surveys have yielded no action or analysis and no updates have been shared with the people who completed them.
Consider the Following Questions:
- Introduction to Evaluation
- The 5Qs of this Toolkit
- 1.0 What is Evaluation?
- 2.0 Planning Your Evaluation
- 2.1 Assessing Readiness
- 2.2 Building an Evaluation Plan
- 2.3 Section Summary
- 3.0 Conducting Your Evaluation
- 3.1 Understanding the Ethics of Data Collection
- 3.2 Designing the Tools and Collecting your Data
- 3.3 Inputting, Analyzing and Interpreting Data
- 3.4 Section Summary
- 4.0 Sharing and Learning
- 5.0 Evaluation Projects
- Resource List
- Partner Resources
- Bibliography and References
- Appendix 1: Glossary of Terms
- Appendix 2: Case Study Answers
- Appendix 3: Worksheets & Templates
- 6.0 Apply to the Evaluation Capacity Program for 2023/2024