- Outputs refer to the immediate results produced by completing planned activities. Once an activity has been conducted, the outputs are the resulting consequences following that activity. Outputs are things you can count and answer the question “how many/much did we do?”
- Outcomes refer to the effects your activity has had over time and speaks to short, intermediate or long-term changes that result from your work. Outcomes answer the question of ‘what happened as a result of what we did? Shorter-term outcomes are often about changes in knowledge, skills or awareness. Intermediate and longer-term outcomes can often be about changes in behaviours, actions taken, ability to cope or address life situations as they arise
- An Indicator is a way to actually measure what was done or what happened as a result of the work you do. In relation to evaluation work, indicators refer to the ways we measure outcomes (note that you will be developing indicators when you develop your data collection plan!).
2.2.3 Develop or Review Logic Model
An essential component of evaluation preparation is to review your logic model. If you do not already have one is place now is the time to develop one. Developing a logic model that your program or service is based on offers the opportunity to articulate the planned activities, outputs and outcomes of your work.
Start with revisiting the goals of your service or initiative. Clarifying your program goals will ensure that your logic model is relevant, and the activities, outputs and outcomes identified are aligned with your goals and priorities.
What is a Program Goal?
A program goal is defined as a broad statement (not too broad!) about the long-term expectation of what should happen as a result of your program (the desired result). It serves as the foundation for developing your program objectives. Goal statements “identify” the specific target group and provide the “what” information as distinct from the “how” the goal will be achieved or when it will come about. Examples of program goals include:
“The Fairview College student counselling program offers students the skills and supports they need to successfully complete their PSE experience.”
“Mountain Top University’s Accessibility Plus Program offers students with mental illness dedicated supports to enhance their socio-recreational experience on campus.”
What is a Logic Model?
A logic model is a visual representation of the change we hope to achieve with the resources we have and the activities we plan. They offer a simple and yet powerful way to illustrate a program or service in a way that captures all its key elements. There are several tools available to help build a logic model and while these may vary, they all have some common components illustrated below. For a logic model template and sample indicators see Worksheet #2 below.
Core components of a logic model include:
- Resources or Inputs: What you invest in your program or service (i.e. dollars, time, staffing)
- Activities: What you plan to do (i.e. workshops, develop materials, training)
- Outputs: What you actually do (i.e. # of activities or # of people reached)
- Outcomes: Changes you see as a result of your work. Can be short term (1-2 years), intermediate (2-3 years) or long term (3+ years)
- Impact: Long term outcomes can also be referred to as impacts, and usually cannot be solely attributed to the program alone
A strong program logic model shows the underlying theory of change. It articulates what your program can and cannot control. It explicitly describes how and why you think your counseling program, for example, is expected to help students by pointing to causal assumptions or concrete evidence. For instance, if you think increasing students’ knowledge of available mental health support services leads to increased use of those services by students, is that an assumption or are you able to point to evidence to show it is the case. As well, a strong program logic model helps frame evaluation and can be used as a tool to design evaluation. The components of a strong program logic model are problem statement, program rationale, Inputs, Activities/Strategies, outputs, outcomes, and impact.
|Problem Statement||Problem Rationale||Inputs||Activities or Strategies||Outputs||Outcomes||Impacts|
See below for additional resources on logic models below and Worksheet #2 for a useful logic model template and sample content under each section. Your completed logic model will set the groundwork for what comes next in identifying what you want to evaluate, what types of data to collect, and how.
The difference between an output, outcome and indicator
Case Example 3
Read the following case example to learn more about outputs and outcomes:
Ruby has been working at the Student Accessibility Office at Waverly University as a Mental Health Educator for over five years. As part of her work she coordinates an annual mental health fair and speakers’ event to engage students, school staff and faculty on key issues related to student mental health and addictions. Every year she partners with mental health service providers from organizations on and off campus such as the Canadian Mental Health Association, Community Health Centres, Student Counselling Services and/or Health Clinics to attend the event as guest speakers and exhibitors. Her goals are:
- To reduce stigma associated with mental health and addictions on campus among students and administrative staff by offering a platform for open dialogue and learning
- To increase student knowledge of mental health such as ‘recognizing signs of depression’, supportive ways of acknowledging mental health’ or ‘dealing with anxiety’
- To connect students to relevant services on and off campus to help them navigate and access the services they need
The event is in its fifth year and Ruby wants to conduct an evaluation to see whether they are meeting their goals. Through her event logs Ruby knows that 360 students and campus staff have attended the event in the last five years, and through the feedback forms completed at the event each year she knows that 70% of attendants learn something new about mental health. She has also fostered partnerships by securing speakers and exhibitors for her event, 23 in total. This has increased the number of external services students may be referred to by 40% when presenting with a mental health issue at the Student Accessibility Office where she works. Over the last five years her office has also noted an increase of students accessing their in-house peer counsellors, an increase in referrals and an increase in the number of student leaders who want to volunteer or support mental health work on campus.
Here are some output and outcome related questions that Ruby could consider useful in her evaluation. Consider how Ruby might answer these questions using the information above.
Worksheet #2: Sample Logic Model & Exercise
- The 5Qs of this Toolkit
- 1.0 What is Evaluation?
- 2.0 Planning Your Evaluation
- 2.1 Assessing Readiness
- 2.2 Building an Evaluation Plan
- 2.3 Section Summary
- 3.0 Conducting Your Evaluation
- 3.1 Understanding the Ethics of Data Collection
- 3.2 Designing the Tools and Collecting your Data
- 3.3 Inputting, Analyzing and Interpreting Data
- 3.4 Section Summary
- 4.0 Sharing and Learning
- 5.0 Evaluation Projects
- Resource List
- Partner Resources
- Bibliography and References
- Appendix 1: Glossary of Terms
- Appendix 2: Case Study Answers
- Appendix 3: Worksheets & Templates