- Introduction to Evaluation
- The 5Qs of this Toolkit
- 1.0 What is Evaluation?
- 2.0 Planning Your Evaluation
- 3.0 Conducting Your Evaluation
- 4.0 Sharing and Learning
- 5.0 Evaluation Projects
- Resource List
- Partner Resources
- Bibliography and References
- Appendix 1: Glossary of Terms
- Appendix 2: Case Study Answers
- Appendix 3: Worksheets & Templates
- 6.0 Apply to the Evaluation Capacity Program for 2023/2024
Home Evaluation Toolkit 3.0 Conducting Your Evaluation 3.3 Inputting, Analyzing and Interpreting Data
3.3 Inputting, Analyzing and Interpreting Data
For beginner evaluators, inputting data should be kept relatively simple. An excel spreadsheet can be used for both quantitative and qualitative data as follows:
Surveys – create a spreadsheet as seen below:
| Participant # | Q.1 (satisfaction rating) | Q.2 (what could have been improved) | Q.3 (Change in confidence) | Q.4 (actions taken) |
|---|---|---|---|---|
| 1 | 4 | 0 | 5 | 1 |
| 2 | 3 | 4 | 5 | 3 |
Interviews/Focus Groups
| Participant # | Q.1 (what was liked about service) | Q.2 (number of referrals accessed post-counselling) | Q.3 (confidence rating) | Q.4 (actions taken) |
|---|---|---|---|---|
| 1 | Friendliness of staff | Location not accessible to someone with a walker | I know how to stand up for myself if someone isn’t giving me a fair shake | I failed a test but instead of dropping the class I talked to the professor about re-taking it |
Analysis & Interpretation
Once you have your data in hand, the illuminating task of analysis and interpretation can begin. Analysis allows you to see what your data is telling you and can identify what changes have happened as a result of you service, what is working really well that you should do more of or what you might want to or improve.
Your analysis can include both simple and/or complex processes to categorize the information you have collected and determine patterns in the data.
Data Analysis: Key Terms & Definitions
Below are some key terms and definitions related to data analysis to help get you started.
| Sample | Refers to the group that was used to collect data i.e. a sample of the student population that was surveyed or a sample of peers who attended a focus group. The size of your sample will have some implications for data analysis. For example, if your sample size for a satisfaction survey was representative of only 10% of the students you accessed the service, this may lead one to question the reliable of your results. |
| Reliability | Data gathered is considered reliable if it consistently produces the same results or findings |
| Validity | Refers to how sound your data is and if it the data you have collected is a true measure of what you are trying to learn. For example, five students in a focus group all believe that mental health is not a major issue on campus, this will be recorded as data collected in your session but it does not make your finding true |
| Transcribe | Refers to the act of recording discussions from focus groups or interviews verbatim to capture the exact words used in the session |
| Coding | Refers to the process of reviewing transcribed documents and looking for patterns in thoughts shared. Each common thought is coded and if the same code is repeated multiple times, you can determine that to be a common theme |
Validity and Reliability of Qualitative Data
Note that the validity and reliability of a quantitative measure can be ensured by using a well-tested or validated instrument. However, instruments of qualitative measures are difficult to test or validate beforehand. There are ways staff can ensure the validity and reliability of a qualitative measure when conducting data analysis: The following are examples of how you can improve the validity and reliability of qualitative data:
- Using two or more staff to analyze the data:
- The use of negative cases, which refers to seeking out negative cases in a qualitative data that go against the main theme or finding.
- The use of rival explanations, which refers to the search for alternate reasons for the findings turning out the way they did.
- The use of Triangulation method, which refers to the use of more than one data source and data collection method
Analyzing Quantitative Data
Analyzing quantitative data can cover simple methods of placing your information or data in a spreadsheet and tracking changes or looking for patterns. The results or findings for quantitative information can typically look like changes in percentages, volume of use and/or mapping service access points.
Very often, quantitative data is used to create descriptive statistics. These statistics are meant to give a bigger picture relative to your evaluation questions. For example, the mean, median and mode are all descriptive statistics that you can use to analyze your data:
| Mean | The “average” number; found by adding all data points and dividing by the number of data points. | 3+12+6+14+5+8 = 48/6 – the mean is 8 |
| Median | The middle number; found by ordering all data points and picking out the one in the middle (or if there are two middle numbers, taking the mean of those two numbers). | 3,5, 6, 8, 12, 14 – the 2 middle numbers are 6 + 8 (they are added and divided by 2 to get the median) = 7 |
| Mode | The most frequent number—that is, the number that occurs the highest number of times. | 3, 3, 3, 4, 4, 4, 4, 5, 8, 9 – the mode is 4 |
| Range | The difference between the highest and the lowest scores | 3,3,3,4,4,4,5,8,9 – the range is 9-3=6 |
| Variance | The measure of how spread out a data set is from the average value | 3, 4, 8 – the variance is 7 |
| Standard Deviation | Refers to how far the typical case or response deviates from the mean score | 3, 4, 8 – the standard deviation is 2.6 |
You can also calculate simple frequencies and percentages to paint a broad picture through your data. Beyond descriptive statistics, data analysis is inferential. Most people will need special training to do inferential analysis. The Additional Resources section on the next page has a great guide to get you started.
Analyzing Qualitative Data
Analyzing qualitative data requires a careful reading of your data to systematically identify overarching themes from the data gathered. If you have transcribed responses from a focus groups, the answers to each question discussed in the group will need to be read line and line as you look out for repeating themes.
For example, if one of the questions at the focus group is ‘what are the factors that contribute to student resilience in dealing with anxiety on campus?’ you may find that students repeatedly mention these factors during several focus groups: ‘supportive faculty’, ‘teachers who understand mental health’ or ‘openness of professors to discuss mental health.’ These common responses speak to the theme of a faculty that is responsive or aware of the mental health challenges students may face as a factor that contributes to student resilience.
Picking out themes in qualitative data is done through the process of “coding” the data. This means determining broader headings or key themes and then arranging data under those headings/themes. As you go through this process, you may find that new themes emerge, in which case you create a new heading/key theme and start including data under there when appropriate. You may go through your data a few times In order to ensure you have captured all the themes and/or synthesize /create new themes. While this process can be time consuming and as the transcribed data has to be reviewed line-by-line, professional transcription services can be employed, especially for large volumes of data.
TIP
When conducting qualitative analysis, it is a good idea to have more than one person people read through and code the data to ensure you catch and account for any individual bias.