Data Collection, Assessment and Evaluation
Monitoring through systematic data collection and assessment (formative and summative) is extremely important in evaluating the effectiveness of any strategy. Monitoring involves the collection and interpretation of data to determine if a student has, or students have, achieved a specific goal or outcome. It is also a means of determining progress made and effectiveness of strategies implemented. To effectively monitor student progress and use of strategies, assessment and data collection should be conducted both before and after the strategy is implemented. It is also helpful to collect some data during implementation of the strategy to determine if any adjustment or revision of the strategy is needed.
The first step in monitoring strategies and students’ progress is to set achievable goals/objectives. In order to set achievable goals, it is critical that an initial assessment be conducted and data collected to identify a student’s baseline skills or behaviours (the student’s current performance of skills or behaviours). It is important to know what the student’s current skills are as a starting point for measuring progress, and to know what is reasonable to expect given their current level of knowledge and skill. Collecting data will also help to determine whether the student has the pre-requisite skills and knowledge (see Check Student has All Prerequisite Skills and Knowledge) needed before working on a specific skill or goal. Once the baseline of their skills/behaviours have been ascertained, this data is then used to determine the goal or desired outcome that the student can reasonably be expected to achieve within a given period of time, and whether the adjustment or strategy is effective (see SMART Goals). Once a goal has been set, information needs to be collected regularly in order to monitor progress in student learning.
Means of data collection
Data collected prior to, during and/or following implementation of a strategy should document how, how often and/or to what extent the student(s) perform(s) the desired skill. It is very important to have a systematic means of recording this information to allow for later interpretation. Data can be collected by teachers, teaching assistants or by the students themselves in self-monitoring records. Below are some examples of data collection methods (see also examples of data collection sheets at the end of this entry):
- Measures of frequency – Measuring frequency entails recording the number of times that a student engages in a particular behaviour during a set length of time. It is important to remember that to record the frequency of a behaviour, you need to have a clear description of the behaviour. Behaviours for frequency counts must have a clear beginning and end. For example, incidences of calling out usually have a clear start and stopping point, and thus can be counted. Other examples include recording the number of times that a student gets out of their seat during a 40 minute maths lesson or the number of words a student reads correctly in a minute of reading. The following is an example of a data sheet that can be used to record frequency of behaviour:
- Interval recording– Exactly noting frequency of behaviours can be challenging, particularly, if these behaviours happen regularly or repeatedly. As an alternative, interval recording can be used to track behaviours. Interval recording entails dividing a period of time into intervals and then recording if the behaviour occurred at any time within that interval (partial interval). In this case, a teacher or assistant would record if the behaviour was demonstrated at any point within the interval rather than recording how many times the behaviour occurred. Intervals can be anywhere from 1 minute to an entire session. Measuring within timed intervals can be difficult for teachers without assistance. However, teaching assistants or special education personnel could assist in recording behaviours within specific intervals (see Teaching Assistants to Undertake and Communicate More Detailed Data Collection). A good way to keep track of interval lengths is to have a timer that goes off at the end of each interval and prompts the recorder to start a new one. A number of inexpensive phone apps are available to make recording easier. When on their own, having intervals that are linked to specific activity periods is a bit easier for teachers to keep track of. For example, a teacher might divide the day into intervals of maths, English, history and art, and then record if the student demonstrated the behaviour at any time during any of those specific activities.
Another form of interval recording is whole interval recording in which the behaviour is only recorded if it occurred for the entire interval. As such, it is very similar to duration recording, and is more suitable to monitor behaviours without a distinctive beginning and end, such as time on task. However, interval recording is very hard to do unless you have a person who is available to specifically record data.
The following is an example of an interval recording sheet with two examples of intervals:
- Measures of duration – Measuring duration entails determining how long a student demonstrates a specific action or behaviour. For example, recording how long students work before requesting help or how long students work on a task before stopping. Below is an example of a duration recording sheet. Like frequency recording, duration recording must be conducted with behaviours that have a clear beginning and end. It is also worth noting that whole interval recording may be an easier means of estimating duration than measuring the amount of time (duration) that someone engages in the behaviour. The following is an example of a duration recording form:
- Measures of accuracy – Measuring accuracy entails recording how correctly a student performs a skill or demonstrates knowledge of a concept. This can be very similar to measuring frequency, but might also include additional notation of approximations or responses that are close to the desired outcome, as well as errors made. For example, recording the number of responses that a student answers correctly and noting the ones that were close or the types of errors made. In another example, recording the steps in a task that a student performs independently and the number, or type, of prompts needed for other steps.
- Measures of quality – Measuring quality generally entails recording a rating of quality against a set criteria. For example, recording quality of sentences written by recording how many include nouns and verbs, correct punctuation and grammar. It may also be helpful to develop assessment rubrics for tasks to help in assessing quality.
- Measures of latency – Measuring latency entails measuring the time between a prompt or stimulus and the student’s response or relevant action. For example, recording the time between the teacher’s direction and when the student starts working.
To obtain these records, you can measure directly through observation in real-time. Alternatively, using video/audio recordings of students as they engage in activities or behaviours (if possible) will eliminate the pressure of recording within a limited time, and it also has the advantage of allowing you to rewind and record different behaviours. It is important to note that video/audio recording will require consideration of privacy of other students and consent of students and parents. You could also gather work samples in order to record the quality or accuracy of the work. As already mentioned, you can also have the student collect their own data, such as noting the time they started and completed an activity or the number of questions they answered correctly.
Having clear records of students’ skills, knowledge and/or behaviours will allow you to compare their performance or behaviours after implementing the identified strategy or adjustment with their performance or behaviours before implementing the strategy or adjustment. This will enable you to determine whether the student(s) have made progress or whether any changes or adjustments to teaching or strategies implemented are needed.
Determining whether a goal has been achieved involves undertaking assessments, collecting data, and evaluating the outcomes. It may be possible to continually collect data through frequent observations or formative assessment (see Formative Assessment) to track progress over time. The student may also be able to use a self-monitoring system to track their own progress (see Self-monitoring). You will need to plan for what data is to collected, how the data will be collected, how often it will be collected and in what contexts it will be collected. You may be able to have a teaching assistant collect information while you are teaching the class, or a small group (see Teaching Assistants to Undertake and Communicate More Detailed Data Collection).
Having an end point evaluation, where you repeat the data collection process used initially, will allow a comparison between the behaviours demonstrated before and after the adjustment or the strategy was implemented. Data can often be graphed, which makes it easier to track changes and see trends over time. For example, you might record the frequency of calling out on consecutive days to determine whether the behaviour is increasing or decreasing in frequency, or whether no pattern is demonstrated. If no pattern is evident, then this may mean that the strategy is not effective or needs to be adjusted, or that staff are not implementing the strategy consistently. An example of a graph of frequency data is presented below:
Keeping records is also important to both 1) gain an understanding of student characteristics that can inform decisions about adjustments or supports, and 2) to track what strategies have been implemented and how effective they have been. If a record is kept of the skills and knowledge a student has mastered, these records can inform decisions about the pre-requisite skills a student possess. Keeping records and consulting these as necessary can help to ensure that effective strategies are continued in a consistent way, and that ineffective strategies are not used repeatedly. Record keeping will also support teachers when they need to write reports for families/caregivers or for other professionals. Many schools will have a procedure in place for keeping records regarding supports implemented for individual students such as individual education plans, or individual learning plans. Schools are also required to collect and record data for the Nationally Consistent Collection of Data on School Students with Disability (NCCD) to demonstrate level of supports needed and what type of adjustments were provided.
It is very important to carefully consider how information about students is recorded and stored, and how it can be shared with relevant school personnel and parents while still maintaining student’s right to privacy or confidentiality. Care should be taken to ensure that private information such as a student’s diagnosis should not be accessible to all staff at the school, but only to those working directly with the student.
Resources for collecting data:
- These templates for recording observations of students are designed for preschoolers, but could be used with older students
- A simple self-monitoring template for students
- A range of data collection forms can be found on these links:
- A template for a task analysis
- This site has a range of measurement resources for academics and behaviour
Resources for keeping records:
- These templates from NCCD may be useful for keeping an ongoing record of adjustments and outcomes for an individual student
- These templates from NCCD may be useful for keeping an ongoing record of adjustments and outcomes for a group of students
- This template is for an individual learning plan (or individual education plan)
Other examples of data collection:
