BCBA Exam Study Topics: Measurement & Interobserver Agreement

Following are some useful pieces of information (from Cooper, Heron, & Heward, 2007) regarding measurement in behavior analysis.

• Event recording encompasses a wide variety of procedures for detecting and recording the number of times a behavior of interest is observed.
• Measurement is reliable when it yields the same values across repeated measurement of the same event.
• Although high reliability does not confirm high accuracy, discovering a low level of reliability signals that the data are suspect enough to be disregarded until problems in the measurement system can be determined and repaired.
• Most investigations in ABA use human observers to measure behavior, and human error is the biggest threat to the accuracy and reliability of data.
• Measuring a dimension of the behavior that is not suited for, or is irrelevant to, the reason for measuring the behavior threatens validity.
• Indirect measurement is measuring a behavior different than the behavior of interest. This threatens the validity of the measurement system due to requiring the researcher or practitioner to make inferences about the relationship between the measures obtained and the actual behavior of interest.

• Time sampling refers to a variety of methods for observing and recording behavior during intervals or at specific moments in time.
• Observers using whole-interval recording divide the observation period into a series of equal time intervals. At the end of each interval, they record whether the target behavior occurred throughout the entire interval.
• Observers using partial-interval recording divide the observation period into a series of equal time intervals. At the end of each interval, they record whether behavior occurred at any point during the interval.
• Observers using momentary time sampling divide the observation period into a series of time intervals. At the end of each interval, they record whether the target behavior is occurring at that specific moment.

• Measuring the behavior after it has occurred by measuring its effects on the environment is known as measurement by permanent product.
• Measurment of many behaviors can be accomplished via contrived permanent products.
• Assessing the reliability of measurement requires a natural or contrived permanent product so the observer can remeasure the same behavioral events.

• The most commonly used indicator of measurement quality in ABA is interobserver agreement (IOA) , the degree to which two or more observers report the same observed values after measuring the same events.
• Researchers and practitioners use measures of IOA to (a) determine the competence of new observers, (b) detect observer drift , (c) judge whether the definition of the target behavior is clear and the system not too difficult to use, and (d) convince others of the relative believability of the data.
• There are numerous techniques for calculating IOA, each of which provides a somewhat different view of the extent and nature of agreement and disagreement between observers.
• Percentage of agreement between observers is the most common convention for reporting IOA in ABA.
• IOA for data obtained by event recording can be calculated by comparing (a) the total count recorded by each observer per measurement period, (b) the counts tallied by each observer during each of a series of smaller intervals of time within the measurement period , or (c) each observer’s count of 1 or 0 on a trial-by-trial basis.
• Total count IOA is the simplest and crudest indicator of IOA for event recording data, and exact-count-per-interval IOA is the most stringent for most data sets obtained by event recording.

Reference: Cooper J.O, Heron T.E, Heward W.L. (2007). Applied behavior analysis (2nd ed.) Upper Saddle River, NJ: Pearson.