Article Club: Selecting Measurement Procedures

Why Measurement Choice Shapes Every Clinical Decision

Most of us choose a data system out of habit. It worked for the last client, so we use it again. This session worked through an article on this problem. Its title: A Proposed Model for Selecting Measurement Procedures for the Assessment and Treatment of Problem Behavior. The model asks you to slow down. You pick the system for this person and this problem behavior (behavior we want to reduce). You do not pick it for the behavior in general.

Why does that matter so much? Because the data drives the plan. Baseline data (the data taken before treatment) sets the bar for progress. The graph then decides if you keep going, change the plan, or check if you ran it right. If the measure is wrong, every call after it is shaky.

The group framed this as an ethics issue, not just a technical one. Weak data can make a plan look like it works when it does not. It can also hide real gains from the people paying for care. Either way, the client is the one who loses.

accurate data is ethical and effective

From the talk — Pauli Catanese & Morgan Johnson

The Frequency Default: Measurement's Most Common Error

The article flags one error above the rest. Teams default to a count because a count is familiar. Frequency recording (a tally of each instance) is the first tool most of us learned. So it becomes the answer for every behavior, even when it fits that behavior poorly.

The room had a sharp example. A behavior tech sits with a clicker and logs hundreds of hits in one session. That same tech is also supposed to block the behavior and run the program. At some point the data work starts to compete with the treatment.

That is a practical question, and the model treats it as a real one. If the count is too high to catch, you are not getting good data anyway. You are getting a guess, plus a tech too busy to teach. Pick a system the person in the room can actually run.

If your data collection is interfering with your implementation of the program?

From the talk — Pauli Catanese & Morgan Johnson

When a Count Hides Real Progress

Here is the example that stopped the room. You count tantrums and get five per session at baseline. You run treatment, and you still get five per session. The graph is flat. No change in level, no change in trend, nothing to show a funder.

Now say those baseline tantrums ran 10 to 15 minutes each. And say the treatment tantrums run two minutes. That is about 60 minutes of tantrum per session down to eight. The count never saw it. Duration (how long the behavior lasts) was the dimension that mattered.

A clinician in the room hit the same wall with aggression. The target moved from a count to partial interval (scored if it happens at all). That gave a decent percent of session. But the topography (what the behavior looks like) changed, and how hard the behavior hit dropped too. The data could not show it, because the milder form still met the written definition. The fix was to add intensity (how hard or severe it is). They used a one to five rating, logged once per session at the session high.

Note what forced that change. The funder said the data showed no measurable outcome. Our graphs are the basis for funding and for medical necessity (proof the care is still needed). If we measure the wrong dimension, the win we can see with our own eyes never reaches the file.

After insurance reminded me that I wasn't showing a measurable outcome, and then I said, wait a minute, I see a measurable outcome, but it's not what I've been collecting data on.

From the talk — Pauli Catanese & Morgan Johnson

The Seven Measurement Procedures the Model Compares

The article lines up seven options. Event recording (a tally of each instance). Duration. Latency (the time from cue to response). Intensity. Permanent product (the lasting mark left behind). Partial interval. And momentary time sampling (a quick look at set moments).

Whole interval recording (scored only if it lasts the whole time) was left out of the model. It tends to underestimate behavior. Since the goal is to show a drop, a measure that already reads low is a poor fit. Partial interval leans the other way and tends to read high.

The room traded uses for each one. Partial interval worked well for tantrums. One episode could run five minutes or 30, so a count of one tells you nothing. That same client kept a separate count for single acts that did not build into a full episode. Permanent product came up for school work, daily living tasks, and damage to property.

Momentary time sampling drew the most doubt for problem behavior. Clinicians had used it in classrooms for on-task work (staying engaged with the assigned task). One had used it to set a token schedule (a system of rewards using tokens) a teacher could take over. Nobody in the room used it to track a behavior that could hurt someone.

there is again no single system that's best

From the talk — Pauli Catanese & Morgan Johnson

Walking the Flowchart: The Questions It Asks in Order

The flowchart starts with the behavior itself. Is it observable, or is it covert (it happens inside the person)? If you cannot see it, you look for a product it leaves behind. Then it asks the key question: will your chosen dimension mask the true change? That is the flat tantrum count from above.

Next come practicality and setting. A teacher watching a full classroom cannot watch one student the whole period. An interval method fits better there. Staffing is part of this too. Our field is still young, and trained staff are thin. If no one is free to count each instance, the chart gives you another road.

Then come time and safety limits. You cannot sample a few moments while someone is hurting themselves. That is why the room kept landing on a count or duration for risky behavior. After that, the chart asks what your main worry is. Worried about how long it lasts? Use duration. Worried about how often? Use a count. Worried about how hard? Use intensity.

One more branch matters. Is this a free operant (it can happen at any time) or a restricted operant (it follows a set event)? If it only shows up after a demand (an instruction to do something), watch that one moment closely. If it can happen at any time, your plan has to cover the whole session.

it's a guide it's not supposed to give you a clear answer

From the talk — Pauli Catanese & Morgan Johnson

Two Worked Examples: Elopement and Vocal Disruptions

The group ran the chart live on elopement (leaving an area without permission). Observable: yes. Discrete and countable with a clear start and end: yes. Observer resources enough to catch each one: yes. Can it happen at any time: yes. Is duration, latency, or intensity the main worry: not really. So the chart lands on a count.

The clinician had already built severity in. They tracked high risk and low risk elopement as two targets. Running the chart backed that up. It also showed where duration could still earn its keep, such as how long an elopement episode runs.

The second case came from a practice vignette in the article. A seven year old in a clinic yelled and made loud noises during work. The noise cut into instruction and into peer time. It was easy to see and hear, happened many times per session, and could be logged right after each one. The chart says event recording, so a count.

Then the room pushed back, and that is the useful part. Vocal noise is rarely one clean burst. It is often a string of short sounds with pauses in it. Several clinicians said they would move to partial interval once they watched it. The chart can tell you a behavior is countable. Only your own eyes tell you if the end of one instance is clear.

Sometimes the behavior seems like it's discrete until you actually start tracking it.

From the talk — Pauli Catanese & Morgan Johnson

Splitting a Target by Shape, Severity, or Function

The talk kept circling back to one move: split the target. When one label holds hitting, kicking, biting, and scratching, the graph swings wildly day to day. One clinician pulled biting and scratching out into their own target. Aggression then showed a clear drop, and biting stood out as most of what was left.

You can split by how severe it is too. One target for the mild form, one for the strong form, each with its own count. One clinician used low risk and high risk, since one form disrupts a classroom and the other threatens safety. If there is no clear start and stop inside an episode, partial interval does the same job.

The idea that landed hardest was splitting by function (the reason behavior keeps happening). A clinician logged vocal disruption as two separate targets. One covered escape and access (avoiding a task or getting something). The other covered automatic (the behavior feeds itself). The escape and access version stayed low. The automatic version ran high with no demands and free access to preferred items. The escape and access form dropped out first, and the automatic form took much longer.

The room compared that setup to a slow functional analysis (a test of why behavior happens). It is not one, but it sorts the data the same way. And it pays off right away.

Because if you have different functions, you want to have different interventions.

From the talk — Pauli Catanese & Morgan Johnson

Making the Model Work Inside Your Data App

Good choices still have to fit the software, and the room was blunt about the gaps. Intensity often has no field at all. The workaround: enter it as a count of one through five. Five is defined as the top level and one as the lowest. The app does not know it is intensity. You do, so you can read the graph.

Latency may have no setting either. You can run it as duration. Start the timer with the demand, then stop it when the response begins. Duration per instance has its own catch, since the app may only show a session total. The count of episodes is in there, but you have to dig for it.

Rate (count over time) was the biggest gap. Forty instances across five hours is not the same as six across one hour. If the app only shows raw counts, a long session reads like a bad day. The group tracked that by hand.

Two more field notes. If your interval scoring slips while you de-escalate, go back and edit the data. Or carry a stopwatch and run duration instead. And when the behavior changes shape, make the choice on purpose. Pick between a phase change line (a mark showing a plan change) and a brand new target. A new target means a new graph, and sometimes a fresh baseline. One team even tested interval length first. They took a week of data on both a two minute and a five minute interval. The five minute one fit the behavior better, so they kept it.

it doesn't calculate the rate for you

From the talk — Pauli Catanese & Morgan Johnson