Interobserver Agreement (IOA): A Practical Guide

Interobserver agreement, often called IOA, checks whether your data can be trusted. It asks a simple question. If two people watched the same behavior, would they record the same thing? When they agree, your data is more believable.

This matters for every BCBA, RBT, teacher, and parent who tracks behavior. Decisions rest on that data. Programs change based on it. If the numbers are shaky, the whole plan is shaky too. IOA is how you catch that early.

Why IOA is worth your time#

IOA is not just a research rule. It is a basic quality check on clinical work. Stephanie Peterson pushes hard on this idea.

Shouldn't we be taking IOA data anyway on our clinical practice? Like if we're not taking IOA data on our clinical practice, that's probably not a good thing. From the talk. Stephanie Peterson

Think about the choices that flow from your data. Some lead to medication changes or restrictive plans. Those are big calls. They deserve data you can defend, not data one person guessed at.

IOA as a first thing to rule out#

A supervisor rarely sees every session. That gap creates uncertainty. When the data looks strange, IOA is a smart first check.

Matt Harrington describes this reality plainly.

is my data accurate as a supervisor? I don't get to see a hundred percent of the sessions. I get to see 20%, 30% if I can slide it through Medicaid. From the talk. Matt Harrington

Before you build a complex clinical story, test the simple thing first. Maybe the data is off because two people scored it differently. He suggests a short, focused check.

I'm going to take IOA for a little bit. Give me a week. Let me rule out IOA. From the talk. Matt Harrington

This is the parsimonious move. Rule out the simple problem before chasing a hard one. A week of IOA is cheap. A wrong clinical guess can cost much more.

Low agreement points to a fixable issue. Maybe the target behavior needs a clearer definition. Maybe two staff read the same act in two ways. Once you find the gap, you can retrain and move on with trust in your data.

Using IOA to coach your team#

IOA is also a teaching tool. It gives supervisors a clear, trackable metric for feedback. Mellanie Page treats it as a core supervision skill. She pairs IOA collection with behavioral skills training when placing staff with a child. You can even have an RBT score IOA on a recording of their own work. This turns a data check into a growth moment.

You can ask them to take IOA data as they watch with you so they can pinpoint areas of consistency with expectation and then areas where they need to make improvements. From the talk. Mellanie Page

Now the RBT sees exactly where their scoring drifts. That is a concrete target for growth. It also builds a habit of self-checking. Over time, staff learn to catch their own errors early.

IOA fits neatly into behavioral skills training, often called BST. You model the scoring, practice it, and give feedback. The IOA number gives that feedback a clear anchor. It turns a vague "do better" into a specific goal.

When getting IOA feels impossible#

Many practitioners say they cannot collect IOA. There is no second observer and no time. Peterson hears this all the time. Teams tell her they cannot publish because they lack IOA data and have no capacity to get it.

There are real workarounds. You can score archived video later. You can use permanent products, like completed worksheets, that stay put. You can train an aide who is already in the room.

Sometimes IOA truly is not possible. Amber Valentino shared one honest example from her own work.

This is the only study I've ever had published. It didn't have IOA, which I know sounds a little funny to you, but it was a sensitive thing. We were teaching something very private and personal, and we just couldn't justify having another person in there or video recording. From the talk. Amber Valentino

So privacy can override IOA in rare cases. But that should be the exception, not the habit.

What the research says#

Practitioners collect IOA on client behavior far more than on their own procedures. One review looked at studies in two major journals. Almost all reported IOA for participant behavior. Very few reported agreement for their procedural fidelity data (Essig, L., Rotta, K., & Poling, A. (2022). Interobserver agreement and procedural fidelity: An odd asymmetry). That gap means we check the child's data but rarely check whether we ran the plan correctly.

Good news for busy clinicians comes from a study on how much IOA is enough. Using functional analysis data, researchers found no meaningful difference in IOA across different numbers of scored sessions with highly trained observers (Hausman, N. L., et al. (2022). Interobserver agreement: A preliminary investigation into how much is enough?). Consistency mattered more than sheer volume.

Even untrained observers can sometimes hit acceptable agreement. At a wolf sanctuary, untrained visitors recorded wolf behavior on their phones. Their IOA reached 80% or higher across the board (Anckner, C. M., & Jacobs, K. W. (2024). Interobserver Agreement among a Staff Member and Visitors at a Wolf Sanctuary). A stricter measure fell short for some who tracked two behaviors at once. This hints that simple targets are easier to score reliably.

FAQ#

What is a good IOA percentage in ABA? Many practitioners aim for 80% agreement or higher. The right target depends on the behavior and the setting. Consistency across sessions matters as much as any single score.

How do you calculate interobserver agreement? Two people record the same behavior at the same time. You then compare their records and figure out how often they matched. Different methods exist, such as exact agreement and interval-by-interval agreement.

How often should you collect IOA? There is no single rule for every case. Research suggests that steady, consistent checks matter more than a huge number of sessions. Many teams collect IOA on a set share of sessions across the program.

If you want the supervision angle, Mellanie Page turns IOA into coaching in Dunder Mifflin's Guide to BCBA Supervision: Lessons from The Office. Matt Harrington connects data checks to daily practice in Research to practice - extending past the pages.