Teaching to the Test in ABA: Why Scores Can Mislead
Teaching to the test means drilling the exact items on an assessment. Scores go up, but real skills and daily life may not change. Here is why.
Key takeaway
Teaching to the test means teaching a learner the exact items on a skills assessment, instead of teaching the broader skill. The learner's score goes up.
genArete: Milestone based comparison criteria in Skill Assessment
Free learning path on this topic
Child Development for Behavior Analysts
Typical development for behavior analysts. Kristen Byra's overview, then Kelly Brzak's deep dives on ages 2 to 5, 6 to 8, and 9 to 11, with milestones, common problem behaviors, and programming tips for each band.
Experts who cover this
2 recordings on openceu cover teaching to the test ABA.
- Featured talk1 Learning CEU
genArete: Milestone based comparison criteria in Skill Assessment
Mark MaladyThe Behaviorist Bookclub
Malady describes new clinicians filling in blank assessment cells by directly teaching the missing items (sometimes using the assessment itself as the teaching…
- 1 Learning CEU
Mastering the VB-MAPP: A Deep Dive into the Milestones, Barriers, and Transitions Assessments
Dr. Jessica OsosCopper Consulting Group
Presented as a common clinical pitfall: using VB-MAPP/ESA assessment items directly as teaching targets, which devalues the assessment's sampling purpose and…
On this page · 9 sections▾
Teaching to the test means teaching a learner the exact items on a skills assessment, instead of teaching the broader skill. The learner's score goes up. But the change may not show up anywhere else in their life.
In ABA, this pitfall often appears after tools like the VB-MAPP are run. The VB-MAPP is a common assessment that checks a child's language and social skills. A clinician sees blank spots on the grid and works to fill them. Two presenters, Mark Malady and Dr. Jessica Osos, describe how this happens and why it matters.
What teaching to the test looks like#
Mark Malady says this worry has come up in general talks for about ten years. It is often voiced about new clinicians. The pattern is simple. A clinician runs an assessment. Then they make a list of every skill the learner missed. Then they teach that list, item by item.
Sometimes it goes one step further. The clinician uses the assessment's own testing method as the way to teach. So the learner practices the same task, in the same format, that the test will later check.
new clinicians will run an assessment and then they'll just kind of start creating a list of all the skills that are missing. Sometimes they just directly teach to that list. They might even use the actual assessment methodology to teach a skill.
From the talk — Mark Malady
Malady sums up the result in a short phrase. Filling the space becomes the goal. The blank cells on the form stop being a clue about the learner. They turn into a to-do list for the clinician. That shift is the heart of teaching to the test.
Why scores rise while daily life stays the same#
The trouble is not that scores go up. The trouble is what the higher score hides. Malady describes the learner scoring higher six months later on the same assessment. On paper, it looks like progress. The form is filling up.
Then someone asks a different question. How is the learner really doing? Are they having new experiences in their life? Malady points to the Vineland here. The Vineland is a broad, or "global," measure of everyday functioning. It looks at the whole person, not one test grid.
when we actually ask, hey, how are they doing? Are they experiencing new things in their life? When we look at really global measures like the Vineland, we then don't see any alteration or any meaningful differentiation being reported by support network members or their family.
From the talk — Mark Malady
So the family and support network do not report real change. The score moved, but the life did not. This is the core danger. A rising number can make a team feel good while the learner gains little that matters outside the session.
How the shape of a tool pulls clinicians along#
Malady does not blame new clinicians alone. He looks at the tools themselves. Many skill assessments show information in a fixed visual layout. He describes a "center to outer" arrangement. That layout creates a linear path, meaning a straight line of steps from one level to the next.
Lines carry a message for people. Malady notes that we learn this long before we become behavior analysts. We learn it as complex verbal organisms, or simply as people who use language. When you see a line, you feel you should fill it. You move from one step to the next step to the next step.
He also names a second problem. The X and Y axes, the two edges of a chart, are set ahead of time. Clinicians do not choose how the skills are grouped or shown. That limits their choices. It makes their actions more predictable. Malady says predictable action can be good at times. But it can also do harm when the predictable action is not the action the learner needs. The tool, in other words, can quietly steer the clinician toward filling cells.
Why trained behavior analysts still fall into it#
This part puzzles Malady most. Behavior analysts learn about generalization early in their training. Generalization means a skill shows up in new places, with new people, and with new materials. They also learn about related ideas like extension and induction. These are ways one learned skill spreads into other, similar skills.
how do we get to the point of teaching to a test when we as behavior analysts understand the complexities of generalization and extension, induction, all these great concepts that we learn in our early foundational education, how did we accidentally fall into this pattern?
From the talk — Mark Malady
Malady's answer is, in his words, "way too simple." Our tools tap into other parts of our history. That history evokes the action of filling the line. In plain terms, the form prompts a habit, and the habit wins over the training.
This framing matters. It moves the fix away from blaming people. Instead, it asks the field to look hard at its tools. Malady invites clinicians to ask whether a given feature serves them. If it does not, he suggests building other options.
A sampling tool is not a curriculum#
Dr. Jessica Osos comes at the same pitfall from a hands-on angle. She talks about the ESA, the echoic skills tool found inside the VB-MAPP. "Echoic" means repeating sounds or words that someone else says. Osos calls using ESA targets to set echoic goals one of the most common pitfalls she sees.
Her key point is clear. The ESA is a sampling tool. It is not a curriculum. A sampling tool checks a small set of items to give you a picture of a bigger skill. A curriculum is the full plan of what to teach. Mixing these two up is where teaching to the test begins.
Osos explains what the ESA is good for. It helps you decide whether echoic training should be a priority. It also shows which areas are weak. For example, it can hint at which sounds are harder or easier for that child. That is useful information. But it is information for planning, not a list of goals.
I would not recommend using the targets directly from the ESA as teaching targets for echoic. Cause then you're teaching to the test.
From the talk — Dr. Jessica Osos
What gets lost when you teach the sample#
Osos names two costs. First, it devalues the assessment. If the learner practices the exact test items, they may master them. Then the true reason for giving the assessment loses its value. The test can no longer tell you what it was built to tell you.
Second, the skill may not spread. Osos notes that the child may not be able to echo other similar sounds that hit the same skill. If you only teach the sampled targets, you may get only those targets. The broad skill the sample was meant to stand for may still be missing.
This lines up closely with what Malady describes. In his account, scores rise but global measures do not change. In Osos's account, test items get mastered but similar sounds do not follow. Both describe the same gap. The learner gets better at the test, not at the skill the test was meant to sample.
Where the two experts agree#
Malady and Osos speak in different talks with different focus. Malady looks at how assessment layouts shape clinician behavior. Osos looks at one specific tool and one specific skill area. Still, they land in the same place. Neither one defends teaching the test items. There is no real disagreement between them to report.
Their views fit together well. Malady explains why the pitfall is so easy to fall into. The design of the tool invites it. Osos explains how to spot it in daily work. When you finish the ESA and feel the pull to copy its targets, stop. Use the results to guide priorities instead.
Taken together, they give a fuller picture. One speaker covers the cause. The other covers the clinical moment where the mistake happens. Both point to the same risk: a score that grows without real, useful change for the learner.
Choosing better targets instead#
So what should a clinician do? Osos offers a direct answer. Use the assessment to decide whether a skill area should be a priority. Then pick targets based on functional communication needs. Pick what is socially significant for that child, meaning what will truly matter in their life.
She stresses one word: functional. Echoic targets should be functional and fit the child's development. She also reminds clinicians to think about culture and language. Consider the client and their family unit when doing assessments.
Osos adds that no single tool tells the whole story. When you combine several assessments, you get a holistic view of the child's learning profile. That fuller view helps you choose goals that matter. For a deeper look at the VB-MAPP, see the course Mastering the VB-MAPP.
Malady's advice works at the level of tools. Ask whether a feature of your assessment serves you. If it pushes you toward filling space, look for other options. And check progress against broad measures and family reports, not just the grid.
FAQ#
What does "teaching to the test" mean in ABA?
It means teaching a learner the exact items on a skills assessment. The clinician fills in blank cells by drilling those items. Scores rise, but the broader skill may not grow. Mark Malady notes that global measures like the Vineland may show no real change.
Why is it a problem if the learner's score goes up?
The score stops meaning what it should. Dr. Jessica Osos explains that tools like the ESA are sampling tools. If the learner masters the sampled items, the test loses its value. The child may still struggle with similar skills that were never taught.
How can clinicians avoid teaching to the test?
Use assessment results to set priorities, not to write the goal list. Choose functional, socially significant targets for each learner. Consider family, culture, and language. Combine several assessments for a fuller view. Also check progress with broad measures and family reports.
Turn this topic into a CEU
You just studied this. Now get credit for it.
Watch genArete: Milestone based comparison criteria in Skill Assessment with Mark Malady, BCBA and earn 1 free BCBA CEU. Detailed certificate, delivered the moment you finish.
Keep going with a free learning path
Child Development for Behavior Analysts
Typical development for behavior analysts. Kristen Byra's overview, then Kelly Brzak's deep dives on ages 2 to 5, 6 to 8, and 9 to 11, with milestones, common problem behaviors, and programming tips for each band.