Clean Run - August 2017 - 16

extinction were our goal, it's difficult
for humans to perform the necessary
randomized schedules. Third, in the
real world there are many alternative
sources of reinforcement (we call them
distractions). That means even when
done correctly, the possible value of a
variable reinforcement schedule can
be demolished by something called the
Matching Law.

Extinction Trials
Much of the information we have on
the effects of variable schedules comes
from lab experiments called extinction
trials (Mowrer & Jones, 1945). An animal confined in a small area is trained
to perform a behavior. A monkey may
press a lever, a pigeon may peck a disk,
or a rat may run down a chute to jump
on a platform at the end. The animal
performs the behavior repeatedly, and
the behavior is reinforced each time.
After these reinforced repetitions, the
reinforcement schedule is thinned according to a preplanned formula. Reinforcement is gradually reduced, and in
some experiments taken down to zero.
The pattern of the animal's response is
recorded under these conditions of reduced reinforcement.
It's true that many studies have shown
that a variable reinforcement schedule
is comparably more resistant to extinction. (Although it's important to note
that not all studies have shown that. Recent studies have shown that richer reinforcement schedules can lead to better
resistance to extinction, a phenomenon
called "behavioral momentum" (Nevin,
Mandell, & Atak, 1983).)
When a behavior is resistant to extinction, an animal will keep performing
it as reinforcement becomes sporadic.
But there is no guarantee that the behavior will happen when we want it. A
behavior that is resistant to extinction
is only more likely to be performed after decreased reinforcement. Also, just
because a behavior is resistant to extinction doesn't mean that it will be performed eagerly, enthusiastically, or with
low latency. These are all qualities we
value and need in our dogs' behavior.
Finally, many extinction trials are
performed in what is called a "free

operant" setup. In this setup there is a signal to the animal that reinforcement may
be available for a certain behavior the animal has already learned. The signal stays
on for a period and the animal is free to perform the behavior multiple times. The
performances of the behavior are reinforced or not, according to the schedule. But
counting free operant responses yields data that have little relevance to most of our
training situations. Real world training usually incorporates what are called "discrete
trials." That is, we give one cue and we need the dog to perform a behavior right
then. If instead, the dog waits 90 seconds and then performs it three times, those
would count as "responses" in a free operant trial. In the lab, they would count towards "resistance to extinction." But in real life they wouldn't help us at all (if we
happened to wait long enough to find out about them).

Random Schedules
In order to attempt to get the resistance to extinction that can be tied to variable reinforcement schedules, we need to follow a precise plan.
First, we need to train the behavior to fluency. Behavioral fluency is defined as a
combination of accuracy plus speed of responding (Binder, 1996). Fluency is a much
bigger challenge in the real world than in the lab because we need our dogs to be
able to respond in so many different situations. There's a lot of generalization work
to do before we can reduce the schedule. And our dog might never achieve the fluency that an animal alone in a Skinner box could.
Second, after the behavior is fluent and generalized, we need to change the schedule gradually. That's one thing the science is in agreement on. For instance, if we
changed from continuous reinforcement directly to a schedule where the dog was
reinforced every eighth time on average, the dog would likely give up rather than
transitioning to the new schedule (Schwartz, 2002, p. 219). Resistance to extinction
only occurs if the thinning of the schedule is gradual.
And consider what withholding reinforcement means to the dog. When teaching a
new behavior, we withhold reinforcement when the dog responds incorrectly. But
when we switch to a variable schedule, we will withhold reinforcement when the dog
responds correctly. We need a plan for explaining the new rules to the dog. Removal
of reinforcement is a known cause of frustration and even aggression in animals.
Finally, we need a method to compute and track the schedule. It must average the
right number of reinforcements and must be random.
Randomizing is hard for humans. Let's say we've decided that our goal is to reinforce the dog for one out of every four sits, that is, 25% of the time. But it has to be
random. So if the dog is going to sit 20 times, we will plan to reinforce five of them,
but we can't do it in a pattern.
If we try to wing it, we'll likely become predictable. That's what humans do. We
may reinforce more often in the kitchen than in the den, or more often when the dog
looks at us a certain way. Or we'll reinforce when the dog sits in a more difficult
situation and consistently skip it during the easier times. And the dog will learn the
pattern, because that's what dogs do. So in those situations in which we tend not to
reinforce, they will tend not to respond.
How would we address this problem? By preparing beforehand. We can use a random number generator or do it by hand. For example, we could plan ahead to reinforce sits #3, #9, #10, #15, and #18 of the dog's first 20 sits of the day.
After we've memorized the sit numbers, what about the times we ask the dog for eye
contact or to get on a mat? We will need a plan for those, too. Good luck with memorizing all that.
This is no joke. The data about the effects of variable reinforcement come from precisely computed schedules. If we are going to try to use variable ratio reinforcement,
we need to use the methods that make it work.

Clean Run | August 17

Table of Contents for the Digital Edition of Clean Run - August 2017