Logan:  Basic Principles of Learning

                PRINCIPLE: SECONDARY NEGATIVE REINFORCEMENT

          There will be an increase in the likelihood of a response 
          that is shortly followed by the termination of a stimulus
          that has previously been associated with an aversive state 
          of affairs.

     The Principle of Secondary Negative Reinforcement is typically
studied in the context of AVOIDANCE CONDITIONING, in which the organism
learns to emit a response that prevents the occurrence of some aversive
event such as an electric shock.  In the laboratory, for example, a rat
might be placed in a box containing a grid floor and a small bar 
protruding from the wall.  The rat is scheduled to receive an occasional
electric shock through the grid floor unless he presses the bar at the
appropriate times.  He can, therefore, avoid the shock by pressing the bar.

     There are two basic procedures in avoidance conditioning: cued and
non-cued.  In general, a cue is a stimulus that conveys information, and
in the CUED AVOIDANCE procedure, some exteroceptive stimulus event, such
as the onset of a light or a tone, precedes the scheduled occurrence of
the shock.  If the rat presses the bar after this warning signal has
occurred and before the shock is delivered, the warning signal goes off
and the shock does not occur.  In the NON-CUED AVOIDANCE procedure, there
is no warning signal and the shock is simply scheduled to occur on a
regular basis, such as every thirty seconds.  In this situation, pressing
the bar postpones the next scheduled shock for some length of time.  In
either situation, a perfectly adaptive organism could avoid all aversive
events by emitting the avoidance response (AR) at appropriate times.  Few
organisms are quite that successful, but avoiding over 90% of the
scheduled shocks is a common laboratory finding.

     Which fact raises the "avoidance paradox".  How can a response be
maintained if it is followed by nothing happening?  We may view the
aversive stimulus as the unconditioned stimulus in the avoidance
conditioning paradigm, but whenever an anticipatory response occurs, the
US does not occur.  According to the Principle of Experimental
Extinction, the non-occurrence of the US should lead to a decrease rather
than an increase in the likelihood of the response.  The solution of the
avoidance paradox also explains why the principle is called "secondary
negative reinforcement".

     Consider first what happens on nonavoidance trials, i.e., trials
when the AR fails to occur.  The antecedent stimulus is followed by the
aversive stimulus, which does at least two and perhaps three things.
First, it elicits any typical unconditioned response innately associated
with the US.  Second, it arouses the motivational state of fear.  Finally,
it may lead on to an escape response (ER) depending on the specific
procedure used in the situation.  (Sometimes the aversive event is very
brief and its duration is independent of the organism's behavior.)
According to the Principle of Classical Conditioning, each of these
responses should tend to become anticipatory and occur with the onset of
the S and prior to the onset of the US.
 
     For our immediate purposes, the critical component of this
conditioning process is that the antecedent stimulus, the warning signal,
becomes a secondary motivating, fear-arousing stimulus in its own right.
This means that the AR is reinforced by the fear reduction resulting from
the termination of the warning S.  This type of reinforcement is
"negative" because it results from the termination of a stimulus, and it
is "secondary" because the stimulus has acquired its motivating
properties.  Where turning shock off is primary negative reinforcement,
turning off a stimulus associated with shock is secondary negative
reinforcement.

     Note, however, that not only fear, but the UR to the aversive event,
and any required ER also tend to become anticipatory.  The effects of
these will depend on whether they are compatible or incompatible with the
AR.  It is easier to train an AR that is the same as the escape response
and it is difficult to train an AR that is incompatible with the
unconditioned response to the scheduled aversive event.

     This latter effect is somewhat easier to understand if the US is a
beneceptive (pleasant) one rather than an aversive one.  Specifically,
suppose we modify the Pavlovian classical conditioning paradigm, in which
a metronome regularly precedes the delivery of food, with the rule that,
if the dog begins to salivate before the food US delivery, it will not be
given.  This is called OMISSION TRAINING; the US is omitted if a CR
occurs.  The dog finds it very difficult to learn not to salivate as an
anticipatory CR.  That is to say, a not-salivating CR is incompatible
with a salivation UR, leading to continually conflicting tendencies.

     An especially important factor in avoidance conditioning is what has
been termed species specific defense reactions.  Most organisms have some
innate responses to danger signals, and attempting to train an avoidance
response that is incompatible with these defense reactions will likely
fail.  For example, a rat tends to freeze motionless when afraid, and
hence may have difficulty learning to press a bar to avoid shock.

     Yet another factor that affects the ease of avoidance conditioning
is past experience.  If an organism is first exposed to an extensive
series of unavoidable, inescapable aversive events, then that organism
will at least be retarded, and perhaps may never learn an avoidance
response when given an opportunity to do so.  This effect is termed
LEARNED HELPLESSNESS. . . having initially learned that nothing one does
has any effect, it is difficult later to learn in a situation in which
one's responses do have an effect.

     Avoidance behavior is an extremely common feature of our everyday
lives.  Locking one's house or car are avoidance responses intended to
prevent the aversive event of a theft.  Putting on adequate clothing for
a winter's day, paying one's utility bill, and even using contraceptives
are avoidance responses maintained by secondary negative reinforcement.

TERMS: Avoidance conditioning, cue, learned helplessness, omission
training, reinforcement (secondary, negative), avoidance response.