Journal of the Experimental Analysis of Behavior
Aversive control is an important yet understudied process of learning. One reason aversive control may be relatively understudied is ethical concerns about painful stimuli (e.g., electric shock). High decibel broad-band noise and 22-kHz vocalizations both demonstrably affect rodent behavior while not necessarily being painful. The goal of this study was to determine if 100-dB 22-kHz-pure tones were differentially more effective in reducing operant response rates in rats. We examined whether 22-kHz pure tones would function as aversive stimuli, specifically as positive punishers. The effects of response-dependent as well as continuously presented 22-kHz and 1-kHz tones on rate of response maintained by variable interval 30-s food deliveries were assessed across several conditions. We found that response rates were lower when tones were presented response dependently than when tones were presented continuously throughout a session. We also found that the lower response rates obtained with response-dependent 22-kHz tones were not significantly different from response rates obtained with response-dependent 1-kHz tones. The primary conclusion of this experiment is that both 1-kHz and 22-kHz tones functioned as punishers, but that the 22-kHz tones were not differentially more effective in reducing response rate.
Noncontingent reinforcement is a commonly used procedure to decrease levels of problem behavior. Goals of this intervention are to decrease motivation, responding, and the functional relation between behavior and consequences, but it could also possibly compete with performance of alternative desirable responses. In the current study, we assessed the effects of noncontingent reinforcement arranged from 0% to 100% of sessions on performance of alternative responding across two experiments. Experiment 1 assessed manding (i.e., requests) maintained by attention and tangibles with a child with developmental disabilities and Experiment 2 assessed keypecking maintained by food with six pigeons. We extended previous research by (a) showing that noncontingent reinforcement competes with both the acquisition and maintenance (performance) of an alternative response, (b) extending the generality of the findings across nonhuman and human participants, and (c) eliminating influence of sequence effects through random manipulations of noncontingent value in pigeons. Overall, greater amounts of noncontingent reinforcement competed with both acquisition and maintenance of alternative responding.
Spatial reasoning, where novel spatial relationships are inferred based on trained relationships, can be conceptualized as arbitrarily applicable spatial relational responding. Here, we conducted two experiments to develop and validate, for the first time, a laboratory procedure to establish arbitrarily applicable spatial relational responding in adult humans. In Experiment 1, participants were trained on nonarbitrary spatial relational tasks designed to establish contextual cues for left of, right of, above, and below. Contextual cues were then used to train a series of arbitrary spatial relations involving four abstract shapes. Following training in a subset of arbitrary relations (A is left of B, B is above C, C is right of D), subsequent testing examined the emergence of untrained spatial relations (B is right of A, C is below B, D is left of C, D is below A and A is above D). When absent in initial tests, spatial relational responding was facilitated by a remedial training procedure incorporating nonarbitrary relational guidance. Participants showed patterns of spatial relational responding consistent with test relations. In Experiment 2, a variant reversal design yielded predictable, reversed spatial relational responses. Overall, the present procedures represent the first empirical demonstration of arbitrarily applicable spatial relational responding and thus, arguably, the first functional analytic model of spatial reasoning.
In a replication and extension of Conger and Killeen's (1974) widely cited demonstration of matching in conversations, we evaluated nine participants’ allocation of speech and gaze to two conversational partners. German speakers participated in two 90-min sessions in which confederates uttered approval on independent variable-interval schedules. In one of the sessions, confederates uttered approval contingent upon and contiguous with eye contact whereas in the other session approval was uttered independent of the participant's gaze. Several measures of participants’ verbal behavior were taken, including relative duration and rate of speech and gaze. These were compared to confederates’ relative rate of approval and relative duration and rate of talk. The generalized matching equation was fitted to the various relations between participants’ behavior and confederates’ behavior. Conger and Killeen's results were not replicated; participants’ response allocation did not show a systematic relation to the confederates’ relative rate of approval. The strongest relations were to overall talk, rather than approval. In both conditions, the participant talked more to the confederate who talked less—inverse or antimatching. Participants’ gaze showed the same inverse relation to the confederates’ talk. Requiring gaze to be directed toward a confederate for delivery of approval made no difference in the results. The absence of a difference combined with prior research suggests that matching or antimatching in conversations is more likely due to induction than to reinforcement.
Behavior analysis and neuroscience are disciplines in their own right but are united in that both are subfields of a common overarching field—biology. What most fundamentally unites these disciplines is a shared commitment to selectionism, the Darwinian mode of explanation. In selectionism, the order and complexity observed in nature are seen as the cumulative products of selection processes acting over time on a population of variants—favoring some and disfavoring others—with the affected variants contributing to the population on which future selections operate. In the case of behavior analysis, the central selection process is selection by reinforcement; in neuroscience it is natural selection. The two selection processes are inter-related in that selection by reinforcement is itself the product of natural selection. The present paper illustrates the complementary nature of behavior analysis and neuroscience through considering their joint contributions to three central problem areas: reinforcement—including conditioned reinforcement, stimulus control—including equivalence classes, and memory—including reminding and remembering.
Robotics is emerging as a promising tool for aiding research on animal behavior. The possibility of generating customizable, controllable, and standardized robotic stimuli has been demonstrated through a number of behavioral assays, involving vertebrates and invertebrates. However, the specific appraisal of the nature of robotic stimuli is currently lacking. Here, we attempt to evaluate this aspect in zebrafish, through a within-subject design in which experimental subjects are faced with three experimental conditions. In the first test, we investigated sociability by measuring zebrafish response to a conspecific separated by a one-way glass. In the second test, we studied zebrafish behavior in response to a 3D-printed zebrafish replica actuated along realistic trajectories through a novel four-degree-of-freedom robotic platform. Last, we investigated fear responses in a shelter-seeking test. In agreement with our expectations, zebrafish exhibited an equivalent preference for live and robotic stimuli, and the degree of preference for the robotic replica correlated negatively with the individual propensity to seek shelter. The equivalent preference for the replica and conspecific suggests that the appraisal of the target stimuli is analogous. The preliminary evidence of a correlation between behavioral responses across tests points to the readability of robotics-based approaches to investigate interindividual differences.
Resurgence is defined as an increase in the frequency of a previously reinforced target response when an alternative source of reinforcement is suspended. Despite an extensive body of research examining factors that affect resurgence, the effects of alternative-reinforcer magnitude have not been examined. Thus, the present experiments aimed to fill this gap in the literature. In Experiment 1, rats pressed levers for single-pellet reinforcers during Phase 1. In Phase 2, target-lever pressing was extinguished, and alternative-lever pressing produced either five-pellet, one-pellet, or no alternative reinforcement. In Phase 3, alternative reinforcement was suspended to test for resurgence. Five-pellet alternative reinforcement produced faster elimination and greater resurgence of target-lever pressing than one-pellet alternative reinforcement. In Experiment 2, effects of decreasing alternative-reinforcer magnitude on resurgence were examined. Rats pressed levers and pulled chains for six-pellet reinforcers during Phases 1 and 2, respectively. In Phase 3, alternative reinforcement was decreased to three pellets for one group, one pellet for a second group, and suspended altogether for a third group. Shifting from six-pellet to one-pellet alternative reinforcement produced as much resurgence as suspending alternative reinforcement altogether, while shifting from six pellets to three pellets did not produce resurgence. These results suggest that alternative-reinforcer magnitude has effects on elimination and resurgence of target behavior that are similar to those of alternative-reinforcer rate. Thus, both suppression of target behavior during alternative reinforcement and resurgence when conditions of alternative reinforcement are altered may be related to variables that affect the value of the alternative-reinforcement source.
Timeouts are sometimes used in applied settings to reduce target responses, and in some circumstances delays are unavoidably imposed between the onset of a timeout and the offset of the response that produces it. The present study examined the effects of signaled and unsignaled timeouts in rats exposed to concurrent fixed-ratio 1 fixed-ratio 1 schedules of food delivery, where each response on one lever, the location of which changed across conditions, produced both food and a delayed 10-s timeout. Delays of 0 to 38 s were examined. Delayed timeouts often, but not always, substantially reduced the number of responses emitted on the lever that produced timeouts relative to the number emitted on the lever that did not produce timeouts. In general, greater sensitivity was observed to delayed timeouts when they were signaled. These results demonstrate that delayed timeouts, like other delayed consequences, can affect behavior, albeit less strongly than immediate consequences.
Effects of shifts in food reinforcement context on rats’ consumption of concurrently available water or sucrose solution
The purpose of this study was to investigate the effects of signaled transitions from relatively rich to lean conditions of food reinforcement on drinking concurrently available water or sucrose-sweetened water in rats. Past research demonstrated that these negative incentive shifts produce behavioral disruption in the form of extended pausing on fixed-ratio schedules. Four male Long-Evans rats operated on a two-component multiple fixed-ratio fixed-ratio schedule. In one manipulation, the ratio was held constant and the components arranged either a large six-pellet reinforcer (rich) or small one-pellet reinforcer (lean). In a second manipulation, the components both produced a one-pellet reinforcer but differed in terms of the ratio requirement, with the rich and lean conditions corresponding to relatively small and large ratios. In both manipulations, components were pseudorandomly presented to arrange four transitions signaled by retractable levers: lean-to-lean, lean-to-rich, rich-to-rich, and rich-to-lean (the negative incentive shift). During experimental conditions, a bottle with lickometer was inserted in the chamber, providing concurrent access either to tap water or a 10% sucrose solution. The negative incentive shift produced considerably more drinking than the other transitions in all rats during both manipulations. The level of drinking was not polydipsic; rather, it appears that the negative incentive shift enhanced the value of concurrently available reinforcers relative to food reinforcement.
Are positive and negative reinforcement “different”? Insights from a free-operant differential outcomes effect
Although theoretical discussions typically assume that positive and negative reinforcement differ, the literature contains little unambiguous evidence that they produce differential behavioral effects. To test whether the two types of consequences control behavior differently, we pitted money-gain positive reinforcement and money-loss-avoidance negative reinforcement, scheduled through identically programmed variable-cycle schedules, against each other in concurrent schedules. Contingencies of response-produced feedback, normally different in positive and negative reinforcement, were made symmetrical. Steeper matching slopes were produced compared to a baseline consisting of all positive reinforcement. This free-operant differential outcomes effect supports the notion that that stimulus-presentation positive reinforcement and stimulus-elimination negative reinforcement are functionally “different.” However, a control experiment showed that the feedback asymmetry of more traditional positive and negative reinforcement schedules also is sufficient to create a “difference” when the type of consequence is held constant. We offer these findings as a small step in meeting the very large challenge of moving negative reinforcement theory beyond decades of relative quiescence.
In a frequently used suboptimal-choice procedure pigeons choose between an alternative that delivers three food pellets with p = 1.0 and an alternative that delivers ten pellets with p = 0.2. Because pigeons reliably choose the probabilistic (suboptimal) alternative, the procedure has been proposed as a nonhuman analog of human gambling. The present experiments were conducted to evaluate two potential threats to the validity of this procedure. Experiments 1 and 2 evaluated if pigeons obtained food at a lower unit price (i.e., pecks per pellet) on the suboptimal alternative than on the optimal alternative. When pigeons worked under this suboptimal procedure they all preferred the suboptimal alternative despite some pigeons paying a higher price for food on that alternative. In Experiment 2, when the unit price ratio more closely approximated the inverse of the expected value ratio, pigeons continued to prefer the suboptimal alternative despite its economic suboptimality. Experiment 3 evaluated if, in accord with the string-theory of gambling, the valuation of the suboptimal alternative was increased when pigeons misattributed a subset of the suboptimal no-food trials to the optimal alternative. When trial sequences were arranged to minimize these possible attribution errors, pigeons still preferred the suboptimal alternative. These data remove two threats to the validity of the suboptimal choice procedure; threats that would have suggested that suboptimal choice reflects economic maximization.
Despite the success of exposure-based psychotherapies in anxiety treatment, relapse remains problematic. Resurgence, the return of previously eliminated behavior following the elimination of an alternative source of reinforcement, is a promising model of operant relapse. Nonhuman resurgence research has shown that higher rates of alternative reinforcement result in faster, more comprehensive suppression of target behavior, but also in greater resurgence when alternative reinforcement is eliminated. This study investigated rich and lean rates of alternative reinforcement on response suppression and resurgence in typically developing humans. In Phase 1, three groups (Rich, n = 18; Lean, n = 18; Control, n = 10) acquired the target response. In Phase 2, target responding was extinguished and alternative reinforcement delivered on RI 1 s, RI 3 s, and extinction schedules, respectively. Resurgence was assessed during Phase 3 under extinction conditions for all groups. Target responding was suppressed most thoroughly in Rich and partially in Lean. Target responding resurged in the Rich and Lean groups, but not in the Control group. Between groups, resurgence was more pronounced in the Rich group than the Lean and Control groups. Clinical implications of these findings, including care on the part of clinicians when identifying alternative sources of reinforcement, are discussed.
“Watch out!”: Effects of instructed threat and avoidance on human free-operant approach–avoidance behavior
Approach–avoidance paradigms create a competition between appetitive and aversive contingencies and are widely used in nonhuman research on anxiety. Here, we examined how instructions about threat and avoidance impact control by competing contingencies over human approach–avoidance behavior. Additionally, Experiment 1 examined the effects of threat magnitude (money loss amount) and avoidance cost (fixed ratio requirements), whereas Experiment 2 examined the effects of threat information (available, unavailable and inaccurate) on approach–avoidance. During the task, approach responding was modeled by reinforcing responding with money on a FR schedule. By performing an observing response, participants produced an escalating “threat meter”. Instructions stated that the threat meter levels displayed the current probability of losing money, when in fact loss only occurred when the level reached the maximum. Instructions also stated pressing an avoidance button lowered the threat level. Overall, instructions produced cycles of approach and avoidance responding with transitions from approach to avoidance when threat was high and transitions back to approach after avoidance reduced threat. Experiment 1 revealed increasing avoidance cost, but not threat magnitude, shifted approach–avoidance transitions to higher threat levels and increased anxiety ratings, but did not influence the frequency of approach–avoidance cycles. Experiment 2 revealed when threat level information was available or absent earnings were high, but earnings decreased when inaccurate threat information was incompatible with contingencies. Our findings build on prior nonhuman and human approach–avoidance research by highlighting how instructed threat and avoidance can impact human AA behavior and self-reported anxiety.
A second type of magnitude effect: Reinforcer magnitude differentiates delay discounting between substance users and controls
Basic research on delay discounting, examining preference for smaller–sooner or larger–later reinforcers, has demonstrated a variety of findings of considerable generality. One of these, the magnitude effect, is the observation that individuals tend to exhibit greater preference for the immediate with smaller magnitude reinforcers. Delay discounting has also proved to be a useful marker of addiction, as demonstrated by the highly replicated finding of greater discounting rates in substance users compared to controls. However, some research on delay discounting rates in substance users, particularly research examining discounting of small-magnitude reinforcers, has not found significant differences compared to controls. Here, we hypothesize that the magnitude effect could produce ceiling effects at small magnitudes, thus obscuring differences in delay discounting between groups. We examined differences in discounting between high-risk substance users and controls over a broad range of magnitudes of monetary amounts ($0.10, $1.00, $10.00, $100.00, and $1000.00) in 116 Amazon Mechanical Turk workers. We found no significant differences in discounting rates between users and controls at the smallest reinforcer magnitudes ($0.10 and $1.00) and further found that differences became more pronounced as magnitudes increased. These results provide an understanding of a second form of the magnitude effect: That is, differences in discounting between populations can become more evident as a function of reinforcer magnitude.
We propose quantitative experimental approaches to the question of whether positive and negative reinforcement are functionally different, and discuss scientific and ethical concerns that would arise if these approaches were pursued.
Extended pausing during discriminable transitions from rich-to-lean conditions can be viewed as escape (i.e., rich-to-lean transitions function aversively). In the current experiments, pigeons’ key pecking was maintained by a multiple fixed-ratio fixed-ratio schedule of rich or lean reinforcers. Pigeons then were provided with another, explicit, mechanism of escape by changing the stimulus from the transition-specific stimulus used in the multiple schedule to a mixed-schedule stimulus (Experiment 1) or by producing a period of timeout in which the stimulus was turned off and the schedule was suspended (Experiment 2). Overall, escape was under joint control of past and upcoming reinforcer magnitudes, such that responses on the escape key were most likely during rich-to-lean transitions, and second-most likely during lean-to-lean transitions. Even though pigeons pecked the escape key, they paused before doing so, and the latency to begin the fixed ratio (i.e., the pause) remained extended during rich-to-lean transitions. These findings suggest that although the stimulus associated with rich-to-lean transitions functioned aversively, pausing is more than simply escape responding from the stimulus.
Children's preference for mixed- versus fixed-ratio schedules of reinforcement: A translational study of risky choice
Laboratory research has shown that when subjects are given a choice between fixed-ratio and bi-valued mixed-ratio schedules of reinforcement, preference typically emerges for the mixed-ratio schedule even with a larger ratio requirement. The current study sought to replicate and extend these findings to children's math problem completion. Using an ABCBC reversal design, four fourth-grade students were given the choice of completing addition problems reinforced on either a fixed-ratio 5 schedule or one of three mixed-ratio schedules; an equivalent mixed-ratio (1, 9) schedule, a mixed-ratio (1, 11) schedule with a 20% larger ratio requirement, and an equally lean mixed-ratio (5, 7) schedule without the small fixed-ratio 1 component. This was followed by a reversal back to the preceding phase in which preference for the mixed-ratio schedule had been observed, and a final reversal back to the mixed-ratio (5, 7) phase. Findings were consistent with previous research in that all children preferred the mixed-ratio (1, 9) schedule over the equivalent fixed-ratio 5 schedule. Preference persisted for the leaner mixed-ratio (1, 11) schedule for three of the four children. Indifference or preference for the fixed-ratio 5 alternative was observed in phases containing the mixed-ratio (5, 7) schedule. These results extend previous research on risky choice to children's math problem completion and highlight the importance of a small ratio component in the emergence of preference for bi-valued mixed-ratio schedules. Implications of these results for arranging reinforcement to increase children's academic responding are discussed.