Fixed-interval pause duration in chained fixed-ratio , fixed-interval schedules

Ten rats were submitted to chained fixed-ratio (FR), fixed-interval (FI) schedules of reinforcement. A FR schedule at one lever produced a discriminative stimulus associated with a FI 60-s schedule of primary reinforcement (water) at the second response lever. In Experiment 1, the FI schedule was kept constant while the FR requirement was changed from one to seven responses under five different experimental conditions for five rats. Increases in the FR requirement resulted in increases in postreinforcement pauses but also decreases in pauses in the FI schedule. Using another five rats, Experiment 2 tested the hypothesis that short pauses in the FI schedule result from the use of the chained schedules procedure. Baseline was a FI 80-s schedule. In the second condition, chained FR 1 FI 80-s schedules were programmed. The third condition was a return to baseline. In baselines 1 and 2, the FI pause was compatible with the literature but decreased considerably when a chained schedule was used. The present results support the hypothesis that the time between primary reinforcement presentations dominates the control of FI pauses over control by the onset of a discriminative stimulus.


Introduction
A review published by Lejeune, Richelle, and Wearden (2006) calls our attention to the importance of the work with the fixed-interval (FI) schedules of Ferster and Skinner (1957) for the study of interval timing in animals (e.g., Staddon & Cerutti, 2003).Timing behavior or the temporal control of behavior is the capacity to adjust behavior to temporal regularities in the environment in the range of seconds to minutes (Machado, Malheiro, & Erlhagen, 2009).Time as part of a three-term reinforcement contingency has been used as a discriminative stimulus (e.g., Arantes & Machado, 2008), definition of responding (e.g., Bejarano & Hackenberg, 2007), and parameter of the reinforcing stimuli (e.g., Todorov, 1973;Alsop & Porritt, 2006), but time as a discriminative stimulus that defines FI schedules has been most studied.Reviews of temporal control in FI schedules have been published during the past 50 years (e.g., Felton & Lyon, 1966;Dews, 1970;Davison, 1974;Shull, 1979;Lowe & Wearden, 1981;Lejeune & Wearden, 1991;Aparício, Lopez & Nevin, 1995;Machado, 1997;Lejeune et al., 2006).
In FI schedules, the possibility of a reinforcing stimulus that follows a response depends on the time since the last reinforcement or since the presentation of a discriminative stimulus (e.g., Ferster & Skinner, 1957;Catania, 1984).The two procedures are named "fixedinterval schedules" either in single or complex schedules as concurrent chains (e.g., Killeen, 1970;Davison, 1976;Moore, 2009).However, typical studies of temporal control use FI schedules defined as the time since the last reinforcement.One of several possibilities of studying timing behavior in FI schedules when the time to the next reinforcement is signaled by some event other than primary reinforcement is chained schedules with FI as the second component (Ferster & Skinner, 1957).Pilot studies (de Souza & Todorov, 1975;Todorov & Teixeira-Sobrinho, 2009) showed an unexpected result in which pauses in the second component of chained fixed-ratio (FR) FI schedules were shorter than typical pauses in single FI schedules (e.g., Sherman, 1959;Schneider, 1969).Pausing appeared to be controlled by the interreinforcement interval (variable) and not by the fixed time to the next opportunity for reinforcement signaled by a discriminative stimulus.Todorov and Teixeira-Sobrinho (2009) studied chained FR n FI 60-s schedules with the FI 60-s kept constant while the FR increased from one to eight responses and reported that pausing in the FI schedule decreased with increases in the FR requirement.Post-reinforcement pauses (PRPs) increased as the FR increased, systematically changing the interval between primary reinforcements.An experiment conducted by de Souza and Todorov (1975) used a similar procedure with rats and reported the effect on PRPs and the absence of pauses during the FI schedule.The present study was a replication and extension of these experiments that sought to understand how and why the discriminative stimulus that signals the constant FI duration does not control pausing in chained schedules while retaining its conditioned reinforcer ability to maintain FR responding.

Subjects
Five naive, male Wistar rats aged 6 months at the beginning of the experiment (described as rats 1, 3, 5, 6, and 7) were used.Rats were born and maintained in the vivarium of the Instituto de Educação Superior de Brasilia and were housed individually in polycarbonate cages (30 × 30 × 50 cm) and maintained on a 12 h/12 h light/dark cycle with constant temperature (22 ± 2°C) and relative humidity (55%).Food was available at all times, and access to water was restricted for 48 before each experimental session.

Apparatus
Five MedAssociates Modular Test Chambers (MedAssociates ENV-008 SN: 3318) for rats were used.The chambers had two standard response levers and access to water controlled by an electromechanical device.Access to water was located centrally between the response levers.During reinforcements, a dipper presented 0.06 ml of water for 3 s.A houselight was located on the wall opposite the wall with the response levers, and two lights could be turned on or off above each lever.All events within the experimental chamber were scheduled and recorded using a computer compatible with IBM-PC interface DIG-700P1 and Windows Med-PC software (SOF-735).Data were recorded using Schedule Manager software using Visual Basic and developed especially for the present study.

Procedure
After the shaping of lever-pressing through the differential reinforcement of successive approximations of that response class, subjects were gradually exposed to chained FR 1 FI 60-s schedules.The number of sessions and hours required to shape lever-press responding for each rat was not recorded.Under this first condition, a response on the right lever (FR 1) turned on a light above the left lever and initiated a 60-s period (FI 60 s).The first response after 60 s from the onset of the light turned off the light above the lever resulted in the presentation of water for 3 s.Sessions occurred three times per week, either on Monday, Wednesday, and Friday for some rats or Tuesday, Thursday, and Saturday for other rats.Rats were exposed to each experimental condition for a minimum of nine sessions.The stability criterion required that the medians of FI pausing for each group of three sessions did not show any tendency for the last nine sessions.For each rat, the median for that group of nine sessions was representative of that experimental condition.In successive experimental conditions, FR requirements were fixed at 2, 3, 5, 7, and 1.
Data were recorded as PRPs, pauses in the FI schedule, IRI, number of reinforcements in the session, and duration of the session.A PRP was defined as the length of time between the end of access to water and the first response in the FR schedule.Pause during the FI was defined as the length of time between the onset of the discriminative stimulus (light) and the first response in the presence of that stimulus.

Results and Discussion
Table 1 shows the number of sessions for each rat in each experimental condition.Figure 1 shows that, for four of five rats, pauses in the FI 60-s schedule of chained FR FI schedules decreased as the FR requirement increased (p = .0196,Kruskal-Wallis analysis of variance [ANOVA]), replicating the data of Todorov and Teixeira-Sobrinho (2009).Pauses in the first experimental condition (FR 1) were lower than the pauses reported in the literature (e.g., Schneider, 1969).As the response requirement increased in subsequent experimental conditions, the pauses systematically decreased with the exception of Rat 1.
Figure 2 shows that in the first return-to-baseline session (chained FR 1 FI 60-s schedules), the FI pauses were very close to those observed in the first experimental condition.Notably, data that refer to the return to baseline are from the first session of the second exposure to the chained FR 1 FI 60-s schedules.Return to baseline levels must have occurred during the beginning of the session because the data in Figure 2 are presented as medians of all FI pauses in that session.Post-reinforcement pauses (i.e., pauses before responding in the FR schedule began) increased with increases in the FR requirement for all five rats (p = .049,Kruskal-Wallis ANOVA; Figure 3).The magnitude of the effect altered the IRI.The obtained IRI was the sum of the PRPs plus a minimum of 60 s of the FI schedule.
Figures 4 and 5 show how the increases in the IRI affected the pattern of responding in the chained schedules.Data are from Rat 7 in the experimental conditions when FR 1 and FR 7 were used.
Figure 4 shows short PRPs and typical responding in the FI component of the chained schedules.In Figure 5, with FR 7, PRPs were longer, and responding in the FI began almost immediately when the schedule was on and sometimes continued at a steady rate until reinforcement or showing positive acceleration (scallop).Data from Rat 1 were atypical and are not included in the analysis of Figure 1.When the FR requirement was 1, FI pauses were short (as observed by de Souza & Todorov, 1975) and increased with increases in FR requirement, probably showing the effect of continuous exposure to the FI 60-s schedule.Thus, for four of five rats, the results of Experiment 1 generally replicated the data of Todorov and Teixeira-Sobrinho (2009).In chained FR FI schedules, PRPs and IRIs increased whereas pauses in the FI decreased as the FR requirement increased.If FI schedules, defined as the time interval between primary reinforcements, control behavior similarly to FI schedules defined by the onset of some other change in the environment, then the FI pause duration in chained FR FI schedules should change as the FR requirement changes.Fixed-interval pauses could increase with continued exposure to the same FI duration (60 s).However, even with an FR 1, pauses in the FI were lower than pauses typically found in single FI schedules (e.g., Schneider, 1969).In the return to baseline, with a change from FR 7 to FR 1, the first session in the second exposure to FR 1 showed pauses that were very similar to those obtained after stability in the first experimental condition.
Data from the cumulative records of Rat 7 showed how the discriminative control of light over the response lever may have been overtaken by other variables.FR 1 had visible pauses in the FI schedules followed by the positive acceleration of responding until primary reinforcement.Large FI pauses were absent with the FR 7, and scallops were sometimes substituted by continuous responding until reinforcement.
Overall, these results could be interpreted as the primacy of IRIs in the control of pausing and responding in FI schedules (Staddon, Wynne, & Higa, 1991;Innis, Mitchell, & Staddon, 1993), even when the FI duration is determined by events other than the last primary reinforcement.However, instead of or in addition to that, the use of two response levers to better define the chained schedules may have interfered with the discriminative control of light over responding during the FI schedule.One way to explain the differences found in FI performance when the pauses began with primary reinforcement or another event is to compare data from Experiment 1 with data obtained with single FI schedules.Experiment 2 used an A-B-A design to compare pauses in single FI schedules (condition A) with pauses in chained FR FI schedules (condition B).

Subjects
Five naive, male Wistar rats aged 6 months at the beginning of the experiment (described as Rats 9, 10, 12, 14, and 15) were used.Rats were born and maintained in the vivarium of the Instituto de Educação Superior de Brasilia and were housed individually in polycarbonate cages (30 × 30 × 50 cm) maintained under a 12 h/12 h light/dark cycle with constant temperature (22 ± 2°C) and relative humidity (55%).Food was available at all times, and access to water was restricted for 48 before each experimental session.

Equipment
Equipment was the same as in Experiment 1.

Procedure
After shaping lever-press responding as in Experiment 1, subjects were exposed to a single FI 80-s schedule of reinforcement associated with the response lever with a light on over it.Stability criterion was the same as in Experiment 1.When the criterion was met, subjects were exposed to chained FR 1 FI 80-s schedules.The light over the lever was off after the reinforcements.A response on the left lever (FR 1) turned on a light above the right lever and initiated an 80-s period (FI 80 s).The first response after 80 s from the onset of the light turned off the light above the bar resulted in the presentation of water for 3 s.One response turned the light on and initiated the FI 80 s.After the stability criterion was met, subjects were again exposed to a single FI 80-s schedule for the number of sessions necessary to reach the stability criterion.

Results and Discussion
Table 2 shows the number of sessions for each rat in each experimental condition.Similar to Experiment 1, the number of sessions required to reach the stability criterion was low, but Figure 6 shows that the effect was considerable.The change to chained FR 1 FI 80-s schedules had an almost immediate effect of shortening the pauses.Return to baseline under the third experimental condition showed pauses in a single FI 80-s schedule that were compatible with those observed in the first baseline.
Figure 7 shows typical responding in the FI 80-s schedule (i.e., scallops).In Figure 8, with a chained FR 1 FI 80-s schedule, responding in the FI began almost immediately when the schedule was on and sometimes continued at a steady rate until reinforcement (i.e., no scallop).by response acceleration until primary reinforcement.Under the second experimental condition, chained FR 1 FI 80-s schedules resulted in a qualitative change in behavior.Pauses in the FI schedule decreased abruptly with durations similar to those in Experiment 1. Short FI pauses in chained FR FI schedules cannot be explained by shortcomings of the experimental procedure.Despite the fact that the same name is given to three different schedules (i.e., FI beginning and ending with primary reinforcement), the best explanation appears to be that in the FI that began with a discriminative stimulus and FI that ended with a conditioned reinforcer/ discriminative stimulus, the crucial variable was not the time to the next reinforcement marked by some event but rather the obtained IRI.In single FI schedules, the minimum IRI is controlled by the experimenter.In chained schedules, the minimum IRI depends on the characteristics of the schedule programmed as the first component.In the present study, the first component of the chained schedules was an FR schedule.Changes in the FR requirement indirectly resulted in changes in the stable relationship between the last reinforcement and presentation of the discriminative stimulus associated with the next reinforcement.

General Discussion
The results from Experiments 1 and 2 and results of de Souza and Todorov (1975) and Todorov and Teixeira-Sobrinho (2009) showed that FI schedules may be useful to study time discrimination when defined as the time between successive presentations of primary reinforcement (e.g., Machado et al., 2009) but not when the FI is the second link in chained schedules (Ferster & Skinner, 1957).In this case, time to the next reinforcement is signaled by the presentation of a discriminative stimulus associated with responding in the FI schedule and a conditioned reinforcer for the response that produces it.A new issue is why temporal control is different in single and chained schedules.
At least part of the explanation for the present results could be a variable suggested by Shull (1970), specifically the number of responses emitted before the last reinforcement.This hypothesis would lead to longer pauses in the FI schedule as the response requirement in the FR schedule increases.However, this is not the case.Responding in the FI schedule began early in the interval, and the pause in the FI schedule was inversely proportional to the FR requirement.
The results from an experiment in which the FI schedule was the first link of a chain and thus began with primary reinforcement and ended with a conditioned reinforcer also reveal discrepancy.Hanson, Campbell, and Witoslawski (1962) used chained FI FR schedules with rats, food as reinforcement, long FI durations (for rats), and extended training before beginning the experiment.Pauses expressed as percentages of interval length are expected to be constant (cf.Machado et al., 2009), but Hanson et al. (1962) reported that pauses   For all five subjects, pauses in the single FI schedule were consistent with the literature (Cumming & Schoenfeld, 1958;Sherman, 1959;Schneider, 1969;Staddon et al., 1991;Innis et al., 1993).Performance in the single FI 80-s schedule was characterized by no responses in the first 50-80% of the interval followed expressed as percentages decreased sharply as the FI duration increased and FR remained constant.Some possible experimental procedures may be used to further understand temporal control in FI schedules-a question that remains without a widely accepted answer (Wearden & Lejeune, 2006).Chained fixed-time, fixed-interval (FT FI) schedules have the advantage of controlling postreinforcement pauses and response requirements before the onset of the discriminative stimulus that signals the beginning of the FI schedule.Chained FT FI schedules establish that no responding is necessary to produce the onset of the FI schedule.Variations in the duration of the FT schedule produce changes in the IRI, independent of the animal's behavior.Another way to describe this procedure is that it is equivalent to multiple extinction FI schedules (Ferster & Skinner, 1957), in which responding is maintained by an FI schedule in one component and subjected to extinction in another component.Concurrent schedules with an FI as one of the components of the concurrent pair may be another useful procedure.In this case, pauses in the FI schedule depend on the reinforcement context.Todorov and Ferreira (1978) reported "Visual aspects of cumulative record [in concurrent FI 50-s, FI 129-s schedules] showed that in the period after a reinforcement in FI 50-s the subjects would switch to responding in FI 102-s; few switches to the longer FI schedules occurred in the end part of FI 50-s" (Todorov & Ferreira, 1978, p.108).The general conclusion was that responding in longer FI schedules could be partially seen as interim activity controlled by the short FI duration.

Figure 1 .
Figure 1.Pauses in the FI 60-s schedule of chained FR FI schedules as a function of response requirement in the FR schedule.

Figure 2 .Figure 4 .
Figure 2. Pauses (in seconds) during the FI schedule in chained FR 1 FI 60-s schedules.Black bars show the medians of the last nine sessions in the first experimental condition.White bars show the median pause for the first return-to-baseline session (FR 1 FI 60-s schedules) after exposure to FR 7.

Figure 3 .Figure 5 .
Figure 3. Pauses after reinforcement in chained FR n FI 60-s schedules as a function of the FR requirement.

Figure 6 .
Figure 6.Pauses under an FI 80-s schedule compared with pauses in the FI schedule of chained FR 1 FI 80-s schedules.The graph shows individual data from five rats.

Figure 7 .
Figure 7. Cumulative response record in a single FI 80-s schedule.Empty circles show reinforcer presentations.Sample data are from Rat 9.

Figure 8 .
Figure 8. Cumulative response record in chained FR 1 FI 80-s schedules.Empty triangles show discriminative stimulus (light) onset.Empty circles show water presentations.Sample data are from Rat 9.

Table 1 .
Number of sessions per experimental condition for each subject in Experiment 1

Table 2 .
Number of sessions per experimental condition for each subject in Experiment 2 *Rat 14 died after the first return-to-baseline session.