Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Neuroeconomics wikipedia , lookup
Verbal Behavior wikipedia , lookup
Psychological behaviorism wikipedia , lookup
Behavior analysis of child development wikipedia , lookup
Experimental psychology wikipedia , lookup
Vladimir J. Konečni wikipedia , lookup
Behaviorism wikipedia , lookup
Psychophysics wikipedia , lookup
ACTA NEUROBIOL. EXP. 1979, 39: 383-394 Opening address delivered a t t h e Warsaw Colloquium o n Instrumental Conditioning and Brain Research M a y 1979 INVOLVEMENT OF THE PARTIAL REINFORCEMENT PROCEDURE IN REWARD TRAINING: OPENING ADDRESS Kazimierz ZIELINSKI Department of Neurophysiology, Nencki Institute of Experimental Biology Warsaw, Poland Abstract. The analysis of the course of reward training presented in early Konorski and Miller's papers (1933, 1936) indicates that the appearance of active instrumental responses is related with the introduction of the partially reinforced, excitatory classically conditioned procedure. It is postulated that the mechanism underlying the effects of this procedure on the shaping of instrumental responses is similar to that proposed by Konorski (1967) for the effects of inhibitory classically conditioned stimuli on the performance of the instrumental response. It Q a great honor and privilege for me to be given the opportunity to deliver an opening lecture about two events of great importance for this country and for the world of science: the establishment olf 'the Nencki Institute of Experimental Biology in 1918, and the publication of the first of Jerzy Konorski's papers on Type I1 Conditioned Reflexes in 1928 (9, 10). These two events were intimately related. For more than a dozen years leaders of the intellectual life in Warsaw, supported by scientists from other countries who were pupils and friends of the great Polish physiologist and biochemist, Marceli Nencki, attempted to create a biological institute in Warsaw. Only after Poland regained independence, however, were the several biological laboratories existing since 1911 along with their Scientific Council and Directorial Board, finally legalized as the Nenchi Institute of Experimental Biology. A number of outstanding saientists organized a very efficient establishment, providing the opportunity to conduct research in physiology, biochemistry, protozoology, hydrobiology, ethology and mathematical statistics (16). The Institute's library, which counted about 30,000 volumes of books and periodicals before its total destruction during the second World War, was the reason for the first contact with the Nencki Institue by J e n y Konorski and Stefan Miller, students of the Medical Faculty at Warsaw University. Konorski wrote in his autobiography (4): "Instead of medicine, we read and reread Pavlov and discussed every detail of the experimental work of his coworkers. Moreover, we found in the library of the Nencki Institute of Experimental Biology the Russian journals in which the papers on conditioned reflexes were published" (p. 186). As a result of their studies, these two young gifted men designed and started on 1 February, 1928, in the Laboratory of Psychology of the Free Polish University, experiments on avoidance, punishment, reward, and omission learning, varieties of conditioned reflexes other than those studied by Pavlov and his pupils. Progress in their studies was amazingly rapid. The first two papers concerning the properties of Type I1 Conditioned Reflexes were delivered to the Warsaw Branch of the French Biological Society by the summer of 1928, and after a few months were published in French in "Comptes Rendus de la SociCte de Biologiae" (9, 10). Saon after, Konorski and Miller were offered the opportunity to continue their research in the physiological laboratonies of the Medical Faculty at Warsaw University, and later in the State Psychiatric H~ospitalin Pruszk6w near Warsaw. Their results from this early period of research were summarized in a monograph, "The foundation of the physiological theory of acquired movements", published in Polish in 1933 (16). However, much earlier, in 1928, Konorski and Miller exchanged letters with Pavlov, who invited the two young scientists to his laboratories in Leningrad. Konorski worked in the Institute of Experimental Medicine for two years (1931-1933), and Miller for only several months. The main theme of the research conducted by Konorski during the Leningrad period was aoncerned with the relations between instrumental and classical oonditioned reflexes. The results obtained were published in Russian sin the "Transactions of Pavlov's Laboratories" in 1936 (7). After returning from Leningrad the only place in Poland able to accept the two visitors to Pavlov's laboratories within its staff was the Nencki Institute. In contrast to electrophysiology, there was no tradition of behavioral research in Poland. However, the Nencki Institute had developed a strong schml of ethology, conducting research on the adaptive behavior of various species from infusoria to amphibians. The Institute appreciated the importance of neurophysiological ~esearchfor biological sciences and consk+ently attempted to inoorporate different groups of neurophysiol~ogists.In addition, there was no "Sovietophobia" at the Institute. On the contrary, the Soviet Union's scientific centers were the most numerous among those with which the Institute exchanged publications. This aspect was rather important for Konorski's future, since Konorski and Miller, in both lectures and publications, presented an impartial picture of the development lof Soviet science as well as the social changes in that country. And finally, Konorski and Miller returned from the Institute of Experimental Medicine, where the physiological section was built by Ivan Pavlov, but biochemical section was organized by Marceli Nencki during the last decade of his life. Such were the reasons why J e n y Konorski was able to work at the Nencki Institute in the Department of Physiology, heated by Professor Kazirnierz Bidaszewiaz. Thereafter, for 40 years, Professor Konorski was a member of the Nenckii Institute. He started to organize chambers for experiments on conditioning, while participating in the investigations of Liliana Lubiliska on the excitability of penipheral nerves and neuromuscular transmission. When the Physiology Department was destroyed by artillery fire during the first days of the battle for Warsaw, Konorski, like other members of the Institute, tried b salvage apparatus and bmks fmm under the wreckage and debris, typing out copies of scientific notes and works still in manuscript, which were then preserved in various places. All of this work was in vain, becoming lost or destroyed by enemy actions. Among the materials lost were chapters of the book prepared by Konorski just before the outbreak {of the war. The book was eventually published in altered form in 1948 under the title "Conditioned reflexes and neuron organization" (3). However, the many losses in human life were the heaviest blow to the Institute. Among others who were killed in action at the front, fell in the Warsaw Uprising or were murdered by the Nazis, was Stefan Miller, the closest friend and coworker of Konorski. Tragically, he committed suicide when Nazi troups entered his flat in the Warsaw Ghetto. Jerzy Komrski and Liliana Lubihska were able to flee Warsaw, and after a short period of work at the Bialystok hospital, they ~eceived an (invitation to Leningrad. From there, they moved to Sukhumi, where Konorski was offered the position of head of the Physiological Department belonging to the Institute of Experimental Medicine. By the beginning of 1945 Kmorski began his yourney back home. A small group of the pre-war staff, only six persons, reactivated the Nencki Institute, first at Mi in 1945 and then in Warsaw since 1954. Konorski organized a Department of Neurophysiology in the Institute, which he headed until the end of his life. Although he did not relish administrative duties, he 2 - Acta Neurobiol. Exp. 6/79 performed many. Since 1946 he' was continously a member of Institute's Board of Directors, and he was the Director of the Institute from 1968 to 1973. No single important event in the Institute was without his participation. He actively influenced the development of other d5partments of the Institute. Research on human perception as well as in neurochemistry were greatly stimulated by him. Today the Nencki Institute numbers 116 scientific workers, and 50 of them were either his pupils and coworkers o r were trained by close collaborators of Konorski. This number illustrates the extent of his influence on the present character of our Institute. The journal "Acta Neurobiologiae Expenimentalis", a continuation of the pre-war journal, "Acta Biobgiae Experimentalis", a c q u i r d an international range under his guidance. Professor Konorski was also a leader in the cooperation of Polish neurophysiobgists with scientists of other countries, and the audience of this conference provides the best proof of the strehgth of ties between the Nencki Institute and many leading neurophysiological centers thmughout the world. Jerzy Konorski was the most distinguished among many prominent scientists and outstanding persons fostered by the Nencki Institute. He united talent with industry, intellectual courage with responsibility. All of us were influenced by his ideas and personality, his arguments in discussions and by his papers and b k s . All of us know many of his scientific achievements. Thus, I would like to present only one of them, and probably the most important, reported for the first time by Miller and Konorski in 1928 i n the three-page paper, "On a particular form of conditioned reflex" (10). This paper contains a description of the experimental procedures leading to the formation of the four varieties of Type I1 Conditioned Reflexes, now usually refered as instrumental reflexes, and their basic differences from the procedures of classically conditioned reflexes. This disoovery was presented and discussed in all monographs written by Jerzy Konorski. When we compare this paper with corresponding phrases in his last book, "Integrative activity of the brain" (4, p. 389), we find a number of differences in the description of the pnocedures for instrumental eonditioning. These differences reflect changes in the understanding of the processes underlying these conditioning procedures. I n a brief form they were summarized by J e n y Konorski within a Postscript to the 1928 paper, when i t was translated and published by Skinner in t'he Journal of the Experimental Analysis of Behavior (11). In their early papers Konorski and Miller used not only Pavlovian terminology, but they also applied the tyrpe of analysis developed by Pavlov's scientific school. They placed emphasis on the stimulus conditions existing during the formation of instrumental responses. Let us consider the reward training procedure. Konorski and Miller (11) noted that an active movement is learned when a compound of two stimuli, external stimulus (A), presented by the experimenter, and the second stimulus (B), consisting of "all the sensations generated by a particular movement such as llifting the leg - that is, a set of muscular, tactile, and other sensations", is accompanied by the food unconditioned sti~mulus (R). However, the external stimulus alone, similar to "sensations accompanied this particular movement" without the external stimulus, is not accompanied by food. In other words, training of the instrumental response by the reward method involves the discrimination of the three arrangements of stimulus conditions: (i) the external stimulus plus the pqrioceptive stimulus plus hod, ,(ii) the external stimulus alone, and (iii) the proprioceptive stimulus alone (A+\B4-R versus A versus B). After some number of presentations of these three stimulus conditions to the subject "... the stimulus A will then be capable of provoking by itself the appearance of the stimulus B - in order to complete the conditioned compound" (p. 187). Appearance of the movement of which the pmprioception is an element of the conditioned compound is the very basis of type I1 (instrumental) conditioning. Pavlov considered the excitatory classically conditioned reflex as an effect of the association between conditioned and unconditioned stimuli. After the association between the two stimuli is formed, according Do Pavlov "the conditioned stimulus is a signal, as if it were a substitution of the uncond~itionedstimulus" (12, p. 117). In the early Konorski and Miller writings, the compound consisting of external and proprioceptive stimuli was considered as a conditioned stimulus for the classically conditioned reflex. Thus, there 'was a basic uniformity between Pavlov's idea of stimulus substitution and Konorski and Miller's (thoughts of the active instrumental response as a source of stimulation which is missed when the passive movement was not elicited. The description of instrumental learning procedures with phrases stressing the complementary role of the proprioceptive stimuli smoothed over the difference between the classical and instrumental conditioning procedures. To my mind, this was one of the reasons responsible for the delay in the acceptance of the specifity of instrumental reflexes by the Pavlovian school. Careful reading of the Pavlov's foreword to the Konorski and Miller paper published in Russian in 1936 seems to confirm this supposition (12). Later experiments conducted by Gbrska, Jankowska, Tarnecki and others (see 4, p. 467-479) showed "that proprioception of a trained movement does not play kyessential role in type I1 conditioning" and "passive movement as a rule cannot be instrumentalized" (Konorski in the "Postscript", 11, p. 189). Konorski's impressions about his relations with Pavlov presented in his Autobiography (5, p. 195) reflected both admiration and some reservations caused by Pavlov's pegative attitude toward the specificity of Type I1 Conditioned Reflexes and calling them "motor oonditioned reflexes" o r "conditioned reflexes of the motor analyser". However, inspection of the Konorski's bibliography (2) indicates that the titles of the two next pqpers by Konorski and Miller published before their visit to Leningrad in 1930 included just these very phrases used to indicate their own primary views on the nature of instrumental reflexes. The contemporary reader of the early papers by Konorshi and Miller has an impression that the authors were too Pavlovian, that their early readings exerted a stnonger influence on them than their own original findings. Only in the course of experimental work did their understanding of instrumental conditioning become more refined and exact. Paradoxically, the Leningrad period was extremely important for the scientific maturation of Konorski as the founder of instrumental conditioned reflexes. There, he used dogs with long training histories in classically conditioned reflexes, which provided an opportunity to investigate the interrelations between instrumental and classi~callyconditioned reflexes. These experiments confirmed the preliminary data obtained i n Warsaw, showing that conditioned stimuli, which elicited classically conditioned alimentary salivation, exerted an inhibitory effect on instrumental responses established by the reward procedure (Fig. 1). On the other hand, conditioned stimuli which inhibited classically conditioned salivation, as a rule, elicited instrumental responses trained by the reward procedure (6, p. 101-128; 7, p. 139-143 and 165-175; 4, p. 369-375). These findings demonstrated the basic differences between classically conditioned and instrumental reflexes. Moreover, they directed the further research of Konorski and his pupils into problems of extinction and inhibition (see, 15). Data on the interrelations between classically conditioned and instrumental reflexes provided arguments in numerous discussions with other interpretations of instrumental conditioning. For instance, they were heavily used in Konorski and Miller's paper published in The Journal (of General Psychology in 1937 (8) where they discussed Skinner's paper published by the same journal two years earlier (14). Our present approach to the distinction between various cmditioning procedures is based on the analysis of contingencies rather than on stimulus conditions. From this point of view we may distinguish several different relations among the conditioned stimulus, the conditioned respone and the unconditioned stimulus. In the case of classically conditioned reflexes, the conditioned stimulus signals the definite probability of the unconditioned stimulus presentation which cannot be changed by subject's behavior. In typical Pavlovian experiments the excitatory conditioned stimulus is a signal for h d or some oth'er unconditioned stimulus i,nevibably given at the end of its action, and presentation of the Fig. 1. The effect of classically conditioned stimuli, CS+ (excitatory) and CS- (inhibitory), on instrumental response established in the experimental situation. From above: a, movements of the hindleg; b, salivation; c, CS presentations. The animal repeatedly lifts its right hindleg, each movement being reinforced by food. Then (mark in line c) food is not presented for a few seconds and thereafter CS+ (bell) is given with the 15-s CS-US interval. The dog momentarily stops performing the movement, waits for food and salivates copiously. The food is presented, CS+ is terminated and the instrumental responses are restored. Then again food is not given for a few seconds, and CS- (metronome, 60 cyclelmin) is presented. The movements are not stopped, but the salivation is much reduced. From Konorski and Miller (7, p. 141). inhibitory stimulus is a signal that the unconditioned stimulus will not be given either during or shortly after its action. In the real world outside the conditioning chamber, such strict correlations between conditi'oned and unconditioned stilmuli are rather exceptional. Normally, conditioned stimuli signals the certain probability of an unconditioned stimulus. Performlance of a specific behavior is necessary for obtaining an attractive unconditioned stimulus or avoiding an aversive one. When only one kind of overt behavior, let us say, the leg flexion, is ubserved, four different relations between the conditioned stimulus, conditioned response and unconditioned stimulus 'may be expected, and all of them were investigated and described by Konorski and Miller in their first experimental paper. In reward training the occurrence of the specific change of behavior (leg flexion) is a necessary condition for presentation of the attractive unconditioned stimulus, whereas in omission training the withholding of this same response during CS action is a condition for obtaining the attractive US after the CS offset. In avoidance training the specific change of behavior is a condition for not presentating the aversive US, and in punishment training the withholding of this very response d u M g CS action is a condition for nonpresentation of the aversive US. The "truly random" pseudoconditioning procedure (13), where CS presentations, US occurrences, and the performance of the CR are not correlated, provides control conditions for both classical and instrumental training procedures. In the consecutive run of conditioning trials, either classical or ifnstrumental, a subject changes its behavior according to the consequences of this behavior ir. previous trials. Let us limit our analysis to the alimentary situation. When excitatory classically conditioned reflexes are trai-ned, not only copious salivation, but also the targeting (orienting) reflex to the food bowl beoome the $dominantmodes of behavior. The behavior of a well trained dog is immediately chsanged after the CS onset: all gross bodily movement become inhibited and the animal's posture indicates that the dog is ready to get the food that will be presented toward the end of the CS. During training of inhibitory classically conditioned reflexes, drowsiness in dogs has been frequently described, and when a large proportion of inhibitory trials are presented during an experimental session, full sleep may even b e observed. In both varieties of classically conditioned reflexes, a subject learns that its behavior has no consequences on food presentation. For successful instrumental learning two conditions have to be fulfilled: (i) the change in behavior chosen by the experimenter as an instrumental response must be performed with some minimal initial probability, and (ii) the method employed must provide an opportunity for the subject to recognize that the specific change in its o m behavior leads to a change in the probability of the unconditioned stimulus. It is obvious that only elements of the normal repertoire of an animal's responses may be used as instrumental movements. The higher the behavior chosen by the experimenter as an instrumental response in the hierarchy of responses emitted in a given experimental situation, the easier is the instrumental training. However, responses occurring with a lower probability provide an opportunity for more detailed analysis of the increase in their frequency and for discovering the conditions that lead to stabilization of the instrumental response. For dogs placed on the Pavlovian stand, the leg flexion has a lower probability of occurrence than the pressing a bar located near the food-tray for rats in an operant chamber or a key-pecking response for pigeons. Inspection of the graphs illustrated in the early Konorski and Miller papers indicates that the method of training employed in their experiments inevitably increased the general motor activity of the trained dogs and, among other bodily movements, the leg flexion was also more frequent. At early stages of reward training conducted acco~dingto the metod of Konorski and Miller, the dog is confronted with two kihds of trials: (i) passive leg flection elicited during the action of the sporadic external stimulus followed by food presentation, and (ii) action of the same sporadic stimulus without leg flection not followed by the alimentary unconditioned stimulus (6, p. 60-69, 10). Taking into account the discoveries, obtained by Konorski and his collaborators in the 19501s,that proprioceptive feedback is not necessary for instrumental conditioning, we may infer that at the beginning of reward training a dog is confronted with the partially reinforced excitatory, classically conditioned reflex situation. In such a situation the central motor behavior system of a hungry dog is aroused and the trained anilmal performs various motor responses. Most often movements located high in the hierarchy of responses are performed; for instance, another instrumental movement acquired during the previous experimental history of the trained animal (6, p. 66-69). All unsuccessful behaviors become inhibited, the hierarchy of responses changes in the process of eliminating inappropriate movements and eventually the required behavior occurs. The importance of the introduction of the partially reinforced excitatory, classic~allyconditioned reflex situation for the course of reward training may be inferred from experiments conducted by Konorski in Leningrad (7, p. 145-187). He showed that during the action of the sporadic stimulus active leg flexions occur only after the introduction of trials in which the sporadic stimulus was not followed by food (Fig. 2). One of these experiments, conducted on dog "Nord", yielded data of special interest for the present analysis. The motor activity of the dog had been drastically increased by caffeine injection. Among various movements, numerous leg flexion responses were performed. Some of them occurred during the action of the sporadic stimulus and according to the rules of reward training they were reinforced by food presentation. However, as seen from Fig. 3, in these circumstances transformation of the excitatory classically conditioned reflex into the instrumental reflex did not occur. The general motor excitement which was independent of the contingencies employed during training did not provide the opportunity for the subject to recognize that performance of the required motor response had any consequences for presentation of the food reinforcement. Only after the introduction of partial reinforcement was a moderate increase of leg flexion responses in the experimental situation observed and on this background instrumental training proceeded successfuly. When Kcmorshi discussed the effects of classically conditioned stimuli on performance of instrumental responses, his line of thinking was almost identical to that presented above for the analysis of the early stages of reward training. In his last monograph (4) he wrote: "... the inhibitory CS in its early stage of development produces a state which differs from that pmduced both by the positive CS and by the firmly established inhibitory CS. Whereas the positive CS produces a pure alimentary re- Sessions Fig. 2. The course of reward training in dog Fingal. Daily sessions from 14 January 1932 are presented. From above: a, active leg flexion in intertrial intervals; b, active leg flexion during a noise stimulus reinforced by food; C, noise presentations not accompanied by leg flexion or food; 4 noise accompanied by passive leg flexion and food. Vertical dashed line indicates the introduction of noise presentations not accompanied by passive leg flexion and food presentation. From Konorski and Miller (7, p. 165). Fig. 3. The course of reward training in dog Nord. Daily sessions from 19 April 1932 are presented. Before the 4th session, 5 cc of 1% caffein solution was injected. Beginning with the 6th session, noise presentations not accompanied by passive leg flection and food presentation were introduced. From the 9th session noise of a higher inteaity was used as a conditioned stimulus. Denotations as in Fig. 2. From Konorski and Miller (7, p. 176). spome resembling that elicited by the US itself, and the well-established inhibitory CS produces the inhibitory response, the early inhibitory CS given rise to a conflict between the two, resulting in an entirely new state" (p. 375). This state is assumed to be the increase of the hunger drive, which in turn causes the rise of excitability of the central motor behavior system activating structures responsible for the performance of motor acts most probable in a given situation. An analysis of the early papers by Konorski and Miller suggests that the some mechanism, i.e., the increase of the hunger drive and the excitation of the central motor behavioral. system - is active during instrumental reward training at this stage when classically conditioned stimuli are partially reinforced with food presentation. There are data showing that the increase in the fear drive together with the excitability of the central motor system accompanies the early stages of avoidance training. Experiments by Gibson (1) were especially illustr,ative. Goats and sheep were trained m i n a large box enabling observation of the animals' behavior. During classically defensive conditioning, subjects emitted various motor responses, changing their repertoire in each consecutive trial. Similar behavi'or was observed at early stages of avoidance training, although, the response pattern chosen by the experimenter as an instrumental response soon became dominant. When the situation was changed i'n such a way that manother motor response was now required for successful avoidance of shock, the rise in excitability of the motor system resulted in increased emission of various motor responses came initially, similar to behavior observed during classically defensive conditioning. All of these data indicate that the behavior of animals is subjected to many changes according to the c,ontingencies employed. Not only during instrumental but also during classical training, changes in the pattern of motor responses occurred. These processes may be overlooked when our attention is directed to only one special class of responses. But once we recognize that our aim is to investigate the integrative activity of the brain and its role in ad,aptive behavior of humans and animals, we will probably share Klonorski's opinion offered in his "Postscript" to the first experimental paper by Miller and Ksonorski (11): "Fi'nally, the sharp distinction between, not only the procedural side of type I and type I1 conditioned reflexes, but also between their physi~logic~al mech~anisms seems to 'me now largely exaggerated. In fact, further investigation shows with i'ncreasing clarity that both types can be explained on the basis of the same general principles of connectionistic pmesses" (p. 189). Over the years the scientific interests of Jerzy Konorski became much broader than at the beginning of his scientific career. New lines of research were undertaken and more precise methods were introduced. However, the main problem of the present conference, the central mechanisms of instrumental conditioning, always was the most exciting for him. Professor Konorski presented results of experiments and defended his ideas to many audiences, listened attentively to others opinions and lively discussed new f a d s and hypotheses. I am sure that the present conference will be both interesting and enjoyable, if we try to emulate the way so typical of Jerzy Konorski, the founder of investigations on inskumential conditioning. REFERENCES 1. GIBSON, E. J. 1952. The role of shock in reinforcement. J. Comp. Physiol. Psychol. 45: 18-30. 2. GLOWACKA, R. 1974. Bibliography of Professor Jerzy Konorski's papers. Acta Neurobiol. Exp. 34: 681-695. 3. KONORSKI, J. 1948. Conditioned reflexes and neuron organization. Univ. Press, Cambridge, 267 p. 4. KONORSKI, J. 1967. Integrative activity of the brain: An interdisciplinary approach. Univ. Chicago Press, Chicago, 531 p. 5. KONORSKI, J. 1974. Jerzy Konorski, In: A history of psychology in autobiography. Vol. 6. Appleton-Century Crofts, New York. 6. KONORSKI, J. and MILLER, S. 1933. Podstawy fizjologicznej teorii ruchdw nabytych. Ruchowe odruchy warunkowe. Ksiqinica Atlas TNSW, Warsaw, 168 p. 7. KONORSKI, J. and MILLER, S. 1936. Conditioned reflexes of the motor analyser (in Russian). Tr. Fiziol. Lab. Akad. I. P. Pavlova 6: 119-288. 8. KONORSKI, J. and MILLER, S. 1937. On two types of conditioned reflexes. J. Gen. Psychol. 16: 264-272. 9. MILLER, S. and KONORSKI, J. 1928. Le ph6nom6ne de la gbn6ralisation motrice. C. R. SBanc. Soc. Biol. 99: 1158. 10. MILLER, S. and KONORSKI, J. 1928. Sur une forme particulikre des rbflexes conditionnels. C. R. Seanc. Soc. Biol. 99: 1155-1157. 11. MILLER, S, and KONORSKI, J. 1969. On a particular type of conditioned reflex. J. Exp. Anal. Behav. 12: 187-189 (English transl. of Ref. 10). 12. PAVLOV, I. P. 1936. Foreword to the paper by J. Konorski and S. Miller (in Russian). Tr. Fiziol. Lab. Akad. I. P. Pavlova 6: 115-118. 13. RESCORLA, R. A. 1967. Pavlovian conditioning and its proper control procedures. Psychol. Rev. 74: 71-80. 14. SKINNER, B. F. 1935. Two types of conditioned reflex and a pseudo type. J. Gen. Psychol. 12: 66-77. 15. ZIELIRSKI, K. 1979. Extinction, inhibition, and differentiation learning. In A. Dickinson and R. A. Boakes (ed.), Associative mechanisms in conditioning. L. Erlbaum Ass., Hillsdale, p. 269-293. 16. ZIELIRSKI, K. 1979. Sixty Years of the Nencki Institute of Experimental Biology. The Review of the Polish Acad. of Sci. No 4: 47-74. Kazimierz ZIELINSKI,Nencki Institute of Experimental Biology, Pasteura 3, 02-093 Warsaw, Poland.