Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Theory of reasoned action wikipedia , lookup
Behavioral modernity wikipedia , lookup
Neuroeconomics wikipedia , lookup
Behavior analysis of child development wikipedia , lookup
Learning theory (education) wikipedia , lookup
Residential treatment center wikipedia , lookup
Behaviorism wikipedia , lookup
Insufficient justification wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Classical conditioning wikipedia , lookup
OPERANT PLACE AVERSION IN THE RUSTY CRAYFISH, ORCONECTES RUSTICUS Rohan Bhimani A Thesis Submitted to the Graduate College of Bowling Green State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE December 2014 Committee: Robert Huber, Advisor Moira van Staaden Verner Bingman © 2014 Rohan Bhimani All Rights Reserved iii ABSTRACT Robert Huber, Advisor Neural mechanisms for learning are largely conserved across taxa, and with relatively simple, modularly organized nervous systems, crustaceans have proven a suitable model for exploring its cellular basis. Associative learning, in particular, is necessary for animals to gain predictability about their surroundings and to display stimulus-appropriate behavior. Crayfish have previously demonstrated the capability to learn through classical conditioning and they possess many traits desirable in a research model, such as the ability to engage in highly structured behaviors using a number of different, specialized appendages. The present study extends these findings to instrumental learning. We have successfully designed and implemented a spatially contingent, operant paradigm in which crayfish learned to avoid a particular substrate that is paired with electro shock punishment. Animals quickly responded with altered substrate preference and a marked change in locomotor activity. iv TABLE OF CONTENTS Page INTRODUCTION………………………………………………………………………..... 1 MATERIALS AND METHODS………………………………………………………..… 7 Animals………………………………….………………………………………..… 7 Surgery…………………………………..………………………………………..… 7 Behavioral Experiment……………………..……………….…..………………..… 7 Statistical Analysis……………………..…………………….…………………..… 9 RESULTS…………………………………………………………………………….…… 10 Unconditioned Place Preference…………………………….…..………………..… 10 Changes in Preference in Response to Shock……………………………..………… 10 Movement Patterns……………………..…………………….…………………..… 10 DISCUSSION………………………………………………..……………………………… 12 REFERENCES………………………………………………..…………………………….. 15 v LIST OF FIGURES Figure Page 1 Initial unconditioned substrate preference in crayfish ............................................... 24 2 Probability of quadrant use in treatment and yoked controls .................................... 25 3 Summary statistics for movement parameters in treatment and yoked groups ....................................................................................... 26 1 INTRODUCTION The ability of an animal to adjust its behaviors to external conditions through learning is widely conserved across taxa, and it represents a major driving force in evolution (Romanes, 1884). Animals may respond to changing environmental conditions with basic, non-associative forms of learning, in which repeated exposure may weaken (habituation) or strengthen (sensitization) behavioral responses, in contrast to associative learning, where an association is formed between a stimulus and response of co-occurring events. This allows an animal to predict environmental stimuli and to judge the subjective value of a particular behavior's outcome. In classical conditioning (Pavlov, 1927) a neutral stimulus (e.g., sound or visual cue) commonly precedes an unconditioned stimulus and over time acquires the ability to evoke an unconditioned response by itself. Used to assess an individual's learned preferences, it requires that an experimenter repeatedly pairs the attribute of interest (e.g., tactile or visual cues) with an unconditioned stimulus (e.g., food reward, foot shock) in an attempt to reinforce, alter or shape the individual's preferences. Conditioned place preference (CPP) utilizes classical conditioning in which a rewarding or punishing stimulus (Place Aversion) is paired with a distinct spatial cue (Tzschentke, 1998; Sitaraman, 2008; Brandes & Menzel, 1990). CPP paradigms determine changes in preference for a conditioned habitat compared to baseline measures and offer a powerful tool for measuring perceptions of reward or punishment. A subsequent testing phase assesses changes in the animal's preferences (Sovik & Baron, 2013) where enhanced contact with the paired environment (Tzschentke, 1998) confirms rewarding properties of a stimulus, and reduced contact demonstrates a perception of punishment. Conditioning paradigms, however, are limited in what they can measure and are frequently confounded by novelty seeking (Bardo & Bevins, 2 2000). Additionally, multiple training sessions are required to pair the unconditioned stimulus with environmental attributes in order to measure how much time individuals spend after priming. Although CPP provides information about the subjective wanting and seeking of a reward, it tells us little about how and what animals learn. Moreover, CPP paradigms ultimately lack many of the metrics that can be obtained through instrumental learning (e.g., number of operant behaviors, movement patterns) and its implications for drug reward remain somewhat unclear in both human and non-human primates (Bardo & Bevins, 2000). Operant conditioning (Skinner, 1974) pairs the actions of an individual with the consequences of a perceived outcome. Rewards increase the likelihood of a behavior while punishment reduces it. The ability to learn from experience thus encourages behaviors that elicit a beneficial outcome and it discourages behaviors with undesirable consequences (Skinner, 1974; Lukowiak et al., 1995). Much of our understanding of instrumental learning has come from convenient vertebrate model organisms (Mackintosh, 1974), including cats (Thorndike, 1911), dogs (Pavlov, 1927), and rats (Skinner, 1974). Despite providing important insights into the mechanisms of learning, the neural complexities of these systems make it difficult to explore the biological underpinning of acquired behavioral modifications, emphasizing a need to study model organisms. Associative learning is not an exclusive phenomenon of mammals and other vertebrates, however. Behavioral experiments on honeybees were able to pair food reward with color and were among the first to demonstrate classical conditioning in an invertebrate model (Von Frisch, 1921; Takeda, 1961; Menzel, 1967; Vareschi, 1971; Masuhr & Menzel, 1972). Similar findings were later documented in blowflies (Frings, 1941) and Drosophila (DeJianne et al., 1985). Learning is accompanied by changes in synaptic plasticity in Aplysia (Castellucci et al., 1970; 3 Carew et al., 1971), providing insights into the inner workings of classical conditioning (Hawkins, 1984). A light shock paired with a tactile stimulus is sufficient to elicit the wellknown gill and siphon withdrawal reflexes (Carew et al., 1981). Flatworms, such as planarians, show an unconditioned negative phototactic response to light. When primed with aqueous methamphetamine in a lit area, they begin to seek out lit areas, even in the absence of the drug (Raffa & Martley, 2005). Due to their relatively simple, modularly organized nervous systems (Lukowiak et al., 1996), invertebrate studies have elucidated different neural implementations of learning, including non-associative (Jennings, 1906; Thompson, 2009), habituation (Carew et al., 1971) and sensitization (Kandel & Schwartz, 1986), and the ability to form associations between frequently co-occurring events in classical (Carew, 1981) and operant conditioning paradigms (Cook & Carew, 1986). Further insights have emerged from operant paradigms (Abramson & Feinman, 1990; Tomina & Takahata, 2010), however, the literature on these are limited, and studies exploring the efficacy of punishment in such model systems remain largely untested. Studies of operant conditioning in invertebrate models (Makous, 1969; Wells and Wells, 1971; Carew & Sahlely, 1986; Brembs et al., 2002) rely on relatively simple behavioral tasks. Instrumental learning with rewards was demonstrated in honey bees (Núñez, 1970; Pessotti, 1972) and lobsters (Tomina & Takahata, 2010) while punishment altered behavior in insects in cockroach (Horridge, 1962) and Aplysia (Cook and Carew, 1986). Instances of avoidance as a result of punishment have been demonstrated in the cockroach (Horridge, 1962); locust (Hoyle, 1982); Aplysia (Susswein & Schwarz, 1984; Cook and Carew, 1986); pond snails (Lukowiak et al., 1996); and Drosophila (Putz, 2002). 4 Invertebrates are also able to learn rather complicated tasks (Putz, 2002), including a spatially contingent operant paradigm via operant place conditioning (OPC). OPC takes elements from CPP paradigms and integrates the animal’s ability to self-administer reinforcement or punishment contingent on location. Learning can be measured by recording the number of an individual’s operant acts in response to the stimulus (Crowder & Hutto, 1992) or by observing a change in place preference in comparison to baseline measures. OPC paradigms allow for the measurement of changes in motivational states as they progress throughout the trial and account for changes due to increased response to the unconditioned stimulus (Feduccia & Duvauchelle, 2010). Within the addiction literature, OPC has been demonstrated to produce a greater degree of place conditioning than CPP (Tzschentke, 1998). In addition, operant place conditioning has been shown to reinforce non-drug administration related behaviors where CPP shows no interaction (Crowder, 1992). The most compelling evidence for OPC in invertebrates involves placing Drosophila in a heat box where their position results in either the presence or absence of heat (Putz, 2002). Despite the more complex nature of operant conditioning, applying its basic principles to CPP paradigms delivers a baseline for whether learning has in fact occurred. Demonstrations of pure instances of operant learning in invertebrates have been difficult to generalize across species or have utilized restrained, decapitated specimens. In this experiment we aim to investigate operant conditioning in an intact, freely moving decapod crustacean, the rusty crayfish. Crustaceans offer a compelling model system to study learning due to their behavioral complexity, degree of stereotypy, and behavioral use of multiple specialized appendages. Additionally, crustaceans exhibit many of the basic forms of learning including habituation (Cyclocypris, Applewhite & Morrowitz, 1966), classical conditioning (Paguroidea, Mikhailoff, 5 1920; Carcinus maenas, Abramson et al., 1988; Orlosk et al., 2011) and food aversion (Palinuridae, Fine-Levy, et. Al, 1988; Paguroidea, Wight et al., 1990). Its modularly organized nervous system has been characterized extensively. The well-documented neural circuitry of the tail-flip escape response in crayfish highlights specific neurons that control this action (Wiersma, 1947). In addition to their motor abilities, crustaceans demonstrate a willingness to explore their environment when placed in a novel arena and they are able to navigate complex testing arenas. A technical advantage of crustaceans is represented by the presence of a sturdy shell for attachment of an indwelling cannula or stimulating electrode. Decapod crustaceans have proven fertile models for studies of neural mechanisms in behavior (Livingston et al., 1981; Edwards et al., 2003), including insights into nerve-muscle synapse for the study of synaptic transmission (Furshpan & Potter, 1959), the role of glutamate and GABA as excitatory and inhibitory neurotransmitters (Iversen et al., 1967; Taraskevich 1971), the neural orchestration of escape behavior (Edwards et al., 1999), the complex coordination of motor networks (Nusbaum et al., 2001), and emergent properties that arise from direct neuron to neuron interactions within networks (Selverston, 1999). Crayfish are capable of all forms of learning (Krasne & Woodsmall, 1969; Tierney & Lee, 2011; Bierbower, 2010) and they readily navigate a CPP paradigm (Panksepp & Huber, 2004). Demonstrations of operant learning via punishment however have stubbornly resisted elucidation. Kawai et al. (2004) attempted avoidance learning in the crayfish by placing the animal in an underwater maze where individuals had to swim either forwards or tail-flip backwards into a compartment to avoid receiving a shock. Very few individuals were able to successfully complete this complex task. 6 The present study aimed to provide a general model paradigm for operant conditioning of crayfish through punishment, and it examined the effectiveness of electro shock in eliciting substrate avoidance. Using a fixed interval schedule for completing an operant, placeconditioning paradigm we examined whether (1) electric shock produces unconditioned effects, (2) the use of punishment affects place preference (3) unconditioned and conditioned effects of learning can be distinguished through the use of a yoked control, and (4) learning curves for completing the paradigm can be modeled. By utilizing well-defined criteria, this study aimed to test whether animals are able to limit electro-shock punishment through their choice of substrates. The results of this work provide valuable insights into the principles governing operant conditioning and provide a behavioral template to investigate the neuronal changes that take place during associative learning. 7 MATERIALS AND METHODS Animals 26 male crayfish (Orconectes rusticus) were wild-caught from the portage river near Pemberville, Ohio (41.378984, -83.476177), and housed in a large aerated tank (~ 2500 L) prior to testing. Animals were placed under a 16:8 light dark cycle and fed weekly. Individuals with all appendages intact and with carapace length of 2.58 ± 0.55 cm and mass of 8.25 ± 3.85g were included in this study. From size-matched pairs, one individual was assigned randomly to the treatment group while the other served as its yoked control. Treatment animals were further assigned to a group in which electric shock was delivered either on the hard or soft substrate. Upon completion of the experiment, crayfish were returned to the wild. Surgery Prior to surgery, animals were isolated in individual plastic containers (Diameter = 160 mm, height = 95 mm) for 24 hours. Animals were cold-anesthetized in ice for twenty minutes. A small hole was drilled into the carapace roughly 10 mm from the rostrum along the midline using a 20-gauge needle. A 31.5 gauge insulated copper wire was inserted 3 mm into the animal and sealed into the carapace with super glue. The indwelling electrode was connected to a computercontrolled relay interface (Model 1017-0, 0/0/8, Phidgets Inc., Alberta, Canada) using 14-gauge speaker wire (Model AH1450SR, RCA, New York, United States.). Implanted animals were then returned to their individual containers for a 24-hour recovery period prior to testing. Behavioral Experiment A circular polyethylene experimental arena (diameter = 502 mm, height = 270 mm) contained four quadrants of two different, diagonally arranged substrates (soft vs. hard) layered over a base of waterproof cement. Soft quadrants featured 5 stacked layers of shelf liner (Non- 8 adhesive Easy Shelf Liner, Duck Brand, Ohio, USA) made up of beige polyester mesh coated with PVC plastic (depth = 10 mm) while hard quadrants contained beige ceramic floor tile (Model # 8646, Mono Serra, Montreal, Canada) of similar depth. Eight holes were arranged in an equally spaced radial arrangement for water circulation when the arena was not in use. During experiments water flow was turned off to prevent turbulences. Four ground wires were spaced equidistant around the arena and connected. An analog video camera (Sony Bullet, Model 800TVL, Sony, Tokyo, Japan) was placed above the tank 1 m above the water. The analog video signal was fed into a DV converter box (Canopus ADVC-300) and interfaced with an Apple Macintosh computer (iMac, 2.5 GHz Intel i5/OSX 10.6.8). Application of electric shock from a 6V DC power supply was administered via a USB-controlled, Relay Interface kit. Animal tracking and control of shock delivery were performed in real time with custom software developed using the JavaGrinders library, a collection of freeware programming functions for the analysis of behavioral data (available at <http://iEthology.com>). Experimental trials lasted 3 hours and locations of the animal were obtained in real-time. When placed into the experimental arena, animals were given a 10-minute acclimation period in which no shock was delivered. Following this, the entry into a predefined area earned the test animal an electric shock (6 Volts, duration 300 ms) controlled by the software. Each shock was followed by a timeout of 9 seconds. The magnitude of the mild electric shock remained uniform throughout the arena and was confirmed with a multimeter. Preliminary data identified these duration and voltage settings as effective punishment without eliciting long-lasting motor defects. Initial responses to the punishment included tail flips in several instances as well as threat displays where the animal stands on its walking legs and raises its claws in the air. A control design was used in which yoked animals received an electric shock that was not 9 contingent upon their own location, but rather on the movement patterns of the animal they were yoked to. Yoked controls were used to confirm changes in responding due to an association between punishment and location, and they are able to assess the pure, unconditioned responses to electric shock. Time stamps, x,y Cartesian coordinates, body orientation and applications of punishments were recorded in a text file for subsequent analysis. Statistical Analysis Following the conclusion of the experiment, behavioral analyses utilized recorded position and time data. Grouped into 18 time segments (10 mines), descriptive statistics tabulated the time spent, distance traveled, mean speed, and shock delivered by quadrant and substrate. Each individual's initial place preferences were determined before punishment and then assessed changes in quadrant use when shock applications commenced. Analyses compared the number of operant behaviors between treated and yoked groups. After initial tests for heterogeneity, data from animals punished in both hard or soft substrates were analyzed together. Statistical analyses were conducted in R (Version 3.0.3, <http://www.R-project.org>) with additionally installed packages: ggplot2, ez, Deducer, DeducerANOVA, and stats. Repeated measures ANOVA was conducted on temporal data and distance travelled using packages ez (Function: ezANOVA) and stats (Function: t.test). 10 RESULTS Unconditioned Place Preference During the initial 10-minute assessment of unconditioned substrate preference prior to the introduction of electric shocks, crayfish did not exhibited significant quadrant and substrate preference (t(25) = -0.239, p = 0.81). Regardless of treatment group, animals spent an average of 52.0% (SE = 4.7%) of the time on hard substrate and 48.0% (SE = 4.6%) of the time in soft substrate quadrants (figure 1). Changes in Preference in Response to Shock Following activation of electric stimulation, individuals earned an average of 8.88 (SE = 2.38) shocks during the first 10 minutes of testing. Crayfish showed a marked change in place preferences in response to these shocks, where time spent in the punished quadrants decreased from 50.4% (SE = 4.1%) to 22.8% (SE = 5.8%). Yoked controls showed no change in place preference and continued to frequent quadrants that were paired with punished in the treatment group for 52.0% (SE = 3.7%) of the time. In the final 10-minute time segment, animals in the treatment group only spent 1.1% (SE = 0.4%) of their time in punished quadrants in either substrate group (figure 2a). Yoked controls spent on average 44.2% (SE = 12.0%) of time in quadrants that were paired with punishment in the treatment group (figure 2b). Changes in substrate preference are significantly different between the groups of treated and yoked crayfish (F(1, 24) = 62.48, p < .001)). In the final time segment the treatment group only earned an average of 0.87 shocks (SE = 0.24), whereas yoked controls still performed an average of 14.93 operant behaviors (SE = 4.14). Movement Patterns 11 Treatment and yoked groups displayed a peak level of activity prior to the onset of punishment, however the treated group learned to primarily utilize the safe substrate whereas the yoked control remained in motion throughout the arena (Figure 3). Before the power supply was connected, treatment animals moved at 0.0154 m/s (SE = 0.83) compared to yoked controls at 0.012 m/s (SE = 0.004). All crayfish displayed a gradual decrease in activity over time (F(1, 24) = 6.39, p = .018). After 3 hours of testing, contingently punished crayfish moved at 27 mm/s (SE = 16) while yoked controls walked considerably faster at 60 mm/s (SE = 36). Treatment animals quickly returned to the safe substrate after they received a shock and generally remained there until they were removed from the testing arena. Yoked controls in contrast showed a spike in locomotor activity following each shock. Crayfish walk less over time in the arena, however treatment animals showed a significantly greater decrease in activity compared to yoked controls, (F(1, 24) = 60.97, p < .001). By the conclusion of the experiment, treated animals had only travelled an average of 1.633 m (SE = 271) while yoked animals travelled more than double the distance at 3.651 m (SE = 613). 12 DISCUSSION Invertebrate model systems have become increasingly prominent in studies of learning. Drosophila, with a wide array of mutant phenotypes for learning and memory, offer valuable insights into neural and molecular substrates. Moreover, research in Aplysia helped connect our understanding of behavior with neuronal physiology (Brembs, 2003). Crustaceans have clearly demonstrated the ability to learn yet their contributions to the learning literature have to date been limited (Krasne, 1973). Compared to more commonly used taxa, they possess a variety of amenable traits for the study of learning including a relatively simple nervous system composed of large central neurons, while at the same time exhibiting complex behavioral patterns. The present study introduces a powerful paradigm for operant learning through place conditioning, demonstrating that crayfish are able to learn environmental cues by linking them with perceived outcomes. Extending previous work on learning in crayfish (Krasne and Woodsmall, 1969; Bierbower, 2010; Tierney and Lee, 2011), we were specifically able to demonstrate that crayfish adjust their behavior to avoid punishment in an operant learning paradigm. When presented with an aversive stimulus, crayfish minimize their chances for encountering these predictive cues. Crayfish receiving punishment contingent on their location exhibited a pronounced change in place preference and thus the results of this study suggest that punishment, as in other animal systems, is an effective method for training a crayfish towards behavioral modification. Through the use of an automated well-defined punishment schedule, weak electric shocks proved to be a sufficiently noxious stimulus for crayfish, who actively learned to avoid punishment by means of trial and error. 13 Punishment associated changes in substrate preference were not observed in yoked controls. Given their lack of information regarding the rules of the paradigm, they have no means of associating the received punishment with a particular substrate. As punishment for yoked controls was controlled by their master's behavior, thus random to these animals, they increased their movement patterns following a shock. Treatment animals settled into the quadrants that were not punished, while yoked animals, in contrast, showed increased locomotion, presumably to gain predictability as to what caused the shocks. Additionally, yoked animals never settled into a particular quadrant and may have associated station holding with punishment, choosing relocating in an attempt to avoid punishment. Using information gained from this study, we can apply this aspect of crayfish learning to other fields of behavioral research. Crustaceans display many behaviors that are affected by learning and experience, and exhibit natural reward mechanisms that are susceptible to human drugs of abuse (Nathaniel et al., 2012). Operant conditioning approaches represent a powerful tool for measuring the strength of drug reward and offer a unique opportunity to study the function of drugs as strong reinforcers of behavior and as their role in highjacking natural reward circuitry in the brain (Gardner, 2011). By allowing animals to self-administer drugs in an operant paradigm, it is possible to measure the reinforcing properties of several substances and alter the schedule and dosage akin to that demonstrated in rat models (Panlilio & Goldberg, 2007). More recently, studies have investigated the ability to suppress drug-seeking behaviors (Pelloux et al., 2007). Vanderschuren & Everitt (2004) have demonstrated that rats exposed to cocaine curb their drug seeking behavior when infusions are paired with electric shocks. Animals that experienced extended exposure to cocaine did not show suppression to the drug seeking behavior and continued to obtain the drug. We believe that use of the operant paradigm presented here may 14 allow us to explore changes in drug seeking and suppression of drug use in crayfish. Demonstrating that crayfish alter their behavior within a punishment framework sets the stage for further neurobehavioral work in this model and opens the door to explore the associated cellular changes. 15 REFERENCES Abramson, C.L., & Feinman, R.D. 1988. Classical conditioning of the eye withdrawal reflex in the green crab. Journal of Neuroscience, 8, 2907–12. Abramson, C.L., & Feinman, R.D. (1990). Lever-press conditioning in the crab. Physiology and Behavior, 48(2), 267-72. Applewhite, P.B. & Morowitz, H.J., (1966). The micrometazoa as model systems for studying the physiology of memory. Yale Journal of Biology and Medicine, 39, 90-105. Bardo, M. T., & Bevins, R. A. (2000). Conditioned place preference: what does it add to our preclinical understanding of drug reward?. Psychopharmacology, 153(1), 31-43. Bierbower, S. M. (2010). Environmental effects on behavior and physiology in crayfish. Bouwma, P., & Hazlett, B. A. (2001). Integration of multiple predator cues by the crayfish< i> Orconectes propinquus</i>. Animal Behaviour, 61(4), 771-776. Brandes, C., & Menzel, R. (1990). Common mechanisms in proboscis extension conditioning and visual learning revealed by genetic selection in honeybees (Apis mellifera capensis). Journal of Comparative Physiology A, 166(4), 545-552. 16 Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science, 296(5573), 1706-1709. Brembs, B. (2003). Operant conditioning in invertebrates. Current opinion in neurobiology, 13(6), 710-717. Carboni, E., & Vacca, C. (2003). Conditioned place preference. Drugs of Abuse, 481-498. Carew, T.J., Castellucci, V.F., Kandel, E.R. (1971) An analysis of dishabituation and sensitization of the gill-withdrawal reflex in Aplysia. International Journal of Neuroscience, 2, 79–98. Carew, T.J., Walters, E.T., Kandel, E.R. (1981) Classical conditioning in a simple withdrawal reflex in Aplysia californica. Journal of Neuroscience, 1, 1426–1437. Carew, T. J., & Sahley, C. L. (1986). Invertebrate learning and memory: from behavior to molecules. Annual Review of Neuroscience, 9(1), 435-487. Castellucci, V., Pinsker, H., Kupfermann, I., & Kandel, E. R. (1970). Neuronal mechanisms of habituation and dishabituation of the gill-withdrawal reflex in Aplysia. Science, 167(3926), 1745-1748. 17 Cook, D. and Carew, T. (1986). Operant conditioning of head- waving in Aplysia. Proceedings of the National Academy of Sciences, 83, 1120–1124. Crowder, W. F., & Hutto Jr, C. W. (1992). Operant place conditioning measures examined using two nondrug reinforcers. Pharmacology Biochemistry and Behavior, 41(4), 817-824. DeJianne, D., T.R. McGuire, and A. Pruzan-Hotchkiss. 1985. Pavlovian counter-conditioning of proboscis extension in Drosophila melanogaster. Journal of Comparative Psychology (90) 74-80. Edwards, D. H., Hitler, W. J. & Krasne, F. B. (1999). Fifty years of command neurons: the neurobiology of escape behavior in the crayfish. Trends in Neuroscience. 22, 153–161. Feduccia, A. A., Kongovi, N., & Duvauchelle, C. L. (2010). Heat increases MDMA-enhanced NAcc 5-HT and body temperature, but not MDMA self-administration. European Neuropsychopharmacology, 20(12), 884-894. Finn-Levy, J.B., Girador, M.N., Derby, C.D., and Daniel, P.C. (1988). Differential associative conditioning and olfactory conditioning in the spiny lobster, Panulirus argus. Behavioral Neural Biology, 49, 315-331. Frings, H., (1941). The loci of olfactory end-organs in the blow-fly, Cynomyia cadaverina Desvoidy. The Journal of Experimental Zoology, 88, 65-93. 18 Furshpan E.E., Potter D.D. (1959). Transmission at the giant motor synapses of the crayfish. Journal of Physiology 145(2): 289-325 Gardner EL (2011) Addiction and brain reward and antireward pathways. Advances in Psychosomatic Medicine, 30, 22–60 Hawkins R. D. A (1984) cellular mechanism of classical conditioning in Aplysia. Journal of Experimental Biology, 112, 113–128. Horridge, GA (1962). Learning of leg position by the ventral nerve cord in headless insects. Proceedings of the Royal Society of London. Series B, Biological sciences (0080-4649), 157 (966), 33. Iversen, L., Otsuka, M.L., Hall, Z.W., & Kravitz, E.A. (1966). Release of gamma-aminobutyric acid from inhibitory nerves of lobster. Proceedings of the National Academy of Sciences of the United States of America, 56(4), 1110. Jennings, H. S. 1906. Behavior of the Lower Organisms. New York, 366. Kawai, N., Kono, R., & Sugimoto, S. (2004). Avoidance learning in the crayfish (< i> Procambarus clarkii depends on the predatory imminence of the unconditioned stimulus: a behavior systems approach to learning in invertebrates. Behavioural Brain Research, 150(1), 229-237. 19 Krasne, F. B. (1973). Learning in crustacea. In Invertebrate learning (pp. 49-130). Springer US. Krasne, F. B., & Glanzman, D. L. (1986). Sensitization of the crayfish lateral giant escape reaction. The Journal of neuroscience, 6(4), 1013-1020. Lukowiak K., Ringseis E., Spencer G., Wildering W., Syed N. (1996). Operant conditioning of aerial respiratory behaviour in Lymnaea stagnalis. Journal of Experimental Biology. 199:683–691. Mackintosh, N. J. (1974). The psychology of animal learning. Academic Press. Makous, W.L. (1969). Conditioning in the Horseshoe Crab. Psychonomic Science, 14-46 Masuhr, T., & Menzel, R. (1972). Learning Experiments on the Use of Side—Specific Information in the Olfactory and Visual System in the Honey Bee (Apis mellifica). In Information Processing in the Visual Systems of Anthropods (pp. 315-321). Springer Berlin Heidelberg. McConnell, J. V. (1966). Comparative physiology: learning in invertebrates. Annual Review of Physiology, 28(1), 107-136. 20 Menzel, R. (1967). Das Erlernen von Spektralfarben durch die Honigbiene. Z. vergl Physiol, 56, 22-62. Núñez, J. (1970). The relationship between sugar flow and foraging and recruiting behaviour of honey bees (Apis mellifera). Animal Behaviour, 18(3), 527-538. Nusbaum, M. P., Blitz, D. M., Swensen, A. M., Wood, D., & Marder, E. (2001). The roles of cotransmission in neural network modulation. Trends in neurosciences, 24(3), 146-154. Orlosk, J. L., Walker, J. M., Morrison, A. L., & Atema, J. (2011). Conditioning the crab Carcinus maenas against instinctive light avoidance. Marine and Freshwater Behaviour and Physiology, 44(6), 375-381. Panlilio, L. V., & Goldberg, S. R. (2007). Self-administration of drugs in animals and humans as a model and an investigative tool. Addiction, 102(12), 1863-1870. Pavlov, I. P. (1941). Conditioned reflexes and psychiatry (Vol. 2). W. H. Gantt, G. Volborth, & W. B. Cannon (Eds.). New York: International publishers. Pelloux, Y., Everitt, B. J., & Dickinson, A. (2007). Compulsive drug seeking by rats under punishment: effects of drug taking history. Psychopharmacology, 194(1), 127-137. Pessotti, L. (1972). Discrimination with light stimuli and a lever-pressing 21 response in Melipona rufiventris. Journal of Apicultural Research, 11(2), 89-93. Putz, G & Heisenberg, M (2002). Memories in Drosophila Heat-box Learning. Learning and Memory, 9, 349-359. Raffa, R. B., & Martley, A. F. (2005). Amphetamine-induced increase in planarian locomotor activity and block by UV light. Brain research, 1031(1), 138-140. Romanes, G (1884). Mental Evolution in Animals. D. Appleton and Company, New York, New York. Sitaraman, D., Zars, M., LaFerriere, H., Chen, Y. C., Sable-Smith, A., Kitamoto, T., & Zars, T. (2008). Serotonin is necessary for place memory in Drosophila. Proceedings of the National Academy of Sciences, 105(14), 5579-5584. Skinner, B. F. (1974). About behaviorism. Søvik, E., & Barron, A. B. (2013). Invertebrate models in addiction research. Brain, Behavior and Evolution, 82(3), 153-165. Takeda, K.(1961) Classical conditioned response in the honeybee. Journal of Insect Physiology, 6, 168–179. 22 Taraskevich, S. (1971). Reversal potentials of L-glutamate and the excitatory transmitter at the neuromuscular junction of the crayfish. Biochemistry, Biophysics and Biomembranes, 241(2), 700-703. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. Macmillan. Tierney, A. J., & Lee, J. (2011). Spatial learning in a T-maze by the crayfish Orconectes rusticus. Journal of Comparative Psychology, 125(1), 31. Tomina, Y., & Takahata, M. (2010). A behavioral analysis of force-controlled operant tasks in American lobster. Physiology & Behavior, 101(1), 108-116 Tzschentke, T. M. (1998). Measuring reward with the conditioned place preference paradigm: a comprehensive review of drug effects, recent progress and new issues. Progress in Neurobiology, 56(6), 613-672. Vanderschuren, L. J., & Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self-administration. Science, 305(5686), 1017-1019. Vareschi, E. (1971). Duftunterscheidung bei der Honigbiene—Einzelzell-Ableitungen und Verhaltensreaktionen. Zeitschrift für vergleichende Physiologie, 75(2), 143-173. 23 Vogt, P. (1969). Dressur von Sammelbienen auf sinusförmig moduliertes Flimmerlicht. Journal of Comparative Physiology. 63(2), 182-203 Watson, J.B. and Rayner, R. (1920). Conditioned emotional reactions. Journal of Experimental Psychology, 3(1), 1–14 Wells, M.J., and Wells, J. (1971). Conditioning and Sensitization in Snails. Animal Behavior, 19, 305-312. Wiersma, C. A. G. (1947). Giant nerve fiber system of the crayfish. A contribution to comparative physiology of synapse. Journal of Neurophysiology, 10, 23-38. 24 Figure 1. Initial unconditioned substrate preference in crayfish. Initial place preference was obtained during the first ten minutes of each experimental trial for animals in both treatment and yoked groups. No punishment was applied during this time and no significant place preference for hard or soft substrates was evident in either treatment or yoked group, t(25) = -0.239, p = 0.81. 25 Figure 2: Probability of quadrant use in treatment and yoked controls. Positional data were recorded for the treatment group (a) and yoked controls (b). Probability of quadrant use (y) is depicted over the time of the experiment. No punishment was applied during the first 10-minute time segment (Pre). Place conditioning was evident in treated crayfish where punishment was paired with a substrate. These animals demonstrated significantly less time spent in punished quadrants compared to yoked controls (F(1, 24) = 62.48, p < .001). Treatment crayfish learned the substrate that was associated with electrical shock (red) and developed a strong preference for the safe quadrants (grey). Yoked controls failed to show a substrate preference and continue to spend nearly equal amounts of time on the two substrates even after they were acquiring weak electric shocks. 26 Figure 3: Summary statistics for movement parameters in treatment and yoked groups. Mean distance travelled (left axis) and mean speed (right axis) are plotted for the treatment (red) and yoked (blue) animals over time. Treatment and yoked control groups showed a progressive decrease in locomotion as the experiment ran, F(1,24) = 60.97, p < .001. Treatment animals showed a significantly greater decrease in movement compared to yoked controls (F(1,24) = 6.39, p = .018) as they learned and predominantly remained in safe quadrants. For yoked controls, punishment was random and it is inferred that they showed higher movement measures following each shock.