“Emotions and learning: first steps with the clicker and what type of foundations we choose to lay”

 

Overview

Following on from the issues discussed during my conference presentation, I will be exploring the concepts of differential reinforcement and extinction, and how they relate to the initial stages of conditioning the secondary reinforcer or ‘charging the clicker’, paying particular attention to context, the emotions and how best to set the horse up for the kind of training that we want to do. I will also talk about the use of ‘start’ and ‘end’ cues and how they can benefit the horse-handler relationship.

 

Introduction

Perhaps one of the most important training sessions that we do with any horse is the first one in which we teach the association between our chosen bridge signal (in this case a click made by a clicker) and food. This is known as ‘acquisition’ but is often referred to as ‘charging the clicker’. During Acqusition, we have the opportunity to build a fantastically strong foundation on which the rest of our training relationship will be anchored. We also have the potential to run into a whole range of problems! So in order to avoid these potential pitfalls, I want to explain the theory behind what I like to do when I am starting out with a new horse.

ACQUSITION: The important facts

Before you can start training using a bridge, you have to ‘charge’ it, which simply means building a strong neurological connection between the bridge and the food. During this process, we are in fact creating permanent brain change by building new neural pathways. It is a critical process, and mistakes made here can have a lasting impact on training. The process involved is known as classical conditioning, and for this ‘acquisition’ or conditioning phase to successfully take place, a number of criteria (the Laws of Association) have to be satisfied:

  1. The bridge must come first, then the food, so that the bridge reliably predicts the arrival of food. This is known as contiguity.
  1. There should be strong ‘contingency’ between the bridge and the food. In other words, acquisition is more likely to occur if the food and the bridge occur more often together than apart. So during acquisition anyway, you must feed every bridge. This is why a unique and previously unheard noise like a clicker can often appear more effective than a verbal bridge- particularly if the word chosen is one like ‘good’ that may already have been used frequently but in a rather unstructured way.
  1. The more frequently the bridge and food are paired, the stronger the association between them will be.
  1. The greater the ‘intensity’ of the food, the stronger the association will be between bridge and food. So using high value treats during acquisition will result in a stronger association.

The optimal delay (for acquisition to occur most efficiently) between occurrence of the bridge and delivery of the food is 0.5 seconds.

The acquisition phase is purely about classical (also known as respondent/ associative/ Pavlovian) conditioning. So in actual fact, we could just click and feed and not pay any attention at all to what the horse is actually doing- but of course food is always reinforcing, and as I pointed out in my first talk, we can’t treat classical and operant conditioning as if they exist in separate bubbles, when in fact they are at play all the time. So we need to be aware of what responses we MIGHT be operantly conditioning, or reinforcing.

Training with FOOD- what does this mean for the horse?

The first, and most important thing, that we need to remember, is that food is a naturally appetitive stimulus, and horses have a natural, unconditional response to it, that is, they do not need to learn about it. Both the smell and the sight of food act as unconditional distal cues of rewards, in other words, they will arouse the SEEKING circuitry.

This causes appetitive activation, in other words, the searching, foraging, investigatory activities that all animals will exhibit before they are in a position to emit consummatory (in this case, eating) behaviours. Simply put, the smell or sight of food will make your horse want to eat the food, and they will do what they can to reach it so that they can eat it. The kinds of searching, foraging or investigative behaviours that they will try will vary from one individual to the next, according to their previous reinforcement history and their personality. These behaviours can include: nipping, nuzzling, licking, biting, nudging, pawing, or any one of a variety of what we describe as ‘mugging’ behaviours.

However, importantly, we know that horses CAN learn that these unwanted attempts to solicit food from a person will be unsuccessful.  Since none of these behaviours are things that we wish to see directed towards a person, we need to work out a way of teaching the horse that they are inappropriate, but that there are other things that they can do that WILL result in reinforcement.

The normal way to do this is through a combination of differential reinforcement and extinction. In other words, the mugging behaviours are put on an extinction schedule, and care is taken to ensure that they do not receive reinforcement, and a decision is made to differentially reinforce another response.

This is most commonly DRI- Differential Reinforcement of an Incompatible behaviour, for example, turning the head away. Teaching a horse to touch a target is also often viewed as using DRI, but this needs to be treated with caution- the first question I would ask is, what is the horse doing just before the target is presented? Since the target becomes a conditioned reinforcer in itself, then the behaviour just before presentation will be reinforced, so we need to be cautious that this behaviour is not any of the ‘mugging’ or overarousal behaviours that we do not wish to see.

Personally, I always like to look for calmness just before the target is presented, so a reasonably straight head, a soft eye and jaw. But you can see, it isn’t ideal to go to using a target too soon, so I am not now personally a fan of using a target during the initial acquisition sessions. In the past, I did use it as a means of focusing busy horses, but I realised that that only worked because I was paying attention to the behaviour BEFORE the presentation of the target. Plus, used this way, the target becomes a discriminative stimulus, or cue, for that calmness, which is very very useful for targeting, but we want that ‘emotional self control’ at other stages of training, and not just when we are using a target.

So why did I stop using the ‘head away’ approach a few years ago? Because, I realised, that if I was teaching head away, I then had to teach head straight, or else I ended up with horses that were standing with their heads turned away every time they weren’t quite sure what else they should be doing. Now I agree that that is a nice safe behaviour, but it wasn’ t always the best start point for the other things that I wanted to train, so about two or three years ago I decided to bypass the head away and go straight to the head straight and calm, something that seemed to work well for my clients and their horses.

BUT I realised that there was a bit of a problem with using this DRI during acquisition- the cue for stand calmly with your head straight, becomes simply the presence of the trainer. I like this for some of my clients, because it is safe and easy for them to work with. In other words, this becomes the default response for the horse, the thing that they do in the absence of any other cue. Why is this sometimes a problem? Because if we then want to go on and look at freeshaping, the horse is less likely to ‘offer’ to try other behaviours. (the same thing happens with the head away). Yet I wasn’t having this problem with my own horses or the horses I was personally training, and that was because I wasn’t actually using a DRI myself, I was using DRO- Differential Reinforcement of Other behaviour. Why wasn’t I teaching it? Because I was trying to protect my clients and keep things simple for them! So what is this DRO and how does it work?

I look for the horse to offer ANY OTHER behaviour, and I reinforce them for it, as LONG as it ISN’T mugging, tension, overarousal, over excitement,frustration or begging. I make an effort not to reinforce the same thing more than once or twice in a row. I look for and gradually shape calmness into the process. This is easier said than done, because we all have a tendency to fall into patterns very easily, and I am no exception!

So I am ignoring (extinguishing) the over arousal, the searching, foraging, investigatory behaviours directed towards people, and differentially reinforcing pretty much anything else. This way I don’t get a horse too fixated on offering any one behaviour, and I keep that creativity and willingness to try in there right from the start.

Start and End Session Signals

First of all, I’d just like to say that EVERY trainer uses start and end signals, whether they are aware of them or not! The horse always recognises the clues in the environment that indicate whether or not there is an opportunity for reinforcement. In some cases, this is the arrival and departure of the trainer. This can work, as long as the trainer is VERY aware of the importance of putting behaviours on cue, and not reinforcing behaviours that are being offered off cue. WHY? Because, if they do get reinforced off  cue, then, while in the presence of the trainer, the horse will always be in the activation phase of SEEKING, without reaching consummation, and this leads to overarousal, and some of the issues that I discussed during my first talk.

In the real world, we all want to be able to be around our horses and have them be calm. When we have a large number of cues trained, then it is relatively easy to cue behaviours and occasionally reinforce with food, to have food on us all the time and have a calm horse. But in the early stages, particularly if the trainer is using freeshaping, this can lead to the difficulties I described above. Hence I like to teach clear start and end cues, that will say to the horse when it is appropriate to ‘try’, to ‘be creative’ and offer responses in the hope of gaining reinforcement. This means I can use freeshaping, and not run the risk of the horse getting frustrated if, outside of the training session, I don’t reinforce them.

It also means, when I am teaching, that I can signal to the horse that they can switch off and have a rest, while I chat to the client, or make a plan, without me having to leave the arena/ walk away from them. In this way I can utilise mini breaks that really are breaks and time for latent learning to begin.

I find that this helps to produce safer, calmer, happier horses that are really good learners.

Control or communication

This structure approach to training and interactions doesn’t create a robotic horse- because of course we still will ‘hang out’ together, have the odd rub or scratching session, at various times when we are together in the same space. There will be a framework and safe boundaries to how this kind of interaction is initiated. This is because communication is a two way street. Cues are the way that we can communicate to the horse what we are thinking of doing next. However horses can also teach us cues, such as an invitation to scratch, or go for walk, or just chill. They can teach us how to understand when they are cold, hungry, thirsty or in pain, as long as we pay attention to them. The key to a happy relationship, to my mind, is one in which we listen to the horse and the horse listens to us. There will be times when we have to over ride their desires,  for safety or long term welfare reasons. In the same way, there may be times when they will over ride our wishes. A thorough understanding of equine ethology will help us to understand when and why this might happen. For example, we may have taught the horse to lift their legs on cue. One of my clients recently described how the farrier came to her horse and gave him the cue to lift his legs, but he didn’t do as asked. She suggested the farrier ask for the opposite foot. It turned out there was a big stone in it. She knew that, because the leg lift had been trained without compulsion, with the aid of appetitives, if he wasn’t lifting it up, he had a reason. Rather than then resort to compulsion, the way a conventional trainer might do, she suggested trying something different. Her observation and ‘listening’ skills were rewarded.

This is what a relationship is- communication and understanding, with respect for the individual.

 

Training Demonstration

Here are two videos of two lovely Exmoor ponies that I worked with at the start of the summer. The video quality isn’t wonderful, but I hope they illustrate some of the points I have just discussed.

Bug: http://youtu.be/CCTIY1ebX8k

Zeno: http://youtu.be/U2V9mAA1V_8

Is Extinction a bad word?

It seems to me sometimes that extinction has become a bad word in the horse training world. Why might this be? Because, in some cases, it can be an aversive emotional experience. This is all about reinforcement history. If a horse has not been previously reinforced for a response, for example, a horse that has never been hand fed or learned to mug, then all we are looking at is non-reinforcement. This generally appears to be quite a neutral experience for the horse, and so they will learn very quickly and easily how to be calm and relaxed around the food. However, if a horse DOES have a previous history of some mugging or begging or overarousal being reinforced, the withdrawal of that reinforcement can be deemed to be an aversive. This can lead to frustration and anger, activation of the RAGE circuitry. So this is why it is really important to take the history of the horse you are working with. The aversiveness is always minimised by careful use of differential reinforcement.

We need to remember that extinction is a natural process. For example, Liebermann (2000) describes how during the salmon migration, which is seasonal, and only lasts for a brief time period, grizzly bears will make daily visits to the river. These visits cease as soon as the salmon go. The bear has a greater chance of survival if they can learn to visit the river only when the fish are present, and to quit visiting when the fish are gone. If we start observing our lives and those of our horses, we will see that extinction is a process that happens routinely, it is a part of life. The better we can understand it, and the emotions that might accompany it, the better we can train.

So if we look at the searching, foraging, investigatory behaviours again, we know that we still need to extinguish these responses- they aren’t safe, and we need to address them if we intend to train with food. By the way, horses can also show the same kind of problems with scratching (horses that have learned to reverse into owners, body block to solicit scratches, and get frustrated if not scratched when they want), so this is about withdrawal of reinforcement per se, not just food.

How do we work with these kinds of horses? Well, we can use protective contact to start off, but, we need to be aware, learning in horses is very context specific, and they don’t easily generalise during the early stages of training. So once we change the context from behind the barrier to being in the same space as the horse, these issues can resurface, and the extinction process will still need to be worked through. It is really really important that trainers are aware of this. An experienced trainer can work through this in relative safety because they are able to read the horse, stay calm, and know how to respond in order to keep themselves safe (i.e. not with confrontation!, and exiting the area when necessary). For this reason, I recommend that the initial stages of working with food should be done either by an experienced trainer or under the supervision of an experienced trainer.

I find that in the long term, it is far easier to address these issues right at the very start of the training process, and not down the line, since down the line they will have become more resistant to extinction because, through inexperience on the part of the trainer, they will have been occasionally (or sometimes frequently!) reinforced.

Learning emotional control

What we are doing is teaching the horse how to manage their emotions, and respond in a safe way. We aren’t asking them to suppress their emotions (or we shouldn’t be). It is essential to be able to read the horse at all times throughout the training process and monitoring how comfortable they are with what we are doing, and responding appropriately to what they are communicating to us. We are simply asking them to behave in a safe way around food, and that THAT is what earns the food, and not the other, unsafe, behaviours that they have been offering.

BUT we need to pay careful attention to the horses’s individual history. One thing to think about is how your horse is managed. Horses are designed to live out in herds 24 hours a day, 7 days a week, 365 days a year, grazing as and when they wish for 14-16 hours spread across a 24 hour time period. If your horse’s natural lifestyle is compromised through restricted feed, turn out etc, they may be more prone to showing frustration around food. In an ideal world I would advise changing the management where possible so that it more closely represents the natural lifestyle, but again, these changes should be undertaken with the guidance of an expert. I do not ever recommend anyone attempts to use food to train a hungry horse.

I find it helpful to ask the following questions BEFORE we commence training with food:

ü  Does my horse stand calmly while I bring them their hay or bucket feed?

ü  Can I stand safely with them while they eat?

ü  Do they make faces at other horses while eating?

ü  Can I remove the feed bucket if necessary mid way through feeding?

ü  How does my horse react if I have a tasty treat in my pocket?

ü  If I hand feed, do they start pestering me for more?

ü  Does my horse crowd my space and push me about?

ü  Do I have a trainer experienced with working with food on hand to advise me?

If you have a polite calm horse in all those situations, then you will probably be ready to safely start work with food. If you have problems in any of these areas I’d advise before you seek appropriate help/ advice before you start. The answers to these questions will tell an expert what kinds of behaviours have been previously reinforced, and how your horse may be likely to respond to the process of extinction/ differential reinforcement.

Finally I’d like to show you a video of Magic, a yearling mare who had a history of being fed titbits at the field gate. She had learned to be an accomplished mugger. Her new owner withdrew all titbits or hand fed treats as a means of dealing with this. So, rather than changing the behaviour, she simply avoided the triggers. This video shows her first half hour training session. As you can see this is a work in progress. She has since had one more session since and shown considerable improvement, however, she will need a few more sessions yet before she can safely be trained with food. This is the exception rather than the rule, but I wanted to show you the kinds of responses you can get with this kind of prior history, particularly in young horses. In the case of Magic, it would have been better to have worked with her either in the field or to have more hay available in the stable. Interestingly, Bug had a similar history as a young horse, and his owner had also withdrawn handfed treats, and as you can see in his session, he responded really well. This difference is, in part, due to training in the field, and also, that he is older and has already learned a degree of emotional control in his interactions with people.

Magic: http://youtu.be/7g309m0HYOs

 

Acknowledgements

Over the past nearly fifteen years I have built on the received wisdom handed down to me by trainers who have gone before, adding to that my academic background and the practical experience I’ve built up with a wide range of horses. I have observed and adjusted my approach a large number of times, and I will probably continue to do so, as we all develop this field together.

I’d just like to take this opportunity to say a particularly heartfelt thank you to Jenni Nellist for allowing me to discuss and bounce ideas around, and who has generously shared the benefits of her own training practice and knowledge- without discussing stuff with her, and of course other trainers over the years, I wouldn’t be half the trainer I am now.  Jenni and I have always felt that calmness should be an integral part of training. This has probably arisen over the years because of our focus on behavioural rehabilitation work, and work with youngstock.

I know for my own part, that emotional awareness, both on the part of the trainer and the horse, is an essential part of training. Thank you to all of my clients over the years who have shown me how valuable ‘clicking for calmness’ really is.

I’d also like to thank Shawna Karrasch for her encouragement and belief in me.

Heather Simpson also deserves a mention for helping me apply all the knowledge gained from my psychology degree to horses, and for her ‘stables as cages’ campaign all those years ago.

Dr Anne McBride, course director on the MSc/ PGDip in Companion Animal Behaviour Counselling, thank you for believing me as a teacher and trainer, I loved teaching on the course and am sad that it is no longer running.

Finally, a huge thank you to all at Hannah Dawson Equine for making the Equine Clicker Conference happen and for providing us with a forum for discussing and sharing ideas for the benefit of all!

Dr Helen Spence has a psychology degree and a PhD on the influence of owner personality and attitude on the behaviour and temperament of the domestic horse. She has been in practice for the last decade as an equine behaviour and training consultant, offering advice and rehabilitation training for problem horses, starting youngsters and teaching owners and professionals how to improve their training skills. She had taught learning theory and equine behaviour from undergrad to postgrad level. For more information visit her website www.helenspencehorsesense.co.uk