Donate
In the first episode of the odor discrimination mini series, we revisit an older episode of K9 Conservationists where Kayla Fratt speaks with Dr. Simon Gadbois to talk about signal detection theory and his conservation work.
Links Mentioned in the Episode:
Where to find Dr. Gadbois: Website | Lab Facebook Page | Lab Facebook Group
You can support the K9 Conservationists Podcast by joining our Patreon at patreon.com/k9conservationists.
K9 Conservationists Website | Merch | Support Our Work | Facebook | Instagram
Summary (AI-Generated)
Introduction: Canine scent detection dogs play a crucial role in various fields, including search and rescue, conservation, and law enforcement. Training these dogs to effectively detect and locate specific scents requires a deep understanding of signal detection theory. In this blog post, we will explore the principles of signal detection theory and its application in training scent detection dogs.
Understanding Signal Detection Theory
Signal detection theory is a framework used to analyze and interpret an organism’s ability to discern relevant signals from background noise or distractors. In the context of training scent detection dogs, signal detection theory helps trainers evaluate the dog’s ability to detect and discriminate the target scent accurately.
The Two Components: Sensitivity and Specificity
Signal detection theory comprises two essential components: sensitivity and specificity. Sensitivity measures the dog’s ability to detect the target scent accurately, while specificity gauges the dog’s capacity to differentiate the target scent from other distractions or non-target scents. These two components provide valuable insights into a dog’s detection capabilities.
Training Considerations
In scent detection training, it is crucial to strike a balance between sensitivity and specificity. Trainers aim to develop dogs that are highly sensitive to the target scent while maintaining a high level of specificity to avoid false alerts. Achieving this balance requires careful training techniques and an understanding of each dog’s unique abilities and motivations.
Grading Responses and Feedback
One aspect of training that often goes overlooked is the ability to grade responses and provide appropriate feedback to the dog. Simon Gadbois emphasizes the importance of teaching dogs to expect graded reinforcement so that you have the ability to give a milder reward for a less-certain find. This can help avoid creating an extremely conservative dog by never rewarding an unclear find, but also avoid giving a jackpot for something potentially incorrect.
Gradually varying the intensity and saliency of reinforcement helps the dog understand the difference between a good response and an exceptional one.
Kayla Fratt shares her approach of using different marker signals, such as “Catch” and “Tug,” to indicate various levels of success. By using different signals and rewards, trainers can provide nuanced feedback to dogs during scent detection tasks. Additionally, the concept of informing the dog, as described by Kayce Cover, can be valuable. By using uncertain pitches or prosody in vocalizations, trainers can convey to the dog when they are close but not exact, promoting adaptability and understanding.
Challenges and Flexibility
The field of scent detection training can sometimes become overly focused on rigid protocols. However, maintaining flexibility and adapting to individual dogs and situations is crucial. Dogs vary in their abilities and preferences, and trainers must be willing to adjust their training approaches accordingly. The goal is to develop versatile dogs capable of adapting to new environments and challenges.
Problems with Percentage Correct
Using proportion or percentage correct as the sole performance metric in training scent detection dogs can be limiting. Signal detection theory offers a more comprehensive understanding of a dog’s detection abilities by considering factors such as sensitivity, specificity, and bias. By incorporating these measures, trainers can gain a deeper insight into a dog’s performance and make more informed training decisions.
Conclusion
Signal detection theory provides a valuable framework for training canine scent detection dogs. By understanding the components of sensitivity and specificity, as well as incorporating graded responses and feedback, trainers can optimize the training process and enhance a dog’s detection capabilities. Flexibility and adaptability are key in ensuring dogs are prepared for a variety of scent detection scenarios. By embracing the principles of signal detection theory, trainers can unlock the full potential of their canine partners in scent detection tasks.
Transcript (AI-Generated)
Kayla Fratt (KF)
Hi all, you may notice that this is a re-release of an older episode, that’s because I’m kicking off a mini-series on discrimination tasks for our conservation detection dogs. Basically, the question is, how do you keep your dog specific to a given target odor in the presence of other similar target odors or distractions. This topic has always fascinated me; in the world of bomb and drug dogs, it’s common to spend a lot of time working on training the dog not to alert to gloves, containers, or other items that co-occur in the training situation, and also work to ensure that the dog is, for example, alerting to an accelerant plus a fuel in the case of a bomb dog or an accelerant dog, not just, say, alerting to sugar or gasoline on their own.
However, it’s often suggested in the conservation dog world that intentionally introducing your dog to an off-target odor in a line-up situation in order to ensure that your dog is not, say, alerting to red fox scat when you want them to find bobcat scat, may later on cause the dog to recognize and alert to that off-target odor in the field.
This has always really piqued my curiosity, as it’s so different from what is taught in other working dog fields, and I spend a lot of time listening to podcasts and reading books from other folks within the detection dog world, not just within detection. There’s a chance that something’s different here, and what we’re hoping to do is investigate this from a lot of different angles with a lot of different people.
This mini-series will start with a re-release of an episode on signal detection theory with Simon Gadbois, which will lay a technical, scientific background for signal detection theory. I know it’s a dense, nerdy one, but I think it’s really important for this topic.
Next week, we’re going to revisit our episode with Paul Bunker about training “all clears,” and other methods for potentially rewarding the dogs in the absence of a target odor. Next up, we’ve got case studies lined up with Conservation Dogs Collective involving New Zealand mud snails, Conservation Dogs of Northern Ireland involving a really cool moss project. We’ve got another episode with Paul Bunker of Chiron K9 and then also going to cover our own past work with Action for Cheetahs in Kenya. I may be able to squeeze another couple of episodes in here as well.
We’re really just trying to explore the topic of how we keep our dogs specific in the presence of other odors, such as species of the same genus. We probably won’t quite get to the bottom of this question, as things are going to vary based on your actual target, your own training plan, your dog’s personality, and so many other things, but we’re going to try our best. Without further ado, here’s our episode with Dr. Simon Gadbois.
Kayla Fratt (KF)
Hello, and welecome to the K9 Conservationists podcast, where we’re positively obsessed with conservation detection dogs. Join us every other week as we discuss ecology, odor dynamics, dog behavior, and everything in between. I’m your host, Kayla Fratt, and I run K9 Conservationists, where I train dogs to detect data.
Kayla Fratt (KF)
Today, I have the absolute joy of talking to Dr. Simon Gadbois about signal detection theory and how it relates to our conservation detection dogs. This conversation is dense and fascinating; if you’re someone who like me tends to listen to podcasts at an accelerated speed, I would recommend slowing down for this one. Potentially listen to it while you’re cooking instead of driving so you can take some notes and pause to look things up. It is a fascinating discussion, I think you’re gonna really enjoy it, but it is it is dense.
Kayla Fratt (KF)
Dr. Gadbois studies olfactory processing in canids and the application of canine scent processing in wildlife conservation dogs. He works with Nova Scotian reptiles like northern ribbon snakes, and wood turtles. He also works with dogs used for biomedical detection, diagnosis, and assistance for diabetes, anxiety, and PTSD.
Kayla Fratt (KF)
Before we get started, I’m going to remind you that our field vehicle repair fundraiser is still going. As we record, the van is finally ready to go and Barley, Niffler, and I are ready to start our field season. Our repairs were about $4,000 over budget; you can read all about that saga over on our Instagram or on K9Conservationists.org, where I’ve kept everyone updated on the repairs and the minor nightmare that’s been. Any support you can give to the fundraiser is super appreciated. I know that money is tight, so even if all you do is share the link, that’s helpful. You can find the link in the show notes and at K9Conservationists.org. You can also help the podcast by leaving us a review on the Apple Podcast store. Your reviews make my day, and they help other people find the podcast.
Kayla Fratt (KF)
Welcome to the podcast! Simon, can you start out by telling us a little bit about your current work in Nova Scotia at Dalhousie University?
Simon Gadbois
Yes, around 2006 we started working with wildlife conservation canines. It’s something I had been doing since the early to mid 90s, but not in a formal way. Around 2012, we also started working on biomedical stuff, including hypoglycemia detection in humans, and more recently, anxiety attacks in humans, in people with PTSD. I’m kind of moving away from the biomedical stuff, mostly because I think electronic noses are going to take over eventually. I don’t think this will happen with the conservation work just because it’s field-based, and I think dogs will keep an advantage there. No doubt.
Kayla Fratt (KF)
Yeah, I agree. I could see how it would be much easier to make an electronic nose for a medical application versus trying to find something that’s self-propelled and waterproof for the field.
Simon Gadbois
There was a documentary released recently, and I was asked to comment on a few things they had done. They were showing how humans are a little bit stupid the way they do things. The person was looking for something with one of those devices in an arena, and it was taking forever. They released a dog halfway into a five minute time span, and the dog found the source of the smell within, something like 10 seconds. If you’re talking about search patterns, about the motor component to it, the looking for something – what we call trailing or air scenting – I think electronic noses may eventually get there, but it’ll probably be decades from now. They don’t work terribly reliably even if they’re right next to the order source. They also happen to be incredibly specialized, which means that, if you need to look for something different, you need to reprogram the whole thing. I think with field work, the dogs will have the upper paw for a long time.
Kayla Fratt (KF)
That’s good to hear for job security on my end, and I assume on yours as well
Kayla Fratt (KF)
Our main topic for today is signal detection theory and how we can use that for training our dogs, testing our dogs, and fielding our dogs. Can we start out by explaining what signal detection theory is? You’re the only person I’ve heard in this field who talks about it, and I don’t think a lot of our scentwork people know enough about it.
Simon Gadbois
Yeah. I know Nathan Hall did a talk about it at some point. We don’t hear much about it because a lot of people that come into this have a background in learning theory. I have some of that, but I also have a background in psychophysics and sensory ecology, and I think I have come more from the sensory processing perspective than from the learning theory side.
Simon Gadbois
Signal detection theory has been around for a long time in psychophysics; Green and Swets is the seminal paper that started this. It was something that initially was of huge appeal for engineers. It’s an interesting continuation of information theory, because it’s about what kind of decisions you make when you’re uncertain. And that’s really the core of it. Because signal detection theory doesn’t work if the conditions are ideal. If it’s really easy to detect and discriminate between odors then you’re wasting your time trying to use it. But when you work with either a challenging stimulus, you know, a stimulus of low saliency, or a very challenging environment where there’s a lot of turbulence, a lot of wind, harsh conditions, or it’s really dry, then it becomes useful because it tells you what kind of errors the animal makes. That’s really the crucial part of signal detection theory.
Simon Gadbois
The paper we wrote on this, which actually was a proceeding for a conference is way too short. But another thing that I think explains why we don’t hear much about signal detection theory in olfactory work is that it has been mostly used for physical stimuli, so auditory, visual, and tacile stimuli. But chemicals stimuli like taste and olfaction? Not that much. But there’s really no reason why it wouldn’t apply. But it’s not very common in the olfactory world. There’s no doubt about that.
Kayla Fratt (KF)
Yeah. So, it wouldn’t be very useful for something that’s really clear. For example, if you were trying to tell the difference between purple and yellow, you wouldn’t really need it because there’s not a lot of overlap. But if you were trying to decide at what point blue becomes aquamarine becomes green. Is that where signal detection theory starts being useful?
Simon Gadbois
Yep. That’s one of the examples I give my students. If there are no errors in the data produced by the subject, the dog in our case, then it’s useless because you need those errors to do the math. It assumes that there’s some level of difficulty to the task, but if not, then percentage correct is perfectly fine. But your percentage correct should be then astronomically good, right? So 98%, 100%, in which case, why would you need signal detection theory? That’s where it really loses its purpose.
Simon Gadbois
There are cases where, with my students, we start using it but then we eventually abandon it, because they say, “Well, I can do the calculations. I don’t have enough error terms here to, you know, to to get even the D prime, and certainly not the bias.” And then I say, “Stop, then just use percentage correct, then you’re fine.” If the, dog starts being at 97 and above, typically, mathematically, you’re kind of losing the capacity to get something meaningful from signal detection theory data.
Kayla Fratt (KF)
Gotcha. So you have to have some amount of error. And then we’re looking at whether the dog is more likely to say that something is something when maybe it’s more like errors of commission versus omission. Basically, that’s kind of what we’re looking at.
Simon Gadbois
In signal detection theory we call it false alarms and misses. You need some misses, and you need false alarms. So, false positive, false negative. There’s an overlap here with diagnostic theory, because a lot of this, in fact, is kind of repackaged in a different terminology in diagnostic theory, which is a way of extracting the kind of mistakes. So again, false negative false positives, and how they come together in judging the subject.
Simon Gadbois
What I like about signal detection theory is that it gives you at least two parameters that are really important. One is the D prime, which is a measure of either, and that’s very important, detectability if it’s a true detection task, or a measure of discriminability, if it’s a discrimination task. Now, this is where in the olfactory world, we’re mixing up all these words, and they don’t mean anything anymore, because in psychophysics, it’s very specific what they mean. In psychophysics detectability is when the subject – human, rat, dog, whatever – just says, “Yes, it’s there,” or “No, it’s not.” That’s it: presence or absence. That’s detection.
Simon Gadbois
But as you know, in our industry and our business, and in the science of it, we say detection for almost anything, including the search work that dogs do in the field. But it’s so much more than that, at least based on the way it’s conceptualized in psychophysics.
Simon Gadbois
The other way to do it is if you have stimulus one and stimulus two, what’s the difference between those two; that’s discriminability. And signal detection theory works here, if you have an S one, or I should say, maybe an S plus and an S minus. When you start having more stimuli, like in a lineup or a carousel, that’s where signal detection theory starts to fall apart, and then it doesn’t make very good predictions. You have to do some mathematical adjustments, and the reason to use it starts to get lost in the complexity of the math. The math is not actually complex, but it’s that then you’re really doing an approximation of an approximation of an approximation. And then you really lose the the purpose of it. this is why in our tasks, typically, we use a lot of scintillation vials. In the field, we collect our samples with this, typically from our reptiles. And the idea would be that you put this in your device, whatever it is, we use funnels on the ground, but it can be any kind of other arrangement. The target scent is there, and the dogs sniffs it and then needs to make a decision. In a real detection task, you would have either an S minus, in this case, there’s nothing in it. And I’ll just use the one with a cap to indicate the S plus, which has the target scent. In this case, the dog would simply keep the nose on, it’s actually not directly on it, but it’s on the front –
Kayla Fratt (KF)
Kind of a sustained nose touch.
Simon Gadbois
Yeah. For five seconds, we count it 1000, 2000. That’s the dog committing to the answer. It’s better in detection tasks, especially for signal detection theory, if you commit the dog to a clear “No” as well. That’s what we call the yes/no procedure. In this case, the S minus the dog would sniff and then “No, it’s not it.” We have them sit back. So that’s “No, that’s not it.” Now, there’s another way to do it, which is the one that we hint at, and I regret this now, but in that little paper from forget one 2016, I think, is to do what we call a go/no go. So the “Go” is, again, commitment. So, the five second hold, and “No Go” is simply walking away. The problem with the “No Go” when they’re just walking away, is that it’s a little bit ambiguous. So again, signal detection theory typically likes a clear “Yes” and a clear “No.” If you don’t have that, you can always throw some questions about what was that really a no? Right?
Kayla Fratt (KF)
Yeah, did the dog perceive it? Was the wind just the wrong direction at that moment? So they didn’t actually say no.
Simon Gadbois
Yeah, now, I have to say this is mostly lab conditions. In lab conditions, my students don’t like that some dogs don’t have a no response, and you need to train it. And obviously, in the field, you don’t want a dog that will always sit back. We need to make sure that the dog understands that this applies to the testing, either during the initial training or in maintenance training later. Because sometimes, you work the field and you’ve got to figure out if the dog is off. Has the scent changed, which happens a lot with our reptiles. Or is it the field conditions today? Although it’s a weak scent, this dog seems to be a little bit off. For instance, we find more turtles than the dog does. Then you go like, what’s going on? That’s when you go back to the lab and retest the dog to see, are they starting to look for something else? Have they forgotten what the odor is? What’s what’s going on?
Simon Gadbois
Technically, you could do the go/no go as we say in that in that paper, and it’s fine. If you’re satisfied with the dog just walking away from the odor as a no, that’s fine. But in strict signal detection theory, the argument I make now is that it’s better if the dog can commit to a no, except if it interferes in the field. If a dog constantly sits back to say “There’s no turtle here.” Well, okay, then I understand. It’s problematic, but most dogs will learn the difference between the lab and the field.
Kayla Fratt (KF)
Yeah, it seems like it’s been pretty obvious to most of the dogs that I’ve asked to do both. I’m trying to imagine doing a plant survey where the dog is like, “Not this one, not this one.” You’re never gonna get anywhere.
Simon Gadbois
It would take forever.
Kayla Fratt (KF)
Yeah, it would take days to clear a 10 meter plot. B
Kayla Fratt (KF)
So, basically, so signal detection theory is a way for us to get an idea of whether our dogs are giving false positives or false negatives, or, misses versus false alarms. And this is something that we would want to do prior to fielding to make sure that we understand where our dog’s at, and potentially to troubleshoot afterwards. Am I following?
The concept we need to add here is the concept of bias, because that’s where signal detection theory shines. It will tell you what kind of dog you have. In other words, is your dog a liberal decision maker, or a conservative decision maker. That’s a lot of different things. It can be temperament as a dimension of personality, it can be the training, it can be the reinforcement history, it can be all kinds of things that make a dog one way or the other. That’s very important, because, as you know, in some areas, you want extremely conservative dogs as opposed to liberal. The typical example I give my students is mine detection. In mine detection, you’d rather have a dog that does a lot of false alarms, because the cost of a miss is that the dog and potential yourself just blow up on the mine. At the same time, there’s other examples. We had a contract, for instance, with the Canadian Forest Service. In that case, they had a problem with larvae from a specific insect that was damaging trees. And the method they were using at the time was producing a lot of false alarms, which meant that they were cutting and destroying more trees than needed. They would cut down trees and then realize, darn, there was actually nothing here. For the lumber industry, that’s actually a potential loss that accumulates with time and becomes a problem. What they actually wanted in this case were more conservative dogs. They didn’t want dogs that were doing a lot of false alarms. Tell us only when you’re really, really sure there’s something. A dog that, when unsure, is more likely to say no than yes; that’s a conservative dog. And the liberal dog says more yes than no when they’re not sure.
Simon Gadbois
But the beauty of this is that, knowing that it’s partly the dog, most dogs will have a bias. But one of my best dogs, Zillah, for instance, was what we call the ideal observer, she was right smack in the middle. She did about an equal amount of misses and false alarms. In some cases, that’s that’s exactly what you want. But many dogs will fall on one side or the other of this bias. They’re either liberal or conservative, and in some cases, extremely liberal and extremely conservative. Sometimes that works for you, sometimes it doesn’t.
Simon Gadbois
The beauty of this is that technically, via training, you can change this. So if you have a dog that you find a bit too conservative, you want that dog to be a little bit more liberal, it’s easy. Typically, what you would do is, when a dog sniffs and seems to be hesitating, and especially if you know that the target happens to be there, you would immediately say, “Good dog,” click, or, you know, here’s the kibble or whatever you do to reinforce your dog. Now, that’s a dog that learns that, “Okay, if I think it’s it, maybe it’s it, and I should say it is it,” as opposed to the other pattern where you may want to withhold any kind of reinforcement if the dog is showing uncertainty.
Simon Gadbois
In wildlife conservation canines, we have a bit of an issue when a dog gets to a scent, and the target is not there, right? This is a typical problem that happens to us. We have to make decisions about what’s good for us or right for us in that case, right? I have a tendency to give the dog the benefit of the doubt, especially if in the context, it makes sense. And usually with wood turtle spawn, since that’s one of our primary research subjects, they often leave some kind of mark on or sign on the ground. That helps me decide, yeah, you’re right, there was a turtle there this morning. I tend to be a little bit more generous with my dogs, because I don’t mind them being a little bit on the side of false alarms, meaning being liberal. And we gather this data as a data point, there was a turtle here this morning or yesterday or whatever. So that becomes a GPS point. But I could see why some people would like to say, “If there’s nothing there, I’m not interested, find me the target,” in which case, obviously, then you want a dog that will just give a strong answer or response only when the target is actually present or extremely close. Or recent in time.
Kayla Fratt (KF)
Yeah, I’m thinking about some of the scat projects I’ve worked on where, we actually need enough scat present that we’re able to collect data to send to the lab. So generally, if the dog is alerting to something that you know was there, but has now been largely eaten, that’s not generally something we want the dog to be focusing on. I was out once with a dog that found a site where a puma had cached a dead sloth and a dead armadillo in an area, but there was no scat, but it reeked of cat. And I was just shadowing a handler at that point, so I can’t remember whether or not they decided to reward or not. But you know, it’s one of those things where it’s like the dog was absolutely correct. We could smell it, it was clearly a cache, but there was no scat to collect. Depending on who your funders are and what the goal of your project is, you may or may not decide to reward.
Simon Gadbois
This is one of those cases where I think a dog that is tolerant of being on intermittent reinforcement is important, right? And this is where a lot of dog trainers these days, don’t understand intermittent reinforcement. But that’s a such a great example of a situation where if the dog has not experienced intermittent reinforcement before, they will indeed get frustrated and stop working.
It’s a great case of, I was going to say negotiation, but that’s not really the word. But there’s a kind of like implicit understanding that you need to have between you and a dog that, “Yeah, I know, you’re not too sure. But yeah, we agree, something is happening.” And then you reinforce or you don’t, but you don’t leave the dog hanging too too much, either. I’ve seen this in the field, and this is exactly where a dog will fall to one side or the other of what we call the ideal observer in signal detection theory. So it’s where you technically control it in the field; in those moments where the dog’s not sure, you’re not sure, but wow, okay, I do smell the cougar here. So yeah, you’re right, but what do I do? What’s the goal of this study? What do the stakeholders actually want? They want scat, but it’s not here, so what do I do? But the cost of this depends a lot on the cost of your false alarms. If there’s really no cost in the dog giving you that false alarm, I say reinforce. After all, you wan the dog to find, you know, cougars scats in this case. So go for it. Just reinforce it. Worst case scenario, you hit a few spots where it does nothing, but it’s better than missing it.
Kayla Fratt (KF)
Yeah, and as kind of a nerdy naturalist, I was pretty excited to find a dead sloth and armadillo, so I don’t mind if my dog tells me about that, as long as it’s not something that’s so frequent, that it’s going to slow down your searches. I think this is where signal detection theory and understanding what errors your dog is likely to make is probably at its most important. The example we use a lot in North America is, say you’re trying to find Bobcat scat, and red fox scat is pretty difficult to distinguish visually from Bobcat scat. As a handler, it can be challenging if your dog is starting to alert to stuff where you’re just like, “Well, I’m not really sure if that’s Bobcat or red fox,” and there are enough red foxes where it would be problematic as far as lab fees and survey time for your dog to be finding every red fox when you’re trying to find Bobcat. So I’m guessing that is where really understanding the sort of errors your dog is likely to be making would be incredibly important.
Simon Gadbois
Yes, and to me, it shows the importance of a dog that does not always expect a reinforcement when it finds the target. Because then you’re creating an extremely conservative dog, which may not be what you want. The other dimension of this that becomes important is, I think you are allowed to grade the way you respond to a good or potentially good response. When we have a target in the field, if the dog gets right to the turtle, we give a huge party, “Yay, good dog!,” Click, kibble – although, honestly, our dogs don’t care about kibble. At that point, they’re so excited and so self-reinforced in the field that they often don’t pay attention to that at all. Contrast that response to “Yeah, good dog.” In other words, they get the point that you get really excited when we find the actual turtle, and it’s right there, yay! Versus Yeah, you’re still supporting me, A for Effort. But it’s crazy, the number of trainers that don’t seem to understand that concept that it’s okay to grade the response. It’s not always the same intensity, the same response, the same saliency? No. Why not be able to grade this in terms of you know, yeah, it’s a B+ versus an A+, great comparison.
Kayla Fratt (KF)
Yeah. My dogs have different marker signals. I don’t usually use a clicker in the field because I’m not gonna walk around with my finger on the button.
Simon Gadbois
People don’t understand we don’t have free hands in the field!
Kayla Fratt (KF)
No, I’m usually using them to keep myself upright! Or pull ticks out of my hair, or something. So we use “Catch” and “Tug,” where one’s for a thrown toy, one’s for tugging. My dogs have different motivations; one dog’s tug is better than throw, and the other is the other way around. And then both of them also have a “Yes,” or a “Find it,” which are either food delivered to their mouth, or food delivered on the ground. Food is less exciting than toys for both of them, so if I’m not totally sure, but I think there’s a good chance that they found the target, then I can grade my response to them. And then if I’m really not sure, I don’t think this is it, then I might just get some like kind of casual verbal praise, we’ll take a water break, and then we’ll keep going. So we can really grade our responses.
Kayla Fratt (KF)
It is really surprising to me how resistant people are to that in this field. And maybe this is something that at some point, someone – you – need to do a paper that shows that we can give a less exciting reward without creating false alarms. I think most people are more concerned about false alerts or false alarms than they are about misses. But maybe that’s not quite accurate.
Simon Gadbois
Kayce Cover has this interesting idea of informing the dog. She calls it hot or cold. It’s a control condition, most likely in the lab or set up in the field. And I think that’s very useful, too. But you would be surprised, again, how many trainers are totally against this idea. I’m thinking why not tell them? You’re close, but not exactly. She does it with this sound, and I’m not sure I would agree with that, but she grades that and faster, or louder, I think something like that. And I think that’s a great idea. I do it with my voice; canids are intrinsically very attuned to vocalizations and what we what we call the prosody in human language. And I think it’s very important to use, and you can modulate it. So if you’re uncertain, you let the dog know that you are uncertain. Using an uncertain pitch, “Yeah, I think so. Good dog. Good dog.” Eventually they go, “Okay, we understand each other. We’re not sure about this.”
Kayla Fratt (KF)
Yeah, Patricia McConnell’s entire PhD work was all about how responsive many of our domestic animals are to consistent changes in our vocalization. I grew up with this beautiful Labrador Retriever that I so wish I could own again today, because she would have just been the perfect working dog. And when we threw her tennis ball, we had a big Chuck-It, and we lived on 40 acres, so we’d lose those tennis balls out in the 40 acre fields all the time. And my dad and I taught her that, if she was at least orienting towards where we thought the ball had last gone, we would kind of go “Yeah, yeah, yeah!” and get louder and then go quiet again if she if she circled away from it. And this is just a lab playing fetch in the backyard, but it worked really well. I haven’t done it with my search dogs as much but that would be an interesting thing to play around with.
Simon Gadbois
Yeah, and especially in early training, or when transferring the dog to a new scent, I think it’s totally justified to guide them a little bit, and do the coaching the same way we would in any kind of other kind of training or learning with our children. Like, “Yep, you’re getting there!” right?
Kayla Fratt (KF)
My puppy is about seven months old now, and he’s getting ready to deploy on his first project. And in the last month, I’ve noticed he’s really good at going out and finding the target. Right now, we’re working on dead bats. I’m trying to build in an actual alert for him, because he was going up, finding the bat, and then running back to me for reward, which could be an alert. That would be just fine. I know that’s what a lot of search rescue people do, but I would prefer to have a dog who stays at target and alerts to it. So in that case, I was reinforcing him a little bit for coming back to me and then guiding him back to it and coaching him into that alert a little bit in a way that I would imagine make some detection trainers cringe.
Kayla Fratt (KF)
I think like the industry has swung a lot. You see things like traditional narcotics vehicle search, where you’re presenting everything to the dog, and you’re right up the dog’s nose. I’ve also seen a swing in the opposite direction where we want the dog to pretend we don’t exist, and we want to be so thoroughly disengaged from the search, which is certainly valuable. And it’s a cool skill for your dog to be able to search independently. But that doesn’t mean that we have to do 100% of either one of them all the time.
Simon Gadbois
I think it depends on the dog. It depends on the situation. I think sometimes we’re just a little bit afraid to be versatile, right? Versatility is important so a dog can adapt to new situations. For example, I’ve had dogs that are fantastic at training in the lab, but as soon as we get into the field, they’re not connecting the dots. In fact, I’ve had two that never did. And I don’t know if it’s because they don’t like to work in the field, they just like the lab part and think that’s fun. And at the other extreme of this is a dog, Flynn, that I brought to the fields, and it was his first time to the field. He watched Zilla work, and an hour later he was finding more turtles than Zilla and he had no training whatsoever. And he became our turtle dog until he died in February. He was our best dog by far. So yeah, dogs are amazing, but you need flexibility. And I think you need to see it. And what I don’t like about this industry sometimes is, it seems we’re not flexible. We’re so obsessed with protocols. This is what you how you do it, and every dog must go through the same procedure. Look, we’re not working with rats and Skinner boxes here. We’re working with a much more socially engaged species with humans. And let’s just take advantage of it, right?
Kayla Fratt (KF)
Yeah, absolutely.
Kayla Fratt (KF)
Hey, everyone, just popping into this episode with an update on our Patreon. We still have the $3 a month Doggie Detector level, which allows you to ask questions for me and the guests to answer each episode. But that also lets you join our monthly training video analysis calls. These calls are going to be recorded, of course, and we’ll also publish the video afterwards for everyone to view and ask questions about prior to the call to ensure that all timezones can have participants fully. So we’ll basically publish the video we’re going to analyze so that you can ask questions and view it and prepare ahead of time, then we’ll have the call where we talk about it, we can have beverages, it’ll be a good time. And then all of that is going to be shared later. So you can participate before, during and after again, just for $3 a month.
Kayla Fratt (KF)
Now, at the $10 a month Scensational Scientist level, you get everything that we got before at the $3 level, plus you get to submit videos of your training sessions for those calls. So this is perfect for the aspiring Canine Conservationist. And your target odor doesn’t really matter here as long as you do communicate what it is so we can think intelligently about your goals. This is going to be great for nosework competitors and other canine handlers as well. We’re really striving to make these video calls super kind and supportive and helpful. It’s going to be a nice safe place on the internet to get good feedback on your training sessions, because I know how much of a struggle that can be especially in the scentwork world.
Kayla Fratt (KF)
Then finally, the Canine Conservationist patrons get everything from those other two tiers, plus a private 30 minute training call with me to go over whatever you’re running into with your dog. That tier is just $25 a month, and that’s cheaper than booking my time at journeydogtraining.com for behavior modification, and that’s just because I love you and I love my patrons. That’s definitely something to check out. You can join that Patreon over at patreon.com/K9conservationists or at the link at K9conservationists.org. It’s a tiny link up in the top bar, and we also drop that link into our show notes. If you’re listening on your podcast app, you should be able to find it right from there. Thank you guys so much, and let’s get back to the episode!
Kayla Fratt (KF)
Let’s circle back a little bit more to signal detection theory. I wanted to talk about the problems with proportion or percentage correct as our performance metric, because I know we alluded to that earlier on. Can you explain why we don’t like the idea of just saying, “All right, we did a bunch of lineups, that’s our lab pre training. Now we’re going to see what proportion that gets correct.” And then we know the dog is ready to field or not. What is the problem with that setup? And we can talk from the proportion correct side and the lineup side, because I know we’ve got things to say about both.
Simon Gadbois
So technically, there’s no problem with proportion correct. Even if you do the single presentation, or what we call Two AFC, a choice between two or three, or four or six, and that becomes the lineup procedure. So there’s nothing wrong with it, you do a male test, and you figure out if they’re above chance or not. But in signal detection theory, if you go with that thinking, you’ll get more; you get a lot more data. You get the bias, or this criterion, as we call it in signal detection theory, and you get that D prime value, which is not terribly intuitive, you’d have to spend a little bit of time looking at what it means. A lot of people think it’s a linear measure, and it’s not. Often, textbooks will tell you it’s between zero and four, which is not exactly true, it can be above four, so two is much better than 50%. So anyway, you need to get familiar with the measure, essentially. So, you get the D prime, you get the different types of criteria and different ways of calculating it, and all that. There’s even a nonparametric version, by the way of signal detection theory. It’s called the A prime instead of D prime. So there’s the whole mathematics of it.
Simon Gadbois
Signal detection theory, from that perspective, is just a quantitative tool that gives you more information about where your dog is at, assuming you’re interested in that. Some people may not be. For diagnostic purposes, it’s very important, in my opinion. But most biomedical research will just report specificity and sensitivity, which is fine. But it’s related to all of this right. Specificity measures how conservative the dog is; is the dog able to differentiate this stimulus from others? Sensitivity measures how detectable a stimuls is. So these are complementary measures. And in most of the calculations that we have, you know, accuracy is actually the average of specificity and sensitivity. So if you have 90%, accuracy, you may actually have 95% sensitivity and 85% specificity, or the reverse.
Simon Gadbois
There’s a reason why I’m going there, by the way, mentioning specificity here. In psychophysics, the lineup is instrinsically a specificity task, because it’s the one s plus, meaning it’s measuring how discriminable – not detectable, it is from others. So you’re biasing a dog towards being very good at specificity. And this is why most of those studies have a higher specificity than sensitivity. There are exceptions, but a lot of dogs in diagnostic work have very high specificity and lower sensitivity. And that’s because you set them up for this.
Simon Gadbois
There have also been studies done with a percentage. Let’s say that you have 10 trials in a session, you would have probably one out of 10 trial where the lineup is blank. So the dogs as the opportunity to say there’s nothing here, right? Because otherwise the dog will think, “Anytime I walk into this room in front of the lineup, there is a response and answer.” It’s not true. And there you really bias the dog to always alert to something when actually it’s possible there’s nothing.
Simon Gadbois
Typically about 10% of your trials would be a blank lineup. The other 10% are lineups where there’s actually two targets. The argument Schoon and Haak (2009) make here is that, often dogs will run the lineup, and as soon as they find a target, they stop. And what you actually want the dog to do is be able to complete the lineup just in case this is not really the target, or there’s a better target. In other words, there’s a scent that smells more like the target scent than this one. That’s something we discussed in Gadbois & Reeve 2014, this idea that you have 10% blank lineup, 10% two targets, and then the rest is one target. But you’re still setting up the dog entering the room to think that most of the time there will be a target in there. When the reality of the field? Nope.
Simon Gadbois
Especially in the kind of work we do with endangered species or species at risk, you can spend a full day looking and finding nothing. By definition, they’re not out there much. So that is potentially a problem. Again, if your dog is used to reinforcement, it may be not just important, but essential, because in the field, they may deal with what we call rare events, right? And even in humans we know this, they’re looking for rare events. The example I often give here is the NSA example. Jeremy Wolf, a former postdoc here at Dalhousie University in cognitive psychology, figured out that one way to keep NSA agents alert and motivated during their work, when they look at all the luggage going in front of them, is to have the computer engage them in a kind of video game. In other words, the system plants, weapons and explosives in there, and the person does a strict detection task. Yes, nope. Yes, no. Now, if you say yes, and the system just planted a grenade. Well, good, you say yay. But oh, by the way, this was just part of the game. But if the person says yes, and the system goes, wait a minute, there was no grenade in there that was planted, you better check that luggage. And that’s simply keeping the person interested. So what’s interesting about the kind of work we do is that we are asking our dogs to perform in rare event situations, it’s the same.
Simon Gadbois
It’s the same with dogs that do hypoglycemia detection. Hypoglycemic events in kids and noctural hypoglycemia may occur once a month. People realize that dogs have stopped working after a while because they never get the event, they never get to be reinforced for it, so that’s why it’s kind of problematic. What we’re basically saying is, the lineup sets the dog up for knowing that there’s a target somewhere in the room when they get in there, when the real world is not like that. This is why detection tasks are different. When a dog gets in the room, there is a container with or without the S plus in there, and says yes or no. And you can adjust the occurrence of a yes to something that closely matches what’s actually happening in the field to make sure that your dog is not too sensitive to that expectation that 50% of the time there’ll be something there versus only 10% of the time.
Simon Gadbois
But in a lineup intrinsically, the dog knows. It’s not a detection task, it’s a discrimination task. It’s more about comparing this to this. And occasionally the real world can be like this, like the work we did with snakes. We were looking for ribbon snakes, but sometime you would find garter snakes which are from the same genus and most likely have the same motor. But the real world of a search dog is rarely like that. A lot of the time it’s literally is it there? Can you find this and you don’t have a comparison.
Simon Gadbois
The way I explain this to my students, I say it’s the same as a difference between a multiple choice exam versus a true/false. In a multiple choice exam, you can compare and contrast A to B, A to C, A to D, etc. Some people say, well, it can be harder, because then you confuse the animal more with possibilities that are very close to the target. True. But again, it depends on the multiple choice test, on the question. So it depends on your lineup, essentially.
Simon Gadbois
I experimented with true/false with my students would final exams, and I can tell you, it’s not necessarily easier to give a real good, true/false answer to a true/false exam is not as straightforward as students actually predict if you were just to ask them is it easier. The score on the true/false exams I have done with my students’ final exams are about the same as with a multiple choice, despite the fact that they say, Oh, this is going to be easy. It’s a true false exam. Yeah, well, it depends on the statements you make, right. So again, it depends on the stimulus that is given to you.
Simon Gadbois
So now, remember that true/false is better if you want to use signal detection theory, than if you have a lineup or a multiple choice. Now, like I said, the books that I have, will tell you that you can actually have a two way FC, so a two alternative forced choice or three alternative forced choice, etc. when applying signal detection theory. But like I said earlier, the problem is, let’s say you have a regular lineup of six items, that’s a six AFC. You lose the predictive predictability of the model, the model becomes a lot more speculative; as we would say in stats, it’s not as robust. So it’s doable, but it’s not advisable. And you’re really kind of guessing what’s going on if you try to apply signal detection theory. If it’s a two-way, you have to do the square root of D prime, basically. So if it’s six AFC, you can see how it gets a bit ridiculous. At some point, you get really strange, low numbers that are kind of meaningless. So that’s why it’s not advisable to do it with with a lineup. Again, theoretically, you can, but most people avoid doing it.
Kayla Fratt (KF)
Yeah. So for people at home who are interested in trying this out with their own dogs, whether they’re competitive scentwork trainers or more on the conservation dog side of things, are there good places to go as far as finding these formulas and learning how to set up these choices for their dogs so that people can actually get a good idea on where their dogs fall?
Simon Gadbois
Well, yes, there are a few resources. One you can find online is from Memorial University, here in Canada, that’s a little Javascript app that I’ve been using with my students when they want to calculate their D prime. The books I just showed, although they get technical, it doesn’t have to be that technical. I know there’s spreadsheets out there that are used by people to do quick calculations. But here’s the thing; the basic thing that you need is this quadrant. You need to be able to fill a matrix based on four scenarios that will give you the hit, the false alarm, the miss, and a correct rejection. So in other terms, true negative, true positive, false, negative, false positive.
Kayla Fratt (KF)
Yeah.
Simon Gadbois
That means that in your setup, you need the situation where the stimulus is there and the dog can say it’s there, that’s a hit or true positive. You also need a situation where the stimulus is there, but the dog says it’s not, that’s a miss. And you so you get the point. You just need to be able to put numbers in all those four in that matrix of four cells. And to calculate a D prime, really all you need is hits and false alarms. And there’s a little formula. You need to do the Z scores, so you need a little bit of stats to be able to do this. But the app that I was talking about earlier, you can just plug in a proportion, the numbers, and they always work for you. I can send you the link.
Kayla Fratt (KF)
Yeah. And we’ll share the links to that. And you know, Schoen and Haak (2009) and Gadbois and Reeve (2014), and the two books that you showed, we’ll have all those links in the show notes for people so they don’t have to panic about trying to track it down themselves.
Simon Gadbois
Just to go back to the lineup thing, to complete your question, outside of signal detection theory, there may be a working memory issue. One of the arguments that we made in Gadbois and Reeve (2014) was, and Lazarowski and others have published more on this recently, especially if you do a matching to sample task, potentially there is a working memory issue in a lineup, and we demonstrated that with two of our dogs at the time. In other words, what we’re saying is that the dog has a very high accuracy early in the lineup, with the first items that it samples. And that accuracy goes down, as you go down the lineup, basically. We show that after the third item in a lineup or carousel, whatever, then really, the dog doesn’t know. Now, this is in matching to sample, meaning that, every time the dog enters the room, the target sample may be different, right? So you’re working in different odors to start with.
Simon Gadbois
What we didn’t explain in that paper, and the other one that you were talking about, is that sensory interference is still an issue in lineups. So the first one is working memory. But even if it’s only one odor, or the contrast between let’s say, two orders, or yes or no, or a urine sample that may or may not contain a marker for cancer, let’s say, the problem is that as the dog samples from one station to the other, there is actually sensory interference, first of all sensory memory. What did I just sample there, you know, previous one, next one, etc. Not to mention all the molecules from the previous sample that are now mixed in the nasal cavity of the dog. So, there’s still an issue here in terms of how you space this out, like physically and temporally. And that’s why again, the detection task is better, because in the detection task, the dog enters the room, there’s one sample, they sniff it, yes or no, that’s it. Just, Is it cancer? Is it not? No comparison, no possibility of mixing up all of these odors and, and then giving the wrong answer.
Simon Gadbois
And that’s the other dimension of it, there’s literally mnemonic, meaning memory interference, but also sensory interference, when you do lineups, especially the way they tend to be done. I often see these dogs doing lineups or carousels, and they’re literally running around it. And I’m thinking well, in terms of sensory sampling, this is good if they can do it, and if they can be accurate, but imagine if the odors were really close, you know, between an S+ and an S-, or have really low saliency. There, suddenly you get this threshold of error that’s totally different.
Kayla Fratt (KF)
Yeah, I mean, I’m imagining, if you blindfolded me and set me loose in the fruit aisle of a grocery store and had me find the citruses I could do that pretty easily, I would imagine. But if you started having the time to tell the difference between lemons and limes, I’m not sure I could do that by scent. Especially if I was running through the grocery story trying to do that.
Simon Gadbois
There’s another analogy I often give my students, especially for the working memory part. When you do the matching to sample, you give the dog an order, you say okay, this is lavender, and now there’s a lineup of other essential oils or whatever smells – go find a lavender one. We’ve seen dogs sometimes that go in like, “Yeah, it’s lavender. It’s very strong. This is obvious.” But halfway through the lineup, they look at you, and you can tell they go like, “What was it again?”
Simon Gadbois
Here’s the comparison. It especially if your potential number of target odors is large, like 8, 10, or more. A student of mine that once said, “Okay, it’s a little bit like when I’m at home, and I change the water in my aquarium. And I know that when I get new water in my aquarium, I need to match the temperature, otherwise, my tropical fish will actually die. So if I put my hand in the initial aquarium, and then in a second one, I can tell easily, it’s the same temperature or not the same. And then I go to the third aquarium, I may remember what, you know, the first aquarium was like, then the fourth, then…” But you know, as you go down after a while, you go like, “Wait a minute, what was the first one again?” And then you have to go and resample the aquarium to match and do that accurate comparison. So, in other words, I think with sensory information, when the target smell can be different every session, I think it’s important to remember that they have a limited capacity at the sensory and mnemonic level to process stuff. Now, if the dog is allowed to resample, great. And again, I think in the kind of work we do, it’s not that important. Usually, I make that point moreso with the biomedical detection dogs that may actually do different kinds of diagnostics in the same day, where it’s really important to remind them, what they’re working on in this session, or this trial.
Kayla Fratt (KF)
Okay, that makes sense. I think we need to wrap up here. Did you have anything else that you wanted to circle back to or revisit as we’re wrapping up? I think I think this is gonna send a lot of people’s head spinning, but hopefully in a good way. I know I’ve enjoyed this. I normally listen to my podcasts at 1.5 speed, and I think I’d have to listen to this one at normal speed. That’s a good thing. We’re helping people learn, and I really appreciate your time. So if people wanted to follow your lab or continue learning more about about you, and the work that you do, where can people find you online?
Simon Gadbois
On Facebook, there’s a page and a group that have roughly the same name. One is the Canid and Reptile Behavior and Olfaction Team, and the other one is Canid and Reptile Behavior and Olfaction Group, I think. One you just have to “Like,” and the other one you have to ask to join. I’m on Facebook myself, you can join although I think I’m at the limit or very close. We are on Instagram, but it’s not very interesting what’s going on there. Same for Twitter. There’s obviously email, and my personal website that you can find by typing my name and Dalhousie University. I think it will come up. So people are welcome to follow us. And hopefully we can get out of this pandemic, because honestly, this has been limiting a lot of our field work in the last year.
Kayla Fratt (KF)
Yeah, I think we’re all looking forward to that. I’m also keeping an eye on the border. I’ve been trying to get up to Bamff for years now. We’ll see if that happens before I leave Montana. So again, thank you so much for your time. And we’ll probably be in touch soon.
Simon Gadbois
No problem. Thank you for inviting me.
Kayla Fratt (KF)
Thank you so much for listening. I hope you enjoyed that conversation with Dr. Gadbois as much as I did. I was taking notes, and my eyes were wide, that whole conversation. He’s just so, so intelligent. And again, I know that some of the stuff on like the D primes and the S pluses and that sort of thing can be a little confusing. I’ll drop notes into the show notes to make sure that everyone can look those up and stay present with the conversation. I hope that you learned a lot, and that you’re feeling inspired to get outside and be a K9 Conservationist in whatever way suits your passions and skillset; maybe play around with some signal detection theory. Again, you can always find those show notes and extra information on this episode at K9conservationists.org.