Scott Caravello: Yeah, and, just as a reminder, because I know we had covered it in depth last time, but the zero trust concept refers to the fact that you can't presume that media is authentic, and that's because of the sophistication of deepfakes.
Katherine Forrest: Right, right, exactly. And so let's talk about the state of deepfake detection at the end of 2025, because it used to be you could count six fingers on the hand, or, you know, somebody would have an arm coming out of their back, or it would look sort of a little bit weird. And, you know, every once in a while, you still run into a few of those things that are obvious tells of a deepfake. But right now, things have gotten really sophisticated in terms of deepfake technology and interactive deepfake technology in particular. So, let's just say, and cutting to the chase, it's hard to figure out whether or not something is a deepfake. There is no single, you know, silver bullet. There's no single tool that you can plug into your phone and have it run against videos that you're watching to determine whether or not something is a deepfake. So... you know, you have to go to a variety of techniques and different technologies, and it's really, in some ways, like whack-a-mole.
Scott Caravello: Yeah, and part of the reason for that is just because there are so many different kinds of deepfakes, and they are driven by different techniques to actually generate them.
Katherine Forrest: Right, and in response, there has to be an equally sophisticated and robust set of ways to detect them. And there are two things that I would say are emerging as essential. The first is real-time verification that what you're watching is in fact not only in real time, but also real. It's real in real time. And authentication, indicia of authenticity and, you know, having some sort of proof attached to the content so that you can actually trust it later that what you're watching is authentic.
Scott Caravello: Yeah, and so, you know, these strategies work together, right? But they're part of then what we would call a “tech stack.” And that's a general term not limited to this conversation. But what we mean by that is a combination of tools. And so these verification and authentication measures can work together within an organization or a tool to help detect the deepfakes.
Katherine Forrest: Right, so let's start with the real-time detection piece. Okay, so the industry has pushed towards real-time—or basically real-time, near-time, near real-time—detection. And let's consider it in connection with sort of a live video and sort of these video and voice calls, which are particular instances that we discussed last time and that create really very significant fraud risk and just sort of like...
Scott Caravello: Sounds great.
Katherine Forrest: Okay, so the industry has pushed towards real-time—or near real-time—detection. And let's consider it in connection with sort of a live video and sort of these video and voice calls, which are particular instances that we discussed last time and that create really very significant fraud risk and just sort of like... creep out risk, right? You’re going to be on a video call and you're wondering whether you're actually the only human on the screen. And in that context, we have innovative methods for real-time verification, including something that's called a “microchallenge.”
Scott Caravello: And so a microchallenge is another one of those things that it sounds like, which is an immediate digital request for verification. And it's kind of like the deepfake video version of a CAPTCHA that we see around the internet all the time.
Katherine Forrest: Right, exactly. And, in fact, in one approach, which is validated by the European Symposium on Security and Privacy, which goes by the easy-to-say acronym, EuroS&P, it's called a: Generative Output Test for Challenge, Human Authentication, or “G.O.T.C.H.A.” And so it's directly and truly picking up on the CAPTCHA equivalent. And up to now, it's been mostly used for research and non-commercial use, rather than being put inside of a commercialized tool that you and I could download from the App Store. Instead, it's available through GitHub. These GOTCHAs present simple but really carefully chosen challenges to try and detect an interactive deepfake. And, for instance, it requires a person or a deepfake to do a head rotation at a specified angle—like, you know, turn your head 90 degrees or— not 90 degrees… That would be weird. Wait, that would be so weird—Turn your head 360 degrees. But anyway, turn your head at a specific angle, or requiring movements like putting your hand over your eye. And then the GOTCHA uses these tests to try to find issues because deepfakes actually have a harder time than humans do in responding in real time to some of these physical microchallenges.
Scott Caravello: Yeah, and so as much as I love that name, GOTCHA, it's not like these are trick questions or strange tasks where someone is, you know, really going to say, “gotcha,” right? It's looking at normal and run-of-the-mill aspects of the video in order to test parts of deepfake generation that can still lead to inconsistencies, notwithstanding how advanced the technology has become. And so then I think if we're still going to stay in sort of research land, there's another developing approach that's worth discussing, and that's what's called “active probing.”
Katherine Forrest: Right, and that's actually a close cousin to the challenge-response framework of GOTCHA.
Scott Caravello: And I think that this one is really cool.
Katherine Forrest: You know, it really is. It's active probing, and it really sort of flips the script. Instead of asking the user to move to measure how they do in response to a challenge, the detection system subtly changes the environment in a measured way and then it watches, if you will, for expected physical responses from the human or the potential deepfake.
Scott Caravello: Yeah, so one version uses illuminations like the screen flashing or putting off a patterned hue to then see how the subject's facial reflections track. And then another method watches corneal reflections, which are the tiny glints of light on the surface of the eyes. And the reason that that's helpful is because for real people, the reflections should be nearly identical, since both of your eyes are looking at the same thing on the screen from very similar angles. And so, you know, it's a really granular detail, but because of that, it's something that's a little more difficult for the deepfake generator to mirror and therefore can be an indication of whether or not it is in fact fake.
Katherine Forrest: Right, so both techniques raise the bar for imposters, but... those are in research land still.
Scott Caravello: Yeah, and, you know, if we then start thinking about commercial deployment—you know, I’m–I'm specifically thinking of the GOTCHA technique—there are some practical drawbacks that you could come up with. Imagine if we were getting on a video call and in every single meeting you had to make each person on each side of the screen go through 30 seconds of facial aerobics to prove that they're actually a real person.
Katherine Forrest: It would prevent people from actually not turning on their cameras. Right?
Scott Caravello: So true.
Katherine Forrest: You know, seriously, for higher-stake interactions—whether it be board meetings or authorizing payments above a certain threshold, the sort of know-your-customer inquiries, or job interviews—you know, maybe if you've had some other indication that the person isn't who you think they are, you might have that kind of GOTCHA microchallenge request. But it's not, I think, really realistic. Can you imagine getting on like a call for an RFP for a new client pitch and having to, like, say, like “twist your head around” and, you know, “flare a nostril or can you raise an eyebrow or, wiggle your ears” and things like that. I mean, that, I think that that's not really a realistic sort of commercial proposition.
Scott Caravello: Totally. But, that being said, maybe it would make sense to then move on to some of the commercially available products.
Katherine Forrest: Right, right, right. So one of the ideas is that no one wants to give their secrets away to attackers. So the information that you can get about the commercial products to detect the deepfakes—the interactive deepfakes in particular—is somewhat limited. You know, they're not putting out system cards that go into long explanations of exactly how these detection tools work. But we can talk about, and will talk about, sort of two tools where there is some information out there on them, at least about what they're doing. One is called “Reality Defender” and the other one is called “Sensity”—S-E-N-S-I-T-Y—AI. And so Reality Defender offers real-time detection for audio, video, and images, and it includes meeting-focused screening. And, you know, it actually does integrate with Teams and with Zoom, which is kind of a big deal when you think about it, because it means that there are some tools that are now being made available and rolled out where people actually have to meet, and you want some form of verification.
Scott Caravello: Yeah, and then so Sensity AI also integrates with Teams; but I don't think it integrates with Zoom, at least so far as I know. But they advertise a number of different technologies that make up its deepfake detection solution. And so that includes pixel-level analysis that pick up on inconsistencies that are in the video that are then basically giving away that facial manipulation has occurred. And so, you know, as I keep talking about that, it makes me realize that we haven't actually talked that much about audio so far. But that's also a really key part of the equation too, because no matter how realistic the video is, the audio has to match and also seem realistic for these deepfakes to be succeeding. And so, therefore, it's a really important part of detection. And so some of these commercial solutions, like Sensity, are also picking up on natural sound patterns.
Katherine Forrest: Right, and the audio piece is so important because when we do trust, it's often based on both visual and audible detection that we are picking up as humans. We're looking at a video, we're listening to the video, and we trust what we hear. And so it's interesting. So we put these two things together and we just implicitly trust. So there is a commercial tool—another commercial tool—that's called “Pindrop,” that detects deepfake audio by listening for things that are outside of the range of human hearing. They are also looking for—this is a little bit, sort of, funny—but for what they call “liveness cues,” which are indications, sort of like in biometric security, that actually indicate whether somebody is alive. And you might be looking, for instance, at signals that are being played from a speaker, rather than from a live human voice.
Scott Caravello: Right, right. And so then there's also the developer-specific detection tools that we've seen, or that we will see, coming out then, and those tie into state regulation. So there's California's SB 942, which, Katherine, I think you talked about on the first deepfake episode, but that comes into effect on January 1st, and it would require the developers of certain AI systems to not only include latent or hidden disclosures in their synthetic outputs, but also to make freely available a detection tool which a user can use to feed output into and then be told whether the developer's AI is responsible for the generation.
Katherine Forrest: Right. And so when we talk about SB 942, which is this California law, I just want to harken back to our last episode on the executive order from December 11 and–and remind our listeners that any of the laws that are currently passed and actually in effect or going into effect, and have not been challenged in some way or are not yet specifically preempted, those laws are still good laws. So don't ignore those laws, right? This is gonna be sort of a step-by-step process. You know, SB 942's latent watermarking approach really is gaining traction, and there are very sophisticated methods that are out there. But another really interesting approach is something called “DeepForgeSeal,” and that's all one word. Deep Forge—D-E-E-P, capital, F-O-R-G-E, and then, capital, S-E-A-L. And why do they have to do that? Why do they have to run like three words together like that? DeepForgeSeal. And it's what's referred to as a “semi-fragile” watermark. And it relies on AI that is trained to make the watermark remain intact when an image is the subject of a benign, common operation like image compression that happens as a matter of course when you're sharing something, but when the video is used in a way that suggests tampering, then the watermark is designed to fail. And that would occur when sort of deepfake image generation and alteration takes place.
Scott Caravello: Yeah, and so you might insert this watermark into real, authentic media. And then, right, I think the idea, Katherine, is that you'd be able to later then tell if it's been manipulated by a malicious actor.
Katherine Forrest: Right, right. You know, it would be a proactive defense, but we should mention that DeepForgeSeal is also—right now—it's not yet sort of commercially deployed, but it's an interesting proposition.
Scott Caravello: Totally. And, you know, so then just connecting this back to our conversation last time about authentication in the courts—because deepfakes create an issue for authenticity when you're actually putting something in front of court—this seems like the type of concept, though, that could actually be really helpful there.
Katherine Forrest: Right, completely. And, you know, it’s something that would really need to be a tried-and-true method before people really can sort of rely on it and before it would be adopted as something which is a recommended method of authenticating something. But it certainly would be helpful.
Scott Caravello: Totally. So on that note, I think we should also mention the Coalition for Content Provenance and Authenticity, or the “C2PA” standard. And we want to mention that because it's really become the standard for cryptographically signed metadata. And, so putting that in much more plainly understandable language, it means that you can embed information in video, images, or audio in order to provide a digital history.
Katherine Forrest: Right, and like the discussion of what AI developers are doing, it's helpful information. But if I'm a bad guy looking to pass off a deepfake as real, I'm seeking out a tool that does not embed this kind of information inside the digital paper trail, right? So it's not a silver bullet, and those other real-time detection techniques remain important lines of defense that, some of which, as we know, are deployed and some of which we're hoping will be deployed.
Scott Caravello: Yeah, and then I think that the last thing that I would sort of mention here is just that these are really a sample of tools and techniques that are out there. And, you know, they’re–they're a notable set, but we don't mean to say that this is the full universe, and, you know, this field is going to continue to develop as the deepfake generation technology develops in step. So with that said, Katherine, any closing thoughts?
Katherine Forrest: You know, just that this is a complicated area where we're going to be playing whack-a-mole for a while. You know, this is, trust–in a whole variety of ways–is put at issue with different kinds of new technologies. And with some of these deepfakes, we've got some real issues that are gonna come into play. And so it's going to be a tug-of-war between fraudsters and legitimate businesses and civil society, and we're going to be trying to find the right kinds of tools that are going to be lasting and that are going to be really useful and accessible—financially accessible—to people. So that's where we are.
Scott Caravello: Well, that's a semi-positive note to put out in the middle of the holiday season!
Katherine Forrest: You know, I just want to say I'm ready for the red-colored—so, you know—whoopie pies to be taken away from me because I cannot hold myself back. So, you know, if I've put out a nice little sort of note for the holiday season of the ability to detect deepfakes, that's a good thing. The fact that the interactive deepfakes are so creepy is a bad thing. But in all events, that's all we've got time for today. I'm Katherine Forrest.
Scott Caravello: And I'm Scott Caravello. Don't forget to like and subscribe.
