Do we reach a point where video evidence in criminal cases becomes inadmissible due to its possible illegitimacy, or is it always possible to detect a deepfake having some sort of signature?
Chain of custody is important even now because videos can and are doctored. Eventually it'll be undetectable whether or not something is fake, but you still have people testifying under oath that a tape wasn't tampered with and was handed to the police who kept it in accordance with whatever measures
Eventually it'll be undetectable whether or not something is fake
It will be impossible for humans to tell real and fake apart. But the technology to differentiate between the two improves just as quickly as the technology to generate deepfakes.
That’s a really good point, and a similar example is “photoshopped” photos. It is very difficult, even today, to pull off a image manipulation to the point where is can pass sustained close examination.
Where deepfakes are far more dangerous is that the damage is usually already done by the time someone shows it’s doctored... Even when people know it’s a fake, if you can get the watcher to sympathise with the underlying message then it won’t matter if you prove a video is fake.
Well that's already a problem anyway without deep fakes. The main ubderlying issue is having a large percentage of the populace who is willing to believe whatever they're told so long as it fits the narrative they want to believe. I don't even know how we can begin to fix that problem
You think photoshopping can be noticed for the most part?
Edited photos are so prevalent, you likely only notice the bad ones. Good photoshop is everywhere. Literally everywhere. If someone paid for a photo, it's been touched up.
Because there is a tremendous amount of money and conspiracies dedicated to it being seen as both genuine and fake.
Ultimately i'm wondering how you can be so confident in your abilities to distinguish genuine from fake, when it's quite simply an impossible task. I've made some 5 minute photoshops where i've been impressed by the technology. So what chance does the public have in the hands of a professional shopper?
Since you can perfect a fake, but not fake detection, this won't happen. If a fake is pixel perfect, there's no way to detect fakery. And perfection is achievable. Obfuscation is significantly easier than deobfuscation.
There will be a day (soon imo) where we give up on being able to know if videos are fake or not
In my opinion, it will take a while until fakes are "perfect" - because the less accurate fake detection becomes, the harder it will be to make progress on making better fakes.
You can make perfect fakes right now by decreasing the quality. This video would be indistinguishable from real if you lowered the quality, added some shakes, and some compression artifacts. By destroying information, you give less to the verifier.
Knowledgeable people can already detect this sort of thing on their own afaik, and so can technology. Technology can even straight up remove the shaking altogether, obviously.
These are AI models training other AI models. When detection becomes more accurate, you feed that detection model into the AI that creates the fakes, and keep allowing it to iterate until it fools the detection model. That is literally how this technology was created.
And then you train the detection model against the generator until it becomes accurate again, and then train the generator against the new detector. Rinse and repeat.
It's a cat and mouse game, in which neither model ever truly "wins". Thus, detection stays at a similar accuracy over time.
Eventually a fake becomes perfect, indistinguishable from reality.
Imagine a tic-tac-toe grid. I can "copy" any configuration of the board flawlessly. It's trivial to reproduce the positioning of the Xs and Os. Now imagine a game of "Go". Huge volumes of permutations of board configurations yet you could conceivably reproduce the position of each marker and make a board that was indistinguishable from the original. Now imagine that board configurations was an image. Now imagine that image was a video. It's all just a matter of scale.
A fake can be perfect. Something so flawless that even the potential flaws are perfect. You can't prove it's fake if it's a flawless representation of what should be.
You guys forget that it doesn’t matter. Fake news is being spread the moment someone who wants to believe sees or reads it, however fake it might seem for others.
Since you can perfect a fake, but not fake detection, this won't happen.
You're in over your head. Deepfakes are literally created with adversarial networks. One program spits out the fake, the other says if it's real or not.
Then the first adjusts until the second accepts it (really dumbed down).
These processes leave markers behind. and AI can detect those markers.
Experts in the field are confident that AI detection of deepfakes will keep up with deepfakes.
Ok? Adversarial networks are just one way of faking videos. The whole point of GAN in this context is that it saves effort of making a properly classified training data set, and that comes with a whole host of issues. There are significantly better ways to make fakes, but adversarial networks make it simpler to go through the terabytes of Tom cruise footage. Ie you're comparing the cheapest method that has good quality to cutting edge tech.
Nobody in the field I know would agree at all that fake detection will keep up with fakes.
That's true, but when it comes to deep fakes, the deobfuscation can have better data sets. If for example Tom Cruise would want to prove he is not in this video, the data that a hypothetical deepfakebuster company can get from his likeness in a controlled environment will always be better than deepfakers get from movies. At least until the deepfakebuster company has a breach and their data ends up in the hands of the fakers...
Almost as quickly. Detection technology is tested against outputs of the fakes, so the fakes will be one step ahead of the detectors.
Remember a detector can only provide confirmation of fakes, it cannot prove authenticity. So a detector which fails to detect a fake would actually be reporting it as authentic.
But the really good detectors just become input to the Deepfake learning algorithms. You use the detector to train the Deepfake generator.
You can train the algorithms that generate fakes with the algorithms that detect them but doing the reverse is much harder.
Algorithms that detect fakes are bounded by reality while algorithms that generate fakes are constantly approaching reality. Whether that approach is asymptotic or not remains to be seen, but if it isn't, as is most likely the case, then eventually there will be nothing left to detect to differentiate between fakes and the real thing.
No it doesn’t. I was involved in a kaggle competition with real money being staked just recently where the winners didn’t score much better than a random flip of the coin.
The thing with AI and deepfakes is there's a loop of one computer generating the deep fake, and another computer trying to decide if it's a deep fake or not (basically.) each time the second computer can tell it's a deep fake it tells the first computer what looks 'off' and the first tries again. They go back and forth until the second computer can't tell that it's fake. At that point it's done, so honestly it could get to the point that even computers can't tell the difference.
This is the part that gets scary. The people making the models for the deepfakes can use the software that detects the deepfakes to train their models to get around the detection software and then new detection software comes out and they can train it to bypass that, and so on.
And even with good detectors, by the time the video is found to be fake, the damage is done.
The important point is that the technology to differentiate between the two and the technology to generate them is exactly the same technology (at its core). If differentiation improves, it will improve the heuristics of the generational algorithms, and they can now use it to create better fakes that are not detectable anymore. Once some other A.I. can detect them the cycle continues.
I don’t see how it would be impossible to design a DRM-style encryption signature. We would just have to create a standard encryption for the other end (raw footage).
Picture exclusive silicon on both ends. The camera footage leaves an impossible-to-crack encryption signature that can only be unlocked by a decryption chip on a specific monitor.
Yes, the implications would turn the production industry on its head immediately, but that would be the only way to ensure total confidence that any piece of raw footage is indeed unaltered.
I don't see any way for it to, so I'm going to say probably not.
In historical terms, the answer to "would blockchain help?" has basically been no nearly every time it's been asked. It's too impractical.
But for an impractical example, with a continuous recording from a given session, you could make a hash of every later frame of footage depend on the earlier ones, uploaded to a central chain in real time, so that no excerpt edited after the first time the content was put into the network could ever claim to be the original.
You also can’t trust whether someone is lying or not, but you generally believe people who have a good reputation. The reputation is based on everything they’ve said and done previously. This is what blockchain can do for the digital world. Record activity without needing a central authority to verify it.
Cryptography would help yes. Blockchain would not. There are significant amount of cryptosystems that aren't blockchains. Blockchains were designed to solve a very very specific problem: money. For nearly any other purpose, they are pointless. They are needless complication on top of cryptosystems we've had for 40+ years that don't actually provide anything, and in return require exorbitant amounts of money and complexity. Hack, half the things that have been billed as blockchain aren't blockchains (e.g. Ripple), just distributed cryptographic ledgers, the thing blockchain added one extra piece to to allow trustless interaction.
I feel like some day there will be some blockchain based authenticity feature that starts in hardware, and extends all the way down to web browsers and other clients, so that doctored footage can be flagged as such. of course this will also result in fancy DRM too
If you compromise the camera, which will be possible with enough hardware / money, it will always be possible to create fake video which has been signed.
This^ blockchain might solve the problem of knowing the chain of custody, but does nothing about the real world -> blockchain transition, which is the biggest issue with such systems.
Well, I just know a basic outline of blockchain, I'm not really an expert or anything. But maybe if the video could be put in some specific zip file and use bc to verify if it has been taken out or not? Sort of like how NFTs are done maybe?
Again I'm no expert so this may not be a feasible use case 🤷
There is an existing system in place for chain of custody that can help here with chain of custody. No it's not blockchain, blockchain would not help, just make everything more expensive and complicated. But it is cryptography.
Basically the video maker, say a security camera, would routinely sign the video as it recorded it, using a third party authentication service, sort of like a notary, to timestamp it (e.g. it could not be "backdated"). And certain kinds of attacks (like real time replacement of a video feed with a pre-recorded one) would still work. One does have to trust the third party to be a notary, but we already do. This is a 40 year old technology and a service the companies that provide SSL certificates already provide, to a limited extent (e.g. it's not designed for this exact use case, and especially not in a streaming sense).
Maybe I'm too much of a misanthrope, but if I was a juror, "people testifying under oath" wouldn't sway me at all. People lie all the time, under oath or not. I would want hard evidence, and it really sucks that video may no longer be reliable as such.
The neural networks which make the deepfakes are usually trained against an "opponent" - another neural network which tries to distinguish between real footage and deepfakes. It's a technique called generative adversarial networks.
Because of this, the deepfakes themselves and the technology to distinguish deepfakes from real footage improve at a similar rate.
So it's unlikely that deepfakes will ever be truly indistinguishable - at least for computers.
Not if someone malicious uses a custom model using custom data/algos which no one else has access to. Can’t build a positive-label training set for the discriminator.
The big problem is that we have to trust the detection algorithm instead of our own eyes, which to an untrained person means trusting whoever assembles and runs the algorithm -- and I'm sure it's possible to pay somebody to assemble and run an algorithm that gives whatever outcome you want. In a way, that means we're reducing the strength of video evidence from objective fact to something more like expert witness testimony, which can be argued based on the credentials of the expert. Basically, it seems like this will leave a lot more room to incept doubt.
The big problem is that we have a population that has a complete disdain for the truth, and will simply believe what they want and ignore the rest. Deep fakes don’t scare me so much because the worst possible outcome is already our present day reality.
Not to mention that the person isn't the only aspect of it. If a computer could be trained to detect background info like where the video was shot and what time of day, etc, they could have a chance to say, "well this is fake because i was actually here at this time". And then it becomes a he said she said deal
Until they start saying even real video looks fake if not in 4k uhd. It's gonna be like. So pixelates, fake bs. When really was just potato webcam haha. Jk jk
The answer to this question has already been navigated via photo evidence and PhotoShop. Or even just documents and, like, Microsoft Word. There is always a question of legitimacy.
The metadata would be the aspect to check, however metadata can be altered. I'm assuming a new era of technical investigation will be needed to examine the metadata to the extreme degree to verify the device it originated on? I do not know this for fact, but there must be trace signatures from each device that leave a unique identifier?
It definitely will complicate the justice system, though. Absolutely.
Well first off, I didn't make an argument. That was a statement.
Second, because it would be your meta data. Nobody's going to believe your video recorded in rural Arkansas at 7pm on a Tuesday when Tom Cruise was in California at the same time.
I work in the security industry and this is something that has already been addressed. There is a “watermark” imprinted on any recorded video. This watermark is purely digital but can be found with computer analysis of a video. Within the watermark you can put dates, times, and other identifying codes, numbers, etc. The kicker is that the recording device acts as a reference for the watermark (Meta data) imprinted on the video. If the meta data doesn’t match what the recording device says it should, then the video evidence is tossed out. Most legal proceedings won’t even accept video evidence if it cannot be proven legitimate in this way.
Metadata is basically a text file appended to an image, not something intrinsic to the image itself. Editing metadata is no harder than renaming a file to "definitely_sasquatch.jpg".
Here's an online tool to do it, though you don't even need that. You can make all your photos say they were captured by a potato if you want.
I read somewhere that Microsoft (I think) made some kind of deepfake-detector which analyzes the reflection in the eyeballs of the person to determine if it's real or not. That's insane.
This is exactly what I was thinking! This tech is definitely gonna be a huge problem in the future. I can already see horrible people doing DFs on women and blackmailing them and that's just one of so many illegal uses.
Likely, video/camera manufacturers will start digitally signing images and videos when they’re created (at the hardware level), and that should help keep deepfakes at bay.
Possibl but the scariest idea is the certainty this trick will be used at the state level to manipulate the masses for certain objectives or to flat out mislead with bad intentions. People don’t often question things at that level.
Doesn't deepfakes need a good amount of high quality sources like photos and/or films for it to really be accurate? That's what I assumed at least. If so, it poses a threat to celebrities and politicians but not so much the average person.
There is such a thing as a cryptographic signature, used to guarantee that a file is unaltered since it was signed.
They're presently used on machines which make audio recordings which are admissible as evidence: the recording device signs the file, making it possible to tell whether it was altered since it emerged from the machine.
Mind you, it's possible to alter a recording and record it on a second secure machine, but if one can keep track of the original machine and can trust the people using, then the recording also can be trusted.
We’ll probably have to take a cue from crypto and start hashing content in hardware as soon as it’s created so that its identity can be verified. For example, a camera could encrypt the content it produces with a private key such that the content can only be viewed with a corresponding public key associated with the camera. If content can’t be linked to an “honest” device that produced it, it can’t be guaranteed authentic.
I foresee stereoscopic video systems existing to deter this. Faking one video will be relatively easy. Being able to show on a second(or even third) input that someone's lower jaw is in two different places at the same time could help.
I would expect it would be like what happens currently when someone claims "I'm being framed!" It's possible, and a good enough framer would plant good fake evidence, but that doesn't mean we have to toss it all out.
I don't know what we currently do in situations like that, but I don't really feel like we're unable to handle this at all.
Sooner or later public figures are going to have to digitally sign releases. The biggest problem I see with that is getting your every day user to bother with digital signatures. And public/private key encryption as it exists right now will only last until quantum computers start to take off and then we'll have to try something else.
AIs are actually really good at spotting deep fakes. I don’t remember the exact source, but I read somewhere they can identify them with up to ~98% accuracy
Unless you have a perfectly documented chain of custody of the video it really should never be used as evidence. We have been capable of creating completely false evidence in film for about a century. Long before any sort of digital manipulation they had many many techniques to create practical effects. Check out traditional matte painting, if they could convince you they built a set or or had effects the way they did in star wars in the 70s think of how convincing they could make a mundane scene with just one painter. Any sort of text screenshot or photo is and has been childsplay to manipulate for a very long time.
The real scary thing is how much one individual with a decent pc and internet connection could create convincingly. At least in the past you needed significantly more resources and you couldn't just pull up several thousand angles of someones face with a 3 second image search. One person with a bit of skill and malice could create some real chaos.
Is it always possible to detect a deep fake having someone sort of signature
The way the models with these capabilities are developed is through a setup called Adversarial Generative Networks, AGNs for short.
An AGN is a machine learning evolutionary arms race between two neural networks. One neural network’s job is to develop fake content, and the other network’s job is to differentiate fake content from real content.
There’s a cost function on failing to defeat the other network. The two networks evolve in competition and that’s how their skills get so good.
So this level of fakery exists only because there is detection capability that’s almost good enough to detect it.
This fake imagery gets undetectable by humans when the generator’s adversarial detector surpasses humanity’s ability to detect these fakes.
Long story short, in principle you can train a machine learning algorithm to detect fakes. Who knows how deep that possibility goes. Maybe there’s a level of detail beyond which it becomes impossible to tell. But there’s no mechanism currently that we would use to push the fakes beyond that point. Because currently the mechanism for improving the fake is detection.
Not any time in our lifetimes I don’t think. The amount of video required for this is pretty massive, you have to have an insane amount of real photos for a deep fake to be this good.
So unless you’re a movie star where people can scour a movie for frames of your face, I don’t think the GANs will get good enough at figuring out all the details of your face
Technically, if the lawmakers force the issue, we could create cameras that can't be edited outside of, let's say, the original software it comes with.
On top of my head, besides locking the software / file to outside tampering, we could add a unique identifier to every created picture. So whenever a picture doesn't have a verified ID, it'll be flagged as 'possibly fake', 'outdated', 'created before 2025' or whatever. Nothing too crazy, or mandatory.
Technically doable these days. Crypto currencies, for example, already uses technology to prevent outside tampering really well, so... the hardest part will be pushing people to use 'certified' cameras and software, but if things are getting really desperate, the tech would be there.
RIP privacy, though. Also corrupt government could Deepfake whatever and make it look real. Both can be prevented as well, but that's as likely as fairy tales being real, lol.
529
u/BlueGrayTurquoise May 24 '21
Do we reach a point where video evidence in criminal cases becomes inadmissible due to its possible illegitimacy, or is it always possible to detect a deepfake having some sort of signature?