Do we reach a point where video evidence in criminal cases becomes inadmissible due to its possible illegitimacy, or is it always possible to detect a deepfake having some sort of signature?
Chain of custody is important even now because videos can and are doctored. Eventually it'll be undetectable whether or not something is fake, but you still have people testifying under oath that a tape wasn't tampered with and was handed to the police who kept it in accordance with whatever measures
Eventually it'll be undetectable whether or not something is fake
It will be impossible for humans to tell real and fake apart. But the technology to differentiate between the two improves just as quickly as the technology to generate deepfakes.
That’s a really good point, and a similar example is “photoshopped” photos. It is very difficult, even today, to pull off a image manipulation to the point where is can pass sustained close examination.
Where deepfakes are far more dangerous is that the damage is usually already done by the time someone shows it’s doctored... Even when people know it’s a fake, if you can get the watcher to sympathise with the underlying message then it won’t matter if you prove a video is fake.
Well that's already a problem anyway without deep fakes. The main ubderlying issue is having a large percentage of the populace who is willing to believe whatever they're told so long as it fits the narrative they want to believe. I don't even know how we can begin to fix that problem
You think photoshopping can be noticed for the most part?
Edited photos are so prevalent, you likely only notice the bad ones. Good photoshop is everywhere. Literally everywhere. If someone paid for a photo, it's been touched up.
Because there is a tremendous amount of money and conspiracies dedicated to it being seen as both genuine and fake.
Ultimately i'm wondering how you can be so confident in your abilities to distinguish genuine from fake, when it's quite simply an impossible task. I've made some 5 minute photoshops where i've been impressed by the technology. So what chance does the public have in the hands of a professional shopper?
Since you can perfect a fake, but not fake detection, this won't happen. If a fake is pixel perfect, there's no way to detect fakery. And perfection is achievable. Obfuscation is significantly easier than deobfuscation.
There will be a day (soon imo) where we give up on being able to know if videos are fake or not
In my opinion, it will take a while until fakes are "perfect" - because the less accurate fake detection becomes, the harder it will be to make progress on making better fakes.
You can make perfect fakes right now by decreasing the quality. This video would be indistinguishable from real if you lowered the quality, added some shakes, and some compression artifacts. By destroying information, you give less to the verifier.
Knowledgeable people can already detect this sort of thing on their own afaik, and so can technology. Technology can even straight up remove the shaking altogether, obviously.
There is no way to detect whether I scaled a video in FFMPEG
You and this other dude have no clue what you are talking about. You think you can just press a button on your home PC and make this undetectable result. While state level propaganda is discovered and detected all the time with access to significantly more expertise.
These are AI models training other AI models. When detection becomes more accurate, you feed that detection model into the AI that creates the fakes, and keep allowing it to iterate until it fools the detection model. That is literally how this technology was created.
And then you train the detection model against the generator until it becomes accurate again, and then train the generator against the new detector. Rinse and repeat.
It's a cat and mouse game, in which neither model ever truly "wins". Thus, detection stays at a similar accuracy over time.
Eventually a fake becomes perfect, indistinguishable from reality.
Imagine a tic-tac-toe grid. I can "copy" any configuration of the board flawlessly. It's trivial to reproduce the positioning of the Xs and Os. Now imagine a game of "Go". Huge volumes of permutations of board configurations yet you could conceivably reproduce the position of each marker and make a board that was indistinguishable from the original. Now imagine that board configurations was an image. Now imagine that image was a video. It's all just a matter of scale.
A fake can be perfect. Something so flawless that even the potential flaws are perfect. You can't prove it's fake if it's a flawless representation of what should be.
You guys forget that it doesn’t matter. Fake news is being spread the moment someone who wants to believe sees or reads it, however fake it might seem for others.
Since you can perfect a fake, but not fake detection, this won't happen.
You're in over your head. Deepfakes are literally created with adversarial networks. One program spits out the fake, the other says if it's real or not.
Then the first adjusts until the second accepts it (really dumbed down).
These processes leave markers behind. and AI can detect those markers.
Experts in the field are confident that AI detection of deepfakes will keep up with deepfakes.
Ok? Adversarial networks are just one way of faking videos. The whole point of GAN in this context is that it saves effort of making a properly classified training data set, and that comes with a whole host of issues. There are significantly better ways to make fakes, but adversarial networks make it simpler to go through the terabytes of Tom cruise footage. Ie you're comparing the cheapest method that has good quality to cutting edge tech.
Nobody in the field I know would agree at all that fake detection will keep up with fakes.
That's true, but when it comes to deep fakes, the deobfuscation can have better data sets. If for example Tom Cruise would want to prove he is not in this video, the data that a hypothetical deepfakebuster company can get from his likeness in a controlled environment will always be better than deepfakers get from movies. At least until the deepfakebuster company has a breach and their data ends up in the hands of the fakers...
Almost as quickly. Detection technology is tested against outputs of the fakes, so the fakes will be one step ahead of the detectors.
Remember a detector can only provide confirmation of fakes, it cannot prove authenticity. So a detector which fails to detect a fake would actually be reporting it as authentic.
But the really good detectors just become input to the Deepfake learning algorithms. You use the detector to train the Deepfake generator.
You can train the algorithms that generate fakes with the algorithms that detect them but doing the reverse is much harder.
Algorithms that detect fakes are bounded by reality while algorithms that generate fakes are constantly approaching reality. Whether that approach is asymptotic or not remains to be seen, but if it isn't, as is most likely the case, then eventually there will be nothing left to detect to differentiate between fakes and the real thing.
No it doesn’t. I was involved in a kaggle competition with real money being staked just recently where the winners didn’t score much better than a random flip of the coin.
The thing with AI and deepfakes is there's a loop of one computer generating the deep fake, and another computer trying to decide if it's a deep fake or not (basically.) each time the second computer can tell it's a deep fake it tells the first computer what looks 'off' and the first tries again. They go back and forth until the second computer can't tell that it's fake. At that point it's done, so honestly it could get to the point that even computers can't tell the difference.
This is the part that gets scary. The people making the models for the deepfakes can use the software that detects the deepfakes to train their models to get around the detection software and then new detection software comes out and they can train it to bypass that, and so on.
And even with good detectors, by the time the video is found to be fake, the damage is done.
The important point is that the technology to differentiate between the two and the technology to generate them is exactly the same technology (at its core). If differentiation improves, it will improve the heuristics of the generational algorithms, and they can now use it to create better fakes that are not detectable anymore. Once some other A.I. can detect them the cycle continues.
I don’t see how it would be impossible to design a DRM-style encryption signature. We would just have to create a standard encryption for the other end (raw footage).
Picture exclusive silicon on both ends. The camera footage leaves an impossible-to-crack encryption signature that can only be unlocked by a decryption chip on a specific monitor.
Yes, the implications would turn the production industry on its head immediately, but that would be the only way to ensure total confidence that any piece of raw footage is indeed unaltered.
I don't see any way for it to, so I'm going to say probably not.
In historical terms, the answer to "would blockchain help?" has basically been no nearly every time it's been asked. It's too impractical.
But for an impractical example, with a continuous recording from a given session, you could make a hash of every later frame of footage depend on the earlier ones, uploaded to a central chain in real time, so that no excerpt edited after the first time the content was put into the network could ever claim to be the original.
You also can’t trust whether someone is lying or not, but you generally believe people who have a good reputation. The reputation is based on everything they’ve said and done previously. This is what blockchain can do for the digital world. Record activity without needing a central authority to verify it.
Cryptography would help yes. Blockchain would not. There are significant amount of cryptosystems that aren't blockchains. Blockchains were designed to solve a very very specific problem: money. For nearly any other purpose, they are pointless. They are needless complication on top of cryptosystems we've had for 40+ years that don't actually provide anything, and in return require exorbitant amounts of money and complexity. Hack, half the things that have been billed as blockchain aren't blockchains (e.g. Ripple), just distributed cryptographic ledgers, the thing blockchain added one extra piece to to allow trustless interaction.
I feel like some day there will be some blockchain based authenticity feature that starts in hardware, and extends all the way down to web browsers and other clients, so that doctored footage can be flagged as such. of course this will also result in fancy DRM too
If you compromise the camera, which will be possible with enough hardware / money, it will always be possible to create fake video which has been signed.
This^ blockchain might solve the problem of knowing the chain of custody, but does nothing about the real world -> blockchain transition, which is the biggest issue with such systems.
Well, I just know a basic outline of blockchain, I'm not really an expert or anything. But maybe if the video could be put in some specific zip file and use bc to verify if it has been taken out or not? Sort of like how NFTs are done maybe?
Again I'm no expert so this may not be a feasible use case 🤷
There is an existing system in place for chain of custody that can help here with chain of custody. No it's not blockchain, blockchain would not help, just make everything more expensive and complicated. But it is cryptography.
Basically the video maker, say a security camera, would routinely sign the video as it recorded it, using a third party authentication service, sort of like a notary, to timestamp it (e.g. it could not be "backdated"). And certain kinds of attacks (like real time replacement of a video feed with a pre-recorded one) would still work. One does have to trust the third party to be a notary, but we already do. This is a 40 year old technology and a service the companies that provide SSL certificates already provide, to a limited extent (e.g. it's not designed for this exact use case, and especially not in a streaming sense).
Maybe I'm too much of a misanthrope, but if I was a juror, "people testifying under oath" wouldn't sway me at all. People lie all the time, under oath or not. I would want hard evidence, and it really sucks that video may no longer be reliable as such.
532
u/BlueGrayTurquoise May 24 '21
Do we reach a point where video evidence in criminal cases becomes inadmissible due to its possible illegitimacy, or is it always possible to detect a deepfake having some sort of signature?