This is the twelfth part in a series on how AI is crossing out of software and into the physical world. This time, Jan Bosch looks at what it takes to establish that something is real when anything can be synthesized. Let me dispose of detection first, because it’s where most of the money and nearly all of the public conversation has gone, and it’s the wrong problem. Detection asks: Is this artefact fake? That’s an adversarial classification task in which your opponent improves continuously, knows your classifier’s behavior and can iterate against it for free. Worse, a detector must generalize to generators that don’t exist yet. You can win this for eighteen months at a time; you can’t win it structurally. Every serious person in the field worked this out some years ago, which is why the industry pivoted to the question you can actually answer: not whether something is fake, but where it came from. That pivot is correct, and the infrastructure is further along than most people realize. The C2PA standard, Content Credentials, is now shipping in hardware. Cameras from Leica, Sony, Nikon, Canon and Samsung sign images at the moment of capture, with Apple and Google announced for later this year. Adobe’s tools read, preserve and write credentials, Google Search surfaces them, Meta reads them on upload, Tiktok labels with them, Linkedin preserves the chain and X added display in March (for premium users). The Content Authenticity Initiative has over two hundred members, and the BBC, New York Times, Reuters, AP and Washington Post are signing their output. So: a real standard, real silicon, real platform support. And then you meet the holes: Email doesn’t preserve credentials. Messaging apps strip metadata. Most content management systems can’t carry the manifest. And, in the understated phrasing of the adoption trackers, the screenshot problem remains unsolved. Which is to say that a cryptographic chain of custody, signed in hardware at the sensor, authenticated through an editing pipeline, is comprehensively defeated by pressing three keys. The other leg of the strategy is watermarking: marks embedded in the output itself, which should survive the journey that metadata doesn’t. Here, the research is genuinely sobering. A July 2026 empirical evaluation by Tamim and Khan tested three text watermarking schemes: KGW, Unigram and the open-source SynthID-Text implementation, and asked a simple question: Would this stand up as forensic evidence? Before any attack at all, missed detection rates ran from 70 to 83 percent. Then they applied a meaning-preserving paraphrase with a small open model. Among texts the detectors had initially flagged, the watermark was removed in 100, 100 and 98.3 percent of cases, respectively. The authors conclude that none of the schemes meet the evidentiary standard courts require. The caveats are real: one attack type, short texts, small samples and the open-source rather than production SynthID. But the direction isn’t ambiguous. A watermark that a 1.5-billion-parameter model can wash out with a paraphrase isn’t a chain of custody. Which produces a collision worth sitting with. Since 2 August 2026, Article 50 of the EU AI Act has been in force. Providers of generative AI must embed machine-readable markings in synthetic audio, image, video and text so that it’s detectable as artificially generated. Deployers must disclose deepfakes and AI-generated text on matters of public interest. Systems already on the market have until 2 December to comply. Penalties reach 15 million euros or 3 percent of worldwide turnover. I want to be careful here, because this isn’t an argument against the regulation. Requiring marking is reasonable, it raises the cost of casual deception, and the alternative, requiring nothing, is worse. But compliance and efficacy have come apart. Providers can be fully, demonstrably compliant with Article 50 while the markings they embed don’t survive a paraphrase. We should be honest that we’ve
Machines that think: trust and provenance
汽车电子
英文
This is the twelfth part in a series on how AI is crossing out of software and into the physical world. This time, Jan Bosch looks at what it takes to establish that something is real when anything can be synthesized.
本站仅提供摘要,完整内容请阅读原文(版权归原媒体所有)
https://bits-chips.com/article/machines-that-think-trust-and-provenance/
相关阅读
11-09 12:47
11-09 11:09