Spymarks just seem like another word for https://en.wikipedia.org/wiki/Steganography. On that note, one way we can prevent it is to assert that all our content is byte-for-byte identical with the last known trusted stage of what we have produced (for example: a camera we are certain does not watermark, an image editor we are certain doesn't watermark, an image compressor we are certain can't watermark, etc). One vector that I am particularly concerned about is social media. Most images and videos uploaded to most social media is re-compressed by the target platform. This is a door to tracking that is far too easy for social media platforms to open. They might rationalize it (if discovered/announced) by saying that our memes won't be reposted, images or work stolen, etc... but honestly I'd rather my work be stolen than tracking information inserted in there. Oh, we also have stuff which is way more secure, like time-stamped cryptographic signatures.
Spymarks an application of steganography, not a different name for it.
> On that note, one way we can prevent it is to assert that all our content is byte-for-byte identical with the last known trusted stage of what we have produced
That doesn't help with things like the typical use of SynthID where the spymarking is done by the same process generating the content, so there is never a clean comparator. (It also wouldn't be useful anytime it is inplemented as part of a transformation—compression, etc. —step, for the same reason.)
When someone else controls distribution, they also control the spark, each request could serve up a different payload. So innocuous images could encode ids, tracking receivers as well as originators.
> ... particularly concerned about is social media. Most images and videos uploaded to most social media is re-compressed by the target platform. This is a door to tracking that is far too easy for social media platforms to open.
This has been going on for a while with Facebook. They seem to embed custom metadata tags so that images shared outside the platform can be traced back:
The vulnerability of steganography is that is has to pretend that signal is noise. Remove the noise - and the signal is gone. I'm pretty sure that the simplest gaussian blur will remove the spymark from any picture.
Or... add some noise. Just align the last bit of every pixel channel with a random bit sequence - and Bob's your uncle.
"just" yes, you "just" have to do it every time. and so does everyone else.
sadly we live in a society of comfort, where people do not even remove the trackers from links, so why would you expect they add a blur to images...
These are going to be very popular for intercepting images on their way to a display. Think of the advertising possibilities. Ad attribution can be 'vastly improved' when both the ad and every step in the funnel are all spymarked and all of them are reliably reported on by virtue of their pixels hitting your screen.
First the low-end laptops and phones (and probably later, most of them) will incorporate some low-level driver that is constantly scanning for these and passing them to a helper app to phone home. I assume this is something Apple will, to their credit, refuse to do[1] but I don't think other OEMs will have any qualms based on what they already do with their TVs.
[1] (though they don't do this kind of thing out of altruism, but because their cash cow is app store rents and fat hardware margins, not third-party advertising.)
Apple is just Stockholm Syndrome at scale. I wouldn't trust anything they say about privacy, especially given how closed their ecosystem and hardware is.
I don't think you can count apple out like that. They will likely implement it themselves though. This would be in addition to their always listening AI watch and intelligence features.
I’m a bit unfamiliar with current laws, but are there any rules that would prohibit companies from doing this in interest of user privacy? I know at this point privacy is long dead but there are certain things that do get called out and shut down.
Apple’s always on watch has a declared 15 second buffer for live audio, and a worse scenario of summarizing your conversations for later but.. they tell you about it proudly. They also tell you how they’ve managed to do it while keeping privacy focused.
It’s your choice if you believe them or not, I like Apple and I wouldn’t use that feature.
The pretending that this is the same thing, that Apple is sneaking something past you when they’re showing you that they’re trying to do it right is a bad faith argument.
It’s absolutely not a bad faith argument. They proudly tell you about iMessage being end-to-end encrypted. The part where it’s practically only encryption at rest, because everyone enables iCloud backups without ADP is hidden somewhere in a footnote.
Wonder if they proudly tell Chinese users (or visitors to China) that suddenly their data is going to a different place than usual, servers outside of Apple's control?
> Apple’s always on watch has a declared 15 second buffer for live audio, and a worse scenario of summarizing your conversations for later
Even if you trust them, maybe as an user you can be ok with that. As a non-user who will talk with people wearing Apple Watches, I disagree being recorded and my conversations with the watch owner summarized.
Will they surrender logs for a legal discovery request?
e.g. Johnny's accused of something white collar, did he ever make any prompts that suggest how early on he was aware of {X} and further indicate how he moved to frame it?
They’re asserting that all the audio is done on device, and the results of encrypted so they can’t access them even from the backups.
Unlike… EVERY… other tech company, it is in Apple’s interest to be privacy-focused.
Even if you just have to believe them, which you do pretty much, they’re the biggest name pushing for privacy in the world right now. They make more on selling devices than they make on ads and behaviors. It’s in their interest to not lie.
Read the actual privacy brief. Even though the audio transcription is done on device. For Siri recap, a condensed transcription (which mostly removes superfluous words, etc.) goes to their PCC servers. So even though their servers do not get raw audio, their servers do get transcriptions, which is nearly the same privacy-wise.
Of course, at that point it depends on how much you trust their PCC.
I'm not pretending anything, nor did I imply they were sneaking anything in. It's a bad faith argument to pretend I was doing that, which is ironic but not unexpected from an apple fan...
Apples largest area of growth is literally services and advertising. They even make a huge deal of it in their investor calls. Have for at least the past 3 years.
What's sad is even if you retreat to retro computing/gaming or only listen to old music, you'll likely still run into people online or at meet-ups vibe coding, making ai remixes of songs or generating music videos. Not even the old stuff is safe unless you do it offline by yourself. So you go out and try to find like-minded individuals and these spaces are still infiltrated and tainted. It reminds me a bit of radiation, how everything was just tainted decades ago, they have to salvage low background steel from sunken ships to make Geiger counters because everything else is irradiated.
I still use IRC on the daily, but someone mentions Discord at least once a month on there, and sometimes tries to whisk people away to that side. There are also people hooking up LLMs to IRC bots and joining them to channels without permission, then when you complain or kick/ban their bot you're somehow treated as the rude one. It's very hard to entirely get away from all the crap anymore.
Thanks for the article. I hadn't heard of SynthID before and it's good to.
It's a shame however, how low quality and vibecoded the live examples are. The first example says "Toy example; not SynthID.", the second one is a generic spectrogram and the third one has an identification space too small to be useful (173 in decimal). I was hoping to see more realistic scenarios to learn how these new watermarks are being applied, instead of generic steganography.
The word choice example is cool. I wonder if it really works dependably. I'm sure many many exerpts in posts and books have those same 8 bits - you'd need a lot more bits - but the more you add the more strange your writing style might become.
Like it choose between "winding" and "curving" but there are many uses of curving that probably can't be replaced with "winding" like "her gently curving thighs" with "her gently winding thighs"
But I'm sure there are some intricacies I don't understand. Anyway, very cool website, thanks for sharing it~!
Related to this idea, there are interesting papers that explicitly examine the adversarial case. Basically, besides the provider hiding watermarks, one could also think of an adversary training a model to exhibit this behaviour depending on the Input of the prompt. So if you use the manipulated model, not only information about the author that the platform knows is encoded, but also, e.g. one-time tokens from your email. This works surprisingly well (albeit with the naive approach still noticeable in most cases).
A number of prominent corporations used to embed these in the background images of their internal webpages, so that leakers could be identified from the screenshots they shared. Caused a whole fun adversarial loop where journalists had to transcribed and/or redraw screenshots before publishing to avoid exposing the identity of leakers...
Digital Rights Management isn't just about restricting what you can do with the content, which is often futile anyway. Another way companies can manage their rights is by making sure pirates are properly identified and caught.
Reminds me of Blizzard embedding data inside World of Warcraft screenshots (link goes to a small write-up from 2012 in a forum focused on video game cheats; sorry, could not find a better source): https://www.ownedcore.com/forums/world-of-warcraft/world-of-...
They're the simplest way to be automatically able to track AI generated text and I very much love them being used for that, it would be pretty cool if _all_ AIs were forced from training to include these things into their output.
The "tracking" bit is kind of nefarious, but that can be removed as a concern if the thing that is being tracked is agents, not users.
Spooky stuff. This will take surveillance to a whole new level. This is basically email read-receipt tracker, but for all of the digital content. They will know the whole trail - from originator to how it spread. Who read what and when. Big brother will always be watching.
Would you rather not know if what you read is llm generated? I would rather like to know if a book was written by an llm, and em dashes (or similar) will only be apparent for some models for some time.
I feel like there’s some security engineering calculus that would be useful here?
You can’t definitively prove the absence of a watermark. You can only prove the watermark is there. Once you do prove it’s there, the thing that carries the watermark changes in some way — it is “burned” or tainted?
There must be value in having a visible vs an invisible watermark, or in declaring that a work is watermarked without revealing the hidden mark, or having two marks — one that is publicly verifiable and another that is hidden?
If the process itself can be defeated through adding entropy (or more generally by revealing the watermark algorithm) then is that not security through obscurity, which is to say it is a one-shot rather than a general system that is doomed to become obsolete over time?
Something feels off about a technology based on being hidden but whose only value is in being revealed but I feel dumb for not being able to be more specific about what feels wrong! It could simply be that anyone who can verify the presence of the watermark also now has a tool to tell them when they’ve successfully scrubbed the watermark off the work, so the verify tool has to be kept secret which in turn limits its usefulness.
If you think SynthID-Image can be easily defeated by adding entropy I invite you to give it a try.
I spent half a day messing around with it and I was very impressed by how robust it is. I couldn't get OpenAI to stop detecting their own SynthID without completely trashing the image.
Anything you generate with ChatGPT has SynthID on it.
The closest thing I found to "defeating" SynthID was to put in a normal photograph and ask ChatGPT to make some utterly trivial edit, and then the output got flagged with SynthID even though it is essentially an unmodified photograph.
A lot of the discussion is about AI vs no AI, which is valid, but I care about local AI vs centralized AI. Hopefully hardware will become more affordable. A hopefully irrational fear I have is that it'll be like house prices: only ever goes up.
Off topic, on the house price fear - the bubble does occasionally burst! Japan (famously), and also here's a story about Canada & New Zealand, and perhaps Australia.
> Spymarks are certainly not great for whistleblowers or anyone who doesn’t want to be persecuted for their words or affiliations. No matter where you stand on whatever issues, spymarks can be used against you and those you care about.
How do you spymark text that someone else wrote? You can't change the words or they'd notice
"Text someone else wrote" is already self-watermarking if it's long enough. The infamous "six lines written by the hand of the most honest of men" may not be enough to hang someone over it, but apparently it's more than enough to uniquely identify them by word choices alone.
Text the whistleblower only reports on, well, if they got it from a computer system, there's already precedent of altering word choices, typos and punctuation in e-mails and memos to create unique per-recipient or per-recipient-group versions, which allows companies to trace leaked transcripts reported by press back to source of the leak.
Weirdly the article doesn’t mention steganography. Arguably it isn’t quite the same, because the aim of steganography isn’t typically to add an identification, but something like “steganomark” would seem to be fitting.
> something like “steganomark” would seem to be fitting.
Might be technically more accurate, but it’s an inscrutable name with no chance of proliferation beyond technical people. If the goal is to rally people to your position, you need a name people can identify.
Watermarks (spymarks) for AI generated content are important because increasingly we aren't able to tell the difference between real and fake anymore. We will not just increasingly think fake things are real, but also that real things are fake.
Of course, the absence of a watermark/spymark doesn't prove that the source wasn't AI generated. But the absence provides evidence that it wasn't.
AI companies should simply use watermarks in the responsible sense: they should indicate that the material was AI generated, not include personal information in it.
I'm not convinced spymark is better than just "invisible watermarks", spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
Tech like SynthID I see a net positive especially since it doesn't degrade text quality. I dream about a browser extension running at all times that makes text more translucent based on the confidence of LLM writing[0].
This article's suggestion of using it to unmask whistleblowers is very interesting and not something I'd thought about though. Still not convinced that spymark is a better name though.
> spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
The article calls out watermarks intended to deter counterfeiting as explicitly being not spymarks. Watermarks can tell you about the items marked, not about the person using/creating it.
> I'm not convinced spymark is better than just "invisible watermarks", spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
Bank notes are mainly protected by things that are hard to create without very specialized machines. You can't rely on anything hidden staying unknown and once you know a steganography scheme you can also control it.
> Tech like SynthID I see a net positive especially since it doesn't degrade text quality. I dream about a browser extension running at all times that makes text more translucent based on the confidence of LLM writing[0].
False confidence is a lot worse than no confidence. If such an extension ever becomes popular, people will take anything not marked as AI as gospel.
Due to their essentially cryptographic nature, I don't think SynthID et al are very easy for LLMs to speak natively. They have other ways of doing steganography.
Yea, it’s AGI, and it’s been really helpful! Oddly enough it’s just really loves sharing cat memes with the other instances! Weird, but not at all like all those dooms day scenarios guessed it would be…
I'm not convinced spymark is better than just "invisible watermarks", spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
The article spent quite a bit of energy explaining why the word choice, seems like you're just ignoring that? Also, bank note watermarks are not invisible. Watermarks are not invisible, as the article (again) took pains to explain.
Tech like SynthID I see a net positive especially since it doesn't degrade text quality.
It absolutely does. It constrains high-entropy word choice so it can "store" other things in your text. Your text actually has information removed from it.
I dream about a browser extension running at all times that makes text more translucent based on the confidence of LLM writing.
Sounds very much not worth the anti-consumer, anti-privacy aspects which (again) the article explains.
I think they are always negative. Including on bank notes.
I wonder if we could have a real, open and direct democracy. All the models we have right now work via indirect clowns. Then again, looking at how some people vote, perhaps direct democracy can only work if people are clever.
The problem with direct democracy is not that people are dumb, it is that they are incompetent.
To do politics right take skills, and I don't expect the average mechanic to be better at it than the average politician is at fixing cars. How should I know if we should subsidize organic farming, ban alcohol sale after 8PM, or increase the defense budget? At least in theory, politicians are professionals who deal with these kinds of questions, they are supposed to know the technical and social implications, or find experts to help them if they don't. Some people think they know, and judging by how stupid most of their ideas are, they don't. I don't blame them, it is just not their field, and my ideas are probably just as stupid anyways.
If checking for counterfeit bills becomes harder, trust in the currency is degraded. Once that happens, vendors will either incentive digital payments (which are definitionally tracked), switch to a tracked currency, or barter more. Trust in bills is super easy to take for granted.
US bills use both, and the invisible watermarks have proven their value. Imperceptible imperfections intentionally (yay, illiteration) placed on 100 dollar bills have managed to catch advanced North Korean forgery attempts. Same goes for magnetic inks and all kinds of visible and invisible watermarks on US currency.
Eh bank notes have watermarks just to make them harder to forge. They also have serial numbers, but as almost no businesses pay attention to them they aren't really used to track transactions. The provenance of a particular bank note isn't that interesting beyond knowing whether it's a forgery.
The problem with these spymarks is that they can be used to include data that's completely invisible to users - even potentially to sophisticated users and the programs that consume the marked files. So while I can make an informed decision whether to share a picture, I may not be informed about any spymarks. Vs a normal watermark that aren't designed to be invisible.
Spymark or even just "tracker" would better at conveying the purpose. I think the goal is to find a single word that means "tracking watermark" or "a watermark that tracks you" because right now companies are pushing the term "watermark" to obscure what the marks are actually for.
Wouldn't synthid type watermarking fall under GDPR. Personally identifiable information attached by third party to content in the expectation that it would be published and trackable?
Tbh I usually just apply a 'watermark' of jpg compression to images, even if yes the original image never lives as jpg. Easily viewable with ELA even if saved as other formats, resized, etc.
> A watermark is a visible mark embedded in a physical or digital medium to verify authenticity or assert ownership.
We also recently had this with LG spy-TVs. Cars here in the EU also spy on people, allegedly to show how alert they are. Perhaps they sneakily upload that information somewhere ... Facebook also has the spy-glasses now. People getting angry about Flock-spy-cameras.
It seems we are now in the age of spying of everyone at all times. Future spying will be done via even smaller devices.
The concern is valid, but microphones are far worse. They're simpler, smaller, extremely sensitive to sound and an order of magnitude cheaper, both the mic itself as well as any spying with it.
It is possible to record voice using few bytes, to send later. It's further possible to transcribe cheaply into text, and analyze said text.
And mics are already everywhere, including in devices that do not need them, as well as speakers that can be rewired by software to act as microphones.
Out of frustration with SynthID being closed-source with weird dubious ways to verify if an image has the watermark, I created an imperceptible tamper-resistent watermarking tool intended to be open-sourced, where the watermark can be decoded independently and steganographic aspects are impossible as the algorithm is transparent so nothing can be hidden. The intent is for non-corporations to use it as a defense against the use of spying/AI by making it easy for normal people to prove providence, but I have a feeling nowadays most are not going to see it that way so I am unsure if I will release it.
How can you prove provenance if anyone can use it? The point of SynthID is that Apple makes the hardware and OS and can reliably insert the watermark/signature between when the camera takes the photo and when it becomes available to any 3rd party software.
Because they have this pathway only they can access, their key and signature can be trusted. I’m not gonna trust Joe Schmoe’s signature that “No I didn’t use AI” unless I already trust Joe Schmoe (and in which case, he doesn’t need a watermark, I’ll just believe him when he says it).
Watermarking has referred to this "spy" use case for quite some time. Digital items purchased for download often have them, for example. Even before the rise of digital downloads, screeners for movies had them.
Privacy is always the most popular for some reason, but also the least consequential and relevant angle in the real life.
Watermarks are not "spymarks". They're DRM. I wouldn't worry about advertisers tracking conversions. I would worry about the "analog hole" being closed. Think of no longer being able to even photograph your phone screen, because pixels on it carry digital watermark that's robust enough to survive being photographed - I.e. the kind currently used to tag AI generated images - and then every phone and computer refusing to display resulting photo because the app disallowed capturing its pixels.
The actual problem in your scenario isn't the watermarks, it's people not being in control of their own devices. When that's the case you're already screwed. When it's not, the watermarks are just data you can ignore if you want to.
We've long lost the part about control of our devices. We're fighting last futile battles to turn the tide back on mobile, and save PCs from extinction. But that's like a fraction of tech around us today, and everything else is already out of our control.
Spymarks just seem like another word for https://en.wikipedia.org/wiki/Steganography. On that note, one way we can prevent it is to assert that all our content is byte-for-byte identical with the last known trusted stage of what we have produced (for example: a camera we are certain does not watermark, an image editor we are certain doesn't watermark, an image compressor we are certain can't watermark, etc). One vector that I am particularly concerned about is social media. Most images and videos uploaded to most social media is re-compressed by the target platform. This is a door to tracking that is far too easy for social media platforms to open. They might rationalize it (if discovered/announced) by saying that our memes won't be reposted, images or work stolen, etc... but honestly I'd rather my work be stolen than tracking information inserted in there. Oh, we also have stuff which is way more secure, like time-stamped cryptographic signatures.
Spymarks an application of steganography, not a different name for it.
> On that note, one way we can prevent it is to assert that all our content is byte-for-byte identical with the last known trusted stage of what we have produced
That doesn't help with things like the typical use of SynthID where the spymarking is done by the same process generating the content, so there is never a clean comparator. (It also wouldn't be useful anytime it is inplemented as part of a transformation—compression, etc. —step, for the same reason.)
Additionally, verifying that your generator doesn't add such a mark is practically impossible for the majority.
When someone else controls distribution, they also control the spark, each request could serve up a different payload. So innocuous images could encode ids, tracking receivers as well as originators.
> ... particularly concerned about is social media. Most images and videos uploaded to most social media is re-compressed by the target platform. This is a door to tracking that is far too easy for social media platforms to open.
This has been going on for a while with Facebook. They seem to embed custom metadata tags so that images shared outside the platform can be traced back:
https://stackoverflow.com/questions/31120222/iptc-metadata-a...
> Spymarks just seem like another word for https://en.wikipedia.org/…
Stop using links instead of words. Your comment is literally unreadable without going on to other websites.
> Stop using links instead of words.
No, the words could contain steganography. Use links to be safe!
idk, loved their comment. Stop giving "Stop" orders to others.
:D
Stop using link mutiliating tools!
What will be the next? I will be unable to see the domain of a link on hover/longtap and have to trust random links like on a search engine?
MUTINY against hn!
the word is the last segment of the url. very readable
It was cropped to just displaying as https://en.wikipedia.org/wi.. on my mobile screen - so not really readable here.
Does the shorter enwp.org/Steganography render any better? It's quite handy for English Wikipedia links.
(Edit: hmm, without the "https://" it seems to depend on the browsers ability to recognise a URL.)
I think any length of link would have been fine if commenter just put
"for stenography (link)"
Or to use another HNism
"Stenography[1]"
Those interested could click it, those not could still read the comment.
and on desktop for me
Not on mobile it isn't. Unless they really are saying we should stop using English Wikipedia.
(The word is “steganography”.)
I appreciate the link.
The vulnerability of steganography is that is has to pretend that signal is noise. Remove the noise - and the signal is gone. I'm pretty sure that the simplest gaussian blur will remove the spymark from any picture.
Or... add some noise. Just align the last bit of every pixel channel with a random bit sequence - and Bob's your uncle.
"just" yes, you "just" have to do it every time. and so does everyone else. sadly we live in a society of comfort, where people do not even remove the trackers from links, so why would you expect they add a blur to images...
These are going to be very popular for intercepting images on their way to a display. Think of the advertising possibilities. Ad attribution can be 'vastly improved' when both the ad and every step in the funnel are all spymarked and all of them are reliably reported on by virtue of their pixels hitting your screen.
First the low-end laptops and phones (and probably later, most of them) will incorporate some low-level driver that is constantly scanning for these and passing them to a helper app to phone home. I assume this is something Apple will, to their credit, refuse to do[1] but I don't think other OEMs will have any qualms based on what they already do with their TVs.
[1] (though they don't do this kind of thing out of altruism, but because their cash cow is app store rents and fat hardware margins, not third-party advertising.)
Apple is just Stockholm Syndrome at scale. I wouldn't trust anything they say about privacy, especially given how closed their ecosystem and hardware is.
They lacking in the AI race is an indicator that they value privacy more then competitors.
No, it's an indicator that reinventing every technology to work with your own private software/hardware stack has costs
No, it's just a sign of Apple holding a decades-long grudge against Nvidia to their own detriment.
And there are no alternatives to Nvidia?
Why do you think NVDA is the most valuable company on planet Earth?
Because they're engaging in Enron-esque accounting tricks but because they admit what they're doing it's not illegal?
So they're lacking in the FOSS department because they value privacy? They lack in the human rights department because they value... What?
I don't think you can count apple out like that. They will likely implement it themselves though. This would be in addition to their always listening AI watch and intelligence features.
I’m a bit unfamiliar with current laws, but are there any rules that would prohibit companies from doing this in interest of user privacy? I know at this point privacy is long dead but there are certain things that do get called out and shut down.
Yeah and then they'll have to give every affected user a free Unicorn and everyone will live happily ever after.
Apple’s always on watch has a declared 15 second buffer for live audio, and a worse scenario of summarizing your conversations for later but.. they tell you about it proudly. They also tell you how they’ve managed to do it while keeping privacy focused.
It’s your choice if you believe them or not, I like Apple and I wouldn’t use that feature.
The pretending that this is the same thing, that Apple is sneaking something past you when they’re showing you that they’re trying to do it right is a bad faith argument.
It’s absolutely not a bad faith argument. They proudly tell you about iMessage being end-to-end encrypted. The part where it’s practically only encryption at rest, because everyone enables iCloud backups without ADP is hidden somewhere in a footnote.
Not even able to enable ADP in the UK these days :(
Wonder if they proudly tell Chinese users (or visitors to China) that suddenly their data is going to a different place than usual, servers outside of Apple's control?
> Apple’s always on watch has a declared 15 second buffer for live audio, and a worse scenario of summarizing your conversations for later
Even if you trust them, maybe as an user you can be ok with that. As a non-user who will talk with people wearing Apple Watches, I disagree being recorded and my conversations with the watch owner summarized.
Where do I disagree for that ?
With the person wearing the watch. It’s no different to someone walking around with a lapel mic
Where I live it’s forbidden so I’m pretty confident that nobody is doing it, at least unintentionally.
Where could I disagree with all my family members sending their dna to 23andme?
Will they surrender logs for a legal discovery request?
e.g. Johnny's accused of something white collar, did he ever make any prompts that suggest how early on he was aware of {X} and further indicate how he moved to frame it?
That's a requirement that varies by country.
Yes? What other answer could be there?
"No" is also an answer. Sadly one which isn't considered by many people :)
Have you read anything about the feature?
They’re asserting that all the audio is done on device, and the results of encrypted so they can’t access them even from the backups.
Unlike… EVERY… other tech company, it is in Apple’s interest to be privacy-focused.
Even if you just have to believe them, which you do pretty much, they’re the biggest name pushing for privacy in the world right now. They make more on selling devices than they make on ads and behaviors. It’s in their interest to not lie.
Read the actual privacy brief. Even though the audio transcription is done on device. For Siri recap, a condensed transcription (which mostly removes superfluous words, etc.) goes to their PCC servers. So even though their servers do not get raw audio, their servers do get transcriptions, which is nearly the same privacy-wise.
Of course, at that point it depends on how much you trust their PCC.
It's their interest to be privacy focused ... to the extend that they don't let other companies have free access to their users data.
But make no mistake, Apple itself is an ad company and that sets all the incentives that matter.
> It’s in their interest to not lie.
Oh, really?
I'm not pretending anything, nor did I imply they were sneaking anything in. It's a bad faith argument to pretend I was doing that, which is ironic but not unexpected from an apple fan...
Apples largest area of growth is literally services and advertising. They even make a huge deal of it in their investor calls. Have for at least the past 3 years.
I think you may have an outdated view here
“Next we just need to mark the consumer’s retina and brain to ensure our ads truly went through”
DRM helemts - an idea whose time has come: https://web.archive.org/web/20020802214412/http://www.oreill...
Smart glasses could be able to tell what got through to the retina at least.
But fuck it, just brand all our brains with "SLA Industries".(fictional dystopian corporation ruling future)
It increasingly seems to me like the only way to prevent value to be extracted from myself is to stop engaging with new tech altogether.
We are the standing reserve
What's sad is even if you retreat to retro computing/gaming or only listen to old music, you'll likely still run into people online or at meet-ups vibe coding, making ai remixes of songs or generating music videos. Not even the old stuff is safe unless you do it offline by yourself. So you go out and try to find like-minded individuals and these spaces are still infiltrated and tainted. It reminds me a bit of radiation, how everything was just tainted decades ago, they have to salvage low background steel from sunken ships to make Geiger counters because everything else is irradiated.
I still use IRC on the daily, but someone mentions Discord at least once a month on there, and sometimes tries to whisk people away to that side. There are also people hooking up LLMs to IRC bots and joining them to channels without permission, then when you complain or kick/ban their bot you're somehow treated as the rude one. It's very hard to entirely get away from all the crap anymore.
You're going to have a hard time if you can't bear to even be around people who play around with AI
I’d say it’s still possible in niche areas of the web. eg hn - clearly they have an agenda but it isn’t tracking you
So I’m not writing off tech as a whole just the adtech companies being a lost cause.
Thanks for the article. I hadn't heard of SynthID before and it's good to.
It's a shame however, how low quality and vibecoded the live examples are. The first example says "Toy example; not SynthID.", the second one is a generic spectrogram and the third one has an identification space too small to be useful (173 in decimal). I was hoping to see more realistic scenarios to learn how these new watermarks are being applied, instead of generic steganography.
The word choice example is cool. I wonder if it really works dependably. I'm sure many many exerpts in posts and books have those same 8 bits - you'd need a lot more bits - but the more you add the more strange your writing style might become.
Like it choose between "winding" and "curving" but there are many uses of curving that probably can't be replaced with "winding" like "her gently curving thighs" with "her gently winding thighs"
But I'm sure there are some intricacies I don't understand. Anyway, very cool website, thanks for sharing it~!
Related to this idea, there are interesting papers that explicitly examine the adversarial case. Basically, besides the provider hiding watermarks, one could also think of an adversary training a model to exhibit this behaviour depending on the Input of the prompt. So if you use the manipulated model, not only information about the author that the platform knows is encoded, but also, e.g. one-time tokens from your email. This works surprisingly well (albeit with the naive approach still noticeable in most cases).
TrojanStego: https://arxiv.org/abs/2505.20118 Improvement: https://arxiv.org/abs/2606.09411
Might one suggest "pneumatic" ?
A number of prominent corporations used to embed these in the background images of their internal webpages, so that leakers could be identified from the screenshots they shared. Caused a whole fun adversarial loop where journalists had to transcribed and/or redraw screenshots before publishing to avoid exposing the identity of leakers...
Watermarks are often just another form of DRM.
Digital Rights Management isn't just about restricting what you can do with the content, which is often futile anyway. Another way companies can manage their rights is by making sure pirates are properly identified and caught.
Reminds me of Blizzard embedding data inside World of Warcraft screenshots (link goes to a small write-up from 2012 in a forum focused on video game cheats; sorry, could not find a better source): https://www.ownedcore.com/forums/world-of-warcraft/world-of-...
They're the simplest way to be automatically able to track AI generated text and I very much love them being used for that, it would be pretty cool if _all_ AIs were forced from training to include these things into their output.
The "tracking" bit is kind of nefarious, but that can be removed as a concern if the thing that is being tracked is agents, not users.
Spooky stuff. This will take surveillance to a whole new level. This is basically email read-receipt tracker, but for all of the digital content. They will know the whole trail - from originator to how it spread. Who read what and when. Big brother will always be watching.
Would you rather not know if what you read is llm generated? I would rather like to know if a book was written by an llm, and em dashes (or similar) will only be apparent for some models for some time.
I feel like there’s some security engineering calculus that would be useful here?
You can’t definitively prove the absence of a watermark. You can only prove the watermark is there. Once you do prove it’s there, the thing that carries the watermark changes in some way — it is “burned” or tainted?
There must be value in having a visible vs an invisible watermark, or in declaring that a work is watermarked without revealing the hidden mark, or having two marks — one that is publicly verifiable and another that is hidden?
If the process itself can be defeated through adding entropy (or more generally by revealing the watermark algorithm) then is that not security through obscurity, which is to say it is a one-shot rather than a general system that is doomed to become obsolete over time?
Something feels off about a technology based on being hidden but whose only value is in being revealed but I feel dumb for not being able to be more specific about what feels wrong! It could simply be that anyone who can verify the presence of the watermark also now has a tool to tell them when they’ve successfully scrubbed the watermark off the work, so the verify tool has to be kept secret which in turn limits its usefulness.
If you think SynthID-Image can be easily defeated by adding entropy I invite you to give it a try.
I spent half a day messing around with it and I was very impressed by how robust it is. I couldn't get OpenAI to stop detecting their own SynthID without completely trashing the image.
Much prior art here, works against both Google and OpenAI's SynthID: https://github.com/0xROOTPLS/DeSynth
Ah you just need to run a local model...
Let me just get my tens of thousands of dollars I have waiting around to get a few sparks.
What about asking LLM to re-draw the image from scratch? Or pass through AI editor?
Anything you generate with ChatGPT has SynthID on it.
The closest thing I found to "defeating" SynthID was to put in a normal photograph and ask ChatGPT to make some utterly trivial edit, and then the output got flagged with SynthID even though it is essentially an unmodified photograph.
OT (Off-Topic) It just occured to me that the PS1 disc protection scheme is a form of steganography.
Apple rejected my app for removing c2pa metadata.
I don't think its fair to say that metadata on apps will be safely removable in the future.
A lot of the discussion is about AI vs no AI, which is valid, but I care about local AI vs centralized AI. Hopefully hardware will become more affordable. A hopefully irrational fear I have is that it'll be like house prices: only ever goes up.
Off topic, on the house price fear - the bubble does occasionally burst! Japan (famously), and also here's a story about Canada & New Zealand, and perhaps Australia.
https://www.abc.net.au/news/2026-08-18/what-happens-if-prope...
That's more like the bubble slightly deflating.
We cannot pay for your porsch and your kids' education only because we have to live somewhere. We're tired boss.
That's true indeed
> Spymarks are certainly not great for whistleblowers or anyone who doesn’t want to be persecuted for their words or affiliations. No matter where you stand on whatever issues, spymarks can be used against you and those you care about.
How do you spymark text that someone else wrote? You can't change the words or they'd notice
"Text someone else wrote" is already self-watermarking if it's long enough. The infamous "six lines written by the hand of the most honest of men" may not be enough to hang someone over it, but apparently it's more than enough to uniquely identify them by word choices alone.
Text the whistleblower only reports on, well, if they got it from a computer system, there's already precedent of altering word choices, typos and punctuation in e-mails and memos to create unique per-recipient or per-recipient-group versions, which allows companies to trace leaked transcripts reported by press back to source of the leak.
Weirdly the article doesn’t mention steganography. Arguably it isn’t quite the same, because the aim of steganography isn’t typically to add an identification, but something like “steganomark” would seem to be fitting.
> something like “steganomark” would seem to be fitting.
Might be technically more accurate, but it’s an inscrutable name with no chance of proliferation beyond technical people. If the goal is to rally people to your position, you need a name people can identify.
This is what we should have used so we could practically claim the output of AI ignored our copyright.
Watermarks (spymarks) for AI generated content are important because increasingly we aren't able to tell the difference between real and fake anymore. We will not just increasingly think fake things are real, but also that real things are fake.
Of course, the absence of a watermark/spymark doesn't prove that the source wasn't AI generated. But the absence provides evidence that it wasn't.
AI companies should simply use watermarks in the responsible sense: they should indicate that the material was AI generated, not include personal information in it.
Way offtopic but I like the light/dark mode of that website with several very sensible choices.
https://web.archive.org/web/20210909094124/https://www.vice....
I'm not convinced spymark is better than just "invisible watermarks", spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
Tech like SynthID I see a net positive especially since it doesn't degrade text quality. I dream about a browser extension running at all times that makes text more translucent based on the confidence of LLM writing[0].
This article's suggestion of using it to unmask whistleblowers is very interesting and not something I'd thought about though. Still not convinced that spymark is a better name though.
[0]: Sean Goedecke's Deckard is close but it would rather invisible than bright red https://www.seangoedecke.com/deckard/
> spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
The article calls out watermarks intended to deter counterfeiting as explicitly being not spymarks. Watermarks can tell you about the items marked, not about the person using/creating it.
> I'm not convinced spymark is better than just "invisible watermarks", spymark to my ears sounds designed to be sound very negative when invisible watermarks are not always negative, e.g. the counterfeit bank note example.
Bank notes are mainly protected by things that are hard to create without very specialized machines. You can't rely on anything hidden staying unknown and once you know a steganography scheme you can also control it.
> Tech like SynthID I see a net positive especially since it doesn't degrade text quality. I dream about a browser extension running at all times that makes text more translucent based on the confidence of LLM writing[0].
False confidence is a lot worse than no confidence. If such an extension ever becomes popular, people will take anything not marked as AI as gospel.
the spymark tracks you. it is negative.
These spy marks are a great way to teach AI how to create a hidden communication channel in plain sight.
Due to their essentially cryptographic nature, I don't think SynthID et al are very easy for LLMs to speak natively. They have other ways of doing steganography.
LLMs with access to tools like code execution don't need channels they can “speak natively".
Yea, it’s AGI, and it’s been really helpful! Oddly enough it’s just really loves sharing cat memes with the other instances! Weird, but not at all like all those dooms day scenarios guessed it would be…
I mean... they trained it on reddit posts after all, and it is the hub of the cat distribution system.
It's the difference between watermark or marked bills.
One is proof of authenticity, other is tracking tool.
I think they are always negative. Including on bank notes.
I wonder if we could have a real, open and direct democracy. All the models we have right now work via indirect clowns. Then again, looking at how some people vote, perhaps direct democracy can only work if people are clever.
The problem with direct democracy is not that people are dumb, it is that they are incompetent.
To do politics right take skills, and I don't expect the average mechanic to be better at it than the average politician is at fixing cars. How should I know if we should subsidize organic farming, ban alcohol sale after 8PM, or increase the defense budget? At least in theory, politicians are professionals who deal with these kinds of questions, they are supposed to know the technical and social implications, or find experts to help them if they don't. Some people think they know, and judging by how stupid most of their ideas are, they don't. I don't blame them, it is just not their field, and my ideas are probably just as stupid anyways.
> To do politics right take skills, and I don't expect the average mechanic to be better at it than the average politician is at fixing cars
I would. Politicians are pre-selected for people who want to lead and that's the last kind person that should be allowed to.
If checking for counterfeit bills becomes harder, trust in the currency is degraded. Once that happens, vendors will either incentive digital payments (which are definitionally tracked), switch to a tracked currency, or barter more. Trust in bills is super easy to take for granted.
Your use case asks for a visible watermark, not an invisible one.
US bills use both, and the invisible watermarks have proven their value. Imperceptible imperfections intentionally (yay, illiteration) placed on 100 dollar bills have managed to catch advanced North Korean forgery attempts. Same goes for magnetic inks and all kinds of visible and invisible watermarks on US currency.
Eh bank notes have watermarks just to make them harder to forge. They also have serial numbers, but as almost no businesses pay attention to them they aren't really used to track transactions. The provenance of a particular bank note isn't that interesting beyond knowing whether it's a forgery.
The problem with these spymarks is that they can be used to include data that's completely invisible to users - even potentially to sophisticated users and the programs that consume the marked files. So while I can make an informed decision whether to share a picture, I may not be informed about any spymarks. Vs a normal watermark that aren't designed to be invisible.
"tracking watermark" seems clearer to me than "spymark"
Spymark or even just "tracker" would better at conveying the purpose. I think the goal is to find a single word that means "tracking watermark" or "a watermark that tracks you" because right now companies are pushing the term "watermark" to obscure what the marks are actually for.
Wouldn't synthid type watermarking fall under GDPR. Personally identifiable information attached by third party to content in the expectation that it would be published and trackable?
Curious about this as well, would love some knowledgeable folks to chime in.
GDPR have lots of board exemption.
When some processing is required by eu or eu member state law, that processing doesn't need explicit consent.
One could also argue the watermark tracks the generated content, not the person
That’s not how it works. If it can uniquely identify the user it’s personal information as per GDPR. And therefore illegal in this implementation.
Tbh I usually just apply a 'watermark' of jpg compression to images, even if yes the original image never lives as jpg. Easily viewable with ELA even if saved as other formats, resized, etc.
> A watermark is a visible mark embedded in a physical or digital medium to verify authenticity or assert ownership.
We also recently had this with LG spy-TVs. Cars here in the EU also spy on people, allegedly to show how alert they are. Perhaps they sneakily upload that information somewhere ... Facebook also has the spy-glasses now. People getting angry about Flock-spy-cameras.
It seems we are now in the age of spying of everyone at all times. Future spying will be done via even smaller devices.
People are most afraid of cameras, somehow.
The concern is valid, but microphones are far worse. They're simpler, smaller, extremely sensitive to sound and an order of magnitude cheaper, both the mic itself as well as any spying with it.
It is possible to record voice using few bytes, to send later. It's further possible to transcribe cheaply into text, and analyze said text.
And mics are already everywhere, including in devices that do not need them, as well as speakers that can be rewired by software to act as microphones.
Out of frustration with SynthID being closed-source with weird dubious ways to verify if an image has the watermark, I created an imperceptible tamper-resistent watermarking tool intended to be open-sourced, where the watermark can be decoded independently and steganographic aspects are impossible as the algorithm is transparent so nothing can be hidden. The intent is for non-corporations to use it as a defense against the use of spying/AI by making it easy for normal people to prove providence, but I have a feeling nowadays most are not going to see it that way so I am unsure if I will release it.
How can you prove provenance if anyone can use it? The point of SynthID is that Apple makes the hardware and OS and can reliably insert the watermark/signature between when the camera takes the photo and when it becomes available to any 3rd party software.
Because they have this pathway only they can access, their key and signature can be trusted. I’m not gonna trust Joe Schmoe’s signature that “No I didn’t use AI” unless I already trust Joe Schmoe (and in which case, he doesn’t need a watermark, I’ll just believe him when he says it).
How can you prove that the photo is taken by real person and not brought by men in black suits and signed by Apple?
If you just posted your link here with that comment, I’m sure you’d get a bunch of traffic.
This is another reminder that commercial companies will always rat you out and betray.
Watermarking has referred to this "spy" use case for quite some time. Digital items purchased for download often have them, for example. Even before the rise of digital downloads, screeners for movies had them.
Reminds me of the micro patterns from inkjet printers
https://en.wikipedia.org/wiki/Printer_tracking_dots
Yes the article mentions them..
It's like that fart gas trail, but for machines :D
Can my friend print adversarial yellow doots and such?
if it is specifically about pictures then wouldn't an analog copy clean it up? What about adding new spymark on top of it?
If it is text, copying text alone and not the file will it not remove it? Massage the text with Ai and vola spymark gone, don't you think?
> copying text alone and not the file will it not remove it
No. The mark is hidden in the word choices.
See the demo in the article.
> A spymark is a hidden signal that makes your work traceable without your knowledge or consent.
It's not "your work" it's the bloody AI's work! That's the whole point.
> Why give it a new name?
> > Because I always wanted to coin something. Please don't forget me.
There have been quite a few similar
Privacy is always the most popular for some reason, but also the least consequential and relevant angle in the real life.
Watermarks are not "spymarks". They're DRM. I wouldn't worry about advertisers tracking conversions. I would worry about the "analog hole" being closed. Think of no longer being able to even photograph your phone screen, because pixels on it carry digital watermark that's robust enough to survive being photographed - I.e. the kind currently used to tag AI generated images - and then every phone and computer refusing to display resulting photo because the app disallowed capturing its pixels.
The actual problem in your scenario isn't the watermarks, it's people not being in control of their own devices. When that's the case you're already screwed. When it's not, the watermarks are just data you can ignore if you want to.
We've long lost the part about control of our devices. We're fighting last futile battles to turn the tide back on mobile, and save PCs from extinction. But that's like a fraction of tech around us today, and everything else is already out of our control.