Maybe my perception is off, but it seems like there's a huge push by the legislature and some people to do anything and everything to prevent CSAM, yet almost nothing seems to be done to prevent CSA.
For CSAM, there's all sorts of monitoring, scanning, identify capturing, etc. But it's all after abuse has taken place, and it seems that many of the people actually arrested are arrested for CSAM and not CSA. This has even extended to fictional CSAM such as AI generated stories and pictures. As an aside, if that gets extended to political speech or other non-CSAM materials that are determined to be undesirable, that's a big concern. I can imagine that a conservative state could pass a law banning all porn because they claim it could encourage illegal activities such as prostitution, rape, or CSA.
On the CSA side, you rarely hear about arrests (they happen but less than CSAM). There doesn't seem to be any real push for educating and protecting kids before it happens. Ironically, the groups doing the most to educate and implement protective strategies are the ones who have been involved in abuse scandals in the past (Churches, Scouts, etc). Even then, a lot of it is just getting clearances, which doesnt prevent people who where not caught or were first timers. Offenders get put on a list/map. This is sort of a half approach. If they are still a threat, they shouldn't be released. Yet if you comb the list and see some of the results, they don't all seem to fit with CSA. I personally know of 1 who took a leak across from a playground at 2am walking home from the bars and was put on the sex offender registry because it was within 500'.
It seems like these laws are more about peddling to the publicist and lawmakers fantasy of incrementally extreme punishment rather than taking a appropriate, data driven, and level-headed approach that actually protects kids. Otherwise they will just keep pushing ham-fisted low-hanging "fixes" like required scanning and IDs to access the internet.
It is crazy people think apple isnt on the side of privacy. Are they perfect? Not even close, but compared to the rest of big tech theyre simply on another level.
Apple could easily not do this stuff and it may even be easier to not.
> It is crazy people think apple isnt on the side of privacy.
> It also ensured pressure from governments and plaintiffs, including CSAM victims, who preferred Apple’s more interventionist approaches, which Apple had voluntarily demonstrated it was willing to do.
I feel that Apple open pandora's box with the client-side scanning. It proved that it was technically feasible, and was "privacy preserving". I use scare quotes there because I don't think that political or religious dissidents would find that the same or similar technology used to discover and persecute them is "privacy preserving". And that's really the problem with Apple here. They provided a model for scanning for any kind of message or material while purportedly maintaining privacy.
> It proved that it was technically feasible, and was "privacy preserving".
Didn't their paper disproved by reversing the perceptual hashes to reveal blurred version of the images being hashed, and Apple basically said "that's fair, it's not as robust as we wanted, let's visit this later"?
If not, I'll happily stand corrected, but please share sources.
The paper you linked doesn’t reveal blurred versions of the images being hashed. It does train a classifier to determine which of 1,000 ImageNet classes an image belongs to, which “achieved a top-1 test accuracy of 4.34%”.
> to reveal blurred version of the images being hashed
Skimmed your linked paper. It seems they were able to classify hashes up to ~8% top-1 accuracy and ~30% top-10.
Not exactly a blurred version, or any images at all.
So for example, they can say that you probably have images of trees, or images of buildings, but without much other data & very low accuracy.
I'd still be a lot more concerned about them simply flagging political images rather than trying to get a broad understanding of what type of photos I have
This is not client side. Images just sitting on your Android phone are probably safe (although Google could push an update at any time). Your images get scanned when you back them up, send them over RCS, etc. I have even seen criminal cases originating from reverse image search - anything that touche the servers of the big tech companies, except apparently Apple, will be scanned using questionable AI and against a secret list to Protect the Children.
This is an image that he did not send off of his device to anyone except his doctor's office, yet Google reached into his private data and scanned it anyway.
Google reported him to the police based on a single false positive.
To add insult to injury, even after the police contacted Google to tell them that they had cleared him of wrongdoing, Google refused to restore access to his account.
Stop making stuff up man, the image was uploaded to Google Photos servers.
> The father uploaded photos of his son’s genitals, which were also backed up on his Google cloud, to the health care provider’s messaging system as requested.
No, he didn’t affirmatively upload it anywhere, it was automatically synced and scanned when he was following his provider’s instructions to upload HIPAA-protected medical photos through using a device that wasn’t designed for that use case and without any reasonable awareness of the technology and risks involved.
I’m sure there are disclaimers buried in privacy policies and terms of service that nobody reads but let’s not pretend a random user is providing well-informed consent.
Did he set up his device to upload his private data to Google, or did Google create an OS that automatically sent everyone's private data to their AI server for scanning without explicit consent?
I thought the client side scanning was to protect children? If it suspects an image is bad, it blurs it and pops up a warning including a link to resources to go to for help.
Very different than trying to narc out users to the authorities.
The other is detection of images that may contain nudity, whether sent or received, when the owner/admin/parent enables the feature. It is relatively uncontroversial and is described here: https://support.apple.com/en-us/105069
That is what they actually deployed. They were planning on performing client-side scanning of all images uploaded to iCloud for CSAM and reporting it to the authorities, but backpedaled after public push-back.
The original proposal was to use a visual hash designed to identify known bad CSAM images even if cropped or otherwise edited. Your computer would scan all your stuff for these images and report you to apple who would report you to the cops. This presented a number of issues.
Accidental false positives could lead to horrific outcomes up to and including oh look bob got shot by the cops for resisting.
It was possible to produce apparently matching innocuous images and then poison people's machines with them.Oops did you click on that picture of a tree have fun with the cops. Like an advanced form of swatting.
Although inspired by a desire to find CSAM Apple could be forced to scan for ANYTHING by repressive regimes including America and China.
Although initially targeting images client side scanning of messages is a pretty obvious next step. Again obvious good motivation exists and is completely justifiable who doesn't want to stop the next mass shooting or terrorist attack... and then we can basically use it to find people critical of the regime. Do remember we are presently prosecuting a political figure for a picture of sea shells and a guy in texas is rotting in prison for distributing political literature.
Yes, in the sense that you have a legal doctor-patient privilege that binds what they can share with whom. There's not really an Apple cloud user privilege.
No, in the sense that your therapist is still required to report you to the police in various situations where you pose an immediate threat to yourself or others, etc.
A better analogy is a storage locker. AFAIK police need a warrant to search "your" storage locker even though it's on someone else's property. I don't see why data in the cloud should be any different. Pre-emptively scanning everyone's data is equivalent to officers rummaging through all the storage lockers in a facility "just in case" they find something illegal.
> No, in the sense that your therapist is still required to report you to the police in various situations where you pose an immediate threat to yourself or others, etc.
And therapists are legally mandated to report you if you told them you viewed or possessed CSAM.
They are mandated to report child abuse. It is the same story with doctors, if an abusive parent brings in a child for care they learn never to give the kid healthcare again after the doctor reports it. It is rooted in good intentions but the effect is it means abused children never get to see doctors, therapists, get a half-ass minimal homeschool instead of going to school, etc so that mandated reporters never enter the picture.
Reporting abuse the client was involved in has a compelling reason. That's different from hearing their client saw a picture of the abuse of a total stranger.
> No matter how or why? That seems like a terrible mandate.
Honestly shocked that anyone would even say this, but even giving you the benefit of the doubt here -- the one case where I could imagine this might not happen would be if you're a police officer investigating such cases. But they also have their own therapists dedicated/trained in police-specific issues.
I think you're demonstrating incredibly poor judgment here. CSAM is a crime with real victims. Even if your patient came across it innocently, someone is out there intentionally distributing it and that needs to be investigated.
Have you considered the implications of what you're saying?
A whistleblower goes a therapist, stressed out over their pending decision to reveal official misconduct. They've been investigating ways to post something on the internet that can't be immediately taken down by the corrupt government officials they want to expose. They express their discomfort, in confidence, to their therapist, about using something they've discovered is also used for CSAM.
You think it's a good thing for the therapist to be required to report this? Should they report that the patient admitted to viewing CSAM with no context so the whistleblower gets investigated and arrested, or should they provide the context -- that the patient is about to expose the corruption of the government receiving the report?
For that matter, consider what it does when someone is actually a pedophile. They find out that if they try to seek therapy to address their perverse attraction to kids, the therapist isn't allowed to keep their confidence and they'll be arrested, so instead of seeking professional help, they keep abusing kids. Is that the result we wanted? There is a reason doctor-patient confidentiality was a thing.
I used to have a good friend that was a stripper (I promise, I wasn't the client...). Some of her biggest customers were people that wanted therapy without the paper trail of going to a licensed therapist. This is a big thing for pilots as well, since their health records and mental care are intensely scrutinized. A stripper will provide comfort, verbal relief, and physical love for $100 hour you can tell them anything and their reputation is bad enough no one will bother to believe them if they say something bad about you.
> Across most states, viewing CSEM alone is generally not a mandated-reporting trigger; reporting becomes obligatory when disclosures involve an identifiable child being abused or used to produce material.
> California’s CANRA imposes a distinct duty to report electronic access (download/stream) with identifying patient information, upheld against privacy challenges based on compelling state interest.
Trying to avoid false positives by not firing until a threshold was hit (was it 20 images?) seemed insane from a PR position... rightly or wrongly, all it would take would be the wrong court case and you can see the headlines:
"Apple says users can have up to 20 CSAM images on their phone before they'll tell police"
The practical problem is, without a threshold, they'd have an order of magnitude more false positives than 'real' detections, making the system useless.
Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM? I think the program would have to say use but morally I'd say no. The issue with client scanning is it has to assume the worst, or they are than liable. If its the person has 20+ different baptism of nude babys well huh that actually might be CSAM because the context of how that concentrated photos implies but even than its hard what if that person actually has 20 God Children its less crazy than one thinks... Especially if they have multiply phones from a single baptism.
You might not like pictures that way but honestly I think more important in procescuting CSAM is to go after the large sources of CSAM generation. Its trafficing in East Asia, and in Europe. I think weirdly America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures. Abuse definitely happens in the US but making policy decisions like this produces bad policy.
Does iCloud rehost the photos to other people I don't really know because I use andriod tbh. If they are being rehosted (I assume to members of your contacts) that can be problematic but I think honestly the issue a lot more complex than just protect the children which the source of critic is a lot attacks against apples are coming from
> America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures
The US has the largest pornography industry in the world by a massive margin, and the largest consumption of online pornography per capita
Meanwhile should we be surprised that CSAM production is higher in countries like the Philippines that have very weak digital policing, abject poverty, high numbers of street children etc?
The US is pretty extreme, you have at the same time easy access to all the pornography you want (unless you’re in states that require age verification), and also a very prude culture. It’s not the only place with such contradictions though
The age of consent in the PI was like 14 until a couple years ago when they changed it to roughly match international norms, in their culture it was considered consensual rather than abuse until very recently.
The proposed mechanism was hash matching against known CSAM images, so the baptism photos would not trip the filter because they wouldn’t be hash matches.
But AI algorithms actually deployed by other cloud services do not. You’d hope that they wouldn’t flag a baptism or bris photo but currently fielded systems are flagging doctor-patient medical photos and have ruined lives, so.
Apple is on the side of privacy if it serves their marketing. Which is why they would rather build and normalize CLIENT SIDE CONTENT SCANNING so they can continue to market iCloud as "secure and private".
If Apple's interests sometimes align with ours then great. I'll take it. But don't attribute to this ~5 trillion dollar company some kind of altruism.
Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy.
At least on Desktop we have usable Linux, but on the phones there is literally nothing usable because thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
Yes its possible to make Andoid spy on you a little less, but even for tech savvy person its damn inconvinient and Google making platform worse with every single release.
Thanks to Google "security" I can use my banking apps on 9 years old device with 6 years outdated firmware, but not on GrapheneOS.
> thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
Sometimes, all you need is a Web browser. I personally don't want any "apps" on my phone that aren't basic utilities.
I am aware some banks in Europe require 2FA on a mobile device. Short of switching banks, my answer to that is a cheap or e-waste Googled Android phone that stays at home and serves that sole purpose.
It's sadly becoming harder. I've been playing that game for quite long and hope to stick to web apps, but still.
Some banks limit functionality on web apps, which is annoying.
More importantly, many refuse to provide a decent 2FA other than push notifications inside the app or SMS, which is insecure and EU has mandated its phaseout.
The thing that works for me is to pretend to be clueless and get a hardware token, but those are susceptible to impersonation attacks on the bank side.
> hanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
I've been using GrapheneOS for years and that hasn't been my experience. There are two financial apps that don't work for me, and that's it. Pretty much everything else I use is fine. But, to be fair, I'm very scrupulous about my apps and tend to avoid installing an app for every little thing that wants me to.
> Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy.
They probably use E2EE just so they don't have to respond to court orders and such.
> It is crazy people think apple isnt on the side of privacy.
Apple is on the side of making money, and the privacy claims are mostly marketing. The entire stack is closed source, which means it is difficult and expensive to independently verify any of the claims made. What's more, the "auto update" universal backdoor means that Apple can forcibly push a user-hostile "feature" like client-side scanning when it wants or is compelled to by a state actor.
Privacy is a natural fit for Apple in that they make money on discrete devices, but services have grown tremendously. That’s where the erosion of privacy happens.
So once there’s a profit motive for violating your privacy, the justification for eroding your privacy will proceed. It’s really the inertia of Apple starting out as privacy-compatible that makes them hesitant to throw that away.
I was an engineer at both MS and Apple. At Apple, privacy was baked into every new feature from the start. At MS, the privacy component was glued on at the very end, if ever.
Like OP said, Apple isn't perfect nor will they ever be, but they do prioritize privacy better than most.
Apple may be on the side of privacy, but since they are competing everybody out of the market with their slick consumer products they actually form a threat to privacy since now the government has to only implement a backdoor at one vendor.
People understood this settlement to mean Apple was spying on their conversations and selling them to advertisers, when it seems to have more to do with people accidentally triggering Siri. But people don’t care about this kind of nuance or actually tallying up all the ways Apple is pro privacy against rare issues like this one. It’s all just tribalism at the end of the day.
> It is crazy people think apple isnt on the side of privacy.
Look up iCloud Keychain:
For years Apple has let and helped Facebook, TikTok etc. track users even after you delete an app, even ACROSS DEVICES and DEVICE RESETS.
There's no way to even see what data the apps have stored on your account on iOS, only through the macOS Keychain Access app. Even then you can't be sure that that's all that being stored.
They temporarily changed course and wiped iCloud Keychain data when deleting apps, but only during a single beta of iOS some years ago, and then reverted to the way it is now.
It is crazy that I do not trust a multi-trillion dollar company who has forced labor in their supply chain to have my best interests in mind? It is crazy to me you would think they do not understand the concept of lip service.
These companies are liars. I do not trust liars. It has served me well.
Apple silently uploads your passwords and keeps them (lapcatsoftware.com)
170 points by ingve on Nov 1, 2024 | 127 comments
And whenever your privacy contradicts their control over "your" device, you are also out of luck, e.g., you can't have Ublock Origin on an iPhone. Relevant discussion: https://news.ycombinator.com/item?id=44804921
It's telling that these come from people trying to implement tracking and high visibility into user behavior, and complaining that Apple won't let them even though Apple conceptually could.
Except ublock, which can't do what it does the way it normally does, for the same reason you can't have any plugin inspecting realtime activity and doing scriptlet injection.
You can have ad blocking. You can't have plugins with that kind of low level access to your browser activity.
You can prefer something that allows dangerous behavior as a trade-off for greater capabilities, but you can't deny it's a safety trade-off where Apple picked what's safer.
IMO "end-to-end encryption" simply isn't possible when the application is run by the same company as the servers the data sits on, is closed source, and can at any time, see the decrypted contents of data it downloads from their servers and do whatever they want with it.
Same issue with Proton, MEGA, and any other e2ee app... it's only useful when the company decides not to mess with the data it could always decrypt locally. Also why people are hesitant to use javascript-based e2ee solutions where the site owner can modify the code at will to do what they want.
Beyond the privacy marketing angle, e2e allows companies with global exposure to sidestep any unpleasantness when they get a subpoena from Bumfuck, Nowhere.
Sure, the NSA, GCHQ and Mossad have a way to exfiltrate the unencrypted data but proprietary e2e is a good thing for most people IMO. Shifts the risk from "my messages are theoretically available to most law enforcement in the globe" to "YOU’RE STILL GONNA BE
MOSSAD’ED UPON"[0]. This is specially good for me because I know the equivalent to the FBI where is live is too cheap to buy a Cellebrite [1] license.
EDIT: I suppose someone could ask about Meta. The reason behind their support for scanning (and removing e2e in facebook msg) is simply regulatory capture. The zucc wishes to have a letter of marque to "protect" your children and remove the "unsafe" competitors.
EDIT2: Used the wrong term, I mixed up exfiltration channel with sidechannel attack.
Watching memory changing on a complex code base without having said code base is near impossible.
1. Run code
2. Watch memory changes
3. Correlate those to real data
If your code is doing anything complicated that's an intense thing to determine. If you're deep enough for a side channel there's likely a lot easier way of getting in.
Brainfart on my part. I was referring to what @majorchord was worrying about, the unencrypted messages in the client get exfiltrated and get sent to the spooks using steganography on some benign request, edited my comment.
My mental model is that most competent intelligence agencies have a PRISM 3.0 deal with FAANG, including on E2E products or at least have devs on the payroll. I imagine that any backdoor is only used on important targets, so no intel sharing with Cletus the deputy.
This is based on a faulty understanding of the underlying systems. The risk with this sort of E2E encryption is not that the service provider pinky promises not to decrypt what they have, it's that they promise they will not insert a new key into your circle of trust to subsequently start decrypting things.
I think you've imagined this faulty understanding. There are many mechanisms by which Apple could actually decrypt the data despite pinky promises not to. You listed one. There are others.
You're suggesting they purposely put a backdoor into all their custom methods? Why?
From a liability standpoint that implies a security breach could result in massive loss of customer data and if it did occur would destroy their privacy image to their customers.
I agree with the point that what you actually trust is the company to not insert maliscous code or keys into your protected path but modern systems actually contain ways to truly lock out the company itself from seeing your data.
Security wise it's amazing. If a company's admin cannot take your data it's excedingly hard for a hacker to do so.
> IMO "end-to-end encryption" simply isn't possible
In a technical sense it's absolutely possible. Owning the servers != transferring keys to the servers. Most E2E apps run both client and servers, it's about if they ever had key access.
Yes that's exactly his point. E2E is often sold as preventing the owners of the server from being able to read the messages at all, even if they are evil and misleading you.
That's obviously only the case if they aren't also the sole providers of the "ends".
There are actual methods to do this though just not sure anyone does it yet.
1. 3rd party audit of a current repo hash
2. Public hosting of hash
3. Modern attested compute can check the current startup and running code hash and return to the user for their own checks.
4. User encrypts the last known hash they used or trust a 3rd party to perform the check like azure's methods.
Another way is to open source it and repeat 2/3/4
The way around that requires either a backdoor in attested hardware which would be wild if discovered because it's the same tech protecting companies and governments most sensitive info so they're all incentivised to audit that.
There's no issue with mega. There are third party apps and as long as you don't login to mega.nz with their website you're fine. And they also have SDK you can use that they'll not be able to control/manipulate without your knowledge.
There is something ironic about US laws that attempt to prevent crime A by outlawing action B. For example:
* A: physical sexual abuse of children. B: possession or distribution of CSAM
* A: drug trafficking or tax evasion. B: structured cash withdrawals
The irony is that the more B is prevented, the less A can be detected and the less B can be used as evidence of A.
It's my understanding that conviction of CSAM-related crimes do not require any physical act to have ever occurred to any real person: one can be convicted of CSAM-related crimes related to paintings/drawings/created_art of fictional people.
It's my understanding that one can be convicted of structured withdrawals that are not driven by, linked to, or in any way related to anything nefarious.
> one can be convicted of CSAM-related crimes related to paintings/drawings/created_art of fictional people.
This isn't necessarily the case in the US, though I believe only for drawings. AI-generated CSAM probably wouldn't fly in a court of law.
Regardless, it's a naive conception of a system of law to think of it as a utilitarian system of restitution in contexts of "this individual harmed this individual". In fact, that would fall under the category of a "tort" rather than a "crime". The law is just as much about enforcing social mores and norms as it is about dealing with individuals harming each other. Hence why locales like Canada outlaw all forms CSAM, even fictional ones. The victim taken is to be society itself. The possession of this material, implicitly entailing enjoyment of it, is so gross a violation of society's norms and mores that it becomes elevated to a legal matter.
For drawings it has to additionally be "obscene" (since obscenity isn't protected by the first amendment). And there is also a specific law that criminalizes even non-obscene realistic computer generated imagery.
Not quite. There can be more restrictions on the distribution or promotion of obscene material, but mere possession of obscene material is protected by the first Amendment: https://en.wikipedia.org/wiki/Stanley_v._Georgia
The reason why the Supreme Court upheld bans on possessing CSAM is not because it's obscene, but because it incentivizes abuse of children to produce it.
I don't think these are the same. Outlawing CSAM gives law enforcement the ability to shutdown markets and prevent commercial distribution of CSAM. Sexually abusing children is heinous, but sexually abusing children for financial gain is even worse.
There are even people involved in commercial distribution of it that claim to not even be interested in children, just in profit or even allegedly “for a sense of community” (someone actually said this after getting caught, he was in his 20s but I can’t remember his name — he might have been one of the red room guys).
On top of that, while there are different types of child abusers, the worst ones almost invariantly collect CSAM to the point of hoarding. So it really isn’t that bad of a proxy.
The root comment is implying that legalizing or decriminalizing csam would somehow help with prosecution of child abuse? I’m kind of speechless. Csam IS child abuse. The fact that there are consumers encourages producers to, well, produce!
There's also the argument that CSAM can act as a gateway leading people from just being a pedophile in their head, to going out and doing something to some child.
Yes, this is an argument that exists. But it's not supported by evidence. It's the same as the old "video game violence should be outlawed because it might cause real violence", which is just as unsubstantiated.
I mean, if there were any truth to it, surely our nation would have seen an uptick, in the past 30–40 years, of new generations picking up guns and just mercilessly mowing down soft targets as if playing GTA.
Thankfully, that is all confined to fantasy in cyberspace!
Yeah, /your/ nation maybe. I wonder if there's any other reasons that could explain this, especially given the pretty uniform distribution of video games, and the extremely uneven distribution of violence.
Oh, and everywhere in your nation? At the same time?
Yes, it’s an argument but there’s zero data to support it, in fact the opposite.
If this were true then widespread availability of pornography on the internet would have resulted in a massive increase in rape of adult females. When in fact, assault numbers have been on a steady decline for decades.
Why would general pornography lead to rape? Most pornography is not rape pornography. Would people watching a bunch of rape pornography lead to more rapes? I don't know, but that seems more likely than general pornography leading to rapes.
60% of respondents who were found looking for CSAM on the dark web stated that they were fearful that consuming CSAM would lead them to do something to a child in real life: https://doi.org/10.54501/jots.v1i2.29
I'm not taking a position here, but the total amount of pornography is irrelevant to the argument. If the amount of rape pornography available has increased 500x as the total amount of pornography has increased 5000x, the proportion of pornography that was rape pornography has been reduced by 90%.
> 60% of respondents who were found looking for CSAM on the dark web stated that they were fearful that consuming CSAM would lead them to do something to a child in real life
Someone looking for child pornography is somebody looking for child pornography, so their opinion about what the search might lead to seems a bit worthless. There's no reason to assume that they have any insight into what will happen next, but we know for a fact what has already happened.
That out of the way, though, and assuming that child pornography leads to child abuse, the good pro-pornography case that has been made is that when pornography is suppressed rather than regulated, the 95% of people who are exclusively interested in adult pornography will have to go through the same underground channels that child pornography flows through, thus having the perverse effect of exposing more people to child pornography (or at least obtaining access to it.) If exposure to child pornography causes child abuse, then the suppression of adult pornography would then lead to more child abuse.
Does it need to be said that by making all pornography easier to access the internet obviously makes rape pornography more prevalent and easier to access as well?
Hence your theory should be easily visible in rape statistics, yet it’s the opposite.
It would be easily visible or not visible in the rape statistics if the world had a single variable. In any case, even if the world was extremely simple, 'rape porn' could increase rapes, and we could still observe a decrease in rapes if 'general porn' decreased the odds of rape to a larger degree than rape porn increased them relative to the population that consumed each.
Yes exactly my point...so it sounds like you're admitting this "availability of porn leading to action" theory isn't a very good one? And that in fact it leads to less action given it satisfies urges at much lower cost.
Our society is pretty aligned that distribution is another kind of harm. Non-consented distribution of sexual images (eg: revenge porn) is also a crime. Children don’t need to be the ones to press charges in child porn unlike with adults. That’s a good thing.
It surely is, but that doesn't change the fact that many cases are not about distribution and the harm is frequently more of a legal fiction, unless by harm you mean something other than suffering inflicted on the victims. I don't necessarily think it's wrong for society to ban certain acts on purely moral grounds (another example: incest between siblings), but let's not pretend it's something else going on.
> Non-consented distribution of sexual images (eg: revenge porn) is also a crime.
There is very compelling empirical evidence that this causes actual harm (suicide ideation in a very big fraction of the victims), even if it is fictional, so here there is no question about the harm.
Meh, I assume "possession" (of drugs) is how the law is worded because otherwise law enforcement would have to catch someone in the act of using or distributing, which must be much harder to do.
Mere possession of a substance is surely not what society cares about.
There have been a handful of convictions based on fictional content, but usually the defendants also possessed real CSAM so there wasn't much point in contesting the charges over fictional images.
Clicking on a page linked in your article, the PROTECT Act of 2003[1] (passed a year later), I see:
> The PROTECT Act includes prohibitions against obscene illustrations depicting child pornography, including computer-generated illustrations, also known as virtual child pornography. Previous provisions outlawing virtual child pornography... had been ruled unconstitutional... The PROTECT ACT attached an obscenity requirement under the Miller test or the variant test noted above to overcome this limitation.
Which, if I'm reading it right, means that GP was correct in saying "conviction of CSAM-related crimes do not require any physical act to have ever occurred to any real person"
No. In short, in US law, CSAM is a visual depiction of a real-world act of child sexual abuse. Visual depictions like you're describing are covered under a different law, and I'm not aware of it having a short name. There's a good expert thread on this with links to the relevant federal laws here: https://bsky.app/profile/rahaeli.bsky.social/post/3lbt7zkvlq...
(a)In General.—Any person who, in a circumstance described in subsection (d), knowingly produces, distributes, receives, or possesses with intent to distribute, a visual depiction of any kind, including a drawing, cartoon, sculpture, or painting, that—
(1)
(A)depicts a minor engaging in sexually explicit conduct; and
(B)is obscene; or ...
(b)...
(c)Nonrequired Element of Offense.—
It is not a required element of any offense under this section that the minor depicted actually exist.
It is not a required element of any offense under this section that the minor depicted actually exist.
I wonder how you'd actually go about prosecuting that. What's the line between crass and illegal?
People have infantilization fetishes - where they wear diapers and shit. You can certainly imagine someone making a cartoon of that in a sexual way.
Adults often don't look their ages. John Mulaney famously had a set about resembling a child when he was 29.
What if someone generated an image that looked like a teenager, but there was a driver's license in the frame that said the person was an adult?
Real people have ages. Imaginary people are imaginary.
What about art from the antiquity when what we would call a teenager would have been treated as an adult? Surely someone painted people having sex before "the age of consent" was a well-defined term. Is it illegal to own that painting?
> CSAM (“see-sam”) refers to any visual content—photos, videos, livestreams, or AI-generated images—that shows a child being sexually abused or exploited. Child sexual abuse material (CSAM) is not “child pornography.” It’s evidence of child sexual abuse [1]
I can't wrap my head around how AI-generated imagery is evidence of child sexual abuse (CAS). How are you abusing a real child by generating an image of a fake one?
People can also distribute heinous things through snail mail, but we are not yet at the point where the government reads all letters looking for wrongthink.
Just because we technically can make a privacy destroying drag net does not mean we should. Had phones existed 250 years ago, I have no doubt the founders would have thought it obvious that a cellphone’s contents were your personal papers which could not be freely searched.
But that's mostly because it's impractical. They do use dogs to sniff for drugs and explosives, so if CSAM smelled or was visible through X-ray then it would probably be a different story. And let's not forget snail mail is by far a more uncommon way to spread that material than the Internet is. The Internet came into broad use just ~15 years after commercial CSAM was openly being sold by mail order in Europe.
Personally, I am on the side of privacy, just to be clear.
More effort should be done to find real-world equivalents of such actions and "think of the children".
An icloud is like a storage locker or a safety deposit box... the owner should go through all your stuff there, just in case you have some CSAM!
Metadata is just tracking info about who, where and with whom... every bartender should take your IDs and log when you came to the bar, who you sat with and how long you talked there.
EU Chat control is like general eavesdropping... every time you sit down and talk with someone, an EU bureaucrat should sit next to you and listen and write down your conversations, just in case.
etc.
Somehow people think that "it's ok if it's on the internet", even when it's stuff they'd never accept in real life.
While I’m decidedly pro-encryption, I don’t like this argument. If something is the right thing, it would still be the right thing when promoted for the wrong reasons, and if it’s the wrong thing, it’s still the wrong thing even when at present nobody has ulterior motives.
When arguing against surveillance, the arguments should be on its merits, not on whether the current proponents happen to have ulterior motives.
> not on whether the current proponents happen to have ulterior motives.
Even if the current proponents have no ulterior motive, and in fact live and die having done nothing negative with such power, it does not stop the next group in power from extending and abusing power, don't base laws on temporary trust of politicians.
One problem with that is that it’s difficult to prove motives. So you’re on shaky and disputable ground. It’s much better to point out how the proposed mechanisms are prone to be misused, which is independent of current motives. Get rid of the shaky ground. Saying “these are disingenuous people proposing this” is exactly an attempt at emotional manipulation, in the sense of an ad hominem fallacy.
So invading the privacy of millions of people, even if it's just automatic scans for some specific thing is a right thing? Does this apply to mandatory drug tests for everyone everywhere? How about drug and weapon seeking drones, doing daily checks in every apartment everywhere? How about mandatory AI powered microphones everywhere that would detect threats, blackmail, any talk about anything illegal, etc.?
If you take a 1000 random people of the street now,how many of them are sharing CSAM via icloud?
If you take a 1000 random politicians, how many of them have corruption scandals? Why not start with them instead, a bodycam and an AI powered microphone that would detect corruption automatically... let them lead as an example, before they apply the laws onto "the rest of us".
You misread what I wrote. My comment is against the argument used, not against what is being argued for. Using the wrong argument diminishes one’s position. I’d prefer the stance against surveillance to not be diminished by such arguments.
Feels like this has gone completely meta and it's now way more common to see people who don't care about children refusing to support that any proposal that might benefit children under the assumption that nobody cares about children so there must be an ulterior motive.
Truly being honest, I think CSAM scanning of private comms is ineffective in the long term anyways. Pedophiles aren't stupid, you'll drag a bunch at first but the networks will be reestablished and sharing will be done via sneakernet.
The primary focus should always in preventing the creation of CSAM.
- Comprehensive Sex Ed starting young so kids can identify grooming and seek help from a trusted adult, even if abuse comes from a family member.
- Fixing schools in general so homeschooling isn't as attractive for parents. Keep a tab on home schooled children and identify social isolation.
- Bigger resources for actual honest to god on the ground investigations.
To be clear I'm not saying that homeschooling = child abuse, simply there's a lack of the mechanisms to detect it in homeschooling settings.
Banks are probably the most Orwellian surveillance apparatus of all. Almost every time I read an arrest warrant there is a whole section where banking records are used to damn someone. The equivalent level of banking privacy to graphene is roughly walking in to the teller to withdraw a few thousand cash once a month and then paying literally everything with that (or an anonymized crypto).
I wonder if the judge would be in favor of companies proactively going into people's houses at random to check on their belongings, if they don't have inappropriate photos somewhere, or whatever.
It's harder to do, but conceptually the same. So sad it's not being done. Very disturbing.
They could do it when people are not at home. There'd no problem, nobody would even notice.
The judge’s dicta about protecting children in her pro-privacy ruling upholding existing law “tips their hand” that they are somehow part of a global conspiracy to eliminate privacy?
I think your conspiracy theory needs work, to be perfectly honest with you.
“Apple created its own proprietary alternative, NeuralHash, which apparently wasn’t as good. So Apple U-turned on its efforts to scan for CSAM in its cloud storage. Instead, Apple implemented end-to-end encryption for iCloud files.”
Wasn’t Apple’s design to explicitly NOT scan in its cloud storage, but look at the file on-device at the moment you wish to upload it to iCloud? This method would make it compatible with Advanced Data Protection; so ADP could have always been in the pipeline rather than Apple u-turning. In fact, NeuralHash may have been proposed because Apple wanted to introduce ADP and saw a potential problem here/get concerns from government agencies about it and saw this as a means to an end(-to-end).
The system was designed pretty elegantly and offers far better privacy protections - including guardrails - than what Microsoft and Google do, but the communication from Apple about it was absolutely horrible and generated enormous backlash. (Not saying I agreed with implementing it, just saying the design was infinitely better than competitors.)
As the creator of mediaden.ca[1] I’ve thought about this. Client side scanning is maybe marginally better than server side scanning, but both paths lead to privacy rot.
Governments need to catch criminals, but they shouldn’t do it at everyone else’s expense.
I simply don't trust services such as iCloud. The legal landscape is too volatile, and Apple's own "terms and conditions" are also subject to constant change. As far as I can tell, most people don't need cloud backups, and iCloud mostly shows up as an annoyance designed to extract more money from customers. In fact, most people probably don't know that Apple and Google vacuum up their files the moment they are created, for their own good, of course.
Is your phone also made of indestructible materials so it can't break, and magnetically linked to your person so it can't be stolen? Backups exist so you don't lose precious photos/data if you lose the device. Most People™ do not have the technical knowledge to set up a self-owned backup system, and/or will not realize how badly their future selves will wish they had backups if the setup friction for a self-owned solution is too high to conveniently do it right this second.
I think we are closing that gap with AI coding tools.
A growing number of people people today can figure out how to run Fable in a co work session, or codex, and that can indeed set up the self-owned backup system for you along with an alternative trustable cloud backup thats not Apple or Google.
For most people "the cloud" is where you backup stuff. If you have a personal backup strategy that doesn't involve the cloud, you are not "most people".
Doing backups the right way take some skill and investment if you want to do it by yourself. It may involve setting up a NAS, and some discipline with physical media. You have to do your own security too. Most people don't want to do that, so, cloud backup it is.
It is not a perfect solution, even beyond the privacy considerations, like you can still lose your data by losing your account, but from my personal experience, people lose their data less often now that they have cloud backups.
I back up files to my own cloud with a Nextcloud integration for Android. That being said, a monthly or so backup of devices via USB/Ethernet, like we used to in the pre-cloud area, would be enough for all intents and purposes. It's not like what people have on their phones is generally very valuable.
Everybody needs cloud backups for their photos at least.
Most people don’t have computers, those who do, do not regularly backup their photos on them.
In both family and extended family many a cry would be avoided if people paid the 5 bucks it costs to backup their photos before your phone gets stolen or lost.
I know creating a throwaway to hide your name for an opinion is a bad manner, but this one is one I really don’t want linked back to me
The VAST majority of “CSAM” is consensually created and exchanged by teens. Their future selves and their parents form this pressure group attacking everyone’s liberty and privacy to try to undo the downsides of choices they made themselves with full knowledge of what could happen.
The criminal and disgusting tail end of this type of material deserves the worst of consequences for the perpetrators and all the support in the world for the victims, but these are mostly - you guessed it - poor and unprivileged children from far away places and they certainly can’t put this much pressure on apple
I think you’re right, but from another angle. In the state where I lived way back when, a state representative put forth a bill to explicitly make e-CSAM illegal. I guess it was already illegal for print media and this covered a gap in the law about cell phone pics, etc. Thing is, it had no allowance for the age of the picture taker, or even whether the picture taker was the photo subject. If a 16 year old girl took a nude selfie and sent it to her boyfriend, she was a felon.
I wrote to the rep and explained my concerns. I wholeheartedly agreed with the intent of the law, but the code was buggy. To my surprise, he wrote back in horror to say he hadn’t considered that and pulled the bill immediately. I’m proud of having done that.
I’m 100% pro yeeting child pornographers into the sun. I still don’t want to throw kids in prison or remove all traces of a right to privacy in our haste to sun-yeet them.
> I still don’t want to throw kids in prison or remove all traces of a right to privacy in our haste to sun-yeet them.
The one messy corner of this is the "strict liability" for this type of material. An underage kid can take a nude photo, send it to an adult, and then the adult can criminally liable for just having it, even if he deleted it as soon as he saw it. Either both parties involved in handing something for which there is "strict liability' need to be held accountable, or "strict liability" has to be changed so a person isn't liable if he deletes or reports the material as soon as he first becomes aware of it. And this isn't likely to happen because it would provide a plausible defense for every one criminally charged.
IANAL, but the notion of "strict liability" horrified me when I first head of it, and I thought it had to be some kind of a misunderstanding. So we're tossing that "innocent until proven guilty" idea out the window, huh?
I'm sure smarter people than me have sussed this out and can explain why it's a good thing, but it sits wrong with me. We can put the subject matter aside for a second: I don't think I could convict someone for having something happen to them, regardless of what the law says. Let's say drug possession was a strict liability law (and maybe it is for all I know). Finding a baggy of meth on the corner of a farmer's lot would mean that, technically, he was guilty of possession and had to prove that it wasn't really is. That's nuts. And looping back to the subject at hand, if the only evidence that someone possessed CSAM was their email inbox, without proof that it was solicited? They want me on their jury.
There should never be a circumstance where someone can't report something that happened to them to the police without a legitimate fear of being arrested. That's bad for the person, and it's bad for society.
But if their hard drive has folders grouped by age or something, prepare the solar catapult.
I completely agree with you, but I do think that these teens do not have good opsec around these photos. It’s like the revenge porn problem but way worse. A teenager sending a nude selfie to a friend who then later shared these images non-consensually is a much bigger problem than if the same thing happened to adults.
I don’t have any ideas for a solution, but I suspect that the heightened focus on CSAM is really compensating for the fact that we don’t have solutions for revenge porn.
Are there statistics backing up the "VAST" majority claim?
While on statistics, I wonder, are there reliable statistics about child abuse of different types? Studying correlations with other social metrics, like sex education, liberal/conservative, policies regarding prostitution, and others can provide support for/against decisions.
Not that I hope these will impact people's and governments' choices, but I want to challenge my intuitions.
Really sad. I read a lot of psychology. But I've never encountered psychology of child sexual abuse (I know there is if I look it up, but that's my point, it should be shared around, discussed, challenged).
Why would some adults find kids sexually attractive? Is it abusive/aggressive behavior manifesting itself in sexuality? Or is it sexuality channeled in the wrong direction? If it's the second, is it out of desperation, and would happen less if the culture makes it easier for them to satisfy their needs with adults, or would it happen regardless? On the victim's side, are the shy and less social ones more in danger, or the socially active ones? From my social scientist friends I hear a lot that most child sexual abuse is domestic. What are measures that a society can take to prevent these, without turning the society into a surveillance state, which will ultimately harm everyone more, including the children? What can be done to make sure children speak up, so that such behavior is dealt with at the beginning (and maybe while the more terrible things have not happened yet), and not turn into a multi-year childhood trauma?
What are signs (and early signs) on the abuser's side and the child's side? How to deal with these signs?
It's probably impossible to get reliable statistics about that given how both groups are trying to keep everything secret. But you can consider how many paedophiles there are vs how many horny teenagers there are. Based on that it would be extremely surprising if he was wrong.
The real question is what happens when a horny teenager sends another a nude. There definitely have been insane cases where they get stitched up for creating child porn. I don't know if that's the normal outcome today though.
I'd say modern AI tools could probably do this pretty effectively. They're very effective at describing anything else about an image. I have a workflow that churns through large amounts of images, describes them, and then looks for things I specifically want (in my case, auction listings that are not described accurately on the auction website).
I need to process a modest amount of imagery (about 25 million images, and growing) for NSFW content and general captioning/description. About 5% of it contains nudity or partial nudity, and about 10% of that 5% contains sexual activity.
In theory, modern vision language models could classify human nudity and sexual activity very thoroughly. But every model I have tried is reluctant to clearly describe what is notable about sexualized/nude images. The models are deliberately under-exposed to nude and sexualized content during training and further RLHF'd away from generating straightforward descriptions of such images.
Models also occasionally hallucinate WTF captions for ordinary adult sexual activity. I recently ran a baseline test with frames extracted from adult videos and about 1/3000 frames was mis-captioned as involving a child according to Gemma 4 12b.
They can't consent but it does make sense that it's true, I would genuinely bet that more nudes are being shared between 17-year olds than some freak, as this is common to the point where I feel a very large portion of all existing teens have done it.
It wouldn't be so bad tbh, would avoid all the leaks and regrets that comes with it. In term of awareness, I would say that a late teen is fully aware of his/her actions but might not calculate consequences properly.
Do you have a citation for that? Sounds plausible, but I'm not sure I've ever seen it stated that way in any of the related media reports on CSAM efforts.
In Germany, the rise in "youth porn" material has been attributed to such kinds of cases where youth send intimate pictures to each other [1].
Our legal systems are not built to deal with that mess, and it may hang around your neck for the rest of your life. Unfortunately, the law is very explicit, leaving barely any avenue for the courts to drag us out of the mess, and politicians - even if they are actually interested in the topic in the first place - won't touch that area with a ten foot pole for fear of getting blamed a pedophile themselves.
I won't pretend to be so knowledgeable about how so much CSAM is being created, but keep in mind, there are laws against distribution & mere possession, too. Revenge porn is an obvious thing to be mindful of in this context in addition to other types of distribution. Consent to create doesn't imply consent to distribute (probably even to cloud storage) & is completely immaterial to issues of possession if it ends up in some 3rd party's hands. So the scope goes way wider than you're letting on. If you're saying all of these these laws are being abused, like they exist primarily to punish a state senator's daughter's ex-boyfriend, I would ask for something to back that up.
Yes, poor and unprivileged children can't really defend themselves here, but this is the system working to find some legal mechanism to do what it can, as a more powerful force. Protecting people from exploitation is a good use of government. If this was shot down for legal reasons, OK, the system is working and I hope there is a way to expand protections that fits into our system.
It's still a shade of gray to me. If I offered some homegrown cloud storage to my friends, and one of them uploaded CSAM to it, you can bet your ass that I would be arrested for it.
Once again, someone (in this case, the judge of this case) asks if we can meet in the middle on whether or not private communications are actually private.
To be clear: this is not a limitation of nerds' imagination. This is a limitation of physics. A person is either party to a communication (and thus can decrypt it) or is not (and thus cannot). If you demand Apple scan encrypted photos for CSAM, what you are demanding is that Apple be party to every communication done with an iPhone. There is no middle ground on encryption, there will never be a middle ground on encryption, and I will hold this truth on my deathbed.
There is no "encrypted but crackable" - if the CIA can crack it at all, we're only a few years away from some kid's gaming rig doing the same thing. There is no "secure golden key" - if there was, you could buy it in the same section of Amazon that sells copies of the TSA master key that opens all luggage locks.
Personally, the next time a government demands decryption keys, I think Apple should just set all iCloud photo libraries in that country to public and say "Sorry, your politicians made private photos illegal, take it up with them". Obviously, telegraph this far in advance and give users time to actually delete their cloud-hosted photos first. But definitely do not pretend like you can keep a secret with a government bureaucracy of hundreds of thousands of people.
But then again, Apple also capitulated (good meaning) to the EU on third-party app distribution, so Apple has a lot less of a spine than they let on. At least Google actually stayed out of China.
Seems like the kids miss their chance at justice because of section 230 allowing platforms the freedom to remove whatever they want but not be responsible for what they keep or amplify. That is the problem with 230. Censorship is permitted and punishing the censor isn't. Twitter and Tiktok are literally microblog platforms that get away with removing good stuff and leaving evil because they "are not a publisher" while the algorithm literally publishes a chosen set of articles to people. Facebook can remove religious freedom material and leave human trafficking groups. Section 230 gives the publishers the cake and the edict too.
You're conflating "what's illegal" with "what a private entity doesn't want". I don't like it any more than you do, but the first is very clear, the second is a bit harder to "solve".
Maybe my perception is off, but it seems like there's a huge push by the legislature and some people to do anything and everything to prevent CSAM, yet almost nothing seems to be done to prevent CSA.
For CSAM, there's all sorts of monitoring, scanning, identify capturing, etc. But it's all after abuse has taken place, and it seems that many of the people actually arrested are arrested for CSAM and not CSA. This has even extended to fictional CSAM such as AI generated stories and pictures. As an aside, if that gets extended to political speech or other non-CSAM materials that are determined to be undesirable, that's a big concern. I can imagine that a conservative state could pass a law banning all porn because they claim it could encourage illegal activities such as prostitution, rape, or CSA.
On the CSA side, you rarely hear about arrests (they happen but less than CSAM). There doesn't seem to be any real push for educating and protecting kids before it happens. Ironically, the groups doing the most to educate and implement protective strategies are the ones who have been involved in abuse scandals in the past (Churches, Scouts, etc). Even then, a lot of it is just getting clearances, which doesnt prevent people who where not caught or were first timers. Offenders get put on a list/map. This is sort of a half approach. If they are still a threat, they shouldn't be released. Yet if you comb the list and see some of the results, they don't all seem to fit with CSA. I personally know of 1 who took a leak across from a playground at 2am walking home from the bars and was put on the sex offender registry because it was within 500'.
It seems like these laws are more about peddling to the publicist and lawmakers fantasy of incrementally extreme punishment rather than taking a appropriate, data driven, and level-headed approach that actually protects kids. Otherwise they will just keep pushing ham-fisted low-hanging "fixes" like required scanning and IDs to access the internet.
It is crazy people think apple isnt on the side of privacy. Are they perfect? Not even close, but compared to the rest of big tech theyre simply on another level.
Apple could easily not do this stuff and it may even be easier to not.
> It is crazy people think apple isnt on the side of privacy.
> It also ensured pressure from governments and plaintiffs, including CSAM victims, who preferred Apple’s more interventionist approaches, which Apple had voluntarily demonstrated it was willing to do.
I feel that Apple open pandora's box with the client-side scanning. It proved that it was technically feasible, and was "privacy preserving". I use scare quotes there because I don't think that political or religious dissidents would find that the same or similar technology used to discover and persecute them is "privacy preserving". And that's really the problem with Apple here. They provided a model for scanning for any kind of message or material while purportedly maintaining privacy.
> It proved that it was technically feasible, and was "privacy preserving".
Didn't their paper disproved by reversing the perceptual hashes to reveal blurred version of the images being hashed, and Apple basically said "that's fair, it's not as robust as we wanted, let's visit this later"?
If not, I'll happily stand corrected, but please share sources.
Addenda:
- Apple's original paper: https://web.archive.org/web/20210807165030/https://www.apple...
- Paper breaking the hash: https://arxiv.org/abs/2111.06628
Edit: The second one is the wrong paper. I’ll find and link the correct one tomorrow. Keeping the link for transparency.
The paper you linked doesn’t reveal blurred versions of the images being hashed. It does train a classifier to determine which of 1,000 ImageNet classes an image belongs to, which “achieved a top-1 test accuracy of 4.34%”.
Then, that’s the wrong paper. I’ll find it and link it as a reply to this comment. Probably tomorrow morning.
> to reveal blurred version of the images being hashed
Skimmed your linked paper. It seems they were able to classify hashes up to ~8% top-1 accuracy and ~30% top-10. Not exactly a blurred version, or any images at all.
So for example, they can say that you probably have images of trees, or images of buildings, but without much other data & very low accuracy.
I'd still be a lot more concerned about them simply flagging political images rather than trying to get a broad understanding of what type of photos I have
It starts at a broad understanding and ends with people permanently giving up their right and ability to keep rogue corporate governments in check.
> I feel that Apple open pandora's box with the client-side scanning.
That box has been open for years now.
Big brother is already watching what you do on your Android device.
> A Dad Took Photos of His Naked Toddler for the Doctor. Google Flagged Him as a Criminal.
https://www.nytimes.com/2022/08/21/technology/google-surveil...
This is not client side. Images just sitting on your Android phone are probably safe (although Google could push an update at any time). Your images get scanned when you back them up, send them over RCS, etc. I have even seen criminal cases originating from reverse image search - anything that touche the servers of the big tech companies, except apparently Apple, will be scanned using questionable AI and against a secret list to Protect the Children.
This is an image that he did not send off of his device to anyone except his doctor's office, yet Google reached into his private data and scanned it anyway.
Google reported him to the police based on a single false positive.
To add insult to injury, even after the police contacted Google to tell them that they had cleared him of wrongdoing, Google refused to restore access to his account.
Stop making stuff up man, the image was uploaded to Google Photos servers.
> The father uploaded photos of his son’s genitals, which were also backed up on his Google cloud, to the health care provider’s messaging system as requested.
No, he didn’t affirmatively upload it anywhere, it was automatically synced and scanned when he was following his provider’s instructions to upload HIPAA-protected medical photos through using a device that wasn’t designed for that use case and without any reasonable awareness of the technology and risks involved.
I’m sure there are disclaimers buried in privacy policies and terms of service that nobody reads but let’s not pretend a random user is providing well-informed consent.
Did he set up his device to upload his private data to Google, or did Google create an OS that automatically sent everyone's private data to their AI server for scanning without explicit consent?
Google is not Apple. Apple customers expect the higher standard.
I thought the client side scanning was to protect children? If it suspects an image is bad, it blurs it and pops up a warning including a link to resources to go to for help.
Very different than trying to narc out users to the authorities.
There were two different technologies. One was client-side scanning for known CSAM, which created a huge backlash and is described here: https://educatedguesswork.org/posts/apple-csam-intro/
The other is detection of images that may contain nudity, whether sent or received, when the owner/admin/parent enables the feature. It is relatively uncontroversial and is described here: https://support.apple.com/en-us/105069
That is what they actually deployed. They were planning on performing client-side scanning of all images uploaded to iCloud for CSAM and reporting it to the authorities, but backpedaled after public push-back.
The original proposal was to use a visual hash designed to identify known bad CSAM images even if cropped or otherwise edited. Your computer would scan all your stuff for these images and report you to apple who would report you to the cops. This presented a number of issues.
Accidental false positives could lead to horrific outcomes up to and including oh look bob got shot by the cops for resisting.
It was possible to produce apparently matching innocuous images and then poison people's machines with them.Oops did you click on that picture of a tree have fun with the cops. Like an advanced form of swatting.
Although inspired by a desire to find CSAM Apple could be forced to scan for ANYTHING by repressive regimes including America and China.
Although initially targeting images client side scanning of messages is a pretty obvious next step. Again obvious good motivation exists and is completely justifiable who doesn't want to stop the next mass shooting or terrorist attack... and then we can basically use it to find people critical of the regime. Do remember we are presently prosecuting a political figure for a picture of sea shells and a guy in texas is rotting in prison for distributing political literature.
Is it different than telling your therapist something in confidence and then finding police waiting for you in the lobby.
Yes, in the sense that you have a legal doctor-patient privilege that binds what they can share with whom. There's not really an Apple cloud user privilege.
No, in the sense that your therapist is still required to report you to the police in various situations where you pose an immediate threat to yourself or others, etc.
A better analogy is a storage locker. AFAIK police need a warrant to search "your" storage locker even though it's on someone else's property. I don't see why data in the cloud should be any different. Pre-emptively scanning everyone's data is equivalent to officers rummaging through all the storage lockers in a facility "just in case" they find something illegal.
It’s a bit more like requiring you to submit to a weapons pat down before you go to your locker I think.
> No, in the sense that your therapist is still required to report you to the police in various situations where you pose an immediate threat to yourself or others, etc.
And therapists are legally mandated to report you if you told them you viewed or possessed CSAM.
No matter how or why? That seems like a terrible mandate.
They are mandated to report child abuse. It is the same story with doctors, if an abusive parent brings in a child for care they learn never to give the kid healthcare again after the doctor reports it. It is rooted in good intentions but the effect is it means abused children never get to see doctors, therapists, get a half-ass minimal homeschool instead of going to school, etc so that mandated reporters never enter the picture.
Reporting abuse the client was involved in has a compelling reason. That's different from hearing their client saw a picture of the abuse of a total stranger.
> No matter how or why? That seems like a terrible mandate.
Honestly shocked that anyone would even say this, but even giving you the benefit of the doubt here -- the one case where I could imagine this might not happen would be if you're a police officer investigating such cases. But they also have their own therapists dedicated/trained in police-specific issues.
Even if someone went browsing for it, yes that's illegal but there's no benefit in their therapist reporting them for just visiting terrible websites.
But also there are definitely ways to get accidentally exposed. That's an absolutely awful thing to call the cops over.
I think you're demonstrating incredibly poor judgment here. CSAM is a crime with real victims. Even if your patient came across it innocently, someone is out there intentionally distributing it and that needs to be investigated.
Have you considered the implications of what you're saying?
A whistleblower goes a therapist, stressed out over their pending decision to reveal official misconduct. They've been investigating ways to post something on the internet that can't be immediately taken down by the corrupt government officials they want to expose. They express their discomfort, in confidence, to their therapist, about using something they've discovered is also used for CSAM.
You think it's a good thing for the therapist to be required to report this? Should they report that the patient admitted to viewing CSAM with no context so the whistleblower gets investigated and arrested, or should they provide the context -- that the patient is about to expose the corruption of the government receiving the report?
For that matter, consider what it does when someone is actually a pedophile. They find out that if they try to seek therapy to address their perverse attraction to kids, the therapist isn't allowed to keep their confidence and they'll be arrested, so instead of seeking professional help, they keep abusing kids. Is that the result we wanted? There is a reason doctor-patient confidentiality was a thing.
I used to have a good friend that was a stripper (I promise, I wasn't the client...). Some of her biggest customers were people that wanted therapy without the paper trail of going to a licensed therapist. This is a big thing for pilots as well, since their health records and mental care are intensely scrutinized. A stripper will provide comfort, verbal relief, and physical love for $100 hour you can tell them anything and their reputation is bad enough no one will bother to believe them if they say something bad about you.
Makes you wonder if there's a strip-joint where they're all licensed therapists, but non-practicing.
Well, in the jurisdictions that I care about the courts and lawmakers have already decided this and the legal requirement is to report.
If its your license to practice on the line you know what choice you're going to make.
That's not generally true. From [0]
> Across most states, viewing CSEM alone is generally not a mandated-reporting trigger; reporting becomes obligatory when disclosures involve an identifiable child being abused or used to produce material.
> California’s CANRA imposes a distinct duty to report electronic access (download/stream) with identifying patient information, upheld against privacy challenges based on compelling state interest.
[0] https://www.psychiatrictimes.com/view/mandatory-reporting-ch...
I still also totally don't get their policy.
Trying to avoid false positives by not firing until a threshold was hit (was it 20 images?) seemed insane from a PR position... rightly or wrongly, all it would take would be the wrong court case and you can see the headlines:
"Apple says users can have up to 20 CSAM images on their phone before they'll tell police"
The practical problem is, without a threshold, they'd have an order of magnitude more false positives than 'real' detections, making the system useless.
Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM? I think the program would have to say use but morally I'd say no. The issue with client scanning is it has to assume the worst, or they are than liable. If its the person has 20+ different baptism of nude babys well huh that actually might be CSAM because the context of how that concentrated photos implies but even than its hard what if that person actually has 20 God Children its less crazy than one thinks... Especially if they have multiply phones from a single baptism.
You might not like pictures that way but honestly I think more important in procescuting CSAM is to go after the large sources of CSAM generation. Its trafficing in East Asia, and in Europe. I think weirdly America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures. Abuse definitely happens in the US but making policy decisions like this produces bad policy.
Does iCloud rehost the photos to other people I don't really know because I use andriod tbh. If they are being rehosted (I assume to members of your contacts) that can be problematic but I think honestly the issue a lot more complex than just protect the children which the source of critic is a lot attacks against apples are coming from
> America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures
The US has the largest pornography industry in the world by a massive margin, and the largest consumption of online pornography per capita
Meanwhile should we be surprised that CSAM production is higher in countries like the Philippines that have very weak digital policing, abject poverty, high numbers of street children etc?
The US is pretty extreme, you have at the same time easy access to all the pornography you want (unless you’re in states that require age verification), and also a very prude culture. It’s not the only place with such contradictions though
The age of consent in the PI was like 14 until a couple years ago when they changed it to roughly match international norms, in their culture it was considered consensual rather than abuse until very recently.
The proposed mechanism was hash matching against known CSAM images, so the baptism photos would not trip the filter because they wouldn’t be hash matches.
But AI algorithms actually deployed by other cloud services do not. You’d hope that they wouldn’t flag a baptism or bris photo but currently fielded systems are flagging doctor-patient medical photos and have ruined lives, so.
> Imagine you have pictures of someones baptism and the kid was nude. Is it CSAM?
In some countries it is as far as I’m aware
Apple is on the side of privacy if it serves their marketing. Which is why they would rather build and normalize CLIENT SIDE CONTENT SCANNING so they can continue to market iCloud as "secure and private".
If Apple's interests sometimes align with ours then great. I'll take it. But don't attribute to this ~5 trillion dollar company some kind of altruism.
Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy.
At least on Desktop we have usable Linux, but on the phones there is literally nothing usable because thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
Yes its possible to make Andoid spy on you a little less, but even for tech savvy person its damn inconvinient and Google making platform worse with every single release.
Thanks to Google "security" I can use my banking apps on 9 years old device with 6 years outdated firmware, but not on GrapheneOS.
> thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
Sometimes, all you need is a Web browser. I personally don't want any "apps" on my phone that aren't basic utilities.
I am aware some banks in Europe require 2FA on a mobile device. Short of switching banks, my answer to that is a cheap or e-waste Googled Android phone that stays at home and serves that sole purpose.
A lot of banks in UK and EU simply dont offer web access at all.
But you can switch to one that does.
It's sadly becoming harder. I've been playing that game for quite long and hope to stick to web apps, but still.
Some banks limit functionality on web apps, which is annoying.
More importantly, many refuse to provide a decent 2FA other than push notifications inside the app or SMS, which is insecure and EU has mandated its phaseout.
The thing that works for me is to pretend to be clueless and get a hardware token, but those are susceptible to impersonation attacks on the bank side.
Yes I can, but then I wont be able to use some of very convinient fintech services.
I also need to maintain my own nextcloud, photo sync infrastructure and backups.
Its inconvinient. This is exactly what I talking about.
> hanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
I've been using GrapheneOS for years and that hasn't been my experience. There are two financial apps that don't work for me, and that's it. Pretty much everything else I use is fine. But, to be fair, I'm very scrupulous about my apps and tend to avoid installing an app for every little thing that wants me to.
> Apple is very much like WhatsApp. Yes you cant perfectly trust their E2EE against state actors, but both in fact put some effort into making world have little bit more privacy.
They probably use E2EE just so they don't have to respond to court orders and such.
Like when Google got tired of handling geofencing warrants.
> It is crazy people think apple isnt on the side of privacy.
Apple is on the side of making money, and the privacy claims are mostly marketing. The entire stack is closed source, which means it is difficult and expensive to independently verify any of the claims made. What's more, the "auto update" universal backdoor means that Apple can forcibly push a user-hostile "feature" like client-side scanning when it wants or is compelled to by a state actor.
Privacy is a natural fit for Apple in that they make money on discrete devices, but services have grown tremendously. That’s where the erosion of privacy happens.
So once there’s a profit motive for violating your privacy, the justification for eroding your privacy will proceed. It’s really the inertia of Apple starting out as privacy-compatible that makes them hesitant to throw that away.
Indeed.
But they are until they are actually defeated. I would rather plan for failure. We are in a global climate where court rulings can be ignored.
I was an engineer at both MS and Apple. At Apple, privacy was baked into every new feature from the start. At MS, the privacy component was glued on at the very end, if ever.
Like OP said, Apple isn't perfect nor will they ever be, but they do prioritize privacy better than most.
Apple may be on the side of privacy, but since they are competing everybody out of the market with their slick consumer products they actually form a threat to privacy since now the government has to only implement a backdoor at one vendor.
I have a suspicion that Google and Android aren't going to just vanish.
I said this in another thread a while ago, and one of these people who thinks Apple isn’t on the side of privacy cited a lawsuit they settled around Siri listened to conversations: https://www.scientificamerican.com/article/apple-settles-cla...
People understood this settlement to mean Apple was spying on their conversations and selling them to advertisers, when it seems to have more to do with people accidentally triggering Siri. But people don’t care about this kind of nuance or actually tallying up all the ways Apple is pro privacy against rare issues like this one. It’s all just tribalism at the end of the day.
> It is crazy people think apple isnt on the side of privacy.
Look up iCloud Keychain:
For years Apple has let and helped Facebook, TikTok etc. track users even after you delete an app, even ACROSS DEVICES and DEVICE RESETS.
There's no way to even see what data the apps have stored on your account on iOS, only through the macOS Keychain Access app. Even then you can't be sure that that's all that being stored.
They temporarily changed course and wiped iCloud Keychain data when deleting apps, but only during a single beta of iOS some years ago, and then reverted to the way it is now.
It is crazy that I do not trust a multi-trillion dollar company who has forced labor in their supply chain to have my best interests in mind? It is crazy to me you would think they do not understand the concept of lip service.
These companies are liars. I do not trust liars. It has served me well.
Apple is on the side of privacy, except when you want privacy from Apple:
Watchdog ponders why Apple doesn't apply its strict app tracking rules to itself (theregister.com)
161 points by Logans_Run on Feb 14, 2025 | 69 comments
https://news.ycombinator.com/item?id=43047952
Apple silently uploads your passwords and keeps them (lapcatsoftware.com)
170 points by ingve on Nov 1, 2024 | 127 comments
And whenever your privacy contradicts their control over "your" device, you are also out of luck, e.g., you can't have Ublock Origin on an iPhone. Relevant discussion: https://news.ycombinator.com/item?id=44804921
It's telling that these come from people trying to implement tracking and high visibility into user behavior, and complaining that Apple won't let them even though Apple conceptually could.
Except ublock, which can't do what it does the way it normally does, for the same reason you can't have any plugin inspecting realtime activity and doing scriptlet injection.
You can have ad blocking. You can't have plugins with that kind of low level access to your browser activity.
You can prefer something that allows dangerous behavior as a trade-off for greater capabilities, but you can't deny it's a safety trade-off where Apple picked what's safer.
Yep, and if you want good Adblock on iPhone use Wipr.
IMO "end-to-end encryption" simply isn't possible when the application is run by the same company as the servers the data sits on, is closed source, and can at any time, see the decrypted contents of data it downloads from their servers and do whatever they want with it.
Same issue with Proton, MEGA, and any other e2ee app... it's only useful when the company decides not to mess with the data it could always decrypt locally. Also why people are hesitant to use javascript-based e2ee solutions where the site owner can modify the code at will to do what they want.
Beyond the privacy marketing angle, e2e allows companies with global exposure to sidestep any unpleasantness when they get a subpoena from Bumfuck, Nowhere.
Sure, the NSA, GCHQ and Mossad have a way to exfiltrate the unencrypted data but proprietary e2e is a good thing for most people IMO. Shifts the risk from "my messages are theoretically available to most law enforcement in the globe" to "YOU’RE STILL GONNA BE MOSSAD’ED UPON"[0]. This is specially good for me because I know the equivalent to the FBI where is live is too cheap to buy a Cellebrite [1] license.
[0] https://www.usenix.org/system/files/1401_08-12_mickens.pdf [1] https://arstechnica.com/gadgets/2025/10/leaker-reveals-which...
EDIT: I suppose someone could ask about Meta. The reason behind their support for scanning (and removing e2e in facebook msg) is simply regulatory capture. The zucc wishes to have a letter of marque to "protect" your children and remove the "unsafe" competitors.
EDIT2: Used the wrong term, I mixed up exfiltration channel with sidechannel attack.
Side channel is academic at best.
Watching memory changing on a complex code base without having said code base is near impossible.
1. Run code 2. Watch memory changes 3. Correlate those to real data
If your code is doing anything complicated that's an intense thing to determine. If you're deep enough for a side channel there's likely a lot easier way of getting in.
Brainfart on my part. I was referring to what @majorchord was worrying about, the unencrypted messages in the client get exfiltrated and get sent to the spooks using steganography on some benign request, edited my comment.
My mental model is that most competent intelligence agencies have a PRISM 3.0 deal with FAANG, including on E2E products or at least have devs on the payroll. I imagine that any backdoor is only used on important targets, so no intel sharing with Cletus the deputy.
This is based on a faulty understanding of the underlying systems. The risk with this sort of E2E encryption is not that the service provider pinky promises not to decrypt what they have, it's that they promise they will not insert a new key into your circle of trust to subsequently start decrypting things.
I think you've imagined this faulty understanding. There are many mechanisms by which Apple could actually decrypt the data despite pinky promises not to. You listed one. There are others.
You're suggesting they purposely put a backdoor into all their custom methods? Why?
From a liability standpoint that implies a security breach could result in massive loss of customer data and if it did occur would destroy their privacy image to their customers.
I agree with the point that what you actually trust is the company to not insert maliscous code or keys into your protected path but modern systems actually contain ways to truly lock out the company itself from seeing your data.
Security wise it's amazing. If a company's admin cannot take your data it's excedingly hard for a hacker to do so.
> IMO "end-to-end encryption" simply isn't possible
In a technical sense it's absolutely possible. Owning the servers != transferring keys to the servers. Most E2E apps run both client and servers, it's about if they ever had key access.
Only if the company misleads and adds a backdoor to the front-end app (thus this entire discussion).
If the company is misleading, any encryption technology is irrelevant anyway.
Yes that's exactly his point. E2E is often sold as preventing the owners of the server from being able to read the messages at all, even if they are evil and misleading you.
That's obviously only the case if they aren't also the sole providers of the "ends".
There are actual methods to do this though just not sure anyone does it yet.
1. 3rd party audit of a current repo hash 2. Public hosting of hash 3. Modern attested compute can check the current startup and running code hash and return to the user for their own checks. 4. User encrypts the last known hash they used or trust a 3rd party to perform the check like azure's methods.
Another way is to open source it and repeat 2/3/4
The way around that requires either a backdoor in attested hardware which would be wild if discovered because it's the same tech protecting companies and governments most sensitive info so they're all incentivised to audit that.
I seem to recall that Apple provided an audit
There's no issue with mega. There are third party apps and as long as you don't login to mega.nz with their website you're fine. And they also have SDK you can use that they'll not be able to control/manipulate without your knowledge.
I am not a lawyer.
There is something ironic about US laws that attempt to prevent crime A by outlawing action B. For example:
The irony is that the more B is prevented, the less A can be detected and the less B can be used as evidence of A.It's my understanding that conviction of CSAM-related crimes do not require any physical act to have ever occurred to any real person: one can be convicted of CSAM-related crimes related to paintings/drawings/created_art of fictional people.
It's my understanding that one can be convicted of structured withdrawals that are not driven by, linked to, or in any way related to anything nefarious.
> one can be convicted of CSAM-related crimes related to paintings/drawings/created_art of fictional people.
This isn't necessarily the case in the US, though I believe only for drawings. AI-generated CSAM probably wouldn't fly in a court of law.
Regardless, it's a naive conception of a system of law to think of it as a utilitarian system of restitution in contexts of "this individual harmed this individual". In fact, that would fall under the category of a "tort" rather than a "crime". The law is just as much about enforcing social mores and norms as it is about dealing with individuals harming each other. Hence why locales like Canada outlaw all forms CSAM, even fictional ones. The victim taken is to be society itself. The possession of this material, implicitly entailing enjoyment of it, is so gross a violation of society's norms and mores that it becomes elevated to a legal matter.
> The law is just as much about enforcing social mores and norms
This shouldn't be the case in a society that supposedly values liberty.
For drawings it has to additionally be "obscene" (since obscenity isn't protected by the first amendment). And there is also a specific law that criminalizes even non-obscene realistic computer generated imagery.
Not quite. There can be more restrictions on the distribution or promotion of obscene material, but mere possession of obscene material is protected by the first Amendment: https://en.wikipedia.org/wiki/Stanley_v._Georgia
The reason why the Supreme Court upheld bans on possessing CSAM is not because it's obscene, but because it incentivizes abuse of children to produce it.
Stanley v. Georgia considered the question from the right to privacy side, not the first amendment. The relevant cases are https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit... and United States v. Williams https://en.wikipedia.org/wiki/PROTECT_Act_of_2003#Supreme_Co...
I don't think these are the same. Outlawing CSAM gives law enforcement the ability to shutdown markets and prevent commercial distribution of CSAM. Sexually abusing children is heinous, but sexually abusing children for financial gain is even worse.
There are even people involved in commercial distribution of it that claim to not even be interested in children, just in profit or even allegedly “for a sense of community” (someone actually said this after getting caught, he was in his 20s but I can’t remember his name — he might have been one of the red room guys).
On top of that, while there are different types of child abusers, the worst ones almost invariantly collect CSAM to the point of hoarding. So it really isn’t that bad of a proxy.
The root comment is implying that legalizing or decriminalizing csam would somehow help with prosecution of child abuse? I’m kind of speechless. Csam IS child abuse. The fact that there are consumers encourages producers to, well, produce!
There's also the argument that CSAM can act as a gateway leading people from just being a pedophile in their head, to going out and doing something to some child.
Yes, this is an argument that exists. But it's not supported by evidence. It's the same as the old "video game violence should be outlawed because it might cause real violence", which is just as unsubstantiated.
Yeah, that old canard is ridiculous!
I mean, if there were any truth to it, surely our nation would have seen an uptick, in the past 30–40 years, of new generations picking up guns and just mercilessly mowing down soft targets as if playing GTA.
Thankfully, that is all confined to fantasy in cyberspace!
Yeah, /your/ nation maybe. I wonder if there's any other reasons that could explain this, especially given the pretty uniform distribution of video games, and the extremely uneven distribution of violence.
Oh, and everywhere in your nation? At the same time?
Sorry, I think I have a button.
Yes, it’s an argument but there’s zero data to support it, in fact the opposite.
If this were true then widespread availability of pornography on the internet would have resulted in a massive increase in rape of adult females. When in fact, assault numbers have been on a steady decline for decades.
Why would general pornography lead to rape? Most pornography is not rape pornography. Would people watching a bunch of rape pornography lead to more rapes? I don't know, but that seems more likely than general pornography leading to rapes.
60% of respondents who were found looking for CSAM on the dark web stated that they were fearful that consuming CSAM would lead them to do something to a child in real life: https://doi.org/10.54501/jots.v1i2.29
> Most pornography is not rape pornography.
I'm not taking a position here, but the total amount of pornography is irrelevant to the argument. If the amount of rape pornography available has increased 500x as the total amount of pornography has increased 5000x, the proportion of pornography that was rape pornography has been reduced by 90%.
> 60% of respondents who were found looking for CSAM on the dark web stated that they were fearful that consuming CSAM would lead them to do something to a child in real life
Someone looking for child pornography is somebody looking for child pornography, so their opinion about what the search might lead to seems a bit worthless. There's no reason to assume that they have any insight into what will happen next, but we know for a fact what has already happened.
That out of the way, though, and assuming that child pornography leads to child abuse, the good pro-pornography case that has been made is that when pornography is suppressed rather than regulated, the 95% of people who are exclusively interested in adult pornography will have to go through the same underground channels that child pornography flows through, thus having the perverse effect of exposing more people to child pornography (or at least obtaining access to it.) If exposure to child pornography causes child abuse, then the suppression of adult pornography would then lead to more child abuse.
Does it need to be said that by making all pornography easier to access the internet obviously makes rape pornography more prevalent and easier to access as well?
Hence your theory should be easily visible in rape statistics, yet it’s the opposite.
It would be easily visible or not visible in the rape statistics if the world had a single variable. In any case, even if the world was extremely simple, 'rape porn' could increase rapes, and we could still observe a decrease in rapes if 'general porn' decreased the odds of rape to a larger degree than rape porn increased them relative to the population that consumed each.
Yes exactly my point...so it sounds like you're admitting this "availability of porn leading to action" theory isn't a very good one? And that in fact it leads to less action given it satisfies urges at much lower cost.
Our society is pretty aligned that distribution is another kind of harm. Non-consented distribution of sexual images (eg: revenge porn) is also a crime. Children don’t need to be the ones to press charges in child porn unlike with adults. That’s a good thing.
It surely is, but that doesn't change the fact that many cases are not about distribution and the harm is frequently more of a legal fiction, unless by harm you mean something other than suffering inflicted on the victims. I don't necessarily think it's wrong for society to ban certain acts on purely moral grounds (another example: incest between siblings), but let's not pretend it's something else going on.
> Non-consented distribution of sexual images (eg: revenge porn) is also a crime.
There is very compelling empirical evidence that this causes actual harm (suicide ideation in a very big fraction of the victims), even if it is fictional, so here there is no question about the harm.
> Our society is pretty aligned that distribution is another kind of harm.
As well as possession. I don't actually know if those are different for CSAM, but I would assume so because they are for drugs.
Meh, I assume "possession" (of drugs) is how the law is worded because otherwise law enforcement would have to catch someone in the act of using or distributing, which must be much harder to do.
Mere possession of a substance is surely not what society cares about.
At least in the US, fictional content is legal even if it depicts minors sexually: https://en.wikipedia.org/wiki/Ashcroft_v._Free_Speech_Coalit...
There have been a handful of convictions based on fictional content, but usually the defendants also possessed real CSAM so there wasn't much point in contesting the charges over fictional images.
Clicking on a page linked in your article, the PROTECT Act of 2003[1] (passed a year later), I see:
> The PROTECT Act includes prohibitions against obscene illustrations depicting child pornography, including computer-generated illustrations, also known as virtual child pornography. Previous provisions outlawing virtual child pornography... had been ruled unconstitutional... The PROTECT ACT attached an obscenity requirement under the Miller test or the variant test noted above to overcome this limitation.
Which, if I'm reading it right, means that GP was correct in saying "conviction of CSAM-related crimes do not require any physical act to have ever occurred to any real person"
[1] https://en.wikipedia.org/wiki/PROTECT_Act_of_2003
No. In short, in US law, CSAM is a visual depiction of a real-world act of child sexual abuse. Visual depictions like you're describing are covered under a different law, and I'm not aware of it having a short name. There's a good expert thread on this with links to the relevant federal laws here: https://bsky.app/profile/rahaeli.bsky.social/post/3lbt7zkvlq...
Currently it is explicitly against the law[0]:
It is not a required element of any offense under this section that the minor depicted actually exist.[0]https://www.law.cornell.edu/uscode/text/18/1466A
I wonder how you'd actually go about prosecuting that. What's the line between crass and illegal?
People have infantilization fetishes - where they wear diapers and shit. You can certainly imagine someone making a cartoon of that in a sexual way.
Adults often don't look their ages. John Mulaney famously had a set about resembling a child when he was 29.
What if someone generated an image that looked like a teenager, but there was a driver's license in the frame that said the person was an adult?
Real people have ages. Imaginary people are imaginary.
What about art from the antiquity when what we would call a teenager would have been treated as an adult? Surely someone painted people having sex before "the age of consent" was a well-defined term. Is it illegal to own that painting?
a 17 year old can take a nude selfie and be charged as an adult in possession of CSAM
> CSAM (“see-sam”) refers to any visual content—photos, videos, livestreams, or AI-generated images—that shows a child being sexually abused or exploited. Child sexual abuse material (CSAM) is not “child pornography.” It’s evidence of child sexual abuse [1]
I can't wrap my head around how AI-generated imagery is evidence of child sexual abuse (CAS). How are you abusing a real child by generating an image of a fake one?
[1] https://rainn.org/get-the-facts-about-csam-child-sexual-abus...
A: cyber crimes or other digital crimes. probably applies to many of the other crimes you mentioned too. B: privacy
The judge called the outcome disturbing, as it leaves victimized children as "collateral damage" of privacy protections.
As sad as this is, end to end encryption means no CSAM scanning.
As an alternative Apple previously tried to do scanning on the phones locally but caught hell for that too.
This is one of those unfortunate tradeoffs but I see no alternative to privacy taking priority.
People can also distribute heinous things through snail mail, but we are not yet at the point where the government reads all letters looking for wrongthink.
Just because we technically can make a privacy destroying drag net does not mean we should. Had phones existed 250 years ago, I have no doubt the founders would have thought it obvious that a cellphone’s contents were your personal papers which could not be freely searched.
But that's mostly because it's impractical. They do use dogs to sniff for drugs and explosives, so if CSAM smelled or was visible through X-ray then it would probably be a different story. And let's not forget snail mail is by far a more uncommon way to spread that material than the Internet is. The Internet came into broad use just ~15 years after commercial CSAM was openly being sold by mail order in Europe.
Personally, I am on the side of privacy, just to be clear.
If East Germany can, why can't we? /s
Police can absolutely open snail mail with an appropriate warrant when investigating traffickers.
With a signed warrant being the key differentiator vs invading privacy by default.
More effort should be done to find real-world equivalents of such actions and "think of the children".
An icloud is like a storage locker or a safety deposit box... the owner should go through all your stuff there, just in case you have some CSAM!
Metadata is just tracking info about who, where and with whom... every bartender should take your IDs and log when you came to the bar, who you sat with and how long you talked there.
EU Chat control is like general eavesdropping... every time you sit down and talk with someone, an EU bureaucrat should sit next to you and listen and write down your conversations, just in case.
etc.
Somehow people think that "it's ok if it's on the internet", even when it's stuff they'd never accept in real life.
Children are often used as a weapon to erode freedoms, like privacy and speech. Those pushing it rarely actually care about the children.
While I’m decidedly pro-encryption, I don’t like this argument. If something is the right thing, it would still be the right thing when promoted for the wrong reasons, and if it’s the wrong thing, it’s still the wrong thing even when at present nobody has ulterior motives.
When arguing against surveillance, the arguments should be on its merits, not on whether the current proponents happen to have ulterior motives.
> not on whether the current proponents happen to have ulterior motives.
Even if the current proponents have no ulterior motive, and in fact live and die having done nothing negative with such power, it does not stop the next group in power from extending and abusing power, don't base laws on temporary trust of politicians.
Calling out the ulterior motives can help clear the deck to focus on what is right or wrong, without as much emotional manipulation in the picture.
One problem with that is that it’s difficult to prove motives. So you’re on shaky and disputable ground. It’s much better to point out how the proposed mechanisms are prone to be misused, which is independent of current motives. Get rid of the shaky ground. Saying “these are disingenuous people proposing this” is exactly an attempt at emotional manipulation, in the sense of an ad hominem fallacy.
So invading the privacy of millions of people, even if it's just automatic scans for some specific thing is a right thing? Does this apply to mandatory drug tests for everyone everywhere? How about drug and weapon seeking drones, doing daily checks in every apartment everywhere? How about mandatory AI powered microphones everywhere that would detect threats, blackmail, any talk about anything illegal, etc.?
If you take a 1000 random people of the street now,how many of them are sharing CSAM via icloud?
If you take a 1000 random politicians, how many of them have corruption scandals? Why not start with them instead, a bodycam and an AI powered microphone that would detect corruption automatically... let them lead as an example, before they apply the laws onto "the rest of us".
You misread what I wrote. My comment is against the argument used, not against what is being argued for. Using the wrong argument diminishes one’s position. I’d prefer the stance against surveillance to not be diminished by such arguments.
Feels like this has gone completely meta and it's now way more common to see people who don't care about children refusing to support that any proposal that might benefit children under the assumption that nobody cares about children so there must be an ulterior motive.
Truly being honest, I think CSAM scanning of private comms is ineffective in the long term anyways. Pedophiles aren't stupid, you'll drag a bunch at first but the networks will be reestablished and sharing will be done via sneakernet.
The primary focus should always in preventing the creation of CSAM.
- Comprehensive Sex Ed starting young so kids can identify grooming and seek help from a trusted adult, even if abuse comes from a family member.
- Fixing schools in general so homeschooling isn't as attractive for parents. Keep a tab on home schooled children and identify social isolation.
- Bigger resources for actual honest to god on the ground investigations.
To be clear I'm not saying that homeschooling = child abuse, simply there's a lack of the mechanisms to detect it in homeschooling settings.
> end to end encryption means no CSAM scanning
Not true. There is the option of scanning on the device.
Owning your device (instead of the manufacturer, a set of unlisted governments, big software corporations, etc.) means no scanning.
It also means no banking app will work
Banks are probably the most Orwellian surveillance apparatus of all. Almost every time I read an arrest warrant there is a whole section where banking records are used to damn someone. The equivalent level of banking privacy to graphene is roughly walking in to the teller to withdraw a few thousand cash once a month and then paying literally everything with that (or an anonymized crypto).
I don't think you understand banking
https://www.ncontracts.com/nsight-blog/risk-management-strat...
I wonder if the judge would be in favor of companies proactively going into people's houses at random to check on their belongings, if they don't have inappropriate photos somewhere, or whatever.
It's harder to do, but conceptually the same. So sad it's not being done. Very disturbing.
They could do it when people are not at home. There'd no problem, nobody would even notice.
Kind of a bad analogy (not service related, no associated liability).
A better one: what about rental property, like a business? Can the landlord randomly check for criminal behavior?
> Can the landlord randomly check for criminal behavior?
Absolutely not.
-
The judge’s dicta about protecting children in her pro-privacy ruling upholding existing law “tips their hand” that they are somehow part of a global conspiracy to eliminate privacy?
I think your conspiracy theory needs work, to be perfectly honest with you.
> As sad as this is, end to end encryption means no CSAM scanning.
I think it depends on your definition of e2ee and where the "end"s are.
If the locally running application can decrypt the data, it could always do whatever it wanted. Is that really how you define e2ee?
The locally running application is one of the 'ends' of the end to end encryption.
Then in that case I think the previous statement of "end to end encryption means no CSAM scanning" would be false.
A win for privacy and freedom
Indeed. And a rare one at that.
“Apple created its own proprietary alternative, NeuralHash, which apparently wasn’t as good. So Apple U-turned on its efforts to scan for CSAM in its cloud storage. Instead, Apple implemented end-to-end encryption for iCloud files.”
Wasn’t Apple’s design to explicitly NOT scan in its cloud storage, but look at the file on-device at the moment you wish to upload it to iCloud? This method would make it compatible with Advanced Data Protection; so ADP could have always been in the pipeline rather than Apple u-turning. In fact, NeuralHash may have been proposed because Apple wanted to introduce ADP and saw a potential problem here/get concerns from government agencies about it and saw this as a means to an end(-to-end).
The system was designed pretty elegantly and offers far better privacy protections - including guardrails - than what Microsoft and Google do, but the communication from Apple about it was absolutely horrible and generated enormous backlash. (Not saying I agreed with implementing it, just saying the design was infinitely better than competitors.)
Also, Apple's E2E iCloud encryption vastly predated the NeuralHash efforts.
As the creator of mediaden.ca[1] I’ve thought about this. Client side scanning is maybe marginally better than server side scanning, but both paths lead to privacy rot.
Governments need to catch criminals, but they shouldn’t do it at everyone else’s expense.
1. https://mediaden.ca
I simply don't trust services such as iCloud. The legal landscape is too volatile, and Apple's own "terms and conditions" are also subject to constant change. As far as I can tell, most people don't need cloud backups, and iCloud mostly shows up as an annoyance designed to extract more money from customers. In fact, most people probably don't know that Apple and Google vacuum up their files the moment they are created, for their own good, of course.
> most people don't need cloud backups
What world do you live in?
The world where the operating system on my phone (GrapheneOS) isn't conspiring against me or uploading my files to someone else's computer.
Is your phone also made of indestructible materials so it can't break, and magnetically linked to your person so it can't be stolen? Backups exist so you don't lose precious photos/data if you lose the device. Most People™ do not have the technical knowledge to set up a self-owned backup system, and/or will not realize how badly their future selves will wish they had backups if the setup friction for a self-owned solution is too high to conveniently do it right this second.
I think we are closing that gap with AI coding tools.
A growing number of people people today can figure out how to run Fable in a co work session, or codex, and that can indeed set up the self-owned backup system for you along with an alternative trustable cloud backup thats not Apple or Google.
That must also be the world where more than a tiny number of people are using GrapheneOS
But you still need backups right?
For most people "the cloud" is where you backup stuff. If you have a personal backup strategy that doesn't involve the cloud, you are not "most people".
Doing backups the right way take some skill and investment if you want to do it by yourself. It may involve setting up a NAS, and some discipline with physical media. You have to do your own security too. Most people don't want to do that, so, cloud backup it is.
It is not a perfect solution, even beyond the privacy considerations, like you can still lose your data by losing your account, but from my personal experience, people lose their data less often now that they have cloud backups.
> But you still need backups right?
I back up files to my own cloud with a Nextcloud integration for Android. That being said, a monthly or so backup of devices via USB/Ethernet, like we used to in the pre-cloud area, would be enough for all intents and purposes. It's not like what people have on their phones is generally very valuable.
Everybody needs cloud backups for their photos at least.
Most people don’t have computers, those who do, do not regularly backup their photos on them.
In both family and extended family many a cry would be avoided if people paid the 5 bucks it costs to backup their photos before your phone gets stolen or lost.
I gotta say handling my ever growing photo collection is a pain in the ass but I'm just not okay with uploading it to some server I don't control.
Ah, the CSAM saga. Very poorly handled by Apple. Suspect it may have taken Hair Force One off the shortlist of CEO succession.
IANAL but I thought the whole reason scanning worked was it wasn't required so there weren't fourth amendment issues.
I know creating a throwaway to hide your name for an opinion is a bad manner, but this one is one I really don’t want linked back to me
The VAST majority of “CSAM” is consensually created and exchanged by teens. Their future selves and their parents form this pressure group attacking everyone’s liberty and privacy to try to undo the downsides of choices they made themselves with full knowledge of what could happen.
The criminal and disgusting tail end of this type of material deserves the worst of consequences for the perpetrators and all the support in the world for the victims, but these are mostly - you guessed it - poor and unprivileged children from far away places and they certainly can’t put this much pressure on apple
I think you’re right, but from another angle. In the state where I lived way back when, a state representative put forth a bill to explicitly make e-CSAM illegal. I guess it was already illegal for print media and this covered a gap in the law about cell phone pics, etc. Thing is, it had no allowance for the age of the picture taker, or even whether the picture taker was the photo subject. If a 16 year old girl took a nude selfie and sent it to her boyfriend, she was a felon.
I wrote to the rep and explained my concerns. I wholeheartedly agreed with the intent of the law, but the code was buggy. To my surprise, he wrote back in horror to say he hadn’t considered that and pulled the bill immediately. I’m proud of having done that.
I’m 100% pro yeeting child pornographers into the sun. I still don’t want to throw kids in prison or remove all traces of a right to privacy in our haste to sun-yeet them.
> I still don’t want to throw kids in prison or remove all traces of a right to privacy in our haste to sun-yeet them.
The one messy corner of this is the "strict liability" for this type of material. An underage kid can take a nude photo, send it to an adult, and then the adult can criminally liable for just having it, even if he deleted it as soon as he saw it. Either both parties involved in handing something for which there is "strict liability' need to be held accountable, or "strict liability" has to be changed so a person isn't liable if he deletes or reports the material as soon as he first becomes aware of it. And this isn't likely to happen because it would provide a plausible defense for every one criminally charged.
IANAL, but the notion of "strict liability" horrified me when I first head of it, and I thought it had to be some kind of a misunderstanding. So we're tossing that "innocent until proven guilty" idea out the window, huh?
I'm sure smarter people than me have sussed this out and can explain why it's a good thing, but it sits wrong with me. We can put the subject matter aside for a second: I don't think I could convict someone for having something happen to them, regardless of what the law says. Let's say drug possession was a strict liability law (and maybe it is for all I know). Finding a baggy of meth on the corner of a farmer's lot would mean that, technically, he was guilty of possession and had to prove that it wasn't really is. That's nuts. And looping back to the subject at hand, if the only evidence that someone possessed CSAM was their email inbox, without proof that it was solicited? They want me on their jury.
There should never be a circumstance where someone can't report something that happened to them to the police without a legitimate fear of being arrested. That's bad for the person, and it's bad for society.
But if their hard drive has folders grouped by age or something, prepare the solar catapult.
I completely agree with you, but I do think that these teens do not have good opsec around these photos. It’s like the revenge porn problem but way worse. A teenager sending a nude selfie to a friend who then later shared these images non-consensually is a much bigger problem than if the same thing happened to adults.
I don’t have any ideas for a solution, but I suspect that the heightened focus on CSAM is really compensating for the fact that we don’t have solutions for revenge porn.
Are there statistics backing up the "VAST" majority claim?
While on statistics, I wonder, are there reliable statistics about child abuse of different types? Studying correlations with other social metrics, like sex education, liberal/conservative, policies regarding prostitution, and others can provide support for/against decisions.
Not that I hope these will impact people's and governments' choices, but I want to challenge my intuitions.
That's the fun of it. Try to find out? Straight to prison.
Really sad. I read a lot of psychology. But I've never encountered psychology of child sexual abuse (I know there is if I look it up, but that's my point, it should be shared around, discussed, challenged).
Why would some adults find kids sexually attractive? Is it abusive/aggressive behavior manifesting itself in sexuality? Or is it sexuality channeled in the wrong direction? If it's the second, is it out of desperation, and would happen less if the culture makes it easier for them to satisfy their needs with adults, or would it happen regardless? On the victim's side, are the shy and less social ones more in danger, or the socially active ones? From my social scientist friends I hear a lot that most child sexual abuse is domestic. What are measures that a society can take to prevent these, without turning the society into a surveillance state, which will ultimately harm everyone more, including the children? What can be done to make sure children speak up, so that such behavior is dealt with at the beginning (and maybe while the more terrible things have not happened yet), and not turn into a multi-year childhood trauma?
What are signs (and early signs) on the abuser's side and the child's side? How to deal with these signs?
It's probably impossible to get reliable statistics about that given how both groups are trying to keep everything secret. But you can consider how many paedophiles there are vs how many horny teenagers there are. Based on that it would be extremely surprising if he was wrong.
The real question is what happens when a horny teenager sends another a nude. There definitely have been insane cases where they get stitched up for creating child porn. I don't know if that's the normal outcome today though.
The proposed CSAM scanning used perceptual hashing to try to identify CSAM material known to law enforcement.
It was not a tool to identify private images as being underage. That’s an impossible task.
It's impossible to do with perfect accuracy, but that doesn't mean it isn't done.
Just ask the dad who was investigated for taking pictures of his toddler for the doctor: https://www.koffellaw.com/blog/google-ai-technology-flags-da...
I'd say modern AI tools could probably do this pretty effectively. They're very effective at describing anything else about an image. I have a workflow that churns through large amounts of images, describes them, and then looks for things I specifically want (in my case, auction listings that are not described accurately on the auction website).
I need to process a modest amount of imagery (about 25 million images, and growing) for NSFW content and general captioning/description. About 5% of it contains nudity or partial nudity, and about 10% of that 5% contains sexual activity.
In theory, modern vision language models could classify human nudity and sexual activity very thoroughly. But every model I have tried is reluctant to clearly describe what is notable about sexualized/nude images. The models are deliberately under-exposed to nude and sexualized content during training and further RLHF'd away from generating straightforward descriptions of such images.
Models also occasionally hallucinate WTF captions for ordinary adult sexual activity. I recently ran a baseline test with frames extracted from adult videos and about 1/3000 frames was mis-captioned as involving a child according to Gemma 4 12b.
Children cannot legally consent to most things in most places, especially until near the end of their teen years.
They can't consent but it does make sense that it's true, I would genuinely bet that more nudes are being shared between 17-year olds than some freak, as this is common to the point where I feel a very large portion of all existing teens have done it.
In the near future, when a 17 year old asks her phone to take a nude selfie, the phone will say "no".
It wouldn't be so bad tbh, would avoid all the leaks and regrets that comes with it. In term of awareness, I would say that a late teen is fully aware of his/her actions but might not calculate consequences properly.
Do you have a citation for that? Sounds plausible, but I'm not sure I've ever seen it stated that way in any of the related media reports on CSAM efforts.
In Germany, the rise in "youth porn" material has been attributed to such kinds of cases where youth send intimate pictures to each other [1].
Our legal systems are not built to deal with that mess, and it may hang around your neck for the rest of your life. Unfortunately, the law is very explicit, leaving barely any avenue for the courts to drag us out of the mess, and politicians - even if they are actually interested in the topic in the first place - won't touch that area with a ten foot pole for fear of getting blamed a pedophile themselves.
[1] https://www.n-tv.de/panorama/KI-treibt-Jugendporno-Fallzahle...
I won't pretend to be so knowledgeable about how so much CSAM is being created, but keep in mind, there are laws against distribution & mere possession, too. Revenge porn is an obvious thing to be mindful of in this context in addition to other types of distribution. Consent to create doesn't imply consent to distribute (probably even to cloud storage) & is completely immaterial to issues of possession if it ends up in some 3rd party's hands. So the scope goes way wider than you're letting on. If you're saying all of these these laws are being abused, like they exist primarily to punish a state senator's daughter's ex-boyfriend, I would ask for something to back that up.
Yes, poor and unprivileged children can't really defend themselves here, but this is the system working to find some legal mechanism to do what it can, as a more powerful force. Protecting people from exploitation is a good use of government. If this was shot down for legal reasons, OK, the system is working and I hope there is a way to expand protections that fits into our system.
If these judges are so righteous, they should go further and mandate the OS to do mandatory scanning of personal hd.
It's still a shade of gray to me. If I offered some homegrown cloud storage to my friends, and one of them uploaded CSAM to it, you can bet your ass that I would be arrested for it.
Does Sundar get arrested if someone uploaded CSAM to GDrive?
No, because Google will report it to the law enforcement.
Don't worry, we'll get there soon enough.
sigh
Once again, someone (in this case, the judge of this case) asks if we can meet in the middle on whether or not private communications are actually private.
To be clear: this is not a limitation of nerds' imagination. This is a limitation of physics. A person is either party to a communication (and thus can decrypt it) or is not (and thus cannot). If you demand Apple scan encrypted photos for CSAM, what you are demanding is that Apple be party to every communication done with an iPhone. There is no middle ground on encryption, there will never be a middle ground on encryption, and I will hold this truth on my deathbed.
There is no "encrypted but crackable" - if the CIA can crack it at all, we're only a few years away from some kid's gaming rig doing the same thing. There is no "secure golden key" - if there was, you could buy it in the same section of Amazon that sells copies of the TSA master key that opens all luggage locks.
Personally, the next time a government demands decryption keys, I think Apple should just set all iCloud photo libraries in that country to public and say "Sorry, your politicians made private photos illegal, take it up with them". Obviously, telegraph this far in advance and give users time to actually delete their cloud-hosted photos first. But definitely do not pretend like you can keep a secret with a government bureaucracy of hundreds of thousands of people.
But then again, Apple also capitulated (good meaning) to the EU on third-party app distribution, so Apple has a lot less of a spine than they let on. At least Google actually stayed out of China.
the judge is indeed wise
Seems like the kids miss their chance at justice because of section 230 allowing platforms the freedom to remove whatever they want but not be responsible for what they keep or amplify. That is the problem with 230. Censorship is permitted and punishing the censor isn't. Twitter and Tiktok are literally microblog platforms that get away with removing good stuff and leaving evil because they "are not a publisher" while the algorithm literally publishes a chosen set of articles to people. Facebook can remove religious freedom material and leave human trafficking groups. Section 230 gives the publishers the cake and the edict too.
You're conflating "what's illegal" with "what a private entity doesn't want". I don't like it any more than you do, but the first is very clear, the second is a bit harder to "solve".