This filter isn't specific to polls. Anything that reaches people through reshares gets selected on how surprising or pleasing it is, so the visible sample skews toward the tails. Marketing has the same problem with screenshots of campaign results: the ones that travel are the outliers, and anyone calibrating on what they've seen shared ends up with expectations nobody can hit.
Curious whether the effect survives controlling for pollster. Some pollsters have consistent house effects and some have much bigger followings, so part of the slope could be "polls from well-known outfits get reposted more" rather than "favorable margins get reposted more".
Almost all polls are worthless, because the decision to respond to the poll is itself a major portion of the signal.
It takes extensive work to get around that problem alone, let alone "viral" and other worthless push-polls - work that even the very best in the industry, at the highest level, can make major mistakes on.
>Almost all polls are worthless [...] because the decision to respond to the poll is itself a major portion of the signal.
But most polls I see are from professional pollsters like Pew or YouGov? For them, correcting for this is opinion polls 101. That's why you see "representative sample". They're not just randomly polling 1000 people and taking the average. They're adjusting for a bunch of other demographic factors as well.
And there's no clear way to adjust for the most basic factor that people who vote for one candidate are less likely to say so.
In order to account for that, you need you know how many people who are less likely to respond are voting for that candidate. And then you're back to square one.
This is why professional polls have had a consistent bias over the last few years, as this hidden unobservable factor's effect has gotten stronger and stronger.
Because they can correct against the election results.
If you're trying to do a scientific poll that is not about an election, it can become exponentially harder unless you can find independent ways to obtain data that helps you adjust (like home ownership, which is reported to the government).
Then you're mostly looking at those who "try" - but if you look wider you'll see polls used way more often than that (think of the companies entirely built around it, customer satisfaction surveys, Facebook "polls", etc, etc).
Doing a proper poll is expensive and (frankly) most of the time is pretty predictable or you can get 90% of the results with 10% of the effort, and most times you don't actually care about the results because you're building it to get a result you want anyway.
(Interestingly enough elections themselves almost by definition have notable discrepancies to a "true" perfect sampling; but it's usually overcome by the sheer volume.)
This is totally wrong. Accounting for partisan non-response bias is a standard part of a pollsters job, not some insurmountable gotcha. Some are better than others at this. Polls aren't oracular, nor are they claiming to be. It's almost always better to look at an average of polls rather than a single one.
There is a lot of public data on past elections that anyone can analyze and see exactly how accurate polls have been in the past. If you're expecting polls to be a perfectly accurate prediction then you are misunderstanding their purpose. What they are great at is giving a probabilistic estimate of the outcome, within a certain range.
It's likely that any key race you care about next week will end up within 4 points of the current polling average. It's extremely likely that it will end up within 8 points. That's useful information!
When they are 3 or 4 percentage points off they are ripped to shreds.
The polls for the recent Brazil elections were about 49% for Lula, the result was 52%, that's a 3 point difference, not a massive error margin. With brexit the polling pointed to about 52% for remain, actual result was 48%.
But those are proper polling companies, not worthless online ones.
All polls are flawed. Remember last US pres election polls by CNN? Completely wrong. Why? Because they surveyed mostly left-"learing" participants.
Right and supposedly middle (this doesn't exist) media does the same.
That means we gotta find the average as you suggested.
---
Side note
> Given that Musk has absolutely destroyed the Twitter API for academic use
Why bring up Musk, when you can simply say, "Twitter lowered Academic API rate limit because ...". You sound like you need someone to blame not professional and politically motivated (yes the post is about the political poll but this shows the "bias" and his supposed poll will be biased depending on how he gathered data).
Twitter would have the same way of generating metrics for the justification as you are trying to justify the flaw in the poll outcome.
Most likely the decision is based on their internal metrics/dashboard report
- All reputable firms try get valid results, they have professionals, they know who they are talking to and the limits. What we see in the last 2-3 weeks of the elections are the polls they want to release however, those that err in one direction or another.
- They definitely accommodate for misrepresentation samples - if more of their respondents indicate 'registered republicans' than in the general public they will adjust.
- It's very hard to measure 1) the people that are hard to reach and statistically more likely to not trust pollsters and 2) likelihood of actual voting aka enthusiasm.
- there are plenty of centrist institutions and pollsters
- It's fair to call out Elon because he's making the policies
Twitter is just a bad way to measure anything except for the Zeitgeist of Twitter readers.
In 2021 Twitter was encouraging free academic use of the API [0].
This was in line with long standing behaviors in this space [1].
Musk bought Twitter in 2022. In 2023, they began substantially restricting API access versus how it had been for years.
It is not unfair at all to look at that timeline, look at Musk's acquisition as the biggest change that had happened to the company in that timeframe, and draw a relationship.
> Remember last US pres election polls by CNN? Completely wrong.
I do. I don't remember that at all. I think you might have some odd definition of completely wrong though.
But please, by all means, share these 100% wrong polls that showed, for example, Trump having 0% chance of winning. Most of the polls I saw were fairly accurate within their margin of error and made it clear the race was tight (which, in the end, it was). And nothing I've seen has changed that, but if you have polls from CNN specifically that you can point to, by all means.
This filter isn't specific to polls. Anything that reaches people through reshares gets selected on how surprising or pleasing it is, so the visible sample skews toward the tails. Marketing has the same problem with screenshots of campaign results: the ones that travel are the outliers, and anyone calibrating on what they've seen shared ends up with expectations nobody can hit.
Curious whether the effect survives controlling for pollster. Some pollsters have consistent house effects and some have much bigger followings, so part of the slope could be "polls from well-known outfits get reposted more" rather than "favorable margins get reposted more".
The very nature of the term 'viral' obviously would be disqualifying or hint that something is very awry.
The entire objective of a polls is to get a probabilistic sample. That's the whole thing. Anything that deviates that ... creates problematic outcome.
Of course polls that show web unexpected lead for an individuals favorite candidate are more likely to be shared by that individual.
Almost all polls are worthless, because the decision to respond to the poll is itself a major portion of the signal.
It takes extensive work to get around that problem alone, let alone "viral" and other worthless push-polls - work that even the very best in the industry, at the highest level, can make major mistakes on.
>Almost all polls are worthless [...] because the decision to respond to the poll is itself a major portion of the signal.
But most polls I see are from professional pollsters like Pew or YouGov? For them, correcting for this is opinion polls 101. That's why you see "representative sample". They're not just randomly polling 1000 people and taking the average. They're adjusting for a bunch of other demographic factors as well.
And there's no clear way to adjust for the most basic factor that people who vote for one candidate are less likely to say so.
In order to account for that, you need you know how many people who are less likely to respond are voting for that candidate. And then you're back to square one.
This is why professional polls have had a consistent bias over the last few years, as this hidden unobservable factor's effect has gotten stronger and stronger.
And even the best pollsters, who over decades learn to correct for drift like that, get blown up by a sea-change that they didn't expect.
I don't see how they can adjust for the type of people not responding to polls unless they can force them to answer.
Because they can correct against the election results.
If you're trying to do a scientific poll that is not about an election, it can become exponentially harder unless you can find independent ways to obtain data that helps you adjust (like home ownership, which is reported to the government).
non-response is factored in as well. https://www.pewresearch.org/methods/2026/09/16/why-many-poll...
Then you're mostly looking at those who "try" - but if you look wider you'll see polls used way more often than that (think of the companies entirely built around it, customer satisfaction surveys, Facebook "polls", etc, etc).
Doing a proper poll is expensive and (frankly) most of the time is pretty predictable or you can get 90% of the results with 10% of the effort, and most times you don't actually care about the results because you're building it to get a result you want anyway.
(Interestingly enough elections themselves almost by definition have notable discrepancies to a "true" perfect sampling; but it's usually overcome by the sheer volume.)
This is totally wrong. Accounting for partisan non-response bias is a standard part of a pollsters job, not some insurmountable gotcha. Some are better than others at this. Polls aren't oracular, nor are they claiming to be. It's almost always better to look at an average of polls rather than a single one.
There is a lot of public data on past elections that anyone can analyze and see exactly how accurate polls have been in the past. If you're expecting polls to be a perfectly accurate prediction then you are misunderstanding their purpose. What they are great at is giving a probabilistic estimate of the outcome, within a certain range.
It's likely that any key race you care about next week will end up within 4 points of the current polling average. It's extremely likely that it will end up within 8 points. That's useful information!
"because the decision to respond to the poll is itself a major portion of the signa"
Yes - they are aware of that.
> Almost all polls are worthless
Do you have one example of a poll that isn't?
The only poll that matters is the one on election day.
Most professional pollers do a good job.
When they are 3 or 4 percentage points off they are ripped to shreds.
The polls for the recent Brazil elections were about 49% for Lula, the result was 52%, that's a 3 point difference, not a massive error margin. With brexit the polling pointed to about 52% for remain, actual result was 48%.
But those are proper polling companies, not worthless online ones.
All polls are flawed. Remember last US pres election polls by CNN? Completely wrong. Why? Because they surveyed mostly left-"learing" participants.
Right and supposedly middle (this doesn't exist) media does the same.
That means we gotta find the average as you suggested.
---
Side note
> Given that Musk has absolutely destroyed the Twitter API for academic use
Why bring up Musk, when you can simply say, "Twitter lowered Academic API rate limit because ...". You sound like you need someone to blame not professional and politically motivated (yes the post is about the political poll but this shows the "bias" and his supposed poll will be biased depending on how he gathered data).
Twitter would have the same way of generating metrics for the justification as you are trying to justify the flaw in the poll outcome.
Most likely the decision is based on their internal metrics/dashboard report
These are all problematic statements.
- CNN doesn't do polling, they have firms do that
- All reputable firms try get valid results, they have professionals, they know who they are talking to and the limits. What we see in the last 2-3 weeks of the elections are the polls they want to release however, those that err in one direction or another.
- They definitely accommodate for misrepresentation samples - if more of their respondents indicate 'registered republicans' than in the general public they will adjust.
- It's very hard to measure 1) the people that are hard to reach and statistically more likely to not trust pollsters and 2) likelihood of actual voting aka enthusiasm.
- there are plenty of centrist institutions and pollsters
- It's fair to call out Elon because he's making the policies
Twitter is just a bad way to measure anything except for the Zeitgeist of Twitter readers.
In 2021 Twitter was encouraging free academic use of the API [0].
This was in line with long standing behaviors in this space [1].
Musk bought Twitter in 2022. In 2023, they began substantially restricting API access versus how it had been for years.
It is not unfair at all to look at that timeline, look at Musk's acquisition as the biggest change that had happened to the company in that timeframe, and draw a relationship.
[0] https://newsonline.library.vanderbilt.edu/2021/03/use-twitte...
[1] https://arxiv.org/abs/2404.07340
> Remember last US pres election polls by CNN? Completely wrong.
I do. I don't remember that at all. I think you might have some odd definition of completely wrong though.
But please, by all means, share these 100% wrong polls that showed, for example, Trump having 0% chance of winning. Most of the polls I saw were fairly accurate within their margin of error and made it clear the race was tight (which, in the end, it was). And nothing I've seen has changed that, but if you have polls from CNN specifically that you can point to, by all means.