No. Below is my emotional opinion based on my own experience:
Currently the models are totally helpless with plot and emotions.
They make epistemic, logistical and temporal mistakes.
But:
They are very good in finding _your_ epistemic, logistical and temporal mistakes. They are great as adversarial reviewers. They can validate grounding. They can test voices. They can build complex parallel constructions representing timestamped inner monologues, dialogues and events.
A process consisting of many models representing characters + states + epistemic isolation + reviewers could be great to validate or disprove ideas.
Models can write you formal models of your writing. They can run solvers against your formal models to - again - validate and disprove ideas. Literally, you could validate your writing with TLA+, SAT solvers, queries against formal ontologies.
They can build corkboards of unimaginable complexity.
A moderately good writer who plays well as a part of human-machine writing apparatus could be a competitor even for a top-tier author.
When you write with a model, you define disciplines. You write your prose. You let your model turn your prose into formal systems and verify it. You let your model criticize you. Sometimes you let your model to polish your crude language into something sleek.
So, at current state of LLM development, writer as source of ideas and governing process is safe. Writer as a vision-expansion machine is not.
I've been very hostile to LLMs in literature. I've started to write a short novel about that. As a joke I tried to use models after half of the text was done. I've applied my engineering skills. That changed both me and my opinion. Models are awesome if you know their advantages and weaknesses.
I have a feeling the author framed this the wrong way.
my take is - the author wanted to express that there's always a demand for human prose / writing that captures the subtleties of expression, thought & ideas. which is very true. & we can already see this e.g by people opting out of LinkedIn for it's A.I driven long posts. whereas engagement is high on X where posts are likely to be human generated.
I wouldn't say writing as a job is protected - as corporations will always take shortcuts.
For now. And that is because humans are the only customers engaging with content. As agentic AI takes hold and is given credit card authority, bots will will engage. Bots will read the posts of other bots and use those when making purchase decisions for thier humans. If we are lucky, the bots will develop thier own twitter-specific language and we humans can finally wave goodbye to tweets as a concept. The bots will play the twitter game for us.
There may well be some writing jobs that are safe for the reasons given in this post, but that doesn't help all the writers I know who have already lost their jobs and are struggling to find work.
It's not enough that humans can tell the difference and feel an ick, there also need to be enough organizations willing to pay money for that difference. From my vantage point, there are not. It turns out that for a ton of the writing produced by companies, the quality of the prose wasn't really "load-bearing" as Claude puts it. That writing is there to occupy a space and look professional at a glance, the same way elevator music is tolerable for the duration of an elevator ride.
I have worked in the publishing industry, translation and editing (got out to a more dependable career just in time), and what is astonishing is how even "load-bearing" text is not safe from the cost-cutting pressures. That is, even some respectable publishers for the last several years will publish a book one is expected to pay good money for, with minimal editing and proofreading compared to the old days, and if it is an originally foreign book, with machine translation and minimal post-editing.
I've got some heavy reader friends and they keep complaining that many of Spains's fantasy book translations are ridiculously bad and blatantly machine translated now: Senseless terms that require looking up the original to understand, e.g. "fall" translated as in falling when it meant autumn; proper nouns, including character names, sometimes translated sometimes not; and many sentences that mix up the grammatical gender because the machine wasn't given enough context to infer it.
But these publishers don't get punished because they keep licensing popular foreign franchises, which aren't quite fungible, so consumers don't want to miss out and thus don't vote with their wallets.
A lot of the replies are insisting AI will get better at writing with more development but I don't see it. Even if you have a mathematically perfect writing AI you still run into the same problems you would have if you handed off your writing task to someone on fiverr or something. It can't magically know what you want to say, it only has the information you gave it. A prompt complex enough where it won't get any wrong ideas has to contain as much information as the output would have.. so just write it.
I also believe this. Post-training LLMs with vague metrics can only be achieved with RLHF, which is not impossible, but extremely costly and difficult. Instead, companies will opt for RLVR, focusing on math and programming tasks. This pushes objectives away from writing quality; often far away. That is why older models, in my view, actually read better than newer ones. It's by design.
I think for this argument to be true, the axiom that supports it is that the models have just as much context as they will ever have, and you cannot see being able to give them more / enough to be able to understand your perspective. That feels unlikely to be a position that doesn't change. As a society we're giving more and more context each day to this, and that makes this a valid opinion now, but one that erodes over time.
its not about dumping more and more info into the context, its about the intention. whats not in the context is just as important as what is. and i dont see how that can be automated.
also were seeing models become worse at writing as they get smarter.
Size of context is not the entire story here, it's ability to properly feed and index the context that's needed on this sort of thing. E.g. your entire slack/discord/email/github/jira/zoom meeting/coffee chat ... history is the context that you bring to the table on this sort of thing. Most of this is unindexed. Much of this will not be in the future.
> The latest models got even worse.
Which models? This is one of those things that likely has both model and domain specific aspects that impact your experience. In my experience with OpenaAI models predominantly (I previously worked there), they've improved significantly over the last 6-12 months. My experience with Claude is worse, but I haven't spent as much time getting into a mechanical sympathy there. They're still not perfect though and I have many steering docs that help avoid the biggest problems in the models I use when generating docs.
Claude writing quality got unbelievably bad with Opus 4.7, with no improvement in Fable. Opus 4.6 was fine. Im starting to see it as a security risk - my brain just can't process its word vomit, so just tell it to go on, implement whatever
Workaday copywriting is dead (was moribund, now has been shot in the head). Prestige literary writing is zero-sum and therefore eternal, in the same way that equities trading (not formation and not business-building) is zero-sum and therefore the actual success of LLM has not given anyone a signal advantage anywhere there, because everyone else has LLM too.
Copywriting struggles with the issue that businesses know they need it, but they don't value it. It's absolutely shocking to see how poorly companies, and governments, communicate and no amount of AI/LLM usage will change it. For decades it's been neglected and relegated to "cover my ass"- writing.
The majority of AI deployment in businesses could have been avoided, if more attention had been placed on good communication. LLMs don't yield better communication or corporate writing, just more of it, because no one in business seems to understand the value or have the ability to recognize good writing.
Equity trading is not zero sum. There are real businesses and assets behind the equities they represent that have actual value and produce actual money.
theoretically equity trading is about distributing stonks most optimally so that productive companies get more and less productive ones get less, improving productivity of society as a whole
Derivatives (options, futures, swaps, etc) are a zero sum market, there is always a winner and a loser necessarily because the contracts have an expiration date.
The same is not true for trading equities, they are not zero sum. There are dividends, buybacks, companies will sometimes spin off a part or parts as separate companies that you get newly issued shares of stock from (GE splitting into parts is a recent example), public companies get taken private at a premium to the market price, etc.
Futures are zero sum, but can also be a win for both sides, because what they gain is stability.
"I will buy y tons of corn from you in April for £x" - now I don't have to worry about how much my corn will cost and you don't have to worry about what your income will be.
The market has been heavily engineered and gamed to be positive sum. But sometimes I wonder could it turn out to be negative sum on certain timescales. That is that overall money would be lost on markets.
Writing has an audience, purpose, constraints, and desired outcome. Creating tests for writing is more difficult than creating a unit test, but it's not undefinable. It's just engineering.
If you're good at it, which extremely few people are. It's my belief that not enough good writing exists for the tech companies to train an LLM. At least not a generally applicable on.
Tech companies are training off the shadow libraries, which have the bulk of English-language human writing in book form from the twentieth century, and that traditionally published stuff with an ISBN usually got a proper editor of some kind. So, it is hard to believe LLM companies don't have enough material; it's not as if they are limited to low-quality stuff on the web.
Exactly. And it's not only about whether a specific job is safe from AI, but whether it's safe from the effects of those that aren't safe: competition from laid off people and downward wage pressure. Then there's the backpressure into the educational pipeline.
Similar, much narrower crises were managed for manufacturing jobs during the worldwide industrialization era. The solution for those displaced by trade was retraining.
Unfortunately, at the moment we have no clue of what workers should be retrained to. Not writing.
It's a pretty reliable career for hundreds if not thousands of mid-list authors who generally stick to genre fiction. It's entirely possible that AI could core out the middle of the pack as well as the bottom, which would threaten not just jobs but a wealth of institutional knowledge that those authors share among their peers and especially new up-and-comers.
If I found the best human writer in the world, and had him write every article on the internet, wiping his memory between each one, his writing would start to get pretty noticeable and boring!
I agree. LLMs write really well, but probably too consistently in terms of style. That said, I did recently ask AI to do an editing pass over some writing I did, and I was surprised that it missed a tense mismatch that I had (not within a sentence, but within a paragraph).
The real thing being said here is that the author can tell good writing from bad, and assumes everyone can... I'm a visual artist and at generative art is blindingly obvious.... To me.... But not to many folks around me! Including a few artists and art adjacent folks.
It extends well beyond creative stuff too. It makes me think of Gell-Man amnesia [1].
People can tell when something they are experts in is being done poorly, but others can't, and it frustrates the experts. Then, those same people think something completely different is being done well despite what experts in that respective field say. It's like when someone from your family reads an article about your profession and then proceeds to tell you how your job works. lol
It's a really pervasive issue in society IMO. And a few prompts in someone's favorite LLM just reinforces it to people that don't know (any better|what they don't know).
Have you tested this systematically, or is it possible that you are experiencing survivorship bias? If there were any generative art pieces that you didn't notice, you would have thought that they were human-made. Therefore, all the pieces you identified were "obvious" to you. Not to mention false positives.
I imagine that they haven't done a peer-reviewed scientific study of their ability to notice generative art.
We can replace their statement by "I'm a visual artist and at generative art is often blindingly obvious.... To me....". In other words, they can notice some or most generative art but other people familiar with art can't, or at least can't as often as they do. I think what they are trying to say isn't too changed by that.
Even if it’s obvious to some at this time, as time passes artificially generated writing will become mainstream and what becomes the normal writing style. If you read IX century literature, it reads differently, the vocab is different, the sentences are built differently. It was totally normal back then. It’s not normal now. In ten years the new artificial style will become commonplace and accepted -I mean unless we artificially always make it deviate.
LLM prose has degraded with each model update, the models are now being RL'd into wall of texts that only makes sense to other agents. I think its going to become more obvious as we move forward that is llm generated because labs seem to only be focused on tool-calling/agentic-coding environments, as that's the only thing that drives revenue.
There may come players who focus on models that are good at writing for technical writing/docs , copyrighting ect but I think people will lean towards not using them and will rather have the "human touch" for the things that directly impact brand perception.
Keep in mind, every single AI company that is selling the idea that you don't need to hire designers and web design is "solved" have $100k retainer designers crafting their landing pages.
Yeah, this tracks with my experiments. About every 6 months or so for the past year-and-a-half I've tried using the "frontier" LLMs to as-near-as-possible autonomously write novel length stories, because I find it fascinating.
When I had Gemini 2.5 write a novel, it wasn't really objectively "good" by any stretch of the imagination, but while the prose was very purple and full of cliches and, well, bad writing I guess, it still felt ... subjectively good, at least for what it was.
Last week I did a run with GPT-5.6, and wow. On the one hand, it managed to produce 110,000 words that were "shockingly" coherent. The model was able to maintain state and plot lines and background details extremely well, much better than older models.
But I just don't like the prose. I haven't really liked _any_ prose that GPT-5.6 produces. It's significantly better at "instruction following" and keeping track of things, but, wow.
> “The sequence is consistent with their voluntary choices.” Mara enlarged the uncertainty field rather than the result. “It does not prove what happened to anyone we can’t observe. It does not prove contact did this. And it does not turn the Shard into treatment.”
GPT-5.6 in particular becomes so fixated on certain ideas like "consent" and epistemology, that by the end of the narrative, the prose and dialogue are all just "agent speech", despite the prompt/harness specifying that it's a _novel_ with narrative prose and such.
Interestingly, the model itself produces an accurate critique of its own output:
> The draft has become a *consent-centered medical, legal, and logistical procedural*. The important drift is therefore not that many events were omitted. It is that the retained events now prove a different thesis.
Which begs the question of if it would do better with a couple rounds of output -> critique -> revision. But I think I've had enough LLM prose for a bit...
Claude is the one model family I've not really used. Which yeah, feels like a backwards thing to say in a world where seemingly everyone using LLMs is using Claude Code.
I've used some Opus 4.5/4.6 via Antigravity and Sonnet by the web chat. I'm torn because as far as LLMs go, it does feel more ... "literate".
But maybe too literate, judging by how many people are complaining about "Claudeisms". I suspect Claude would be just as susceptible, if not more, to the sort of ... "moralizing" that GPT seems to gravitate towards (for lack of a better term).
I have a weird background (product, development, writing + devrel). I write a lot of code and a lot of articles.
I think there is a lot of overlap in how people who write code or articles (documentation, books, etc.) use AI. On one side of the spectrum, you have people who just blindly input some prompt, accept the output and move on with their lives. You can likely predict how that is going for them (not great). On the other side of the spectrum, you have people who outright reject all AI and are continuing to plod on with how they have always done things.
In the center is a more reasonable approach that leverages AI to create without blindly accepting the output. This applies very much to writing.
The workflow that I've adopted over the past two years or so has been to leverage AI to help with the research and outline process. Once I'm happy with the structure I go and I write what I need to write.
This maps pretty closely to the code that I write. It's fine.
If we're doing book recommendations, I quite recommend the essays in https://academic.oup.com/mit-press-scholarship-online/book/1... and particularly the philosophical discussion there of whether "robot sex" is an oxymoron, as a robot almost by definition cannot dissent, so it might only be "robot-assisted masturbation"
If you thought all of the money being poured into data centers was excessive, wait until people can order up a proto-sentient sex bot to their front door. Oh boy.
I think there is good chance that AIs have already replaced parts of the certain aspects of industry. And with some tuning could probably replace lot more of at least virtual parts...
Yeah I had GPT5.6 Sol act as an editor on my new chapter and it did find decent issues with PoV and narrative distance but it was so bad at prose. The best way I can describe it is, it was too clean. Often times humans write about things unsaid and left for reader to interpret or deliberate awkward sentences to tease character psychology. The AI just straight up corrects these without second thought. This will likely be the case with general purpose models unless we see some genre specific fine tunes.
I don't think writing is an "AI-complete" problem. It's just that the models are inefficient at it right now, due to training and/or architecture. Good writing requires thinking, reflection, and doing multiple passes. This currently translates to using a high reasoning effort and burning lots of tokens, but big AI labs like Anthropic are struggling with handling the load, so they're doing the exact opposite and finding whatever cheap trick they can to reduce token usage. One such trick is condensing ideas into as few tokens as possible, which in my opinion is one of the reasons Claude sucks so much at writing.
That's because it's cheaper to automate than pay a software engineer. The economics has to make sense. Writing is tricky. Prose quality isn't there yet for many genres. You can tell AI written prose apart. Maybe it won't matter. Maybe there is demand for it even. Or Maybe we will create genre specific finetunes if there is enough demand for it.
First, I actually had the author of this post as my distributed systems professor back in college. He's not just a super smart dude on a technical level, but he's probably the only CS professor I've ever had who seems to appreciate writing in itself (which may be evident from his blog). I think the combination of these two qualities puts him in a good spot to make the claims he is.
In terms of what he's presenting here, I'll say this: his line of logic highlighting the "dual-mind problem" of writing seems, to me, like the most compelling aspect of this argument. When I write something (especially to a specific audience, including an individual), I must practice cognitive empathy if I want what I write to be consumed properly and effectively. This is a cold way of putting it, but even in text message responses which span a few words towards family or a friend, I can put quite a bit of thought into how it will be received, how they will read it, interpret it, etc. My cadence in just texting, alone, can shift dramatically from one message to a next based on who I'm sending it to, what I'm trying to convey, etc.
All of that is fine and not hard to understand, but I suppose the interesting part is this: when I'm just trying to get information across (i.e., instructions, directions, etc.), my messages will resemble something written by an AI. It's not enjoyable to consume, but that's not the point. But as soon as I want to add a bit of "fun" to a message, the task I'm performing is completely different. It's no longer just an exchange of information, but it's an attempt to invoke specific feelings, visualizations, memories, etc., in the other person/people.
I do have a hard time imagining what it will take for AIs to be able to do THAT effectively. I think they're fine at conveying information, but I think people are already becoming very aware that conveying information, in itself, is not enough to be effective. These LLMs don't "care" to "entertain" you with what they're writing at you about. They're just spitting out the mathematically derived facts with, seemingly, no meaningful ability to invoke deeper thoughts in their audiences' minds. And that might not seem particularly important outside the context of writing fiction, etc., but I think it's actually pretty critical even in "dry" settings, like explaining code, because the thoughts, feelings, emotions, etc., invoked through reading IS the output of reading (even if it doesn't seem that way).
Perhaps AI will be able to do this more effectively if they're trained on direct brain signals or something. Like, train the AI not just to convey accurate information, but also encourage it to do so in a way which stimulates different areas of the brain with different intensities based on the premise that, doing so, is actually what people are getting out of that text.
Go on to Ao3 and a popular fandom, say like A Song of Ice and Fire: 9 out of 10 fanfictions written in the past two years stink to high heavens of AI. 9 out of 10. From some that are obvious AI to even normies, to some that slip one or two ai tropes after a couple of chapters, because the author got lazy and stopped editing carefully.
It's sad, infuriating, discouraging. A deluge of slop drowning the last embers of authentic human creativity.
I appreciate what this piece is trying to do directionally but it shares a logical flaw with lots of other pieces about AI and knowledge work. It assumes that in order to disrupt a particular field, AI needs to print serviceable work unassisted; that it's "vibe-shipping".
AI today is most effective when it's not vibing, but rather copiloting a skilled operator. I don't necessarily want my agent to build an entire system for me, even if it's ultimately the author of almost every line of code; I'm actively making decisions throughout. There's obviously a spectrum here but at most points on the spectrum the amount of assistance available is still a step change in the economics.
So too with writing.
The first rule of accelerating writing with AI is that you're not allowed to use a single word the AI suggests. Even if what the model comes up with is great, better than what you could have done, as soon as the AI suggests it it's poisoned. At least with current models, readers can detect LLM prose in the parts per trillion, and as soon as they do you've lost them.
The second rule of writing with AI is that AI encouragement is toxic. A structural consequence of RL is that models are exquisitely tuned to generate responses that make their users perceive value. We recognize this in a gross sense in "sycophancy", but the problem recurs fractally in at finer-grained levels, where stuff like "this part is really strong" will subtly allow the model to set a course for your writing and you'll confidently ship crap.
With those two rules in mind, models are incredibly valuable for writing, more valuable in my experience than the professional copywriters I've worked with. The trick is to get them to make suggestions at a higher level than just writing alternatives:
* Do the sentences in these paragraphs end with the new idea or information?
* Are the real actors in each sentence the grammatical subjects?
* From paragraph to paragraph is there a clear flow of topics, or are things jumping around?
* Is this piece crudded up with metadiscourse like "it's important to note"?
I've had a stack of notecards for ages that I took down from Joseph Williams "Style: Towards Clarity And Grace", the most programmer-brained writing book ever written, I love it very much. For the past year or so I've been feeding them through GPT and Claude one by one, and it's drastically increased the speed at which I can knock out a completed piece.
I think it's pretty hard to argue that AI isn't going to have an impact on the writing profession. It's just not the most obvious impact everyone assumes it will have, where it, like, writes whole op-eds or whatever. At least not yet.
I was thinking about this yesterday, while going through some slop documentation generated by deepseek. If you compare claude fable 5, side by side with the deepseek, the differences in prose are glaring. It's not even close. Deepseek prose reads like fragmented shorthand, where claude fable 5 comes fairly close to human, certainly not superior to human quality writing.
I watched a podcast with a cognitive scientist and one of main contributors to the theory of linguistic relativity, Lera Boroditsky.
She said something to the effect that, "in this very moment, we are speaking in ways that were never spoken before. We are saying things that no other person has said before...."
Language models are not sample efficient and cannot adapt to evolving language, unless it's documented in large amounts of examples.
So whatever isn't documented, whatever isn't in the training dataset or the rag corpus, the model will always be incredibly different in expression from humans.
I recently have been trying out frontier models on writing. Just for laughs, to bring an idea I have into the world acting more as director than author. I have a multi agent setup and am getting different models to argue about rating the story for clarity and whether the plot is complete.
I agree with the article. Even the frontier models call out that all the characters tend to sound the same. It also started at some point making huge changes to the core premise, and also adding characters willy nilly. The issues it called out with the plot (the ones it actually consulted me on) also made me realize how terrible of a writer I actually am.
Overall, it has been an interesting experience. I look forward to reading my own book!
I understand a lot better now why people are bemoaning KDP being filled with absolute garbage AI slop.
> ... and model checkers can instantly catch errors.
Lol. If that were true software would've been a lot better historically... Model checkers don't scale to 90% of the software we write. Typically you have to 1) heavily abstract the program and 2) put it in some sort of harness to specify how you want to model the outside world (which will always fall short of practice). And probably 10 other workarounds since most model checkers are Research Grade Software™. Not saying they're not tremendously useful, but that bullet doesn't hold up at all.
"Humans are going to be significantly better at writing things intended for other humans to consume" is completely at odds with "And organizations care enough about the gap, and are willing to pay for the difference".
This feels premature. The thing about the LLMs people use today is that it's a handful of super expensively trained models serving 1000s of use cases ranging from frontier math to recipe planning.
We get to learn the foibles and language of Claude and ChatGPT as a result. The slop is almost detectable if you provide no steering prompts about story structure, narrative structure, or stylistic cues. And most writers are not finetuning the weights to their LLMs explicitly.
If you invest time into doing all that (not really trivial stuff), the results will be better.
The safest jobs are the ones that can't be done behind a screen. I'm a technician and there's no robot that could replace me and there won't be for a long time.
You can't make 1000+ trash robots just to produce one that works okay sometimes and somehow make the economics work from that, so I very much doubt it.
Edited to say "I think". I think that because I have been looking hard for developments in humanoid robotics generalization and new impressive demos and capabilities keep popping up all the time.
For example the recent Skild AI fairly general imitation capability. It really is a huge amount of effort on increasing the generality of humanoid robots going on now and a lot of demonstrations coming out showing off progress.
With the current trajectory there is no reason to think it will stop soon. There is a lot of motivation to innovate and train and it is pushing things forward.
If you're not looking for evidence of improvement, or rather looking for failure cases, then you're not going to see that trajectory.
And physical damage is lot more expensive to repair in vast majority of use cases. Have fun when your humanoid window washing robot punches through your window. Or any other such cases.
Except that in the worst-case AI scenarios where a huge portion of the professionals who work "behind a screen" lose their jobs, the entire labor market will have to deal with that.
There's so much focus in the US on vocations but Americans have no idea how little mechanics, plumbers, etc. in other countries make. There are a variety of reasons for this but one of the biggest is that you can't charge $200/hour to fix a leak when $200 represents a double-digit percentage of a worker's monthly salary.
The reason LLMs use the same cadence and cliches is that their RLHF does not emphasize writing well, the way it emphasizes, say, coding. And the reason for that should be pretty obvious: coding is where the big money is at.
So the author may be correct, but for a different reason: unless writing starts being very valuable as a profession, it's unlikely the labs will spend significant resources making their AI models better at it.
No, they use "the same cadence and cliches" because they inflate a short and ambiguous prompt into long and specific prose by making statistical assumptions about what best fills in the gaps. It's not a training problem, it's an information theory problem, and it's not really surmountable.
Any given model will always have some distinct implicit voice that its biased towards for that infill content, and so a popular model will always become exhaustingly common, painfully familiar, and cliche. Users can use more elaborate prompts that shift the voice away from the most normative and towards some other nodes, but they need to put in special effort for that, and what people-at-scale specifically want from these tools is to put in very little effort, so we can expect that overwhelming number of casual and naive users will always be generating cliche slop with them.
Code escapes this problem not because of training but because it specifically benefits from cliche (boilerplate, patterns, etc) and so an model whose code "voice" reflects your own taste as a coder (or your toolchain's taste as a vibecoder) is going to feel like productive output rather than slop. But it's still cliche.
No. Reinforcement Learning is doing a lot here. Anyone who played with these models before the Davinci intstruct-tuning (completion) era can tell you the same. In some ways, SOTA models have gotten better at writing, but the neuroticism of instruct-tuning has still not been resolved.
"The reason LLMs use the same cadence and cliches is that their RLHF does not emphasize writing well, the way it emphasizes, say, coding. And the reason for that should be pretty obvious: coding is where the big money is at."
Software can be checked for being 'written well' by compilers / linters etc. There is no equivalent for well-written natural prose. Spelling and grammar checkers haven't a clue about prose semantics.
They just haven't got to it yet. It's not a high priority but eventually they will make a solid effort on easier style guidance and more humanization. They will probably drop the most worn ou constructs like "not X, not Y, but Z" that have become tells and target a lot of known issues in reinforcement training.
It's slightly weird how confident writers are that it won't get improved.
What's interesting is that we still read AI's output and that's still the primary interface between a human and the AI. Implicitly that means that writing is important and I think we're already starting to see this with the complaints around how Claude "talks"
Claude "talk" is particularly annoying if you're using Claude to perform a job function (write code, analyze a spreadsheet, etc.) but average people reading an ad, Facebook post, etc. don't really care from what I can tell. In fact, I think most of them have no idea what Claude "talk" is, even if some of them notice common patterns that have emerged recently.
I think it's likely to be lawyers who are the safest. Lawyers are the ones primarily in charge of the government, and I think it's likely that as soon as they feel like the legal profession is threatened, they will pass laws making it illegal to use AI for legal issues. It's not all sunshine and roses (paralegals are probably hosed), but the lawyers seem like they'll do ok.
Unless the model watermarks text, it will become increasingly difficult to tell the difference between human legalese and AI legalese. Especially if they make it illegal, that will create a strong incentive to find-tune to imitate human lawyers more convincingly.
Eh, I think we're going to run into the same issues we're seeing in software. That is, junior and lower status positions (so SDETs in software, paralegals in law) are rapidly being downsized, and that's going to have a big issue in the future where there isn't the same talent funnel. And then in the future those that work their way up will basically be dependent on working with AI.
It is great time to start business as downsized paralegal. While ago there was an app that objected parking ticket fines in NYC. Anyone can find similar niche to specialise in, and just produce boring repetitive law work using LLM templates and mostly automated work.
> I think it's likely to be lawyers who are the safest. Lawyers are the ones primarily in charge of the government
Given what seems like an increasingly inevitable deprecation of these outdated, lumbering nation-states, it seems to me that these two assertions are mutually exclusive.
> Since LLMs lack an active mental model of a specific human reader... They cannot empathize with the human reader, as they don't have the human lived experience. And they have zero skin in the game.
LLM is perfectly capable of empathy, it just never told to do so.
Most writers today lack empathy and have no lived experience. Young californian uni graduates have strong opinions on everything, but produce repetitive boring preachy cringe stuff.
I will take well prompted LLM generated writing anytime over thst!
I don't think there's any chance people will be employed to sell the output of their brains in 20 years, outside of "athletes", or whatever we end up calling people who think for sport. We mostly succeeded at doing this with muscles, and today there's no market for hauling rocks out of quarries manually. We're in the process of doing that for brains too.
AI will outdo people in all practical uses. We're already there for debugging and getting very close for coding, and we're in the middle of the largest investment in human history to expand that to everything else.
If we do a good job of alignment, AI will treat people like those cats "in charge" of train stations in Japan: our every need will be accommodated, but we won't be controlling things we don't understand.
Of course. But they're going to be machines. I don't think there's magic that makes the human brain impossible to automate. And engineering is going to lead to faster improvements than evolution could.
No. Below is my emotional opinion based on my own experience:
Currently the models are totally helpless with plot and emotions.
They make epistemic, logistical and temporal mistakes.
But:
They are very good in finding _your_ epistemic, logistical and temporal mistakes. They are great as adversarial reviewers. They can validate grounding. They can test voices. They can build complex parallel constructions representing timestamped inner monologues, dialogues and events.
A process consisting of many models representing characters + states + epistemic isolation + reviewers could be great to validate or disprove ideas.
Models can write you formal models of your writing. They can run solvers against your formal models to - again - validate and disprove ideas. Literally, you could validate your writing with TLA+, SAT solvers, queries against formal ontologies.
They can build corkboards of unimaginable complexity.
A moderately good writer who plays well as a part of human-machine writing apparatus could be a competitor even for a top-tier author.
When you write with a model, you define disciplines. You write your prose. You let your model turn your prose into formal systems and verify it. You let your model criticize you. Sometimes you let your model to polish your crude language into something sleek.
So, at current state of LLM development, writer as source of ideas and governing process is safe. Writer as a vision-expansion machine is not.
I've been very hostile to LLMs in literature. I've started to write a short novel about that. As a joke I tried to use models after half of the text was done. I've applied my engineering skills. That changed both me and my opinion. Models are awesome if you know their advantages and weaknesses.
I have a feeling the author framed this the wrong way.
my take is - the author wanted to express that there's always a demand for human prose / writing that captures the subtleties of expression, thought & ideas. which is very true. & we can already see this e.g by people opting out of LinkedIn for it's A.I driven long posts. whereas engagement is high on X where posts are likely to be human generated.
I wouldn't say writing as a job is protected - as corporations will always take shortcuts.
To the contrary: Twitter has the lowest content engagement rates out of any of the networks.
For now. And that is because humans are the only customers engaging with content. As agentic AI takes hold and is given credit card authority, bots will will engage. Bots will read the posts of other bots and use those when making purchase decisions for thier humans. If we are lucky, the bots will develop thier own twitter-specific language and we humans can finally wave goodbye to tweets as a concept. The bots will play the twitter game for us.
There may well be some writing jobs that are safe for the reasons given in this post, but that doesn't help all the writers I know who have already lost their jobs and are struggling to find work.
It's not enough that humans can tell the difference and feel an ick, there also need to be enough organizations willing to pay money for that difference. From my vantage point, there are not. It turns out that for a ton of the writing produced by companies, the quality of the prose wasn't really "load-bearing" as Claude puts it. That writing is there to occupy a space and look professional at a glance, the same way elevator music is tolerable for the duration of an elevator ride.
I have worked in the publishing industry, translation and editing (got out to a more dependable career just in time), and what is astonishing is how even "load-bearing" text is not safe from the cost-cutting pressures. That is, even some respectable publishers for the last several years will publish a book one is expected to pay good money for, with minimal editing and proofreading compared to the old days, and if it is an originally foreign book, with machine translation and minimal post-editing.
I've got some heavy reader friends and they keep complaining that many of Spains's fantasy book translations are ridiculously bad and blatantly machine translated now: Senseless terms that require looking up the original to understand, e.g. "fall" translated as in falling when it meant autumn; proper nouns, including character names, sometimes translated sometimes not; and many sentences that mix up the grammatical gender because the machine wasn't given enough context to infer it.
But these publishers don't get punished because they keep licensing popular foreign franchises, which aren't quite fungible, so consumers don't want to miss out and thus don't vote with their wallets.
A lot of the replies are insisting AI will get better at writing with more development but I don't see it. Even if you have a mathematically perfect writing AI you still run into the same problems you would have if you handed off your writing task to someone on fiverr or something. It can't magically know what you want to say, it only has the information you gave it. A prompt complex enough where it won't get any wrong ideas has to contain as much information as the output would have.. so just write it.
I also believe this. Post-training LLMs with vague metrics can only be achieved with RLHF, which is not impossible, but extremely costly and difficult. Instead, companies will opt for RLVR, focusing on math and programming tasks. This pushes objectives away from writing quality; often far away. That is why older models, in my view, actually read better than newer ones. It's by design.
One of the biggest lessons in life to learn is there are no shortcuts.
Doesn’t stop people trying.
> It can't magically know what you want to say
I think for this argument to be true, the axiom that supports it is that the models have just as much context as they will ever have, and you cannot see being able to give them more / enough to be able to understand your perspective. That feels unlikely to be a position that doesn't change. As a society we're giving more and more context each day to this, and that makes this a valid opinion now, but one that erodes over time.
its not about dumping more and more info into the context, its about the intention. whats not in the context is just as important as what is. and i dont see how that can be automated.
also were seeing models become worse at writing as they get smarter.
Context went from 8,192 tokens on GPT 4 to 1M tokens currently with zero improvement. The latest models got even worse.
Size of context is not the entire story here, it's ability to properly feed and index the context that's needed on this sort of thing. E.g. your entire slack/discord/email/github/jira/zoom meeting/coffee chat ... history is the context that you bring to the table on this sort of thing. Most of this is unindexed. Much of this will not be in the future.
> The latest models got even worse.
Which models? This is one of those things that likely has both model and domain specific aspects that impact your experience. In my experience with OpenaAI models predominantly (I previously worked there), they've improved significantly over the last 6-12 months. My experience with Claude is worse, but I haven't spent as much time getting into a mechanical sympathy there. They're still not perfect though and I have many steering docs that help avoid the biggest problems in the models I use when generating docs.
Claude writing quality got unbelievably bad with Opus 4.7, with no improvement in Fable. Opus 4.6 was fine. Im starting to see it as a security risk - my brain just can't process its word vomit, so just tell it to go on, implement whatever
Workaday copywriting is dead (was moribund, now has been shot in the head). Prestige literary writing is zero-sum and therefore eternal, in the same way that equities trading (not formation and not business-building) is zero-sum and therefore the actual success of LLM has not given anyone a signal advantage anywhere there, because everyone else has LLM too.
Copywriting struggles with the issue that businesses know they need it, but they don't value it. It's absolutely shocking to see how poorly companies, and governments, communicate and no amount of AI/LLM usage will change it. For decades it's been neglected and relegated to "cover my ass"- writing.
The majority of AI deployment in businesses could have been avoided, if more attention had been placed on good communication. LLMs don't yield better communication or corporate writing, just more of it, because no one in business seems to understand the value or have the ability to recognize good writing.
Equity trading is not zero sum. There are real businesses and assets behind the equities they represent that have actual value and produce actual money.
The business-building is not zero sum, but the trading is zero-sum, and the societal purpose of the thing is to provide liquidity
theoretically equity trading is about distributing stonks most optimally so that productive companies get more and less productive ones get less, improving productivity of society as a whole
Derivatives (options, futures, swaps, etc) are a zero sum market, there is always a winner and a loser necessarily because the contracts have an expiration date.
The same is not true for trading equities, they are not zero sum. There are dividends, buybacks, companies will sometimes spin off a part or parts as separate companies that you get newly issued shares of stock from (GE splitting into parts is a recent example), public companies get taken private at a premium to the market price, etc.
Futures are zero sum, but can also be a win for both sides, because what they gain is stability.
"I will buy y tons of corn from you in April for £x" - now I don't have to worry about how much my corn will cost and you don't have to worry about what your income will be.
The market has been heavily engineered and gamed to be positive sum. But sometimes I wonder could it turn out to be negative sum on certain timescales. That is that overall money would be lost on markets.
This is more or less what Ray Dalio was getting at with The Changing World Order, IMO.
Presumably they mean HFT / "locals" / "sell side."
These folks compete in a reasonably zero-sum game.
They're market makers who provide liquidity. So it's not exactly zero-sum.
Writing has an audience, purpose, constraints, and desired outcome. Creating tests for writing is more difficult than creating a unit test, but it's not undefinable. It's just engineering.
Has writing ever been a safe job, even before AI?
If you're good at it, which extremely few people are. It's my belief that not enough good writing exists for the tech companies to train an LLM. At least not a generally applicable on.
Tech companies are training off the shadow libraries, which have the bulk of English-language human writing in book form from the twentieth century, and that traditionally published stuff with an ISBN usually got a proper editor of some kind. So, it is hard to believe LLM companies don't have enough material; it's not as if they are limited to low-quality stuff on the web.
If you are Steven King, sure.
From what I heard it's a bloodbath at the bottom.
Exactly. And it's not only about whether a specific job is safe from AI, but whether it's safe from the effects of those that aren't safe: competition from laid off people and downward wage pressure. Then there's the backpressure into the educational pipeline.
Similar, much narrower crises were managed for manufacturing jobs during the worldwide industrialization era. The solution for those displaced by trade was retraining.
Unfortunately, at the moment we have no clue of what workers should be retrained to. Not writing.
It's a pretty reliable career for hundreds if not thousands of mid-list authors who generally stick to genre fiction. It's entirely possible that AI could core out the middle of the pack as well as the bottom, which would threaten not just jobs but a wealth of institutional knowledge that those authors share among their peers and especially new up-and-comers.
If I found the best human writer in the world, and had him write every article on the internet, wiping his memory between each one, his writing would start to get pretty noticeable and boring!
I agree. LLMs write really well, but probably too consistently in terms of style. That said, I did recently ask AI to do an editing pass over some writing I did, and I was surprised that it missed a tense mismatch that I had (not within a sentence, but within a paragraph).
The real thing being said here is that the author can tell good writing from bad, and assumes everyone can... I'm a visual artist and at generative art is blindingly obvious.... To me.... But not to many folks around me! Including a few artists and art adjacent folks.
It extends well beyond creative stuff too. It makes me think of Gell-Man amnesia [1].
People can tell when something they are experts in is being done poorly, but others can't, and it frustrates the experts. Then, those same people think something completely different is being done well despite what experts in that respective field say. It's like when someone from your family reads an article about your profession and then proceeds to tell you how your job works. lol
It's a really pervasive issue in society IMO. And a few prompts in someone's favorite LLM just reinforces it to people that don't know (any better|what they don't know).
[1] https://en.wiktionary.org/wiki/Gell-Mann_Amnesia_effect
> generative art is blindingly obvious.... To me
Have you tested this systematically, or is it possible that you are experiencing survivorship bias? If there were any generative art pieces that you didn't notice, you would have thought that they were human-made. Therefore, all the pieces you identified were "obvious" to you. Not to mention false positives.
I imagine that they haven't done a peer-reviewed scientific study of their ability to notice generative art.
We can replace their statement by "I'm a visual artist and at generative art is often blindingly obvious.... To me....". In other words, they can notice some or most generative art but other people familiar with art can't, or at least can't as often as they do. I think what they are trying to say isn't too changed by that.
Even if it’s obvious to some at this time, as time passes artificially generated writing will become mainstream and what becomes the normal writing style. If you read IX century literature, it reads differently, the vocab is different, the sentences are built differently. It was totally normal back then. It’s not normal now. In ten years the new artificial style will become commonplace and accepted -I mean unless we artificially always make it deviate.
LLM prose has degraded with each model update, the models are now being RL'd into wall of texts that only makes sense to other agents. I think its going to become more obvious as we move forward that is llm generated because labs seem to only be focused on tool-calling/agentic-coding environments, as that's the only thing that drives revenue.
There may come players who focus on models that are good at writing for technical writing/docs , copyrighting ect but I think people will lean towards not using them and will rather have the "human touch" for the things that directly impact brand perception.
Keep in mind, every single AI company that is selling the idea that you don't need to hire designers and web design is "solved" have $100k retainer designers crafting their landing pages.
Yeah, this tracks with my experiments. About every 6 months or so for the past year-and-a-half I've tried using the "frontier" LLMs to as-near-as-possible autonomously write novel length stories, because I find it fascinating.
When I had Gemini 2.5 write a novel, it wasn't really objectively "good" by any stretch of the imagination, but while the prose was very purple and full of cliches and, well, bad writing I guess, it still felt ... subjectively good, at least for what it was.
Last week I did a run with GPT-5.6, and wow. On the one hand, it managed to produce 110,000 words that were "shockingly" coherent. The model was able to maintain state and plot lines and background details extremely well, much better than older models.
But I just don't like the prose. I haven't really liked _any_ prose that GPT-5.6 produces. It's significantly better at "instruction following" and keeping track of things, but, wow.
> “The sequence is consistent with their voluntary choices.” Mara enlarged the uncertainty field rather than the result. “It does not prove what happened to anyone we can’t observe. It does not prove contact did this. And it does not turn the Shard into treatment.”
GPT-5.6 in particular becomes so fixated on certain ideas like "consent" and epistemology, that by the end of the narrative, the prose and dialogue are all just "agent speech", despite the prompt/harness specifying that it's a _novel_ with narrative prose and such.
Interestingly, the model itself produces an accurate critique of its own output:
> The draft has become a *consent-centered medical, legal, and logistical procedural*. The important drift is therefore not that many events were omitted. It is that the retained events now prove a different thesis.
Which begs the question of if it would do better with a couple rounds of output -> critique -> revision. But I think I've had enough LLM prose for a bit...
I'm curious if you've tried Claude. Subjectively, I've always preferred Claude's writing over ChatGPT and Gemini.
Claude is the one model family I've not really used. Which yeah, feels like a backwards thing to say in a world where seemingly everyone using LLMs is using Claude Code.
I've used some Opus 4.5/4.6 via Antigravity and Sonnet by the web chat. I'm torn because as far as LLMs go, it does feel more ... "literate".
But maybe too literate, judging by how many people are complaining about "Claudeisms". I suspect Claude would be just as susceptible, if not more, to the sort of ... "moralizing" that GPT seems to gravitate towards (for lack of a better term).
Brother, AI was already a threat to writing before LLM arrived
Mmm, I doubt it.
I have a weird background (product, development, writing + devrel). I write a lot of code and a lot of articles.
I think there is a lot of overlap in how people who write code or articles (documentation, books, etc.) use AI. On one side of the spectrum, you have people who just blindly input some prompt, accept the output and move on with their lives. You can likely predict how that is going for them (not great). On the other side of the spectrum, you have people who outright reject all AI and are continuing to plod on with how they have always done things.
In the center is a more reasonable approach that leverages AI to create without blindly accepting the output. This applies very much to writing.
The workflow that I've adopted over the past two years or so has been to leverage AI to help with the research and outline process. Once I'm happy with the structure I go and I write what I need to write.
This maps pretty closely to the code that I write. It's fine.
Safest job from AI is also the oldest job.
Machine sex and other stories by Candas Jane Dorsey - https://archive.org/details/machinesexothers0000dors/page/76...
If we're doing book recommendations, I quite recommend the essays in https://academic.oup.com/mit-press-scholarship-online/book/1... and particularly the philosophical discussion there of whether "robot sex" is an oxymoron, as a robot almost by definition cannot dissent, so it might only be "robot-assisted masturbation"
For any wondering, that book is also available on internet archive.
https://archive.org/details/robotsexsocialet0000unse
That's not obvious at all, particularly given the myriad movies trying to scare us about the dangers of robots taking over that.
Mostly because selling and/or using it is illegal or morally offensive enough to be penalized by your community if too many people know.
If you thought all of the money being poured into data centers was excessive, wait until people can order up a proto-sentient sex bot to their front door. Oh boy.
Of course, stone mason
I think there is good chance that AIs have already replaced parts of the certain aspects of industry. And with some tuning could probably replace lot more of at least virtual parts...
The safest jobs are the ones that don't feed a training algorithm with data, any work that remains more of a mystery.
Yeah I had GPT5.6 Sol act as an editor on my new chapter and it did find decent issues with PoV and narrative distance but it was so bad at prose. The best way I can describe it is, it was too clean. Often times humans write about things unsaid and left for reader to interpret or deliberate awkward sentences to tease character psychology. The AI just straight up corrects these without second thought. This will likely be the case with general purpose models unless we see some genre specific fine tunes.
> This will likely be the case with general purpose models unless we see some genre specific fine tunes.
Are we at the point in the hype cycle to go "there's a skill for that" yet?
I don't think writing is an "AI-complete" problem. It's just that the models are inefficient at it right now, due to training and/or architecture. Good writing requires thinking, reflection, and doing multiple passes. This currently translates to using a high reasoning effort and burning lots of tokens, but big AI labs like Anthropic are struggling with handling the load, so they're doing the exact opposite and finding whatever cheap trick they can to reduce token usage. One such trick is condensing ideas into as few tokens as possible, which in my opinion is one of the reasons Claude sucks so much at writing.
> Good writing requires thinking, reflection, and doing multiple passes
Same with code, but that doesn't seem to matter anymore.
There's a lot of writing that is meant to be read by humans.
Code's ultimate goal, on the other hand, is to be run by computers.
I highly disagree. Code is still meant to be and still is read more by humans.
That's because it's cheaper to automate than pay a software engineer. The economics has to make sense. Writing is tricky. Prose quality isn't there yet for many genres. You can tell AI written prose apart. Maybe it won't matter. Maybe there is demand for it even. Or Maybe we will create genre specific finetunes if there is enough demand for it.
> That's because it's cheaper to automate than pay a software engineer
For now. It also seems like we're still paying software engineers.
Even safer jobs might be the ones don't get automated because nobody really cares about, e.g. fishing lobster on a boat
First, I actually had the author of this post as my distributed systems professor back in college. He's not just a super smart dude on a technical level, but he's probably the only CS professor I've ever had who seems to appreciate writing in itself (which may be evident from his blog). I think the combination of these two qualities puts him in a good spot to make the claims he is.
In terms of what he's presenting here, I'll say this: his line of logic highlighting the "dual-mind problem" of writing seems, to me, like the most compelling aspect of this argument. When I write something (especially to a specific audience, including an individual), I must practice cognitive empathy if I want what I write to be consumed properly and effectively. This is a cold way of putting it, but even in text message responses which span a few words towards family or a friend, I can put quite a bit of thought into how it will be received, how they will read it, interpret it, etc. My cadence in just texting, alone, can shift dramatically from one message to a next based on who I'm sending it to, what I'm trying to convey, etc.
All of that is fine and not hard to understand, but I suppose the interesting part is this: when I'm just trying to get information across (i.e., instructions, directions, etc.), my messages will resemble something written by an AI. It's not enjoyable to consume, but that's not the point. But as soon as I want to add a bit of "fun" to a message, the task I'm performing is completely different. It's no longer just an exchange of information, but it's an attempt to invoke specific feelings, visualizations, memories, etc., in the other person/people.
I do have a hard time imagining what it will take for AIs to be able to do THAT effectively. I think they're fine at conveying information, but I think people are already becoming very aware that conveying information, in itself, is not enough to be effective. These LLMs don't "care" to "entertain" you with what they're writing at you about. They're just spitting out the mathematically derived facts with, seemingly, no meaningful ability to invoke deeper thoughts in their audiences' minds. And that might not seem particularly important outside the context of writing fiction, etc., but I think it's actually pretty critical even in "dry" settings, like explaining code, because the thoughts, feelings, emotions, etc., invoked through reading IS the output of reading (even if it doesn't seem that way).
Perhaps AI will be able to do this more effectively if they're trained on direct brain signals or something. Like, train the AI not just to convey accurate information, but also encourage it to do so in a way which stimulates different areas of the brain with different intensities based on the premise that, doing so, is actually what people are getting out of that text.
Go on to Ao3 and a popular fandom, say like A Song of Ice and Fire: 9 out of 10 fanfictions written in the past two years stink to high heavens of AI. 9 out of 10. From some that are obvious AI to even normies, to some that slip one or two ai tropes after a couple of chapters, because the author got lazy and stopped editing carefully.
It's sad, infuriating, discouraging. A deluge of slop drowning the last embers of authentic human creativity.
I appreciate what this piece is trying to do directionally but it shares a logical flaw with lots of other pieces about AI and knowledge work. It assumes that in order to disrupt a particular field, AI needs to print serviceable work unassisted; that it's "vibe-shipping".
AI today is most effective when it's not vibing, but rather copiloting a skilled operator. I don't necessarily want my agent to build an entire system for me, even if it's ultimately the author of almost every line of code; I'm actively making decisions throughout. There's obviously a spectrum here but at most points on the spectrum the amount of assistance available is still a step change in the economics.
So too with writing.
The first rule of accelerating writing with AI is that you're not allowed to use a single word the AI suggests. Even if what the model comes up with is great, better than what you could have done, as soon as the AI suggests it it's poisoned. At least with current models, readers can detect LLM prose in the parts per trillion, and as soon as they do you've lost them.
The second rule of writing with AI is that AI encouragement is toxic. A structural consequence of RL is that models are exquisitely tuned to generate responses that make their users perceive value. We recognize this in a gross sense in "sycophancy", but the problem recurs fractally in at finer-grained levels, where stuff like "this part is really strong" will subtly allow the model to set a course for your writing and you'll confidently ship crap.
With those two rules in mind, models are incredibly valuable for writing, more valuable in my experience than the professional copywriters I've worked with. The trick is to get them to make suggestions at a higher level than just writing alternatives:
* Do the sentences in these paragraphs end with the new idea or information?
* Are the real actors in each sentence the grammatical subjects?
* From paragraph to paragraph is there a clear flow of topics, or are things jumping around?
* Is this piece crudded up with metadiscourse like "it's important to note"?
I've had a stack of notecards for ages that I took down from Joseph Williams "Style: Towards Clarity And Grace", the most programmer-brained writing book ever written, I love it very much. For the past year or so I've been feeding them through GPT and Claude one by one, and it's drastically increased the speed at which I can knock out a completed piece.
I think it's pretty hard to argue that AI isn't going to have an impact on the writing profession. It's just not the most obvious impact everyone assumes it will have, where it, like, writes whole op-eds or whatever. At least not yet.
I was thinking about this yesterday, while going through some slop documentation generated by deepseek. If you compare claude fable 5, side by side with the deepseek, the differences in prose are glaring. It's not even close. Deepseek prose reads like fragmented shorthand, where claude fable 5 comes fairly close to human, certainly not superior to human quality writing.
I watched a podcast with a cognitive scientist and one of main contributors to the theory of linguistic relativity, Lera Boroditsky.
She said something to the effect that, "in this very moment, we are speaking in ways that were never spoken before. We are saying things that no other person has said before...."
Language models are not sample efficient and cannot adapt to evolving language, unless it's documented in large amounts of examples.
So whatever isn't documented, whatever isn't in the training dataset or the rag corpus, the model will always be incredibly different in expression from humans.
I recently have been trying out frontier models on writing. Just for laughs, to bring an idea I have into the world acting more as director than author. I have a multi agent setup and am getting different models to argue about rating the story for clarity and whether the plot is complete.
I agree with the article. Even the frontier models call out that all the characters tend to sound the same. It also started at some point making huge changes to the core premise, and also adding characters willy nilly. The issues it called out with the plot (the ones it actually consulted me on) also made me realize how terrible of a writer I actually am.
Overall, it has been an interesting experience. I look forward to reading my own book!
I understand a lot better now why people are bemoaning KDP being filled with absolute garbage AI slop.
> ... and model checkers can instantly catch errors.
Lol. If that were true software would've been a lot better historically... Model checkers don't scale to 90% of the software we write. Typically you have to 1) heavily abstract the program and 2) put it in some sort of harness to specify how you want to model the outside world (which will always fall short of practice). And probably 10 other workarounds since most model checkers are Research Grade Software™. Not saying they're not tremendously useful, but that bullet doesn't hold up at all.
"Humans are going to be significantly better at writing things intended for other humans to consume" is completely at odds with "And organizations care enough about the gap, and are willing to pay for the difference".
No, that's wishful thinking.
This feels premature. The thing about the LLMs people use today is that it's a handful of super expensively trained models serving 1000s of use cases ranging from frontier math to recipe planning.
We get to learn the foibles and language of Claude and ChatGPT as a result. The slop is almost detectable if you provide no steering prompts about story structure, narrative structure, or stylistic cues. And most writers are not finetuning the weights to their LLMs explicitly.
If you invest time into doing all that (not really trivial stuff), the results will be better.
The safest jobs are the ones that can't be done behind a screen. I'm a technician and there's no robot that could replace me and there won't be for a long time.
I think the ChatGPT moment for humanoid robotics is coming within a year or two.
You can't make 1000+ trash robots just to produce one that works okay sometimes and somehow make the economics work from that, so I very much doubt it.
I think you're off by at least an order of magnitude.
So was ChatGPT back in 2021.
> The ChatGPT moment for humanoid robotics is coming within a year or two.
How long after that moment did the ChatGPT robots need to invent the time travel machine which let you come back to 2026 to confidently claim this?
Edited to say "I think". I think that because I have been looking hard for developments in humanoid robotics generalization and new impressive demos and capabilities keep popping up all the time.
For example the recent Skild AI fairly general imitation capability. It really is a huge amount of effort on increasing the generality of humanoid robots going on now and a lot of demonstrations coming out showing off progress.
With the current trajectory there is no reason to think it will stop soon. There is a lot of motivation to innovate and train and it is pushing things forward.
If you're not looking for evidence of improvement, or rather looking for failure cases, then you're not going to see that trajectory.
Robotics cost way more to produce and support, unlike software
And physical damage is lot more expensive to repair in vast majority of use cases. Have fun when your humanoid window washing robot punches through your window. Or any other such cases.
When a robot can figure out how to contort it's body to climb inside a machine to replace an encoder, I'll start to worry.
Except that in the worst-case AI scenarios where a huge portion of the professionals who work "behind a screen" lose their jobs, the entire labor market will have to deal with that.
There's so much focus in the US on vocations but Americans have no idea how little mechanics, plumbers, etc. in other countries make. There are a variety of reasons for this but one of the biggest is that you can't charge $200/hour to fix a leak when $200 represents a double-digit percentage of a worker's monthly salary.
The reason LLMs use the same cadence and cliches is that their RLHF does not emphasize writing well, the way it emphasizes, say, coding. And the reason for that should be pretty obvious: coding is where the big money is at.
So the author may be correct, but for a different reason: unless writing starts being very valuable as a profession, it's unlikely the labs will spend significant resources making their AI models better at it.
No, they use "the same cadence and cliches" because they inflate a short and ambiguous prompt into long and specific prose by making statistical assumptions about what best fills in the gaps. It's not a training problem, it's an information theory problem, and it's not really surmountable.
Any given model will always have some distinct implicit voice that its biased towards for that infill content, and so a popular model will always become exhaustingly common, painfully familiar, and cliche. Users can use more elaborate prompts that shift the voice away from the most normative and towards some other nodes, but they need to put in special effort for that, and what people-at-scale specifically want from these tools is to put in very little effort, so we can expect that overwhelming number of casual and naive users will always be generating cliche slop with them.
Code escapes this problem not because of training but because it specifically benefits from cliche (boilerplate, patterns, etc) and so an model whose code "voice" reflects your own taste as a coder (or your toolchain's taste as a vibecoder) is going to feel like productive output rather than slop. But it's still cliche.
No. Reinforcement Learning is doing a lot here. Anyone who played with these models before the Davinci intstruct-tuning (completion) era can tell you the same. In some ways, SOTA models have gotten better at writing, but the neuroticism of instruct-tuning has still not been resolved.
"The reason LLMs use the same cadence and cliches is that their RLHF does not emphasize writing well, the way it emphasizes, say, coding. And the reason for that should be pretty obvious: coding is where the big money is at."
Software can be checked for being 'written well' by compilers / linters etc. There is no equivalent for well-written natural prose. Spelling and grammar checkers haven't a clue about prose semantics.
They can use human writers and also LLM-based multi-step scoring systems with analytical guidelines.
They just haven't got to it yet. It's not a high priority but eventually they will make a solid effort on easier style guidance and more humanization. They will probably drop the most worn ou constructs like "not X, not Y, but Z" that have become tells and target a lot of known issues in reinforcement training.
It's slightly weird how confident writers are that it won't get improved.
What's interesting is that we still read AI's output and that's still the primary interface between a human and the AI. Implicitly that means that writing is important and I think we're already starting to see this with the complaints around how Claude "talks"
Claude "talk" is particularly annoying if you're using Claude to perform a job function (write code, analyze a spreadsheet, etc.) but average people reading an ad, Facebook post, etc. don't really care from what I can tell. In fact, I think most of them have no idea what Claude "talk" is, even if some of them notice common patterns that have emerged recently.
I think it's likely to be lawyers who are the safest. Lawyers are the ones primarily in charge of the government, and I think it's likely that as soon as they feel like the legal profession is threatened, they will pass laws making it illegal to use AI for legal issues. It's not all sunshine and roses (paralegals are probably hosed), but the lawyers seem like they'll do ok.
If that was the logic then software engineers would stop using AI to "protect their jobs" but that hasn't been the case.
There will always be people that work against their own profession and colleagues for short term gains
I know of a few software engineers who have stopped using AI beyond using it selectively as a search / troubleshooting / stack overflow replacement.
If you are taking steps to protect your brain maybe in a way you are also protecting your job?
Who knows how this all turns out years from now.
I am one of those engineers. I just don't want to be dependent on it. I want to still do the work and learn how to be a better programmer
Unless the model watermarks text, it will become increasingly difficult to tell the difference between human legalese and AI legalese. Especially if they make it illegal, that will create a strong incentive to find-tune to imitate human lawyers more convincingly.
Eh, I think we're going to run into the same issues we're seeing in software. That is, junior and lower status positions (so SDETs in software, paralegals in law) are rapidly being downsized, and that's going to have a big issue in the future where there isn't the same talent funnel. And then in the future those that work their way up will basically be dependent on working with AI.
It is great time to start business as downsized paralegal. While ago there was an app that objected parking ticket fines in NYC. Anyone can find similar niche to specialise in, and just produce boring repetitive law work using LLM templates and mostly automated work.
> I think it's likely to be lawyers who are the safest. Lawyers are the ones primarily in charge of the government
Given what seems like an increasingly inevitable deprecation of these outdated, lumbering nation-states, it seems to me that these two assertions are mutually exclusive.
It’s orating.
Yes write, but then orate.
Look at all of our AI feeds.
They want to be us so bad.
> Since LLMs lack an active mental model of a specific human reader... They cannot empathize with the human reader, as they don't have the human lived experience. And they have zero skin in the game.
LLM is perfectly capable of empathy, it just never told to do so.
Most writers today lack empathy and have no lived experience. Young californian uni graduates have strong opinions on everything, but produce repetitive boring preachy cringe stuff.
I will take well prompted LLM generated writing anytime over thst!
It’s just a guessing next word program.
It has no feelings, therefore no empathy.
Best come up out of the rabbit hole for some fresh air & sunshine brother.
I don't think there's any chance people will be employed to sell the output of their brains in 20 years, outside of "athletes", or whatever we end up calling people who think for sport. We mostly succeeded at doing this with muscles, and today there's no market for hauling rocks out of quarries manually. We're in the process of doing that for brains too.
AI will outdo people in all practical uses. We're already there for debugging and getting very close for coding, and we're in the middle of the largest investment in human history to expand that to everything else.
If we do a good job of alignment, AI will treat people like those cats "in charge" of train stations in Japan: our every need will be accommodated, but we won't be controlling things we don't understand.
There will always be creative superstars.
Of course. But they're going to be machines. I don't think there's magic that makes the human brain impossible to automate. And engineering is going to lead to faster improvements than evolution could.
lol yeah sure. AI sucks and always will.