Sharing this in hopes it helps other founders who are scared l10n is expensive/bad quality when using Claude Code and other coding agents, it doesn't have to be.
Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace.
I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default.
Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologizing over a direction they chose for the upcoming 6th game in the series:
- "People complained, and we're going in a different direction."
- "We had this translated from Spanish by an LLM."
The 11 paragraphs of text are superfluous, because... (1) anything you could learn from them you could also learn from the two-bullet-point summary on the second page; and (2) the text style clearly doesn't reflect anything that the spokesperson for Ironhide Games wrote. This is a document written specifically to prevent any information from coming through other than the two bullet points in the summary.
And that's a style that companies often invoke on purpose. PR statements are vetted for this kind of thing, which is why everyone hates them. But it appears to be the only style that LLMs want to produce, whether they're making a PR statement or not, and I feel sure that almost none of the people using them for translations actually want this.
I also feel sure that the "LLM style" is specifically intended by the LLM providers, and that it wouldn't exist if they didn't spend enormous effort causing it to.
It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all?
I would leave it at that, but then I consider what you're actually using it for...
> two-person founding team running a language-learning app in dozens of languages, so the copy is read by people who are unusually sensitive to it.
They aren't "unusually sensitive to it". Language learners don't fucking know what native speech in the target language sounds like, they're learning it! Nor does your two person team. How could you possibly claim to objectively evaluate how good translations are in dozens of languages you don't speak? 58, according to your site! This is beyond scummy even for the usual AI boosters, and will deceive people trying to learn, who don't know better, with low-grade slop. Disgusting.
Compared to who? Compared to a professional with a language degree, with a feel and love for both languages and an understanding of the subject area, LLM translation is shit, no question. But such professionals don't get hired by startups to localize their websites, too expensive. When comparing to paid- by-the-word contractors who do website localization, I am not so sure that LLM is worse - because in practice too many biologically-human-localized websites and apps are translated horrifically.
> LLM translation is shit but it's borderline free
I don't think this is true. LLM translation will generally satisfy the core goals of (1) communicating the information that's present in the source document; and (2) not communicating information that isn't present in the source document.
There are some style issues -- I have a comment sidethread complaining about them -- but those are very much second-order effects compared to "can I get a foreigner to understand what my document says?"
Possibly, if it's a strictly formulated medical, business, or legal document (but if it's a medical, business, or legal document, being right 99% of the time isn't good enough!). Google Translate could already do that much, incidentally. If it's not a formal document - if it's anything vaguely resembling casual speech or literary writing - an LLM will butcher it in all kinds of ways no matter how much context it has.
I'm so confused with the "Upgrade" example. Why does it lose the title case and why does it end up phrased as "upgrade to" instead of "Upgrade"?
Sharing this in hopes it helps other founders who are scared l10n is expensive/bad quality when using Claude Code and other coding agents, it doesn't have to be.
Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace.
I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default.
Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologizing over a direction they chose for the upcoming 6th game in the series:
https://cdn.discordapp.com/attachments/524968371494846484/15...
https://cdn.discordapp.com/attachments/524968371494846484/15...
To me the main message is twofold:
- "People complained, and we're going in a different direction."
- "We had this translated from Spanish by an LLM."
The 11 paragraphs of text are superfluous, because... (1) anything you could learn from them you could also learn from the two-bullet-point summary on the second page; and (2) the text style clearly doesn't reflect anything that the spokesperson for Ironhide Games wrote. This is a document written specifically to prevent any information from coming through other than the two bullet points in the summary.
And that's a style that companies often invoke on purpose. PR statements are vetted for this kind of thing, which is why everyone hates them. But it appears to be the only style that LLMs want to produce, whether they're making a PR statement or not, and I feel sure that almost none of the people using them for translations actually want this.
I also feel sure that the "LLM style" is specifically intended by the LLM providers, and that it wouldn't exist if they didn't spend enormous effort causing it to.
It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all?
I would leave it at that, but then I consider what you're actually using it for...
> two-person founding team running a language-learning app in dozens of languages, so the copy is read by people who are unusually sensitive to it.
They aren't "unusually sensitive to it". Language learners don't fucking know what native speech in the target language sounds like, they're learning it! Nor does your two person team. How could you possibly claim to objectively evaluate how good translations are in dozens of languages you don't speak? 58, according to your site! This is beyond scummy even for the usual AI boosters, and will deceive people trying to learn, who don't know better, with low-grade slop. Disgusting.
> LLM translation is shit
Compared to who? Compared to a professional with a language degree, with a feel and love for both languages and an understanding of the subject area, LLM translation is shit, no question. But such professionals don't get hired by startups to localize their websites, too expensive. When comparing to paid- by-the-word contractors who do website localization, I am not so sure that LLM is worse - because in practice too many biologically-human-localized websites and apps are translated horrifically.
> LLM translation is shit but it's borderline free
I don't think this is true. LLM translation will generally satisfy the core goals of (1) communicating the information that's present in the source document; and (2) not communicating information that isn't present in the source document.
There are some style issues -- I have a comment sidethread complaining about them -- but those are very much second-order effects compared to "can I get a foreigner to understand what my document says?"
Possibly, if it's a strictly formulated medical, business, or legal document (but if it's a medical, business, or legal document, being right 99% of the time isn't good enough!). Google Translate could already do that much, incidentally. If it's not a formal document - if it's anything vaguely resembling casual speech or literary writing - an LLM will butcher it in all kinds of ways no matter how much context it has.
[flagged]