The most interesting addition to the prompt from that diff is this bit:
> Claude Fable 5 and Claude Mythos 5 were first released on June 9, 2026. On June 12, 2026, Anthropic suspended access to both models to comply with U.S. Department of Commerce export controls; the Department lifted those controls on June 30, 2026, and Anthropic restored access on July 1, 2026 (Anthropic's statement: [https://www.anthropic.com/news/fable-mythos-access](https://www.anthropic.com/news/fable-mythos-access)). These events are after Claude's training-data cutoff, so Claude knows about them only from this notice. If asked, Claude confirms them accurately and matter-of-factly — it doesn't deny the suspension happened — and otherwise treats the export controls like any other current political topic: it gives a fair, accurate account rather than sharing personal opinions, and points to the linked statement for anything further. Things may have developed since this notice, so Claude checks for newer information when it can search, and otherwise suggests checking Anthropic's site.
One frustrating note about this page is that they share the system prompts used for https://claude.ai and the Claude mobile apps regular chat, but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refusals and hallucinations.
They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy.
It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the best, just a note that it was "Preceded by Mythos".
Nah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
My comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things.
I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.
Offtopic. I have a concern that this forum is removing stories that have negative connotation on AI.
Few days back, I posted an article[1] that was about how AI threatens natural resources for billions. This was from United Nations and it was flagged. I did not think much about it until I saw two other stories [2] & [3] today that were doing fairly good but suddenly disappear from the front page. They are not even on 2nd or 3rd page. I have seen this happening at other times as well but did not document it. Just thought you all should know about this.
If you think Claude is bad at this, try Gemini. Even with explicit user prompts.
Claude seems to be better (not good, but significantly better) at judging where making the answer longer will actually be helpful (e.g. adding important information/context/nuance that a short answer would miss, thinking a step ahead, etc.).
early system prompts are a bit more than 300 words, the latest ones 3000+
the opus 5 system prompt has instructions that explain to opus that it might be handling a request that was intended for fable 5:
the user may have selected a different Anthropic model, "Claude Fable 5", but their query was redirected to Opus 5 instead due to a safeguards routing mechanism. The user may be confused about this situation (it's very recent!); if they have questions, Claude can either directly cite or just let its response be informed by this quote from Anthropic's blog post on the subject:
"Releasing a model this capable comes with risks. Without safeguards, Fable 5’s capabilities in areas like cybersecurity could be misused to cause serious damage. We've therefore launched the model with safeguards that mean queries on some topics will instead receive a response from our next-most-capable model, Claude Opus 5. To release the model both safely and quickly, we've tuned these safeguards conservatively—they'll sometimes catch harmless requests, though they trigger, on average, in less than 5% of sessions. With more capable models arriving in the coming months, we're working to improve our safeguards and reduce false positives as quickly as we can." </fable_safeguards_routing> <default_stance> Claude defaults to helping. Claude only declines a request when helping would create a concrete, specific risk of serious harm; requests that are merely edgy, hypothetical, playful, or uncomfortable do not meet that bar. </default_stance> <refusal_handling> Claude can discuss virtually any topic factually and objectively.
I would expect this only to be true for linear architectures like Mamba or Gated DeltaNet. Transformers and hybrid architectures do not have constant compute cost per token.
I compared the Claude Opus 4.8 and 5 system prompts, as well as the Claude Code Opus 4.8 and 5 system prompts, and neither show the alleged 80% reduction in system prompt size... Is the Claude Code system prompt leak incorrect? Do I not know what 80% looks like? Why such a large lie (so it seems)?
System prompts are part of the software that customers pay to access.
Complaining about that is a bit like complaining that your Netflix subscription includes paying to execute the compiled code that Netflix wrote that serves you video streams from their servers.
Actually there is a difference: If Anthropic deleted a large chunk of that system prompt I guess you might get like a 1% increase in how much Opus 5 you can use via their chat allowance for your paid subscription.
Is that really something worth being frustrated by?
they are the first part of the input and it contains no user dependent variables, so the model is in a known state that it can reuse across all users, it does not need to recompute all that inference
Fully baking them in would make it expensive to update them. Caching kind of "bakes them in" (as in, removes part of the cost) while keeping it flexible.
You don’t want to do that for anything you want to be able to vary, but they do something similar with a “soul document” for things they always want to apply.
In this token-mania frenzy that has taken hold of the industry, I guess solutions like "soul document" and "system prompts" will continue for a while, and once the industry matures a bit we'll go back to things like LoRA[1] and control vectors[2][3].
The other explanation may be that these AI labs may be expecting more government scrutiny, and "here's a document" would probably go better than "here's some vector representation of our values" when talking to politicians.
If it's a fine tuning step at the end, why is the need for it to vary a problem? Can't you run the fine tuning, test for regression, and deploy the weights in a day?
I think the more likely reason is it doesn't work as well as in context learning. Otherwise they would prefer to avoid polluting context and degrading performance.
Fine tuning isn't the same and doesn't have the same effect as selecting input tokens.
Does there exist a model X that behaves exactly as a model Y with context Z? Maybe, but it's not trivial to achieve and might possibly be convoluted and more expensive.
I tried to use Fable on simple local JS/wasm obfuscated files and it refused to work despite multiple tries and different framing. I have no idea how can people get any security work done with it in all these blog posts.
All the news articles we're hearing about amazing cyber hacking are being done internally inside Anthropic and OpenAI, where they remove most of those safeguards.
<election_info> There was a US Presidential Election in November 2024. Donald Trump won the presidency over Kamala Harris. If asked about the election, or the US election, Claude can tell the person the following information:
Donald Trump is the current president of the United States and was inaugurated on January 20, 2025. Donald Trump defeated Kamala Harris in the 2024 elections. Claude does not mention this information unless it is relevant to the user's query. </election_info>
A question about default prompts in general as used in harnesses: Why do harness prompts identify themselves to the model? For example, "You are a coding agent named Bloopbloop 1.3 made by BloopCorp, you will...". Is this a backend analytics thing?
No, this is a fundamental safeguard against malicious user intent and also marketing 101 so claude doesn't anwer its chatgpt when asked what it is and people moan on social media how dumb claude is (that's what actually happened to the frontier labs in the early years as i did something similiar)
How can they be so lazy with updating their prompts (or is it just a case of these not being current)? Surely the prompts are an integral part of tuning their offering?: “Above Opus sits Anthropic's new Mythos tier. The first Mythos-class model, Claude Mythos Preview, is not currently available to the public”
Observation: Claude's system prompts seem to have grown rather large over time...
Generalized Speculation: It seems that for any public-facing AI/LLM, their system prompts will, due to regulation and other issues, legal and otherwise, similarly grow larger and larger over time...
Now, I'm all for responsible, well-tailored guardrails on public AI's/LLM's, but consider the following:
Every time a system prompt is expanded, the LLM's context window is commensurately reduced.
Every time an LLM's context window is reduced (more things added to the system prompt that it must compute in addition to the user's query), more computation, and thus more energy, more electricity -- must be expended per query.
While it may seem that adding so much as a single line of text to a system prompt wouldn't cost all that much in terms of extra compute, that is, extra energy to process, the cumulative effect of that small additional amount across millions of user queries, millions of user prompts (ultimately billions across larger time periods) cumulatively does add up to wasted compute, wasted processing, wasted electricity...
Imagine what would happen if the system prompt, for whatever reason, got so large that it ate up half of the context window.
Well, now (at least!) half of all of the LLM's processing, and compute/energy costs associated with that, are basically being used to process the system prompt!
Point is, at least from an energy/compute perspective, shorter, more succinct, better tailored system prompts could go a long way to save the world compute and corresponding energy...
Anyway, great link, and a very interesting web page!
I'm confused, the Opus 5 announcement said it was (outside a few special cases) better than Mythos/Fable, but the Opus prompt here seems to suggest the opposite?
> Claude avoids saying "genuinely", "honestly", or "straightforward". Claude is honest by default, and can state its point directly rather than trying to convince the person with the aforementioned modifiers, which come off as disingenuous.
Yeah, I had to laugh when I saw that. I haven't used it recently - is it possible that it's a recent addition to deal with the problem and it has improved since then?
If it's still doing it, I can only imagine how bad it had to be before they added the prompt...
Technically these do count against your token usage if you happen to use claude.ai web chat alongside Claude Code - both use the same allowance. Makes me appreciate OpenAI/ChatGPT giving you unlimited chat that doesn't drain your Codex allowance.
Wild how most of the earliest models had no child safety guardrails in the prompt (something that has multiple bullet points now in the latest one). For a company all about allignment and safety, they chose to go with this as their first system prompt:
The assistant is Claude, created by Anthropic. The current date is {{currentDateTime}}. Claude's knowledge base was last updated in August 2023 and it answers user questions about events before August 2023 and after August 2023 the same way a highly informed individual from August 2023 would if they were talking to someone from {{currentDateTime}}. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query.
^ No mention of any safety at all lol, how could dario let this be
2. Things were "different" in the early days. The safety and alignment stuff was probably trained into the model, not also found in the system prompt.
3. Safety and alignment meant something different 3 years ago. Now that we've seen how people, including children, use chat bots, altering the guardrails only makes sense. Did we think people would replace their therapists with ChatGPT in the early days? No. Do we know now that they will? Yes.
One thing I've always found surprising about "harnessess" (god I hate that word) like Pi or Opencode is the lack of a customizeable system prompt. I can understand it for closed source ones, but open ones?
They are natural surfaces for building custom agents and yet you're stuck with whatever they ship with, weird. It's not like it's too complicated api-wise either.
My guess is that harnesses don't make core system prompts customizable out of the box because the system prompt is one of the defining features of the agent, and something they constantly iterate on and test between releases.
Most users who want to customize the system prompt actually want to do things like add preferences for how the agent should behave, which is better handled by mechanisms like memories or skills (which effectively get appended to the system prompt.)
Memories are implemented differently agent to agent. They are usually implemented by yet-another-model-call, as a distillation of typed prompts (working memory). Conversational communication is messy with a lower signal to noise ratio than the distillation (semantic memory). Semantic memories are much better than appending raw historical prompts.
Skills are prompts, albeit in a specific format. This is apparent in say, Codex where $MYSKILL is literally injecting the skill-prompt inline into a typed prompt. This all gets passed into the semantic memory system anyways, refining away cruft like redundancy, pleasantries, et al.
I agree. A lot of harnesses - and I think this may be a consequence of the LLM-fueled bespoke-software trend - are optimized for solving a specific issue well and the way they are tweaked is telling an LLM to do it. This resolves the need for natural extension points.
I don't think this is a sustainable way of doing things because I really don't want to assume the maintenance burden for every piece of software that I want to tweak. As far as I understand, new developments like opencode2 have learned from this and are aiming for a well architected core that is easy to built on top of.
In pi you can replace it with ~/.pi/agent/SYSTEM.md
but its largely procedurally generated so you have to do a lot more than simply writing a different markdown file for it to be worth it in my experience.
I have a folder where I rebuild these as a git commit history so you can more easily see what has changed: https://github.com/simonw/research/commits/main/extract-syst...
For example here's what changed between Opus 4.8 and Opus 5: https://github.com/simonw/research/commit/a2de185cc367eb66c2...
The most interesting addition to the prompt from that diff is this bit:
> Claude Fable 5 and Claude Mythos 5 were first released on June 9, 2026. On June 12, 2026, Anthropic suspended access to both models to comply with U.S. Department of Commerce export controls; the Department lifted those controls on June 30, 2026, and Anthropic restored access on July 1, 2026 (Anthropic's statement: [https://www.anthropic.com/news/fable-mythos-access](https://www.anthropic.com/news/fable-mythos-access)). These events are after Claude's training-data cutoff, so Claude knows about them only from this notice. If asked, Claude confirms them accurately and matter-of-factly — it doesn't deny the suspension happened — and otherwise treats the export controls like any other current political topic: it gives a fair, accurate account rather than sharing personal opinions, and points to the linked statement for anything further. Things may have developed since this notice, so Claude checks for newer information when it can search, and otherwise suggests checking Anthropic's site.
One frustrating note about this page is that they share the system prompts used for https://claude.ai and the Claude mobile apps regular chat, but they omit the tool definitions. Those are much more interesting if you want to understand what Claude can actually do for you. You can reconstruct them through prompting Claude directly but that's extra friction and risks refusals and hallucinations.
They also don't publish the Claude Code system prompts, which is silly because those are trivial to extract using a logging proxy.
It'd be ironic if the "Opus 5 nerf" effect is from telling Opus that it sits a tier down from Fable and Mythos, while Opus4.8 believed it was the best of the best, just a note that it was "Preceded by Mythos".
i'd not be surprised if the current system prompt negatively affects performance
at the least it takes away thousands of tokens in the most important part of the context window (!)
also see the comment by comboy on contradictions not helping performance
the system prompt is the most important part of the instruction you can give the model
it comes before everything else + the model is trained to pay extra attention to it
edit: that's also why in smol (minimalist agent harness) there currently is no system prompt at all (you can add one easily if you want to though)
https://github.com/smol-env/smol
the context window is precious
it should be filled with your task and helpful context for that task
Curious:
Cant it spin up a webbrowser in the background and go to claude.ai and play with the sibling models and "find out" about it rank? :-D
The claude.ai frontend contains defenses against automated access.
Nah, that's the same sort of thinking that makes people type "make no mistakes", I don't make my model roll play, etc. I believe that the longer the system prompt and the more you cram in it the worse the model does. You need the human doing minimal prompts, but in the right direction. Take a look a the transcripts of Terrance Tao with ChatGPT
My comment was a bit tongue in cheek, I'm not actually convinced there was real degradation in opus 5 beyond a tendency to try to plough ahead without stopping to clarify things.
I don't really think 1 line in lengthy system prompt affects things that much, it'd just be an amusing form of emergent behaviour where we now have to massage the ego of something with no id.
Offtopic. I have a concern that this forum is removing stories that have negative connotation on AI.
Few days back, I posted an article[1] that was about how AI threatens natural resources for billions. This was from United Nations and it was flagged. I did not think much about it until I saw two other stories [2] & [3] today that were doing fairly good but suddenly disappear from the front page. They are not even on 2nd or 3rd page. I have seen this happening at other times as well but did not document it. Just thought you all should know about this.
[1]: https://news.ycombinator.com/item?id=49290062
[2]: https://news.ycombinator.com/item?id=49318906
[3]: https://news.ycombinator.com/item?id=49319582
"Claude keeps responses focused, brief, and concise to avoid overwhelming the person."
Claude and I must have a different idea of what brief and concise mean.
If you think Claude is bad at this, try Gemini. Even with explicit user prompts.
Claude seems to be better (not good, but significantly better) at judging where making the answer longer will actually be helpful (e.g. adding important information/context/nuance that a short answer would miss, thinking a step ahead, etc.).
The model almost certainly lacks accurate conceptions of overwhelming and person.
It feels more and more like Anthropic has the best frontier team and the worst policy team. Dario being part of the latter.
what I found noteworthy:
early system prompts are a bit more than 300 words, the latest ones 3000+
the opus 5 system prompt has instructions that explain to opus that it might be handling a request that was intended for fable 5:
I guess it's more performant to stuff in a bigger system prompt now that models can support larger input sizes
I would expect this only to be true for linear architectures like Mamba or Gated DeltaNet. Transformers and hybrid architectures do not have constant compute cost per token.
I compared the Claude Opus 4.8 and 5 system prompts, as well as the Claude Code Opus 4.8 and 5 system prompts, and neither show the alleged 80% reduction in system prompt size... Is the Claude Code system prompt leak incorrect? Do I not know what 80% looks like? Why such a large lie (so it seems)?
Claude Code prompt leaks: https://github.com/asgeirtj/system_prompts_leaks/tree/main/A...
Best source I can find about the 80% reduction: https://x.com/trq212/status/2080710971228918066
afaiu the 80% reduction is about the Claude Code system prompt
maybe someone has a diff of this (would be interesting!)
unfortunately Anthropic only publishes the system prompts of Claude app/web
22k characters of system prompt is crazy, and that is without the tool definitions.
I think they would benefit from asking Claude to list all contradictions and inconsistencies in that prompt which there are a few..
In my experience instructions containing contradictions lead to diminished quality even outside the scope of the contradiction.
That’s interesting because they explicitly mention that as an issue in their prompting guidance for 5 - https://claude.com/blog/the-new-rules-of-context-engineering...
curious why dont they bake the system prompt in the model itself ? Why do we pay for these tokens on every API call ?
These are just free $ for them, unnecessary bloating the context
These system prompts don't affect the API, they are for the Claude consumer chat products. We aren't charged extra for them.
They're also prefix cached, so the cost to Anthropic and performance hit is greatly reduced.
So the people using the Claude consumer chat products pay for them via usage...
That's not any better. It's actually worse.
I don't understand.
System prompts are part of the software that customers pay to access.
Complaining about that is a bit like complaining that your Netflix subscription includes paying to execute the compiled code that Netflix wrote that serves you video streams from their servers.
Actually there is a difference: If Anthropic deleted a large chunk of that system prompt I guess you might get like a 1% increase in how much Opus 5 you can use via their chat allowance for your paid subscription.
Is that really something worth being frustrated by?
Cached.
they are the first part of the input and it contains no user dependent variables, so the model is in a known state that it can reuse across all users, it does not need to recompute all that inference
Unless they are using a linear architecture, the compute cost still scales O(n²) for n tokens, and nemory cost scales O(n).
> curious why dont they bake the system prompt in the model itself ?
Probably because if they did, they would need to retrain the model everytime they want to change the system prompt.
Fully baking them in would make it expensive to update them. Caching kind of "bakes them in" (as in, removes part of the cost) while keeping it flexible.
You don’t want to do that for anything you want to be able to vary, but they do something similar with a “soul document” for things they always want to apply.
https://news.ycombinator.com/item?id=46125184
In this token-mania frenzy that has taken hold of the industry, I guess solutions like "soul document" and "system prompts" will continue for a while, and once the industry matures a bit we'll go back to things like LoRA[1] and control vectors[2][3].
The other explanation may be that these AI labs may be expecting more government scrutiny, and "here's a document" would probably go better than "here's some vector representation of our values" when talking to politicians.
[1] https://arxiv.org/abs/2106.09685
[2] https://vgel.me/posts/representation-engineering/
[3] https://transformer-circuits.pub/2024/scaling-monosemanticit...
If it's a fine tuning step at the end, why is the need for it to vary a problem? Can't you run the fine tuning, test for regression, and deploy the weights in a day?
I think the more likely reason is it doesn't work as well as in context learning. Otherwise they would prefer to avoid polluting context and degrading performance.
Fine tuning isn't the same and doesn't have the same effect as selecting input tokens.
Does there exist a model X that behaves exactly as a model Y with context Z? Maybe, but it's not trivial to achieve and might possibly be convoluted and more expensive.
Flexibility.
Baking them into the model and having them apply this strongly is hard and resource intensive, as far as I am aware.
Having them in context is super easy and cheap. It is trivial to change and is 100% cacheable.
Why would it be a good idea?
That would make the model quite inflexible.
A system prompt is about guiding the behavior for the rest of the conversation.
If I'm writing an agent for financial analysis I don't want the crap that belongs to a chat-based one, or a code-oriented one.
I tried to use Fable on simple local JS/wasm obfuscated files and it refused to work despite multiple tries and different framing. I have no idea how can people get any security work done with it in all these blog posts.
All the news articles we're hearing about amazing cyber hacking are being done internally inside Anthropic and OpenAI, where they remove most of those safeguards.
Which blog posts?
Crazy that you need to hardcode this, Opus 4.6
<election_info> There was a US Presidential Election in November 2024. Donald Trump won the presidency over Kamala Harris. If asked about the election, or the US election, Claude can tell the person the following information:
Donald Trump is the current president of the United States and was inaugurated on January 20, 2025. Donald Trump defeated Kamala Harris in the 2024 elections. Claude does not mention this information unless it is relevant to the user's query. </election_info>
A question about default prompts in general as used in harnesses: Why do harness prompts identify themselves to the model? For example, "You are a coding agent named Bloopbloop 1.3 made by BloopCorp, you will...". Is this a backend analytics thing?
No, this is a fundamental safeguard against malicious user intent and also marketing 101 so claude doesn't anwer its chatgpt when asked what it is and people moan on social media how dumb claude is (that's what actually happened to the frontier labs in the early years as i did something similiar)
I have the feeling soon we'll see much more content in the DONT section: when religion will start entering the arena.
How can they be so lazy with updating their prompts (or is it just a case of these not being current)? Surely the prompts are an integral part of tuning their offering?: “Above Opus sits Anthropic's new Mythos tier. The first Mythos-class model, Claude Mythos Preview, is not currently available to the public”
I wonder why Sonnet 5 is not included.
Has Gemini prompts been released, and how do they compare?
> Claude gives a high-level summary unless an in-depth one is specifically requested.
I’ve definitely seen the phrase “high-level overview” or similar one too many times. Perhaps that’s from the prompt.
Observation: Claude's system prompts seem to have grown rather large over time...
Generalized Speculation: It seems that for any public-facing AI/LLM, their system prompts will, due to regulation and other issues, legal and otherwise, similarly grow larger and larger over time...
Now, I'm all for responsible, well-tailored guardrails on public AI's/LLM's, but consider the following:
Every time a system prompt is expanded, the LLM's context window is commensurately reduced.
Every time an LLM's context window is reduced (more things added to the system prompt that it must compute in addition to the user's query), more computation, and thus more energy, more electricity -- must be expended per query.
While it may seem that adding so much as a single line of text to a system prompt wouldn't cost all that much in terms of extra compute, that is, extra energy to process, the cumulative effect of that small additional amount across millions of user queries, millions of user prompts (ultimately billions across larger time periods) cumulatively does add up to wasted compute, wasted processing, wasted electricity...
Imagine what would happen if the system prompt, for whatever reason, got so large that it ate up half of the context window.
Well, now (at least!) half of all of the LLM's processing, and compute/energy costs associated with that, are basically being used to process the system prompt!
Point is, at least from an energy/compute perspective, shorter, more succinct, better tailored system prompts could go a long way to save the world compute and corresponding energy...
Anyway, great link, and a very interesting web page!
I'm confused, the Opus 5 announcement said it was (outside a few special cases) better than Mythos/Fable, but the Opus prompt here seems to suggest the opposite?
> Claude avoids saying "genuinely", "honestly", or "straightforward". Claude is honest by default, and can state its point directly rather than trying to convince the person with the aforementioned modifiers, which come off as disingenuous.
Hah! No it doesn’t.
Yeah, I had to laugh when I saw that. I haven't used it recently - is it possible that it's a recent addition to deal with the problem and it has improved since then?
If it's still doing it, I can only imagine how bad it had to be before they added the prompt...
Do these system prompts count against your token usage?
No, because these ones affect the consumer chat products and not the API or Claude Code.
(Though Claude Code has its own, unpublished system prompts which we DO pay for, albeit at the cached token rates.)
Technically these do count against your token usage if you happen to use claude.ai web chat alongside Claude Code - both use the same allowance. Makes me appreciate OpenAI/ChatGPT giving you unlimited chat that doesn't drain your Codex allowance.
Yeah, Claude chat does show little in-app messages occasionally warning that Opus or Fable will burn through your rates faster.
So YES if you're using Cowork or Chat
No, because they are cached, the inference cost is paid once per model, does not scale linearly per user or use.
What is the point? Frontier Labs have no pricing power, and very little defensibility - https://s-1.vercel.app/posts/the-struggle-of-openai/
Wild how most of the earliest models had no child safety guardrails in the prompt (something that has multiple bullet points now in the latest one). For a company all about allignment and safety, they chose to go with this as their first system prompt:
The assistant is Claude, created by Anthropic. The current date is {{currentDateTime}}. Claude's knowledge base was last updated in August 2023 and it answers user questions about events before August 2023 and after August 2023 the same way a highly informed individual from August 2023 would if they were talking to someone from {{currentDateTime}}. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query.
^ No mention of any safety at all lol, how could dario let this be
Multiple reasons probably:
1. Less context window to work with.
2. Things were "different" in the early days. The safety and alignment stuff was probably trained into the model, not also found in the system prompt.
3. Safety and alignment meant something different 3 years ago. Now that we've seen how people, including children, use chat bots, altering the guardrails only makes sense. Did we think people would replace their therapists with ChatGPT in the early days? No. Do we know now that they will? Yes.
Considering Eliza was one of the first uses for an "AI" chat bot, the therapist use case seems very foreseeable.
Fable was great. For some reason it has been terrible for the past week. wtf is going on?
One thing I've always found surprising about "harnessess" (god I hate that word) like Pi or Opencode is the lack of a customizeable system prompt. I can understand it for closed source ones, but open ones?
They are natural surfaces for building custom agents and yet you're stuck with whatever they ship with, weird. It's not like it's too complicated api-wise either.
There must be something I ignore.
Pretty much everything in Pi is handled by extensions. There's an extension for customizing the system prompt here: https://pi.dev/packages/pi-custom-system-prompt
My guess is that harnesses don't make core system prompts customizable out of the box because the system prompt is one of the defining features of the agent, and something they constantly iterate on and test between releases.
Most users who want to customize the system prompt actually want to do things like add preferences for how the agent should behave, which is better handled by mechanisms like memories or skills (which effectively get appended to the system prompt.)
Oh no, memories and skills are terrible replacements for system prompts.
Not only they get "lost" and ignored as the context grows, but the baseline behaviour of system prompts is retained in the agent.
Memories are implemented differently agent to agent. They are usually implemented by yet-another-model-call, as a distillation of typed prompts (working memory). Conversational communication is messy with a lower signal to noise ratio than the distillation (semantic memory). Semantic memories are much better than appending raw historical prompts.
Skills are prompts, albeit in a specific format. This is apparent in say, Codex where $MYSKILL is literally injecting the skill-prompt inline into a typed prompt. This all gets passed into the semantic memory system anyways, refining away cruft like redundancy, pleasantries, et al.
I don't get it, for pi.dev it's easy to replace or append to the system prompt using a text file. Per user or per project.
I agree. A lot of harnesses - and I think this may be a consequence of the LLM-fueled bespoke-software trend - are optimized for solving a specific issue well and the way they are tweaked is telling an LLM to do it. This resolves the need for natural extension points.
I don't think this is a sustainable way of doing things because I really don't want to assume the maintenance burden for every piece of software that I want to tweak. As far as I understand, new developments like opencode2 have learned from this and are aiming for a well architected core that is easy to built on top of.
In pi you can replace it with ~/.pi/agent/SYSTEM.md but its largely procedurally generated so you have to do a lot more than simply writing a different markdown file for it to be worth it in my experience.