Very interesting article! I love reading about hacking in the classical sense: Using a system in way it’s not intended to accomplish an unexpected/unintended outcome. It’s a shame the media has perverted the term to exclusively refer to cyber attackers (“crackers”).
In some ways, I feel the hacker mindset embodies how I like to approach systems and problems. I like to dig in, get my hands dirty, and see how things work. In programming, this often means a lot of stack traces, breakpoints, and print statements. I’ve jumped into a lot of new areas and figured things out on my own.
But on the other hand, I often feel my hacker mindset isn’t nearly as well-developed as some people I meet. That’s probably a lot of imposter syndrome talking. But the things some people do are amazing, things I’d never even think to fathom, let alone attempt, let alone succeed at.
Being a graybeard now, I see this mindset as a key to early AI adopters who are actually reaping benefits now. The corporate approach is having a strongly defined agentic orchestration process backed up by evals and strict testing and validation, but that is a lot of scaffolding around something much simpler.
It is good to look at and learn from agentic orchestration, but most important to understand the purpose behind the approach. Strictly quantified evals leads to more deterministic understanding of individual prompts. It is a way to build strong building blocks which is important in a large system with lots of other contributors and moving parts. It allows incremental improvement and % success rates and all that.
On my pet projects though, I avoid agentic orchestration. I'm cheap, and most orchestration is deterministic - I start here and want to get to there. Why pay for AI to manage a known process with gates and triggers? That makes it much harder to reason through possible outcomes. It may be more flexible and powerful, but that is the "multiple camera angles" concept, where you build defensively "just in case". If I know what I want the next step to be, just do that directly. I prefer to start with a clear DAG and build on top of it, keeping AI calls narrow and compose outputs not processes.
Yes, you can go to town with agentic processes but eventually you hit the wall of understanding. If you lose the thread of what an agentic process does and how it makes decisions, you lose the ability to find shortcuts. At the root of all hacking is curiosity to know how things work. Taking things apart and manipulating the underlying system is exactly how all of these new technical advancements happen, not by following "best practices" that didn't exist 2 years ago and were built to maintain a manageable engineering department more than maximize AI value at minimal cost.
I remember a specific time when someone very knowledgeable on C++ described friend functions (I think it was) and how there was this thing that he was trying to do. After some thought I came up with some abuse of friend functions to do what he needed. He asked me how I came up with it and that's not how they're supposed to be used. I said I went by what a friend function actually does, not caring why it's called such or prescribed uses.
There's a similar principle about the separation of policy and mechanism which never gets enough exposure. Top-down programmers leak high level concerns all the way down so the underlying parts are only good for the one thing, often naming them in the caller's context rather than by what they literally do. A quote I like is that a poet is one who gives many names to the same thing and a mathematician is one who gives one name (mechanism) to many things (policies) -- Henri Poincaré (paraphrased).
Very interesting article! I love reading about hacking in the classical sense: Using a system in way it’s not intended to accomplish an unexpected/unintended outcome. It’s a shame the media has perverted the term to exclusively refer to cyber attackers (“crackers”).
In some ways, I feel the hacker mindset embodies how I like to approach systems and problems. I like to dig in, get my hands dirty, and see how things work. In programming, this often means a lot of stack traces, breakpoints, and print statements. I’ve jumped into a lot of new areas and figured things out on my own.
But on the other hand, I often feel my hacker mindset isn’t nearly as well-developed as some people I meet. That’s probably a lot of imposter syndrome talking. But the things some people do are amazing, things I’d never even think to fathom, let alone attempt, let alone succeed at.
[dead]
Being a graybeard now, I see this mindset as a key to early AI adopters who are actually reaping benefits now. The corporate approach is having a strongly defined agentic orchestration process backed up by evals and strict testing and validation, but that is a lot of scaffolding around something much simpler.
It is good to look at and learn from agentic orchestration, but most important to understand the purpose behind the approach. Strictly quantified evals leads to more deterministic understanding of individual prompts. It is a way to build strong building blocks which is important in a large system with lots of other contributors and moving parts. It allows incremental improvement and % success rates and all that.
On my pet projects though, I avoid agentic orchestration. I'm cheap, and most orchestration is deterministic - I start here and want to get to there. Why pay for AI to manage a known process with gates and triggers? That makes it much harder to reason through possible outcomes. It may be more flexible and powerful, but that is the "multiple camera angles" concept, where you build defensively "just in case". If I know what I want the next step to be, just do that directly. I prefer to start with a clear DAG and build on top of it, keeping AI calls narrow and compose outputs not processes.
Yes, you can go to town with agentic processes but eventually you hit the wall of understanding. If you lose the thread of what an agentic process does and how it makes decisions, you lose the ability to find shortcuts. At the root of all hacking is curiosity to know how things work. Taking things apart and manipulating the underlying system is exactly how all of these new technical advancements happen, not by following "best practices" that didn't exist 2 years ago and were built to maintain a manageable engineering department more than maximize AI value at minimal cost.
I remember a specific time when someone very knowledgeable on C++ described friend functions (I think it was) and how there was this thing that he was trying to do. After some thought I came up with some abuse of friend functions to do what he needed. He asked me how I came up with it and that's not how they're supposed to be used. I said I went by what a friend function actually does, not caring why it's called such or prescribed uses.
There's a similar principle about the separation of policy and mechanism which never gets enough exposure. Top-down programmers leak high level concerns all the way down so the underlying parts are only good for the one thing, often naming them in the caller's context rather than by what they literally do. A quote I like is that a poet is one who gives many names to the same thing and a mathematician is one who gives one name (mechanism) to many things (policies) -- Henri Poincaré (paraphrased).
"The purpose of a system is what it does" ...and all that jazz
We've already lost it, since "AI agents" are next token predictors instead of syntax tree diffusers or anything else that would actually make sense.
[dead]