> Most AI runtimes can only execute one inference call to a model at a time.
I don't know if the LMAX article makes much sense in this space. The latency of calling the models dominates any sort of local effects and the volume of the requests tends to be very low relative to any kind of meaningful system limits.
More broadly, parallel agent systems are of questionable utility compared to ones that operate over the problem serially. Parallelism bringing any uplift implies that the effects of different parts of the problem don't interact much. If things do depend on other things over time, then you have something that can only be solved with one serial narrative.
The case of high frequency trading works really well because the shared resource (global sequence) is extremely contentious and local. You actually get meaningful benefits out of these principles. OpenClaw is not the same kind of problem. Running an actual inference model on the CPU is maybe a different story.
A single fundamental principle underlies the implementation of “mechanical sympathy” in software: make the topology of the software match the topology of the hardware as closely as possible. All of the various tricks and heuristics are just narrow cases of this single principle. Hardware architecture is immutable but software architecture is not.
It was how I learned to design code for supercomputers and is remarkably effective at generating extreme efficiency and scalability in diverse contexts. Importantly, it generalizes to systems of every shape and size. While it is difficult to get good results on a supercomputer if you don’t understand this, it works just as well for a random web app.
I abused this term a bit to help data scientists & data engineers understand why they should take interest in each others skillsets. I used to liken it to a formula 1 driver (scientist) and the car / pit crew (engineers).
Sure, you can maybe be a great driver without caring about the car or the crew, but it is definitely going to have its limits. Likewise, at the end of the day the crew is there to make the driver shine, and need to be invested in understanding how they operate.
Creates a much better sense of culture and collaboration, and overall better products, when everyone can see the part they play and how important the relationship is to their peers.
I looked for a term like "mechanical sympathy" for years. In fact, I think I even came up with that independently when I was thinking about it. I remember asking on some internet forum if anyone else feels a sense of sympathy towards machines and most people made fun of me. A few admitted they didn't like crunching the gears in their car gearbox.
But I was talking about all machines. I don't like it when a washing machine is off balance and vibrating violently. I don't like to use my tools at, or anywhere near, there mechanical limits. When I observe normies they don't seem to care. They'll force and abuse things all the time. I did wonder if it was part of my predisposition towards engineering.
> Most AI runtimes can only execute one inference call to a model at a time.
I don't know if the LMAX article makes much sense in this space. The latency of calling the models dominates any sort of local effects and the volume of the requests tends to be very low relative to any kind of meaningful system limits.
More broadly, parallel agent systems are of questionable utility compared to ones that operate over the problem serially. Parallelism bringing any uplift implies that the effects of different parts of the problem don't interact much. If things do depend on other things over time, then you have something that can only be solved with one serial narrative.
The case of high frequency trading works really well because the shared resource (global sequence) is extremely contentious and local. You actually get meaningful benefits out of these principles. OpenClaw is not the same kind of problem. Running an actual inference model on the CPU is maybe a different story.
A single fundamental principle underlies the implementation of “mechanical sympathy” in software: make the topology of the software match the topology of the hardware as closely as possible. All of the various tricks and heuristics are just narrow cases of this single principle. Hardware architecture is immutable but software architecture is not.
It was how I learned to design code for supercomputers and is remarkably effective at generating extreme efficiency and scalability in diverse contexts. Importantly, it generalizes to systems of every shape and size. While it is difficult to get good results on a supercomputer if you don’t understand this, it works just as well for a random web app.
I abused this term a bit to help data scientists & data engineers understand why they should take interest in each others skillsets. I used to liken it to a formula 1 driver (scientist) and the car / pit crew (engineers).
Sure, you can maybe be a great driver without caring about the car or the crew, but it is definitely going to have its limits. Likewise, at the end of the day the crew is there to make the driver shine, and need to be invested in understanding how they operate.
Creates a much better sense of culture and collaboration, and overall better products, when everyone can see the part they play and how important the relationship is to their peers.
I looked for a term like "mechanical sympathy" for years. In fact, I think I even came up with that independently when I was thinking about it. I remember asking on some internet forum if anyone else feels a sense of sympathy towards machines and most people made fun of me. A few admitted they didn't like crunching the gears in their car gearbox.
But I was talking about all machines. I don't like it when a washing machine is off balance and vibrating violently. I don't like to use my tools at, or anywhere near, there mechanical limits. When I observe normies they don't seem to care. They'll force and abuse things all the time. I did wonder if it was part of my predisposition towards engineering.