At the very least, you can (and probably should) put a plaintext netlist and BOM into an LLM to triple-check your work.
It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly, nets are named and linked correctly, buses have the correct and consistent termination, and so on.
A personal data point: I had Claude Opus 4.8 design a fairly textbook circuit that outputs a monochrome image burned in an EEPROM over standard 640x480 VGA using only 74 series logic and GALs. It designed the circuit and GAL code, and I did the routing, and got it made through JLC for $6. After it came back, there was one error that was not caught, which I could blue-wire, and it works just fine otherwise. I was fairly impressed.
I got a flexpcb that validates in JLC and PCBWay DRC tools from the KiCAD MCP Server and Codex.
I have yet to order any or program it, but it was enough to make me push on with a PCB art project for ST-style guitar pickguards - no netlist, no problems.
Flying home today and got DHL notification that my vibe PCBs are on my porch. Now to try to program, bring up, and see if it works this weekend. $178 for 5 assembled units from PCBway to run this dumb experiment.
I've gone through this same workflow and had success all the way through DRC ruleset passing with JLCPCB-based settings, ordering, fab, and use.
Board was ordered with PCBA (they did assembly of the stock components) and I did SMD for just the oddball module/ICs they don't stock. Works exactly to design.
My boards are for hobby use and are ridiculously simple though compared to anything professional (breakout boards for specific components in FPV drone video transmission subsystems). That's probably an important detail.
I think its like anything else in AI right now. It can do it 90% of the time but that 10% can be really rough and if its a task you can't do or verify yourself, you won't know the difference.
I'm confused. Who made this benchmark and why? It is fascinating to me that people are putting this much work into trying to measure and evaluate LLMs when we've never gone to this level for humans. Sounds like they also built a harness/feedback loop so the LLM knows if it's doing a good job? Again, why didn't anyone ever do this for human engineers?
Humans learn differently from LLMs. LLMs already contain most of the information they need to be able to do PCB layout from reading the entire internet, but they need this kind of fast feedback loop to learn how to extract and use that information in the right way. Humans learn to act at the same time they learn knowledge.
because human engineers are not reproducible, so having an accurate measure of the skills of one of them has a much lower utility return (those benchmarks are very expensive to run, and making them for humans will be at least as much expensive)
I think the new Astra computer use demos show that the models might be able to do things like inspection of real world objects if given a camera.
Super excited to see real world feedback added into the agent loops we have gotten used to working with. Could you
let the model print and test the circuit boards it is prototyping with a jig?
In July I was struggling with writing DIY Rust firmware for an e-ink screen. I mistakenly thought I'd ordered an Inkplate 6 ED060SC7 and actually had the later version, which confounded my efforts. I was also mistaken about the pixel resolution.
The way I found this out is I propped it up next to a webcam so it was more or less full frame, and I had the (then new) Fable write a python script to bezier warp the camera capture to a flat projection of the screen. At that point I couldn't address the whole screen. Once I'd guided the capture script I just left the LLM overnight with the instruction to get full control confirmed by a capture round trip, and it was meaningfully finished in a couple of hours. I don't really have the skills to attempt that myself in a reasonable time frame.
Yes — this is already possible to a useful degree. I’ve been using tscircuit, where you can describe circuits in code and use AI agents to generate and iterate on schematics and PCB layouts. I’ve seen designs from this workflow go all the way to fabricated, working hardware.
Interesting. AI may be pretty good at managing the gazillion different things that make up circuit boards.
Frankly, I am surprised that it isn't already a solved problem, as we've had silicon compilers, for many years, and I always figured that IC design is more difficult than PCB.
Just as some anecdatum, I've tried this a couple of times over the last few years. 2024-2025, no body could generate anything of any sort of complexity, however maybe around the Opus 4.7 timeline, I tried having it generate some relatively simple and one slightly more complex circuit around 8-bit PIC micros, and was very pleasantly surprised. The schematics for simpler stuff is usually OK, often with some of the kind of crazy belt-and-suspenders stuff you see in code (how many decoupling caps do I actually need here, Claude?), and PCB layout has basically been terrible every time I've tried it.
Recent models can generate mostly competent schematics if you're using well known parts, feed them data sheets (and you _must_ feed them the errata too) and it's not too complex. Any complexity analog or RF, everything falls down quickly if you know what you're looking at. Maybe Astra will do better? There's still so much implicit knowledge that a good designer (not me by a long shot, but I know a little) will bake into a board, and even as cheap as JCLPCB is now, you don't want to have to spin a half dozen revs because Claude or Codex hallucinated. It will be genuinely interesting to see what happens on this front.
Works very well with EAGLE XML files, especially if you give it the EAGLE.DTD schema that CadSoft provides. Both .sch and .brd files can be extensively manipulated by Claude (and likely everybody else at this point.)
For stuff like this, definitely. It’s already exciting. Claude et al are surprisingly competent with real hardware. I’ve also had good success debugging basic issues with micros - not via the debugger, but giving datasheet and written analysis. I’m sure it could do things like capture from a connected oscilloscope (Salaeae have an MCP integration for their logic tools). Also for SPICE which I never bother with because the tools always felt stuck in the 90s, but having some serious automated analog review would be nice. I’m really excited to play with peripherals like the RP2040 or Beaglebone real time logic blocks, or improving/adding real USB support instead of USB serial.
For circuits I would trust a netlist or DRC/ERC review, checking a design against reference circuit values, probably not placement as that tends to be guided straight from a datasheet where it makes a difference and I like the artistic side of it.
I have tried to use AI on this from GPT3, documenting a few of the attempts on my blog. They know the datasheets and the theory enough to be extremely useful at the start, but the most recent ones are becoming scary good. Routing a PCB is certainly a bit more than joining some wires, you have to accommodate power traces, ground loops, component overlap and all the various RF tricks and tips, but I do believe we are less than a year from prompt to full assembly including enclosure.
I have found that AI is great for sorting out your libraries, drawing and helping double check footprints managing your BOM. I still draw my schematics and layout my boards. I haven’t found it any good for those tasks yet. There are too many stupid errors it makes. I do get it to review my work though and often it catches things I have missed.
The next pain-point is sourcing the components from Digikey, LCSC, etc., and finding suitable substitutes if necessary.
Of course this assumes the LLMs can already read datasheets because that's the biggest pain-point in designing electronics. It's like filling out tax forms.
Finally it would be great if LLMs could extract simulation models from datasheets!
Sourcing parts has been a huge motivation for this. We are still pretty jarred by the supply chain crisis a few years ago. Having the ability to formally verify that an alternative part fulfills the original design constraints is huge.
Not sure whether you played with llm datasheet extraction lately, but they are working their way up there. Even started reasoning about curves and footnotes.
Where do you think is the biggest pain point: part discovery, datasheet verification or substitute determination?
Routing is the easiest part of layout, and there have been auto-routers (which mostly worked) for a long time now; the difficult task is in placing components, and adjusting those placements.
Sure, I was just using "route" to mean also floorplanning because that's what Astra's routing tool also does. Not sure how they name it. Sorry if I was sloppy.
Not at all, but the term for routing and placing the components is ‘layout’. Floorplanning usually refers to the laying out of larger blocks of the circuit (with the blocks themselves already having been laid-out).
If you give it a real solver to drive and business constraints to operate under, it will do pretty good job (which is to map your business rules to constraints and balanced costs, develop deterministic tests, probabilistic tests, acceptance gates, all boring stuff).
Really excited about constraint solvers making a comeback! Especially after seeing Z3 getting a mention in anthropics fermat post.
EEbench is using atopile's internal constraint solver for engineering parameters (think operating temperature, voltage ranges, stacked tolerances etc).
Love what samuel and his team are building over in the Netherlands!
Always thought it would be fun if there was a more beginner friendly version of atopile [1].
Mostly focused on BOM, schematics, and firmware now but slowly expanding to circuit boards if we feel we can deliver real value there. Early experiments with Astra are promising
Have tried this a few times. While AI has been helpful in talking through component choices and other topics, it’s consistently been a spectacular fail each time in actually designing the board.
I’ve had similar results asking AI to design 3D models for 3D printing.
“Coach me on optimizing error on my printer”… helps.
“Build me a print ready file to these specs.”… it’s like a drunk cat attacked my computer. Just confidently puts out total nonsense.
At the very least, you can (and probably should) put a plaintext netlist and BOM into an LLM to triple-check your work.
It will use the part datasheets and decent EE logic to cross reference pins and parts and polarities and generally check that SMD caps and resistors have realistic specifications for their footprint, switching regulators and communication ICs are configured correctly, nets are named and linked correctly, buses have the correct and consistent termination, and so on.
It's game over once AI figures out autorouting.
A personal data point: I had Claude Opus 4.8 design a fairly textbook circuit that outputs a monochrome image burned in an EEPROM over standard 640x480 VGA using only 74 series logic and GALs. It designed the circuit and GAL code, and I did the routing, and got it made through JLC for $6. After it came back, there was one error that was not caught, which I could blue-wire, and it works just fine otherwise. I was fairly impressed.
I got a flexpcb that validates in JLC and PCBWay DRC tools from the KiCAD MCP Server and Codex.
I have yet to order any or program it, but it was enough to make me push on with a PCB art project for ST-style guitar pickguards - no netlist, no problems.
I'm also foolishly toying with NeXTBus dev boards for the Cube. Is it cursed? Probably. https://github.com/itomato/NeXTBus-Dev-Board
Flying home today and got DHL notification that my vibe PCBs are on my porch. Now to try to program, bring up, and see if it works this weekend. $178 for 5 assembled units from PCBway to run this dumb experiment.
I've gone through this same workflow and had success all the way through DRC ruleset passing with JLCPCB-based settings, ordering, fab, and use. Board was ordered with PCBA (they did assembly of the stock components) and I did SMD for just the oddball module/ICs they don't stock. Works exactly to design.
My boards are for hobby use and are ridiculously simple though compared to anything professional (breakout boards for specific components in FPV drone video transmission subsystems). That's probably an important detail. I think its like anything else in AI right now. It can do it 90% of the time but that 10% can be really rough and if its a task you can't do or verify yourself, you won't know the difference.
How do I subsribe to learn the result of this test. I'm very keen to try this.
Sweet, do you have the pickguard project live somewhere? I want to check it out!
npm run art
https://github.com/wjkennedy/stratopcb/tree/main
I'm confused. Who made this benchmark and why? It is fascinating to me that people are putting this much work into trying to measure and evaluate LLMs when we've never gone to this level for humans. Sounds like they also built a harness/feedback loop so the LLM knows if it's doing a good job? Again, why didn't anyone ever do this for human engineers?
Humans learn differently from LLMs. LLMs already contain most of the information they need to be able to do PCB layout from reading the entire internet, but they need this kind of fast feedback loop to learn how to extract and use that information in the right way. Humans learn to act at the same time they learn knowledge.
because human engineers are not reproducible, so having an accurate measure of the skills of one of them has a much lower utility return (those benchmarks are very expensive to run, and making them for humans will be at least as much expensive)
I think the new Astra computer use demos show that the models might be able to do things like inspection of real world objects if given a camera.
Super excited to see real world feedback added into the agent loops we have gotten used to working with. Could you let the model print and test the circuit boards it is prototyping with a jig?
In July I was struggling with writing DIY Rust firmware for an e-ink screen. I mistakenly thought I'd ordered an Inkplate 6 ED060SC7 and actually had the later version, which confounded my efforts. I was also mistaken about the pixel resolution.
The way I found this out is I propped it up next to a webcam so it was more or less full frame, and I had the (then new) Fable write a python script to bezier warp the camera capture to a flat projection of the screen. At that point I couldn't address the whole screen. Once I'd guided the capture script I just left the LLM overnight with the instruction to get full control confirmed by a capture round trip, and it was meaningfully finished in a couple of hours. I don't really have the skills to attempt that myself in a reasonable time frame.
Yes — this is already possible to a useful degree. I’ve been using tscircuit, where you can describe circuits in code and use AI agents to generate and iterate on schematics and PCB layouts. I’ve seen designs from this workflow go all the way to fabricated, working hardware.
Interesting. AI may be pretty good at managing the gazillion different things that make up circuit boards.
Frankly, I am surprised that it isn't already a solved problem, as we've had silicon compilers, for many years, and I always figured that IC design is more difficult than PCB.
Just as some anecdatum, I've tried this a couple of times over the last few years. 2024-2025, no body could generate anything of any sort of complexity, however maybe around the Opus 4.7 timeline, I tried having it generate some relatively simple and one slightly more complex circuit around 8-bit PIC micros, and was very pleasantly surprised. The schematics for simpler stuff is usually OK, often with some of the kind of crazy belt-and-suspenders stuff you see in code (how many decoupling caps do I actually need here, Claude?), and PCB layout has basically been terrible every time I've tried it.
Recent models can generate mostly competent schematics if you're using well known parts, feed them data sheets (and you _must_ feed them the errata too) and it's not too complex. Any complexity analog or RF, everything falls down quickly if you know what you're looking at. Maybe Astra will do better? There's still so much implicit knowledge that a good designer (not me by a long shot, but I know a little) will bake into a board, and even as cheap as JCLPCB is now, you don't want to have to spin a half dozen revs because Claude or Codex hallucinated. It will be genuinely interesting to see what happens on this front.
I never thought about it until now, but as someone with zero skill in this domain I feel interested in making my own vibetronics now.
Works very well with EAGLE XML files, especially if you give it the EAGLE.DTD schema that CadSoft provides. Both .sch and .brd files can be extensively manipulated by Claude (and likely everybody else at this point.)
the answer is a resounding yes: https://github.com/seanwevans/pynq_butterfly
For stuff like this, definitely. It’s already exciting. Claude et al are surprisingly competent with real hardware. I’ve also had good success debugging basic issues with micros - not via the debugger, but giving datasheet and written analysis. I’m sure it could do things like capture from a connected oscilloscope (Salaeae have an MCP integration for their logic tools). Also for SPICE which I never bother with because the tools always felt stuck in the 90s, but having some serious automated analog review would be nice. I’m really excited to play with peripherals like the RP2040 or Beaglebone real time logic blocks, or improving/adding real USB support instead of USB serial.
For circuits I would trust a netlist or DRC/ERC review, checking a design against reference circuit values, probably not placement as that tends to be guided straight from a datasheet where it makes a difference and I like the artistic side of it.
that's FPGA firmware isn't it?
I have tried to use AI on this from GPT3, documenting a few of the attempts on my blog. They know the datasheets and the theory enough to be extremely useful at the start, but the most recent ones are becoming scary good. Routing a PCB is certainly a bit more than joining some wires, you have to accommodate power traces, ground loops, component overlap and all the various RF tricks and tips, but I do believe we are less than a year from prompt to full assembly including enclosure.
I have found that AI is great for sorting out your libraries, drawing and helping double check footprints managing your BOM. I still draw my schematics and layout my boards. I haven’t found it any good for those tasks yet. There are too many stupid errors it makes. I do get it to review my work though and often it catches things I have missed.
I'm excited by the option in Astra to route PCBs.
The next pain-point is sourcing the components from Digikey, LCSC, etc., and finding suitable substitutes if necessary.
Of course this assumes the LLMs can already read datasheets because that's the biggest pain-point in designing electronics. It's like filling out tax forms.
Finally it would be great if LLMs could extract simulation models from datasheets!
Sourcing parts has been a huge motivation for this. We are still pretty jarred by the supply chain crisis a few years ago. Having the ability to formally verify that an alternative part fulfills the original design constraints is huge.
Not sure whether you played with llm datasheet extraction lately, but they are working their way up there. Even started reasoning about curves and footnotes.
Where do you think is the biggest pain point: part discovery, datasheet verification or substitute determination?
Eurisko was used for VLSI chip design, then used rules discovered there to design TCS Traveler winning fleet.
It is unbelievable how artificial intelligence walk in circles.
Routing is the easiest part of layout, and there have been auto-routers (which mostly worked) for a long time now; the difficult task is in placing components, and adjusting those placements.
Sure, I was just using "route" to mean also floorplanning because that's what Astra's routing tool also does. Not sure how they name it. Sorry if I was sloppy.
Not at all, but the term for routing and placing the components is ‘layout’. Floorplanning usually refers to the laying out of larger blocks of the circuit (with the blocks themselves already having been laid-out).
Yes but I'm not so concerned with how people name these steps.
If you give it a real solver to drive and business constraints to operate under, it will do pretty good job (which is to map your business rules to constraints and balanced costs, develop deterministic tests, probabilistic tests, acceptance gates, all boring stuff).
Really excited about constraint solvers making a comeback! Especially after seeing Z3 getting a mention in anthropics fermat post. EEbench is using atopile's internal constraint solver for engineering parameters (think operating temperature, voltage ranges, stacked tolerances etc).
I know a startup [1] that tries to accomplish this.
[1] https://www.schematik.io/
Love what samuel and his team are building over in the Netherlands! Always thought it would be fun if there was a more beginner friendly version of atopile [1].
[1] https:/atopile.io/
Mostly focused on BOM, schematics, and firmware now but slowly expanding to circuit boards if we feel we can deliver real value there. Early experiments with Astra are promising
Love the democratization of access. Still curious how this is better vs gpt 6 astra or grok 4.7 in the future trained on spacex problems ?
Have tried this a few times. While AI has been helpful in talking through component choices and other topics, it’s consistently been a spectacular fail each time in actually designing the board.
I’ve had similar results asking AI to design 3D models for 3D printing.
“Coach me on optimizing error on my printer”… helps.
“Build me a print ready file to these specs.”… it’s like a drunk cat attacked my computer. Just confidently puts out total nonsense.
LLMs like to fail spectacularly at real life measurement and not tell you about it as well