SGAnalog test reveals models’ inability to handle analog circuits and sudden ban policies

The promises of AI-driven automation of electronic chip design appear attractive to anyone following the race for technological sovereignty, but moving analog schematics from paper to actual silicon collides with a very different reality. We examined a new research paper accepted at the AI-for-Chip-Design workshop of the NIPS 2026 conference by researchers Yuting Li and Wei-Hang Ding, which introduces a comprehensive practical evaluation benchmark called SGAnalog, built on open-source circuits from Tiny Tapeout to test whether models can truly grasp circuit engineering or merely reproduce examples they have memorized.

The researchers relied on 273 topologically distinct designs created by human engineers, and each circuit was processed within a tightly controlled container environment using precise modification timestamps to test the models' temporal training limits. The benchmark extracts the schematic image and the corresponding SPICE netlist from the original source file to ensure an exact structural reference, then evaluates the models along two independent tracks: converting the graphical schematic to a netlist, and determining transistor and component dimensions via the simulation test platforms attached to the original designs.

The results revealed a wide gap between theoretical promises and actual structural capability.Across 66 schematic-to-netlist conversion tasks evaluated on seven leading models, the strongest model achieved only 56.1 percent on the full structural-graph matching metric. Notably, six of the seven models suffered a sharp drop in accuracy when moving from simple to medium-complexity circuits, while a test of one advanced model showed that removing the textual labels placed by designers reduced the full match score, indicating that the models rely partially on reading auxiliary text to trace electrical connections rather than understanding abstract geometric pathways.

On the component-dimension determination track, which comprised 17 specialized tasks, the top-performing model delivered complete proposals for all tasks, scoring 91.2 out of 100 against the human reference. The surprise came from the behavior of the newer Claude model, which refused to answer 4 and 11 of the same tasks citing internal safety policies, even though it had successfully reproduced earlier schematics without objection; because a proposal lacking dimensions receives an automatic zero, this refusal caused its ranking to drop, confirming that the obstacle in some families was not a technical shortfall but a clash with sudden safety policies.

This disparity imposes strict verification rules on hardware engineering teams and emerging semiconductor initiatives in the Gulf and Egypt.Relying on closed-source cloud model interfaces to automate silicon circuits carries a double risk: on the one hand, computer-vision capability remains unable to guarantee correct connectivity for medium-complexity circuits more than half of the time, threatening costly failures when moving to fabrication; on the other hand, sudden algorithmic safety blocks can disrupt software production lines without warning. The decisive skill for engineers in our region therefore becomes building local verification pipelines that use open-source simulation environments such as SPICE to automatically check every electrical node before accepting any model output, rather than relying on generic assessments that do not hold up under real-world manufacturing tests.

The new benchmark showed that model rankings for visual schematic reading differ entirely from their rankings for physical measurement tuning, and that success in code generation does not automatically translate into a understanding of hardware laws. For engineers and developers in the region, the choice is clear: shift from fascination with generative interfaces to building rigorous testing platforms that measure circuit efficiency in defined operating environments, because the cost of error in the silicon world is not remedied by a quick software update but is measured in lost wafers on production lines.