ChipAgents has introduced Build-A-Chip-Bench, a benchmark developed in collaboration with NVIDIA (NASDAQ: NVDA) to evaluate whether AI agents can complete digital chip design from initial specifications through physical implementation and verified GDS output.
The benchmark addresses a gap in existing hardware-focused AI evaluations, which have largely concentrated on register-transfer-level (RTL) code generation and functional verification. A chip design must also meet timing, power and area requirements, complete physical verification and produce a layout suitable for manufacturing.
ChipAgents, an agentic AI platform for semiconductor engineering, said the benchmark is intended to measure progress toward autonomous chip engineering by evaluating decisions across the full design flow.
“As chip designs get more complex and time to market pressures increase, we see the need for chip superintelligence that can fully engineer the next generation of silicon,” said William Wang, founder and CEO of ChipAgents.
Initial Results Reveal Implementation Challenges
Initial evaluations found that the tested AI agents passed functional verification on approximately 40% of tasks on average. Only about 10% met the benchmark’s required physical performance, power and area (PPA) targets.
The results highlight the difference between generating RTL that functions as intended and completing a design that meets physical implementation requirements. A design may pass simulation yet fail timing requirements, exceed its area or power budget, encounter routing problems or contain physical-design violations.
Autonomous chip-design systems must also preserve validated behavior as designs change, use specialized engineering tools and account for feedback from computationally expensive physical-design stages.
Build-A-Chip-Bench evaluates these challenges across interconnected stages rather than treating individual engineering tasks as isolated problems. The assessment covers architecture, microarchitecture, RTL, functional verification, logic synthesis, floor planning, placement, clock-tree synthesis, routing, timing and power signoff, and physical verification.
This end-to-end approach is designed to show where AI agents struggle as they move from design intent to physical implementation.
Measuring Progress Across the Design Flow
“Chip design is a defining test of AI’s ability to make decisions that hold up under the demands of the physical world,” said Tim Costa, vice president and general manager of industrial and computational engineering at NVIDIA.
Costa said the benchmark tests whether agents’ decisions remain valid from design intent through physical implementation, giving developers a clearer view of where reasoning and tool use need to improve.
The benchmark’s emphasis on downstream implementation reflects a broader challenge for AI in semiconductor engineering. Progress in code generation alone does not establish that an agent can manage the dependencies, trade-offs and iterative decisions required to complete a chip design.
An agent must reason across multiple levels of abstraction, maintain functional correctness through revisions and respond to results from specialized design tools. These requirements become particularly important when physical implementation reveals problems that require earlier design decisions to be revisited.
Build-A-Chip-Bench provides a framework for assessing these capabilities and tracking improvements in autonomous engineering systems. Its initial findings suggest that achieving reliable end-to-end chip design will require advances beyond RTL generation, particularly in meeting physical design constraints.
ChipAgents did not disclose the number of tasks evaluated, the individual AI systems tested or comparative results by model. These details would help establish how broadly the initial findings apply across different chip designs and agent architectures.
