知Yizhi Daily
← Back to this edition

Conversations

Developer uses coding agents to run AI inference on FPGA and achieves a 33x speedup

Edition 2026-11-21 · 08:00 Beijing time

A developer with a machine learning background documents the process of using coding agents for FPGA development. On an AWS F2 instance, they tried adapting DeepGrove's ternary-weight architecture and, through hundreds of experiments, achieved a 33x speedup over the first full-model bit-exact baseline, at a token cost of about $10000.

The author says coding agents allowed them to enter unfamiliar areas such as low-level software and hardware. They previously had almost no FPGA experience and only knew that FPGAs can change the logic and routing connections on the chip through configuration, thereby gaining performance advantages over general-purpose CPUs; traditional development usually involves substantial C++, Verilog, and manual waveform analysis.

Inspired by demand for AI inference, the author used an AWS F2 instance with FPGA hardware. The instance has 16 GB of FPGA HBM, a relatively limited amount of memory. At the same time, they came across the ternary-weight architecture discussed by DeepGrove's founder, considered it suitable for memory-constrained settings, and tried to get the model running on an FPGA at a useful speed.

The experiments were carried out over several days by cloud agents running in Conductor, with the author mainly offering regular encouragement and a few machine-learning-oriented hints, such as considering speculative decoding or porting out parts running on the CPU. The agents eventually got the complete model running and achieved a 33x speedup relative to the first full-model bit-exact baseline.

According to the author's statistics, throughout the automated research process, the experiment history produced about 3000000 lines of code, parsed about 30000000 lines of output, ran hundreds of experiments, and consumed about $10000 worth of Claude and OpenAI tokens. The code was mainly C++, with some Verilog; the author says they had not written substantial C++ for about a decade and had never written Verilog.

Original sources

Hacker News ↗
Back to this editionOpen interactive reader