Skip to main content

Research project

TileCNN

A reproducible model-to-FPGA framework for running different quantized CNNs on one reusable INT8 accelerator, without rebuilding the FPGA for each network.

TileCNN overview. ResNet-50, MobileNetV2, and other CNNs are quantized by FixQuant, compiled by one target-aware compiler, and run on a single ZCU104 FPGA target without an FPGA rebuild.

TileCNN is my current FPGA-accelerator research platform. It is an open hardware–software framework that maps quantized convolutional neural networks onto a reusable FPGA accelerator. It connects model export, target-aware compilation, tiled execution, deployment, validation, and measurement, so that the FPGA implementation does not have to be regenerated for every network it runs.

Research problem

Neural-network accelerators are often built around one dataflow or a narrow set of layer shapes. That specialization can deliver excellent peak performance, but it makes the design expensive to retarget as networks change: a new model often means a new bitstream. Useful deployment needs both efficiency and enough flexibility to handle layers with different spatial sizes, channel counts, kernel sizes, and strides.

Approach

TileCNN separates the reusable hardware target from the model-specific mapping. The accelerator, its kernel ABI, and the host configuration are fixed once a target is built. Changing networks changes the compiled schedule, parameters, and tensor layout, not the FPGA design.

TileCNN framework architecture. A quantized CNN is compiled per network by a target-aware compiler into descriptor streams, packed parameters, and an activation-memory plan. A host runtime and the TileCNN accelerator are fixed per target. Input samples, outputs, and measurements change per run.
The framework separates what is generated per network (green), what is fixed per target (blue), and what changes per run (orange).

The framework has four parts:

  • FixQuant exports a quantized model in a hardware-independent form. It is the model-preparation side of TileCNN and is described on its own project page.
  • The TileCNN compiler validates and lowers the network graph, selects legal tiles, partitions channels, builds schedules, and emits kernel descriptors for a specific target.
  • The host runtime packs tensors, allocates device memory, runs the graph through XRT, collects outputs, and records measurements.
  • The FPGA kernel executes the descriptor stream on a signed-INT8 tiled datapath that supports both dense and depthwise-separable CNNs.
TileCNN accelerator block diagram. A control unit fetches the schedule and feeds input-feature-map, weight, and residual loaders. These drive a convolution MAC engine with parallel spatial lanes, followed by accumulators and a quantizer, fused tail operations, and an output buffer that writes back to DDR memory.
The accelerator datapath. A descriptor-driven control unit feeds a tiled MAC engine, followed by accumulation, requantization, and fused residual, pooling, and activation stages.

Three versioned artifacts hold the workflow together and make runs reproducible:

ArtifactContents
ModelPackageQuantized graph, tensors, fixed-point metadata, checksums, a validation input, and its integer reference output
TargetPackageMatching host program, FPGA binary, boot image identity, kernel ABI, capabilities, and build provenance
RunRecordModel and target identities, dataset and preprocessing, runtime settings, accuracy or reference status, timing

Current scope

PropertyCurrent support
FPGA platformAMD/Xilinx ZCU104 (XCZU7EV), Vitis/Vivado 2024.2, 200 MHz kernel clock
ArithmeticSigned INT8 activations and weights with wider accumulation
ExecutionBatch-one, descriptor-driven tiled inference
Validated networksResNet-50 and MobileNetV2
ConvolutionsDense 1×1, 3×3, and 7×7 (stride 1 or 2) and depthwise
Other operationsResidual add, max pooling, global average pooling, activation, output quantization

Results

The framework was evaluated on 3,923 ImageNet-mini validation images with two structurally different CNNs. Three hardware targets were built. They differ in the number of parallel spatial lanes and in how partial sums are buffered. Each target runs both networks without a model-specific FPGA rebuild.

Kernel speedup over the P2-PP target compared with added compute. P4-PP doubles the compute: ResNet-50 gains 1.59 times and MobileNetV2 1.16 times. P8-S has four times the compute: ResNet-50 gains 1.94 times and MobileNetV2 1.17 times. Kernel speedup over the P2-PP target compared with added compute. P4-PP doubles the compute: ResNet-50 gains 1.59 times and MobileNetV2 1.16 times. P8-S has four times the compute: ResNet-50 gains 1.94 times and MobileNetV2 1.17 times.
The grey track is the compute each target adds over P2-PP. The coloured bar is the kernel speedup each network actually achieves.

Kernel latency per image and DSP use for each target. The number in each target’s name is its count of spatial lanes. PP means ping-pong partial-sum buffering, and S means a single buffer.

TargetResNet-50MobileNetV2DSP use
P2-PP54.97 ms9.49 ms25.8%
P4-PP34.65 ms8.18 ms40.6%
P8-S28.34 ms8.12 ms70.4%

ResNet-50 keeps benefiting from wider targets, while MobileNetV2 gains little beyond P4-PP. This is the framework’s central finding: a reusable target should be chosen for the workload mix it will actually serve, not for theoretical peak throughput alone.

Status

This is active PhD research at the DeepHERO Lab, Mälardalen University. A manuscript describing the complete framework is in preparation, and a separate study examines layer-dependent scheduling bottlenecks in tiled FPGA CNN acceleration and the schedule optimizations that address them.