Compiler-based Processor Network Generation for Neural Networks on FPGAs

Inhalt:
The fast-moving field of Neural Network (NN) research has brought breakthroughs in application domains like image or natural language processing. However, since new techniques are constantly emerging, fast prototyping of Neural Networks on edge devices like Field-Programmable Gate Arrays (FPGAs) is particularly appealing due to their reconfigurability. Existing compiler-based toolchains generating specialized hardware are rare and restricted in their scope and coverage. In this paper, we propose a novelNNmodel-to-hardware flowthat exploits the capabilities of Application-Specific Instruction Set Processors (ASIPs) and networks that can be built to implement NNs in a scalable and mixed hardware/software-reconfigurable way. Starting with an NN specification in PyTorch, an ASIP network generation toolchain is presented in which each instantiated core is assigned a portion of neurons to execute a given NN in a parallel and pipelined way while tailoring the instruction set of each core to the computations of the neurons. In addition, we propose multiple ML-specific instruction set extensions realized in hardware, including bit width-restricted ALU operations and activation functions with the circuits automatically generated from a given C/C++ implementation. Experimental results include a design space exploration of alternative implementations for the design objectives of FPGA resource usage, latency, and throughput. Our approach is shown to achieve higher speed-ups than similar processor-based approaches while consuming lower amounts of resources.