Overview

Coral NPU offers standards-based, easy-to-use developer tools that support multiple ML frameworks. The tools leverage existing compiler and library support from the broader RISC-V ecosystem, simplifying development.

Coral NPU is the ideal hardware accelerator for running code developed with PyTorch, JAX, or LiteRT (formerly TensorFlow Lite). Because Coral NPU is an open-source architecture, it can also be customized to facilitate software development.

Guidance for software development falls into two main categories:

  • Deploying ML models on small devices such as AR glasses and other wearables that have ambient environment sensing devices.
  • Building MCU micro-applications for edge applications, using ML kernels.

Coral NPU's software development advantages

Development feature Advantages
Coral NPU is designed to be simple for compilers to support even with ML domain specialization.
  • Compute requirements in ML models are dominated by matrix multiplication.
  • Outer-product multiplication circuit enables N*M*K MACs/cycle compute, with typical values of N=8, M=8, K=4, N*M*K=256 (MAC = multiply-accumulate). This can be parameterized to suit the application domain.
  • Convolution, etc., can be expressed using matmuls as sub-operations.
  • In a fixed-function accelerator, only matmuls are performed, with limited control flow.
When programmed in C, Coral NPU:
  • Is an embedded processor capable of single-context compute offload
  • Allows for C++ intrinsics for bootstrapping workloads
For ML micro-kernels, Coral NPU has:
  • Math library containing a combination of hand-tuned and templated Operators (ML)
  • TFLM implementations of common ML kernels
  • IREE support via baseline RISC-V scalar with Operator kernels
For North Star:
  • Graph refactoring to extract optimal memory layouts/paddings
  • IREE polyhedral compilation and LLVM auto-vectorization
  • Push-button / high-leverage tooling

Getting started

To get started developing software for Coral NPU, read the following topics: